Geek Answers Handbook

Run wget and other commands in a shell script

I am trying to create a shell script to download the latest Atomic versions to my server, unzip them, copy them to the right folder, etc.,

For most of the day, I read tutorials and forum posts, and the syntax eluded some of them. I have executed all these commands, and I know that they work if I manually run them.

I know that I need to develop some error checking, but I'm just trying to get the teams to work correctly. The main problem at the moment is the syntax of the wget commands, I have errors regarding missing half colonies, division by zero, unsupported schemes - I tried using different quotes (single and double) and escape characters in different combinations.

Thanks for any help. Raw wget command -

wget --user="jim" --password="xxx-yyy-zzz" "http://updates.atomicorp.com/channels/rules/subscription/VERSION"

 #!/bin/sh update_modsec_rules(){ wget=/usr/bin/wget tar=/bin/tar apachectl=/usr/bin/apache2ctl TXT="Script Run Finished" WORKING_DIR="/var/asl/updates" TARGET_DIR="/usr/local/apache/conf/modsec_rules/" EXISTING_FILES="/var/asl/updates/modsec/*" EXISTING_ARCH="/var/asl/updates/modsec-*" WGET_OPTS='--user=jim --password=xxx-yyy-zzz' URL_BASE="http://updates.atomicorp.com/channels/rules/subscription" # change to working directory and cleanup any downloaded files and extracted rules in modsec/ directory cd $WORKING_DIR rm -f $EXISTING_ARCH rm -f $EXISTING_FILES rm -f VERSION* # wget to download VERSION file $wget ${WGET_OPTS} "${URL_BASE}/VERSION" # get current MODSEC_VERSION from VERSION file and save as variable source VERSION TARGET_DATE=$MODSEC_VERSION echo $TARGET_DATE # wget to download current archive $wget ${WGET_OPTS} "${URL_BASE}/modsec-${TARGET_DATE}.tar.gz" # extract archive echo "extracting files . . . " tar zxvf $WORKING_DIR/modsec-${TARGET_DATE}.tar.gz echo "copying files . . . " cp -uv $EXISTING_FILES $TARGET_DIR echo $TXT } update_modsec_rules $@ 2>&1 | tee -a /var/asl/modsec_update.log RESTART_APACHE="/usr/local/cpanel/scripts/restartsrv httpd" $RESTART_APACHE
+7
bash shell wget
source share
2 answers

Here are some guidelines to use when writing shell scripts.

  • Always specify variables when using them. This helps to avoid the possibility of misinterpretation. (What if the file name contains a space?)
  • Do not trust file manipulation with commands like rm . Use for loops instead. (What if the file name begins with a hyphen?)
  • Avoid subshells whenever possible. Your lines with backquotes make me itchy.
  • Do not follow if you can help. And especially don't expect any parts of your script after your exec will actually run.

I must point out that although your shell may be bash, you specified /bin/sh to execute this script, so this is NOT a bash script.

Corresponding here with some error checking. Add salt to taste.

 #!/bin/sh # Linux wget=/usr/bin/wget tar=/bin/tar apachectl=/usr/sbin/apache2ctl # FreeBSD #wget=/usr/local/bin/wget #tar=/usr/bin/tar #apachectl=/usr/local/sbin/apachectl TXT="GOT TO THE END, YEAH" WORKING_DIR="/var/asl/updates" TARGET_DIR="/usr/local/apache/conf/modsec_rules/" EXISTING_FILES_DIR="/var/asl/updates/modsec/" EXISTING_ARCH="/var/asl/updates/" URL_BASE="http://updates.atomicorp.com/channels/rules/subscription" WGET_OPTS='--user="jim" --password="xxx-yyy-zzz"' if [ ! -x "$wget" ]; then echo "ERROR: No wget." >&2 exit 1 elif [ ! -x "$apachectl" ]; then echo "ERROR: No apachectl." >&2 exit 1 elif [ ! -x "$tar" ]; then echo "ERROR: Not in Kansas anymore, Toto." >&2 exit 1 fi # change to working directory and cleanup any downloaded files # and extracted rules in modsec/ directory if ! cd "$WORKING_DIR"; then echo "ERROR: can't access working directory ($WORKING_DIR)" >&2 exit 1 fi # Delete each file in a loop. for file in "$EXISTING_FILES_DIR"/* "$EXISTING_ARCH_DIR"/modsec-*; do rm -f "$file" done # Move old VERSION out of the way. mv VERSION VERSION-$$ # wget1 to download VERSION file (replaces WGET1) if ! $wget $WGET_OPTS $URL_BASE}/VERSION; then echo "ERROR: can't get VERSION" >&2 mv VERSION-$$ VERSION exit 1 fi # get current MODSEC_VERSION from VERSION file and save as variable, # but DON'T blindly trust and run scripts from an external source. if grep -q '^MODSEC_VERSION=' VERSION; then TARGET_DATE="`sed -ne '/^MODSEC_VERSION=/{s/^[^=]*=//p;q;}' VERSION`" echo "Target date: $TARGET_DATE" fi # Download current archive (replaces WGET2) if ! $wget ${WGET_OPTS} "${URL_BASE}/modsec-$TARGET_DATE.tar.gz"; then echo "ERROR: can't get archive" >&2 mv VERSION-$$ VERSION # Do this, don't do this, I don't know your needs. exit 1 fi # extract archive if [ ! -f "$WORKING_DIR/modsec-${TARGET_DATE}.tar.gz" ]; then echo "ERROR: I'm confused, where my archive?" >&2 mv VERSION-$$ VERSION # Do this, don't do this, I don't know your needs. exit 1 fi tar zxvf "$WORKING_DIR/modsec-${TARGET_DATE}.tar.gz" for file in "$EXISTING_FILES_DIR"/*; do cp "$file" "$TARGET_DIR/" done # So far so good, so let restart apache. if $apachectl configtest; then if $apachectl restart; then # Success! rm -f VERSION-$$ echo "$TXT" else echo "ERROR: PANIC! Apache didn't restart. Notify the authorities!" >&2 exit 3 fi else echo "ERROR: Apache configs are broken. We're still running, but you'd better fix this ASAP." >&2 exit 2 fi

Please note that although I rewrote this to be more reasonable, there is certainly still much room for improvement.

+10
source share

You have two options:

1- changing it to

 WGET1=' --user="jim" --password="xxx-yyy-zzz" "http://updates.atomicorp.com/channels/rules/subscription/VERSION"'

then run

wget $WGET1 the same with WGET2

or

2- encapsulation of $ WGET1 with reverse windows ``. eg:.

 `$WGET`

This applies to any command that you execute from a variable.

Proposed Changes:

 #!/bin/sh TXT="GOT TO THE END, YEAH" WORKING_DIR="/var/asl/updates" TARGET_DIR="/usr/local/apache/conf/modsec_rules/" EXISTING_FILES="/var/asl/updates/modsec/*" EXISTING_ARCH="/var/asl/updates/modsec-*" WGET1='wget --user="jim" --password="xxx-yyy-zzz" "http://updates.atomicorp.com/channels/rules/subscription/VERSION"' WGET2='wget --user="jim" --password="xxx-yyy-zzz" "http://updates.atomicorp.com/channels/rules/subscription/modsec-$TARGET_DATE.tar.gz"' ## change to working directory and cleanup any downloaded files and extracted rules in modsec/ directory cd $WORKING_DIR rm -f $EXISTING_ARCH rm -f $EXISTING_FILES ## wget1 to download VERSION file `$WGET1` ## get current MODSEC_VERSION from VERSION file and save as variable source VERSION TARGET_DATE=`echo $MODSEC_VERSION` ## WGET2 command to download current archive `$WGET2` ## extract archive tar zxvf $WORKING_DIR/modsec-$TARGET_DATE.tar.gz cp $EXISTING_FILES $TARGET_DIR ## restart server exec '/usr/local/cpanel/scripts/restartsrv_httpd' $*;

Pro Tip: if you need string substitution, using $ {VAR} is much better than disambiguating, for example:

 tar zxvf $WORKING_DIR/modsec-${TARGET_DATE}.tar.gz
+1
source share

More articles:


All Articles