Common shell script template

Question

Millions of developers write shell scripts to solve various problems. I use shell scripts to simplify deployment, lifecycle management, installation, or simply as a glue language .

What I noticed is that nobody cares about the style and quality of shell scripts. Many teams spend many hours fixing Java, C ++, ... style problems, but completely ignore the problems in their shell scripts. By the way, there is usually no standard way to implement a shell script within a specific project, so you can find dozens of different, ugly and buggy scripts distributed around the code base.

To overcome this problem in my projects, I decided to create a script shell template that is versatile and good enough. I will provide my templates to make this question more useful. From these templates, you can:

• Handling Command Line Arguments
• synchronization
• Basic help

Argument processing: getopts (latest version: shell- script -template @github )

#!/bin/bash # ------------------------------------------------------------------ # [Author] Title # Description # ------------------------------------------------------------------ VERSION=0.1.0 SUBJECT=some-unique-id USAGE="Usage: command -ihv args" # --- Options processing ------------------------------------------- if [ $# == 0 ] ; then echo $USAGE exit 1; fi while getopts ":i:vh" optname do case "$optname" in "v") echo "Version $VERSION" exit 0; ;; "i") echo "-i argument: $OPTARG" ;; "h") echo $USAGE exit 0; ;; "?") echo "Unknown option $OPTARG" exit 0; ;; ":") echo "No argument value for option $OPTARG" exit 0; ;; *) echo "Unknown error while processing options" exit 0; ;; esac done shift $(($OPTIND - 1)) param1=$1 param2=$2 # --- Locks ------------------------------------------------------- LOCK_FILE=/tmp/$SUBJECT.lock if [ -f "$LOCK_FILE" ]; then echo "Script is already running" exit fi trap "rm -f $LOCK_FILE" EXIT touch $LOCK_FILE # --- Body -------------------------------------------------------- # SCRIPT LOGIC GOES HERE echo $param1 echo $param2 # ----------------------------------------------------------------- 

Shell flags (shFlags) simplifies command line arguments, so at some point I decided not to ignore this possibility.

Argument processing: shflags (latest version: shell- script -template @github )

 #!/bin/bash # ------------------------------------------------------------------ # [Author] Title # Description # # This script uses shFlags -- Advanced command-line flag # library for Unix shell scripts. # http://code.google.com/p/shflags/ # # Dependency: # http://shflags.googlecode.com/svn/trunk/source/1.0/src/shflags # ------------------------------------------------------------------ VERSION=0.1.0 SUBJECT=some-unique-id USAGE="Usage: command -hv args" # --- Option processing -------------------------------------------- if [ $# == 0 ] ; then echo $USAGE exit 1; fi . ./shflags DEFINE_string 'aparam' 'adefault' 'First parameter' DEFINE_string 'bparam' 'bdefault' 'Second parameter' # parse command line FLAGS "$@" || exit 1 eval set -- "${FLAGS_ARGV}" shift $(($OPTIND - 1)) param1=$1 param2=$2 # --- Locks ------------------------------------------------------- LOCK_FILE=/tmp/${SUBJECT}.lock if [ -f "$LOCK_FILE" ]; then echo "Script is already running" exit fi trap "rm -f $LOCK_FILE" EXIT touch $LOCK_FILE # -- Body --------------------------------------------------------- # SCRIPT LOGIC GOES HERE echo "Param A: $FLAGS_aparam" echo "Param B: $FLAGS_bparam" echo $param1 echo $param2 # ----------------------------------------------------------------- 

I think that these templates can be improved to further simplify the life of the developer.

So, the question is how to improve them in order to have the following:

• built-in log
• improved error handling
• improved tolerance
• smaller size
• built-in runtime tracking

Answer 1

I would avoid relying on bash as a shell and modeling your solution on top of the shell syntax defined by POSIX and using /bin/sh in shebang. Recently, we had some surprises when Ubuntu changed /bin/sh to dash .

Another pandemic in the shell world is a general misunderstanding of exit status codes. An exit with clear code is what allows other shell scripts to programmatically respond to specific failures. Unfortunately, there is not much guidance on the header file sysexits.h .

If you're looking for more information on good shell scripting, focus on the Korn shell scripting resources. Ksh programming tends to focus on really programming, not scripting randomness.

Personally, I did not find much use for shell templates. Unfortunately, most engineers simply copy and paste your template and continue to write the same sloppy shell code. The best approach is to create a library of shell functions with well-defined semantics, and then convince others to use them. This approach will also help in managing change. For example, if you find a defect in a template, then every script that was based on it is broken and will require changes. Using the library allows you to fix defects in one place.

Welcome to the world of shell scripting. Shell scripting is a bit of a lost art that seems to be entering the Renaissance. In the late 90s, good books were written on this subject - UNIX Shell Programming by Burns and Arthur , although Amazon reviews for this book seem awful. IMHO, efficient shell code embraces the UNIX philosophy described by Eric S. Raymond in The Art of Unix Programming.

Answer 2

This is the title of my shell script template (which can be found here: http://www.uxora.com/unix/shell-script/18-shell-script-template ).

This is the appearance of man , which is used when using () for diplsay reference.

 #!/bin/ksh #================================================================ # HEADER #================================================================ #% SYNOPSIS #+ ${SCRIPT_NAME} [-hv] [-o[file]] args ... #% #% DESCRIPTION #% This is a script template #% to start any good shell script. #% #% OPTIONS #% -o [file], --output=[file] Set log file (default=/dev/null) #% use DEFAULT keyword to autoname file #% The default value is /dev/null. #% -t, --timelog Add timestamp to log ("+%y/%m/%d@%H:%M:%S") #% -x, --ignorelock Ignore if lock file exists #% -h, --help Print this help #% -v, --version Print script information #% #% EXAMPLES #% ${SCRIPT_NAME} -o DEFAULT arg1 arg2 #% #================================================================ #- IMPLEMENTATION #- version ${SCRIPT_NAME} (www.uxora.com) 0.0.4 #- author Michel VONGVILAY #- copyright Copyright (c) http://www.uxora.com #- license GNU General Public License #- script_id 12345 #- #================================================================ # HISTORY # 2015/03/01 : mvongvilay : Script creation # 2015/04/01 : mvongvilay : Add long options and improvements # #================================================================ # DEBUG OPTION # set -n # Uncomment to check your syntax, without execution. # set -x # Uncomment to debug this shell script # #================================================================ # END_OF_HEADER #================================================================ 

And here are the usage functions:

  #== needed variables ==# SCRIPT_HEADSIZE=$(head -200 ${0} |grep -n "^# END_OF_HEADER" | cut -f1 -d:) SCRIPT_NAME="$(basename ${0})" #== usage functions ==# usage() { printf "Usage: "; head -${SCRIPT_HEADSIZE:-99} ${0} | grep -e "^#+" | sed -e "s/^#+[ ]*//g" -e "s/\${SCRIPT_NAME}/${SCRIPT_NAME}/g" ; } usagefull() { head -${SCRIPT_HEADSIZE:-99} ${0} | grep -e "^#[%+-]" | sed -e "s/^#[%+-]//g" -e "s/\${SCRIPT_NAME}/${SCRIPT_NAME}/g" ; } scriptinfo() { head -${SCRIPT_HEADSIZE:-99} ${0} | grep -e "^#-" | sed -e "s/^#-//g" -e "s/\${SCRIPT_NAME}/${SCRIPT_NAME}/g"; } 

Here is what you should get:

 # Display help $ ./template.sh --help SYNOPSIS template.sh [-hv] [-o[file]] args ... DESCRIPTION This is a script template to start any good shell script. OPTIONS -o [file], --output=[file] Set log file (default=/dev/null) use DEFAULT keyword to autoname file The default value is /dev/null. -t, --timelog Add timestamp to log ("+%y/%m/%d@%H:%M:%S") -x, --ignorelock Ignore if lock file exists -h, --help Print this help -v, --version Print script information EXAMPLES template.sh -o DEFAULT arg1 arg2 IMPLEMENTATION version template.sh (www.uxora.com) 0.0.4 author Michel VONGVILAY copyright Copyright (c) http://www.uxora.com license GNU General Public License script_id 12345 # Display version info $ ./template.sh -v IMPLEMENTATION version template.sh (www.uxora.com) 0.0.4 author Michel VONGVILAY copyright Copyright (c) http://www.uxora.com license GNU General Public License script_id 12345 

You can get the full script template here: http://www.uxora.com/unix/shell-script/18-shell-script-template

Answer 3

If you are concerned about portability, do not use == in tests. Use = instead. Do not explicitly check if $# is 0. Instead, use ${n?error message} first time you call the required argument (for example, ${3?error message} ). This prevents the extremely annoying practice of emitting a use statement instead of an error message. And most importantly, always put the error messages in the right stream and exit with the correct status. For example:

 echo "Unknown error while processing options" >&2 exit 1; 

It is often convenient to do something like:

 die() { echo "$*"; exit 1; } >&2 

Answer 4

I will also talk about my results. The idea behind all these examples is to stimulate overall quality. It is also important to ensure that the end result is safe enough.

Logging

It is very important to have proper logging from the same beginning. I'm just trying to think about using products.

 TAG="foo" LOG_FILE="example.log" function log() { if [ $HIDE_LOG ]; then echo -e "[$TAG] $@" >> $LOG_FILE else echo "[`date +"%Y/%m/%d:%H:%M:%S %z"`] [$TAG] $@" | tee -a $LOG_FILE fi } log "[I] service start" log "[D] debug message" 

Team test

This applies to security, real life and proper error handling. May be optional.

 function is_command () { log "[I] check if commad $1 exists" type "$1" &> /dev/null ; } CMD=zip if is_command ${CMD} ; then log "[I] '${CMD}' command found" else log "[E] '${CMD}' command not found" fi 

Template Processing

It could only be my subjective opinion, but in any case. I used several different ways to create some / etc configuration directly from the script. Perl, sed, and others do the job, but they look a little scary.

I recently noticed a better way:

 function process_template() { source $1 > $2 result=$? if [ $result -ne 0 ]; then log "[E] Error during template processing: '$1' > '$2'" fi return $result } VALUE1="tmpl-value-1" VALUE2="tmpl-value-2" VALUE3="tmpl-value-3" process_template template.tmpl template.result 

Template example

 echo "Line1: ${VALUE1} Line2: ${VALUE2} Line3: ${VALUE3}" 

Result Example

 Line1: tmpl-value-1 Line2: tmpl-value-2 Line3: tmpl-value-3 

Answer 5

There is no more useful thing in a shell script than well-documented behavior with usage examples and a list of known errors. I think that no program can be called bulletproof, and errors can appear at every moment (especially when your script is used by other people), so the only thing I care about is a good coding style and use only those things that really need a script. Youre on the path to aggregation, and it will always become a large system that comes with many unused modules that are hard to port and hard to maintain. And the more the system tries to be portable, the more it grows. Seriously, shell scripts do not need to be implemented that way. They should be as small as possible in order to simplify further use.

If the system really needs something big and bulletproof, its time to think about C99 or even C ++.