Sed does not work from bash script

Question

I read all the similar questions on this topic, but did not find a suitable question for what I am experiencing. We apologize if this has already been answered.

Inside the bash script I wrote, there is a very simple sed command that doesn't seem to work. There are no errors, and the command works great when run from the command line.

In the output from set -x, I see that the sed command runs fine.

GNU bash version 4.3.11 (1) -release (x86_64-pc-linux-gnu)

Bash script: (softened for easier understanding)

#!/bin/bash -x # This script has the exact same sed command as used on cli contact='" [e-mail] "' sed -i "/$contact/d" /home/tim/Desktop/file.txt exit 

Shell Output:

 tim@ubuntu :~/Desktop$ cat file.txt t,b, [e-mail] tim@ubuntu :~/Desktop$ ./test.sh + contact='" [e-mail] "' + sed -i '/" [e-mail] "/d' /home/tim/Desktop/file.txt + exit tim@ubuntu :~/Desktop$ cat file.txt t,b, [e-mail] tim@ubuntu :~/Desktop$ sed -i "/" [e-mail] "/d" /home/tim/Desktop/file.txt tim@ubuntu :~/Desktop$ cat file.txt tim@ubuntu :~/Desktop$ 

I suppose I am missing something very obvious, but I look at him completely, hoping for an answer to jump off the screen and hit me in the face. Please, help: -)

Tim

Answer 1

In the mailing address in the $contact script variable, which is not in the command line call, there are double quotes:

 # case 1 - works # only the sed pattern delimiters are enclosed in quotes and these quotes will be stripped by the shell. sed -i "/" [e-mail] "/d" ./file.txt; cat file.txt # case 2 - fails # escaping with \ turns dquotes #2,3 from shell-level delimiters to char literals w/o special semantics. sed -i "/\" [e-mail] \"/d" ./file.txt; cat file.txt # case 3 - fails # Single quotes enclose the complete sed pattern spec which comprises double quotes enclosing the mail address sed -i '/" [e-mail] "/d' ./file.txt; cat file.txt # case 4 - works sed -i "/ [e-mail] /d" ./file.txt; cat file.txt # case 5 - works sed -i '/ [e-mail] /d' ./file.txt; cat file.txt 

This explains the different behavior of the script compared to calling cli.

The OP pointed out that it needs double quotes in a real script. which may be so, however, if these double letters are not present in the file, there will be no match.

The solution is to pre-process the file (if necessary, work on the copy) with sed:

 sed -i 's/,/","/g; s/^/"/; s/$/"/' ./file.txt 

This command accepts a comma separated list of items on each line without an item containing double quotes. It will wrap each element in double quotes so that they match the search pattern in the original script $contact variable.

Alternative (adapted from this SO answer [which I was not yet an author of]]

Another option is to modify the corresponding part of the script to output the second variable from $contact :

 contact='" [e-mail] "' c2=$(echo $contact | tr -d '"') sed -i "/$c2/d" /home/tim/Desktop/file.txt 

Answer 2

This is just an addition to the collapsing answer that already solved the problem.

When we use sed in a bash script, the bash script acts as a wrapper for sed. This has two goals -

• The sed command can be executed as if it were being run outside of bash.

• The bash script wrapper helps sed communicate with the outside world using environment variables.

For example, suppose a testfile contains two lines

 [e-mail] [e-mail] 

Now, if I wanted to write a bash script that would help sed replace the lines containing [e-mail] my sedscript script would be like this:

 #!/bin/bash contact=' [e-mail] ' sed -i "/$contact/d" $1 

Now I would execute the script as shown below

 ./sedscript testfile 

delete all lines containing [e-mail].

In fact, you can replace $ 1 with the actual file name. But the important point that was mentioned in the previous answer is that whenever we use the bash variable inside the sed command, always enclose the command in double quotes. Only then will bash replace the variable with the appropriate string before passing it to sed.