Geek Answers Handbook

Echo multiple variables in one line of bash

I want to repeat two variables on the same line.
I want to save 2015-03-04.01.Abhi_Ram.txt in the FILENAME variable and 10 in the COUNT variable and repeat them at the same time.

Sample.txt

2015-03-04.01.Abhi_Ram.txt 10
2015-03-04.02.Abhi_Ram.txt 70

Below is the code I came up with:

for line in `hadoop fs -cat sample.txt` do VAR="${line}" FILENAME=`echo ${VAR}|awk '{print $1}'` COUNT=`echo ${VAR}|awk '{print $2}'` COUNT_DT=`date "+%Y-%m-%d %H:%M:%S"` echo db"|"Abhi_Ram"|"record_count"|"${FILENAME}"||"${COUNT}"||"${COUNT_DT} >> output.txt done I want the output as:

db | Abhi_Ram | record_count | 2015-03-04.01.Abhi_Ram.txt || 10 || Db timestamp | Abhi_Ram | record_count | 2015-03-04.02.Abhi_Ram.txt || 70 || Time stamp

 I'm getting the output as:

db | Abhi_Ram | record_count | 2015-03-04.01.Abhi_Ram.txt |||| time stamp
db | Abhi_Ram | record_count | 10 |||| time stamp
db | Abhi_Ram | record_count | 2015-03-04.02.Abhi_Ram.txt |||| time stamp
db | Abhi_Ram | record_count | 70 |||| timestamp

Can someone point me what I am missing?

+4
source share
2 answers

Consider:

 while read filename count do count_dt=$(date "+%Y-%m-%d %H:%M:%S") echo "db|Abhi_Ram|record_count|${filename}||${count}||${count_dt}" done <sample.txt >>output.txt

This creates a file:

 $ cat output.txt db|Abhi_Ram|record_count|2015-03-04.01.Abhi_Ram.json||10||2015-08-10 14:42:39 db|Abhi_Ram|record_count|2015-03-04.02.Abhi_Ram.json||70||2015-08-10 14:42:39

Notes:

  • It is best to use a lower or mixed case for your shell variables. The system uses uppercase variables, and you do not want to accidentally overwrite them.

  • Many double quotes in the echo statement are not needed. The entire output line can be inside the same line with two quotation marks.

  • If you want to read a file one line at a time, it is safer to use the while read ... done <inputfile . The read statement also makes it easy to define filename and count variables.

  • To substitute commands, many prefer the $(...) form in the backtick form. This is because (a) $(...) makes a visual difference between the beginning and end of command substitution, (b) the $(...) form $(...) well placed, and (c) not all fonts explicitly show backlinks that are great from ordinary ticks. (Thanks Chepner.)

  • For efficiency, redirecting to output.txt been moved to the end of the loop. Thus, the file is opened and closed only once. (Thanks to Charles Duffy.)

  • If you don’t need count_dt , updated with every single record, you can place it before the loop and set it only once each time sample.txt . If you have an updated version of bash (without Mac OSX), then the purpose of count_dt can be replaced (thanks to Charles Duffy) with a native bash instruction (no trim required):

     printf -v count_dt '%(%Y-%m-%d %H:%M:%S)T'
+6
source

John1024 explained how to do it right; I would like to see why the original version is not working. The main problem is that for loops over words, not lines. A file has two words on each line (file name and counter), so it starts a loop twice in a line. To see this, try:

 for line in `hadoop fs -cat sample.txt` do echo "$line" done

... and it will print something like:

 2015-03-04.01.Abhi_Ram.txt 10 2015-03-04.02.Abhi_Ram.txt 70

... that's not what you want at all. It also has some other nasty quirks, for example, if the input file contains the word "*", it inserts a list of file names into the current directory.

The while read ... done <file approach is the right way to iterate through the lines in a shell script. It just happens that you can also split each line into fields without having to bind to awk (in this case read filename count does this).

+1
source

More articles:


All Articles