Entering a pipe in a script

I wrote a shell script in ksh to convert a CSV file to an XML table file. It uses the existing CSV file (the path to which is a variable in the script), and then creates a new output .xls file. The script has no positional parameters. The CSV file name is currently hardcoded in the script.

I would like to make changes to the script, so that it can receive input data from a CSV channel, and therefore the output of .xls can also be forwarded or redirected (>) to a file on the command line.

How is this achieved?

I'm trying to find documentation on how to write a shell script to enter the input of the channel. It seems that "reading" is only used to enter std from kb.

Thanks.

Edit: the script is below for information (now fixed to accept input from the channel through the cat according to the answer to the question.

#!/bin/ksh #Script to convert a .csv data to "Spreadsheet ML" XML format - the XML scheme for Excel 2003 # # Take CSV data as standard input # Out XLS data as standard output # DATE=`date +%Y%m%d` #define tmp files INPUT=tmp.csv IN_FILE=in_file.csv #take standard input and save as $INPUT (tmp.csv) cat > $INPUT #clean input data and save as $IN_FILE (in_file.csv) grep '.' $INPUT | sed 's/ *,/,/g' | sed 's/, */,/g' > $IN_FILE #delete original $INPUT file (tmp.csv) rm $INPUT #detect the number of columns and rows in the input file ROWS=`wc -l < $IN_FILE | sed 's/ //g' ` COLS=`awk -F',' '{print NF; exit}' $IN_FILE` #echo "Total columns is $COLS" #echo "Total rows is $ROWS" #create start of Excel File echo "<?xml version=\"1.0\"?> <?mso-application progid=\"Excel.Sheet\"?> <Workbook xmlns=\"urn:schemas-microsoft-com:office:spreadsheet\" xmlns:o=\"urn:schemas-microsoft-com:office:office\" xmlns:x=\"urn:schemas-microsoft-com:office:excel\" xmlns:ss=\"urn:schemas-microsoft-com:office:spreadsheet\" xmlns:html=\"http://www.w3.org/TR/REC-html40\"> <DocumentProperties xmlns=\"urn:schemas-microsoft-com:office:office\"> <Author>Ben Hamilton</Author> <LastAuthor>Ben Hamilton</LastAuthor> <Created>${DATE}</Created> <Company>MCC</Company> <Version>10.2625</Version> </DocumentProperties> <ExcelWorkbook xmlns=\"urn:schemas-microsoft-com:office:excel\"> <WindowHeight>6135</WindowHeight> <WindowWidth>8445</WindowWidth> <WindowTopX>240</WindowTopX> <WindowTopY>120</WindowTopY> <ProtectStructure>False</ProtectStructure> <ProtectWindows>False</ProtectWindows> </ExcelWorkbook> <Styles> <Style ss:ID=\"Default\" ss:Name=\"Normal\"> <Alignment ss:Vertical=\"Bottom\" /> <Borders /> <Font /> <Interior /> <NumberFormat /> <Protection /> </Style> <Style ss:ID=\"AcadDate\"> <NumberFormat ss:Format=\"Short Date\"/> </Style> </Styles> <Worksheet ss:Name=\"Sheet 1\"> <Table> <Column ss:AutoFitWidth=\"1\" />" #for each row in turn, create the XML elements for row/column r=1 while (( r <= $ROWS )) do echo "<Row>\n" c=1 while (( c <= $COLS )) do DATA=`sed -n "${r}p" $IN_FILE | cut -d "," -f $c ` if [[ "${DATA}" == [0-9][0-9]\.[0-9][0-9]\.[0-9][0-9][0-9][0-9] ]]; then DD=`echo $DATA | cut -d "." -f 1` MM=`echo $DATA | cut -d "." -f 2` YYYY=`echo $DATA | cut -d "." -f 3` echo "<Cell ss:StyleID=\"AcadDate\"><Data ss:Type=\"DateTime\">${YYYY}-${MM}-${DD}T00:00:00.000</Data></Cell>" else echo "<Cell><Data ss:Type=\"String\">${DATA}</Data></Cell>" fi (( c+=1 )) done echo "</Row>" (( r+=1 )) done echo "</Table>\n</Worksheet>\n</Workbook>" rm $IN_FILE > /dev/null exit 0 
+7
unix shell pipe ksh
source share
3 answers

Commands inherit their standard input from the process that starts them. In your case, your script provides its standard input for each command that it runs. A simple script example:

 #!/bin/bash cat > foo.txt 

The piping data in the shell script makes cat read this data, as cat inherits its standard input from your script.

 $ echo "Hello world" | myscript.sh $ cat foo.txt Hello world 

The read command is provided by the shell to read text from standard input into a shell variable, unless you have another command to read or process standard input from a script.

 #!/bin/bash read foo echo "You entered '$foo'" $ echo bob | myscript.sh You entered 'bob' 
+21
source share

There is one problem here. If you run the script without first checking to make sure that there is an entrance on the stdin, it will hang until something is typed.

So, to get around this, you can check that stdin is first, and if not, use the command line argument instead.

Create a script, called "testPipe.sh"

 #!/bin/bash # Check to see if a pipe exists on stdin. if [ -p /dev/stdin ]; then echo "Data was piped to this script!" # If we want to read the input line by line while IFS= read line; do echo "Line: ${line}" done # Or if we want to simply grab all the data, we can simply use cat instead # cat else echo "No input was found on stdin, skipping!" # Checking to ensure a filename was specified and that it exists if [ -f "$1" ]; then echo "Filename specified: ${1}" echo "Doing things now.." else echo "No input given!" fi fi on stdin. #!/bin/bash # Check to see if a pipe exists on stdin. if [ -p /dev/stdin ]; then echo "Data was piped to this script!" # If we want to read the input line by line while IFS= read line; do echo "Line: ${line}" done # Or if we want to simply grab all the data, we can simply use cat instead # cat else echo "No input was found on stdin, skipping!" # Checking to ensure a filename was specified and that it exists if [ -f "$1" ]; then echo "Filename specified: ${1}" echo "Doing things now.." else echo "No input given!" fi fi ; #!/bin/bash # Check to see if a pipe exists on stdin. if [ -p /dev/stdin ]; then echo "Data was piped to this script!" # If we want to read the input line by line while IFS= read line; do echo "Line: ${line}" done # Or if we want to simply grab all the data, we can simply use cat instead # cat else echo "No input was found on stdin, skipping!" # Checking to ensure a filename was specified and that it exists if [ -f "$1" ]; then echo "Filename specified: ${1}" echo "Doing things now.." else echo "No input given!" fi fi all the data, we can simply use cat instead #!/bin/bash # Check to see if a pipe exists on stdin. if [ -p /dev/stdin ]; then echo "Data was piped to this script!" # If we want to read the input line by line while IFS= read line; do echo "Line: ${line}" done # Or if we want to simply grab all the data, we can simply use cat instead # cat else echo "No input was found on stdin, skipping!" # Checking to ensure a filename was specified and that it exists if [ -f "$1" ]; then echo "Filename specified: ${1}" echo "Doing things now.." else echo "No input given!" fi fi 

Then to check:

Add some things to the test.txt file and then pass the result to our script.

 printf "stuff\nmore stuff\n" > test.txt cat test.txt | ./testPipe.sh 

Conclusion: Data was piped to this script! Line: stuff Line: more stuff Data was piped to this script! Line: stuff Line: more stuff

Now let's test if you don’t specify any input:

 ./testPipe.sh 

Conclusion: No input was found on stdin, skipping! No input given! No input was found on stdin, skipping! No input given!

Now let's test if you provided a valid file name:

 ./testPipe.sh test.txt 

Conclusion: No input was found on stdin, skipping! Filename specified: test.txt Doing things now.. No input was found on stdin, skipping! Filename specified: test.txt Doing things now..

And finally, let the test using an invalid file name:

 ./testPipe.sh invalidFile.txt 

Conclusion: No input was found on stdin, skipping! No input given! No input was found on stdin, skipping! No input given!

Explanation: Programs such as read and cat will use stdin if they are available in the shell, otherwise they will wait for input.

Credit goes to Mike from this page in his answer showing how to check the login for stdin: https://unix.stackexchange.com/questions/33049/check-if-pipe-is-empty-and-run-a-command -on-the-data-if-it-isnt? newreg = fb5b291531dd4100837b12bc1836456f

+9
source share

If the external program (which you are creating the scripts) is already accepting input from stdin, your script should not do anything. For example, awk reads from stdin, so a short script to count words in a line:

 #!/bin/sh awk '{print NF}' 

Then

 ./myscript.sh <<END one one two one two three END 

exits

 1 2 3 
+4
source share

All Articles