Using sed or awk to parse multipath output

Question

I am trying to parse multipath output on linux for the script I am writing and I cannot find the correct SED or AWK syntax to get it the way I want it. I have looked at various sites and stackoverflow examples many times.

I want to find a way to do:

temp (360a98000572d4d2d5834566c64536b46) dm-4 NETAPP,LUN [size=20G] 2:0:0:4 sde 8:64 [active][ready] 1:0:0:4 sdm 8:192 [active][ready] redo (360a98000572d4d2d5834566c646c366c) dm-3 NETAPP,LUN [size=5.0G] 2:0:0:3 sdd 8:48 [active][ready] 1:0:0:3 sdl 8:176 [active][ready] 

look like this:

 temp (360a98000572d4d2d5834566c64536b46) dm-4 NETAPP,LUN [size=20G] 2:0:0:4 sde 8:64 active][ready] temp (360a98000572d4d2d5834566c64536b46) dm-4 NETAPP,LUN [size=20G] 1:0:0:4 sdm 8:192 [active][ready] redo (360a98000572d4d2d5834566c646c366c) dm-3 NETAPP,LUN [size=5.0G] 2:0:0:3 sdd 8:48 [active][ready] redo (360a98000572d4d2d5834566c646c366c) dm-3 NETAPP,LUN [size=5.0G] 1:0:0:3 sdl 8:176 [active][ready] 

** Edit So, having made this even more difficult, I found multipath configurations without Netapp default settings. This makes NETAPP, LUN not guaranteed to be on line. What I started to do:

 /sbin/multipath -ll | grep -v "round-robin"| sed 's/\[feat.*//g' | sed ':a; $!N;s/\n\([^\n]*\[size\)/ \1/;ta;P;D' 

which puts the size on the main line to give me something else:

 360a98000572d4d2d5834664e68323436 dm-6 NETAPP,LUN [size=50G] \_ 1:0:0:0 sda 8:0 [active][ready] 360a98000572d4d2d5834664e68395951 dm-7 NETAPP,LUN [size=275G] \_ 1:0:0:7 sdb 8:16 [active][ready] 

However, I cannot get any of the examples below to match "G [$" (I know that I will need another line for T if there are volumes of volumes on terabytes) and give me the correct output.

Thanks for all the suggestions below ** Finish editing

I know how to clear the interval, so I will do it after I get the correct result. Lines that start multipath information end with "LUN." Servers can have 1 to 8 paths under each LUN line (sdx devices). The part before "()" can be either text (alias) or numeric.

Answer 1

One of the methods:

The contents of script.awk :

 $1 ~ /^([[:digit:]]:){3}[[:digit:]]$/ { printf "%s %s\n", line, $0; next; } ##$1 ~ /temp|redo/ { $0 ~ /LUN$/ { getline l; line = $0 " " l; } 

Assuming infile with the contents of the question, run the script as:

 awk -f script.awk infile 

With the following output:

 temp (360a98000572d4d2d5834566c64536b46) dm-4 NETAPP,LUN [size=20G] 2:0:0:4 sde 8:64 [active][ready] temp (360a98000572d4d2d5834566c64536b46) dm-4 NETAPP,LUN [size=20G] 1:0:0:4 sdm 8:192 [active][ready] redo (360a98000572d4d2d5834566c646c366c) dm-3 NETAPP,LUN [size=5.0G] 2:0:0:3 sdd 8:48 [active][ready] redo (360a98000572d4d2d5834566c646c366c) dm-3 NETAPP,LUN [size=5.0G] 1:0:0:3 sdl 8:176 [active][ready] 

Answer 2

I don't know the exact input specification, but this one line can help you:

awk '{if (/LUN$/){ prefix = $0; getline; prefix = prefix " " $0 } else {print prefix, $0} }'

• It checks if the current line ends with a LUN (matching it with the regular expression /LUN$/ )
• If there is a match, it concatenates the current line with the next (using getline and the implicit concatenation operator)
• If there is no match, it displays the current record with prefix

PS: you may need additional line filtering, which should not be complicated, just add more if to the else branch.

Answer 3

This may work for you (GNU sed):

 sed '/LUN$/{N;y/\n/ /;h;d};G;s/^\([^\n]*\)\n\(.*\)/\2 \1/' file 

Explanation:

• /LUN$/{N;y/\n/ /;h;d} For lines ending in LUN , add a new line and then the next line to the pattern space (PS), replace the new line with a space, save the PS in hold space (HS) and then remove the PS and start the next cycle.
• G for all other lines (path lines), add a new line followed by the contents of HS in PS.
• s/^\([^\n]*\)\n\(.*\)/\2 \1/ swap something up to the first line of a new line with something that follows it and replace the new line a space, i.e. add header information to the path string.

Answer 4

Using sed:

 sed -n ' # if this is header append the size line /LUN$/{ N; s/\n/ /; h # and remember this }; # if not header then append to header /LUN \[/!{ G; s/\(.*\)\n\(.*\)/\2 \1/; p; #and print }' input_file 

No comments:

  sed -n ' /LUN$/{ N; s/\n/ /; h }; /LUN \[/!{ G; s/\(.*\)\n\(.*\)/\2 \1/; p; }' input_file 

Answer 5

Due to multipath, the nature of the LUN can have 1+ paths, not just two. So getline () can be in an ugly loop. Here is the explicit awk version. All you have to consider is how to mark the LUN line, the SIZE line and the PATH lines.

 awk '/\(.*\)/ {lu=$0} /^\[size/ {size=$0} $2 ~ /sd/ {print lu, size, $0}' 

But multipath -l can have much more complex information, such as policies, options, etc.

Answer 6

This is a general solution for a whole class of problems similar to yours, namely: parsing record sets that span multiple lines, but the number of lines may vary between record sets.

 sed '/([0-9a-f]\{33\})/ i \\' input_file | \ awk ' BEGIN {RS=""; FS="\n"} {for(i=3; i<=NF; i++) {print $1,$2,$i}} ' 

gives

 temp (360a98000572d4d2d5834566c64536b46) dm-4 NETAPP,LUN [size=20G] 2:0:0:4 sde 8:64 [active][ready] temp (360a98000572d4d2d5834566c64536b46) dm-4 NETAPP,LUN [size=20G] 1:0:0:4 sdm 8:192 [active][ready] redo (360a98000572d4d2d5834566c646c366c) dm-3 NETAPP,LUN [size=5.0G] 2:0:0:3 sdd 8:48 [active][ready] redo (360a98000572d4d2d5834566c646c366c) dm-3 NETAPP,LUN [size=5.0G] 1:0:0:3 sdl 8:176 [active][ready] 

---

Properties of this solution

• identifies a hex digest entry
• handles path 1+
• easily extends to implement footer logic in awk, because each multipath entry is treated as an awk entry

Explanation

Imagine your input looked like this:

 temp (360a98000572d4d2d5834566c64536b46) dm-4 NETAPP,LUN [size=20G] 2:0:0:4 sde 8:64 [active][ready] 1:0:0:4 sdm 8:192 [active][ready] redo (360a98000572d4d2d5834566c646c366c) dm-3 NETAPP,LUN [size=5.0G] 2:0:0:3 sdd 8:48 [active][ready] 1:0:0:3 sdl 8:176 [active][ready] 

That would be easy to make out with awk.
You would just say that ..

• each entry is separated by an empty line
• each field should be just a whole line
• then for your desired result, print out the combination "first line, second line, i-th line" until all lines (fields) have been exhausted.

This is exactly what the above awk code does:
String BEGIN {RS=""; FS="\n"} BEGIN {RS=""; FS="\n"} sets the sep entry. to an empty string and field sep. to the line. The line {for(i=3; i<=NF; i++) {print $1,$2,$i}} does exactly what I described earlier than under 3 ..

Now you need to enter blank lines to separate records
For this task I am using sed. I just need an anchor to see the lines, to enter empty lines earlier. Here I use hex digest in parentheses . I guess it is always 33 characters long.

Answer 7

EDIT:

 awk '/\[size=.*G\]/ { array[prev]++ } { prev = $0 } FNR!=NR { if ($0 in array) { line = $0; getline; line = line FS $0; next } else { print line, $0 } }' file.txt{,} 

Explanation:

 ## file.txt{,} ## this is bash shorthand for reading the same file twice ## { prev = $0 } ## keep track of the last line ## /\[size=.*G\]/ { array[prev]++ } ## if the 'size' pattern is found, add the previous line to an array ## now we have an array of lines to search for ## FNR!=NR ## perform the following actions on the second file only ## if ($0 in array) ## if the line is one of our lines that we're searching for ## line = $0; getline; line = line FS $0; next ## store the line, get the next line join it up and go onto the next record ## else { print line, $0 } ## otherwise print out the 'line' and subsequent data 

Results:

 temp (360a98000572d4d2d5834566c64536b46) dm-4 NETAPP,LUN [size=20G] 2:0:0:4 sde 8:64 [active][ready] temp (360a98000572d4d2d5834566c64536b46) dm-4 NETAPP,LUN [size=20G] 1:0:0:4 sdm 8:192 [active][ready] redo (360a98000572d4d2d5834566c646c366c) dm-3 NETAPP,LUN [size=5.0G] 2:0:0:3 sdd 8:48 [active][ready] redo (360a98000572d4d2d5834566c646c366c) dm-3 NETAPP,LUN [size=5.0G] 1:0:0:3 sdl 8:176 [active][ready]