How can I manipulate $ PATH elements in shell scripts?

Question

Is there an idiomatic way to remove elements from PATH-like shell variables?

That is, I want to take

PATH=/home/joe/bin:/usr/local/bin:/usr/bin:/bin:/path/to/app/bin:. 

and remove or replace /path/to/app/bin without compressing the rest of the variable. Extra points for allowing me to introduce new elements into arbitrary positions. The target will be recognized by a clearly defined line and may appear anywhere in the list.

I know that I saw this, and maybe I can do something on my own, but I'm looking for a good approach. Portability and standardization plus.

I use bash, but an example is also welcome in your favorite shell.

---

The context here is the need to conveniently switch between several versions (one for analysis, the other for working with the framework) of a large scientific analysis package that produces a couple of dozen executable files, has data hidden around the file system, and uses an environment variable to help find all this. I would like to write a script that selects the version, and should be able to remove the $PATH elements related to the current active version and replace them with the same elements as for the new version.

---

This is due to the problem of preventing repeated $PATH elements when restarting login scripts, etc.

---

• Previous similar question: How to avoid duplication of a path variable in csh
• Subsequent similar question: What is the most elegant way to remove the path from the $ PATH variable in Bash?

Answer 1

Solution of the proposed solution from dmckee:

• While some versions of Bash may hyphenate function names, others (MacOS X) do not.
• I do not see the need to use return just before the function ends.
• I do not see the need for all half-columns.
• I do not understand why you are exporting the value of path-element-by-pattern. Think of export as the equivalent of setting (or even creating) a global variable - something should be avoided when possible.
• I'm not sure what you expect from " replace-path PATH $PATH /usr ", but it does not do what I expect.

Consider the PATH value that begins with:

 . /Users/jleffler/bin /usr/local/postgresql/bin /usr/local/mysql/bin /Users/jleffler/perl/v5.10.0/bin /usr/local/bin /usr/bin /bin /sw/bin /usr/sbin /sbin 

The result I got (from ' replace-path PATH $PATH /usr '):

 . /Users/jleffler/bin /local/postgresql/bin /local/mysql/bin /Users/jleffler/perl/v5.10.0/bin /local/bin /bin /bin /sw/bin /sbin /sbin 

I would expect to get my original path back, since / usr is not displayed as an element (full), only as part of a path element.

This can be fixed in replace-path by changing one of the sed commands:

 export $path=$(echo -n $list | tr ":" "\n" | sed "s:^$removestr\$:$replacestr:" | tr "\n" ":" | sed "s|::|:|g") 

I used ':' instead of '|' to separate replacement parts since '|' can (theoretically) appear in a path component, whereas by definition by PATH, the colon cannot. I noticed that the second sed can remove the current directory from the middle of PATH. That is, a valid (albeit perverse) PATH value can be:

 PATH=/bin::/usr/local/bin 

After processing, the current directory will no longer be in PATH.

A similar change in the matching binding is appropriate for path-element-by-pattern :

 export $target=$(echo -n $list | tr ":" "\n" | grep -m 1 "^$pat\$") 

I note along the way that grep -m 1 not standard (this is the GNU extension, also available on MacOS X). Indeed, the -n option for echo also non-standard; you would be better off simply removing the trailing colon added by converting the newline from echo to colon. Since path-element-by-pattern is used only once, it has unwanted side effects (it compresses any pre-existing exported variable called $removestr ), it can reasonably be replaced by its body. This, along with a more liberal use of quotes to avoid space problems or an undesirable file name extension, leads to:

 # path_tools.bash # # A set of tools for manipulating ":" separated lists like the # canonical $PATH variable. # # /bin/sh compatibility can probably be regained by replacing $( ) # style command expansion with ` ` style ############################################################################### # Usage: # # To remove a path: # replace_path PATH $PATH /exact/path/to/remove # replace_path_pattern PATH $PATH <grep pattern for target path> # # To replace a path: # replace_path PATH $PATH /exact/path/to/remove /replacement/path # replace_path_pattern PATH $PATH <target pattern> /replacement/path # ############################################################################### # Remove or replace an element of $1 # # $1 name of the shell variable to set (eg PATH) # $2 a ":" delimited list to work from (eg $PATH) # $3 the precise string to be removed/replaced # $4 the replacement string (use "" for removal) function replace_path () { path=$1 list=$2 remove=$3 replace=$4 # Allowed to be empty or unset export $path=$(echo "$list" | tr ":" "\n" | sed "s:^$remove\$:$replace:" | tr "\n" ":" | sed 's|:$||') } # Remove or replace an element of $1 # # $1 name of the shell variable to set (eg PATH) # $2 a ":" delimited list to work from (eg $PATH) # $3 a grep pattern identifying the element to be removed/replaced # $4 the replacement string (use "" for removal) function replace_path_pattern () { path=$1 list=$2 removepat=$3 replacestr=$4 # Allowed to be empty or unset removestr=$(echo "$list" | tr ":" "\n" | grep -m 1 "^$removepat\$") replace_path "$path" "$list" "$removestr" "$replacestr" } 

I have a Perl script called echopath , which I find useful in debugging problems with PATH-like variables:

 #!/usr/bin/perl -w # # "@(#)$Id: echopath.pl,v 1.7 1998/09/15 03:16:36 jleffler Exp $" # # Print the components of a PATH variable one per line. # If there are no colons in the arguments, assume that they are # the names of environment variables. @ARGV = $ENV{PATH} unless @ARGV; foreach $arg (@ARGV) { $var = $arg; $var = $ENV{$arg} if $arg =~ /^[A-Za-z_][A-Za-z_0-9]*$/; $var = $arg unless $var; @lst = split /:/, $var; foreach $val (@lst) { print "$val\n"; } } 

When I run the modified solution on the test code below:

 echo xpath=$PATH replace_path xpath $xpath /usr echopath $xpath echo xpath=$PATH replace_path_pattern xpath $xpath /usr/bin /work/bin echopath xpath echo xpath=$PATH replace_path_pattern xpath $xpath "/usr/.*/bin" /work/bin echopath xpath 

Output:

 . /Users/jleffler/bin /usr/local/postgresql/bin /usr/local/mysql/bin /Users/jleffler/perl/v5.10.0/bin /usr/local/bin /usr/bin /bin /sw/bin /usr/sbin /sbin . /Users/jleffler/bin /usr/local/postgresql/bin /usr/local/mysql/bin /Users/jleffler/perl/v5.10.0/bin /usr/local/bin /work/bin /bin /sw/bin /usr/sbin /sbin . /Users/jleffler/bin /work/bin /usr/local/mysql/bin /Users/jleffler/perl/v5.10.0/bin /usr/local/bin /usr/bin /bin /sw/bin /usr/sbin /sbin 

This looks right to me - at least for my definition of the problem.

Note that echopath LD_LIBRARY_PATH evaluates $LD_LIBRARY_PATH . It would be nice if your functions could do this, so the user could enter:

 replace_path PATH /usr/bin /work/bin 

This can be done using:

 list=$(eval echo '$'$path) 

This leads to code revision:

 # path_tools.bash # # A set of tools for manipulating ":" separated lists like the # canonical $PATH variable. # # /bin/sh compatibility can probably be regained by replacing $( ) # style command expansion with ` ` style ############################################################################### # Usage: # # To remove a path: # replace_path PATH /exact/path/to/remove # replace_path_pattern PATH <grep pattern for target path> # # To replace a path: # replace_path PATH /exact/path/to/remove /replacement/path # replace_path_pattern PATH <target pattern> /replacement/path # ############################################################################### # Remove or replace an element of $1 # # $1 name of the shell variable to set (eg PATH) # $2 the precise string to be removed/replaced # $3 the replacement string (use "" for removal) function replace_path () { path=$1 list=$(eval echo '$'$path) remove=$2 replace=$3 # Allowed to be empty or unset export $path=$(echo "$list" | tr ":" "\n" | sed "s:^$remove\$:$replace:" | tr "\n" ":" | sed 's|:$||') } # Remove or replace an element of $1 # # $1 name of the shell variable to set (eg PATH) # $2 a grep pattern identifying the element to be removed/replaced # $3 the replacement string (use "" for removal) function replace_path_pattern () { path=$1 list=$(eval echo '$'$path) removepat=$2 replacestr=$3 # Allowed to be empty or unset removestr=$(echo "$list" | tr ":" "\n" | grep -m 1 "^$removepat\$") replace_path "$path" "$removestr" "$replacestr" } 

The following revised test is also currently running:

 echo xpath=$PATH replace_path xpath /usr echopath xpath echo xpath=$PATH replace_path_pattern xpath /usr/bin /work/bin echopath xpath echo xpath=$PATH replace_path_pattern xpath "/usr/.*/bin" /work/bin echopath xpath 

It produces the same result as before.

Answer 2

Submitting my answer What is the most elegant way to remove the path from the $ PATH variable in Bash? :

 #!/bin/bash IFS=: # convert it to an array t=($PATH) unset IFS # perform any array operations to remove elements from the array t=(${t[@]%%*usr*}) IFS=: # output the new array echo "${t[*]}" 

or single line:

 PATH=$(IFS=':';t=($PATH);unset IFS;t=(${t[@]%%*usr*});IFS=':';echo "${t[*]}"); 

Answer 3

To remove an item, you can use sed:

 #!/bin/bash NEW_PATH=$(echo -n $PATH | tr ":" "\n" | sed "/foo/d" | tr "\n" ":") export PATH=$NEW_PATH 

will remove the paths containing "foo" from the path.

You can also use sed to insert a new line before or after a given line.

Edit: you can remove duplicates by laying through sorting and uniq:

 echo -n $PATH | tr ":" "\n" | sort | uniq -c | sed -n "/ 1 / s/.*1 \(.*\)/\1/p" | sed "/foo/d" | tr "\n" ":" 

Answer 4

There are several relevant programs in the answers to : “How to avoid duplicating the path variable in csh .” They focus more on avoiding duplicate elements, but I can use the script as:

 export PATH=$(clnpath $head_dirs:$PATH:$tail_dirs $remove_dirs) 

Assuming you have one or more directories in $ head_dirs and one or more directories in $ tail_dirs and one or more directories in $ remove_dirs, then it uses a shell to combine the head, current, and tail parts into a massive value and then deletes each of the directories listed in $ remove_dirs from the result (and not errors if they do not exist), and also eliminates the second and subsequent occurrences of any directory in the path.

This does not apply to placing the components of the path in a specific position (except at the beginning or at the end, but only indirectly). It is reasonable to indicate where you want to add a new element or which element you want to replace is messy.

Answer 5

Just note that bash can do search and replace. It can perform all the usual "once or all" cases [in] of sensitive parameters that you expect.

On the man page:

$ {parameter / pattern / string}

The template expands to create the template in the same way as when expanding the path. The parameter is expanded, and the longest match of the template with its value is replaced with a string. If Ipattern starts with /, all pattern matches are replaced by a string. Usually only the first match is replaced. If the template starts with C #, it must match at the beginning of the extended parameter value. If the pattern starts with%, it must match at the end of the extended parameter value. If the string is NULL, pattern matches are deleted and the pattern / next pattern can be omitted. If the parameter is @ or *, the substitution operation is applied to each positional parameter in turn, and the extension is the resulting list. If the parameter is an array variable, adjusted using @ or *, the replace operation is applied alternately to each member of the array, and the extension is the resulting list.

You can also split the field by setting $ IFS (input field separator) to the desired separator.

Answer 6

OK, thanks to all the respondents. I prepared an encapsulated version of florin's answer. The first pass is as follows:

 # path_tools.bash # # A set of tools for manipulating ":" separated lists like the # canonical $PATH variable. # # /bin/sh compatibility can probably be regained by replacing $( ) # style command expansion with ` ` style ############################################################################### # Usage: # # To remove a path: # replace-path PATH $PATH /exact/path/to/remove # replace-path-pattern PATH $PATH <grep pattern for target path> # # To replace a path: # replace-path PATH $PATH /exact/path/to/remove /replacement/path # replace-path-pattern PATH $PATH <target pattern> /replacement/path # ############################################################################### # Finds the _first_ list element matching $2 # # $1 name of a shell variable to be set # $2 name of a variable with a path-like structure # $3 a grep pattern to match the desired element of $1 function path-element-by-pattern (){ target=$1; list=$2; pat=$3; export $target=$(echo -n $list | tr ":" "\n" | grep -m 1 $pat); return } # Removes or replaces an element of $1 # # $1 name of the shell variable to set (ie PATH) # $2 a ":" delimited list to work from (ie $PATH) # $2 the precise string to be removed/replaced # $3 the replacement string (use "" for removal) function replace-path () { path=$1; list=$2; removestr=$3; replacestr=$4; # Allowed to be "" export $path=$(echo -n $list | tr ":" "\n" | sed "s|$removestr|$replacestr|" | tr "\n" ":" | sed "s|::|:|g"); unset removestr return } # Removes or replaces an element of $1 # # $1 name of the shell variable to set (ie PATH) # $2 a ":" delimited list to work from (ie $PATH) # $2 a grep pattern identifying the element to be removed/replaced # $3 the replacement string (use "" for removal) function replace-path-pattern () { path=$1; list=$2; removepat=$3; replacestr=$4; # Allowed to be "" path-element-by-pattern removestr $list $removepat; replace-path $path $list $removestr $replacestr; } 

Error grabbing in all functions is still required, and I should probably stick to the re-path solution while I'm in it.

You use it while performing . /include/path/path_tools.bash . /include/path/path_tools.bash in a working script and calling replace-path* functions.

---

I am still open to new and / or better answers.

Answer 7

It is easy to use awk.

Replace

 { for(i=1;i<=NF;i++) if($i == REM) if(REP) print REP; else continue; else print $i; } 

Run it using

 function path_repl { echo $PATH | awk -F: -f rem.awk REM="$1" REP="$2" | paste -sd: } $ echo $PATH /bin:/usr/bin:/home/js/usr/bin $ path_repl /bin /baz /baz:/usr/bin:/home/js/usr/bin $ path_repl /bin /usr/bin:/home/js/usr/bin 

Append

Inserts at the specified position. By default, it is added at the end.

 { if(IDX < 1) IDX = NF + IDX + 1 for(i = 1; i <= NF; i++) { if(IDX == i) print REP print $i } if(IDX == NF + 1) print REP } 

Run it using

 function path_app { echo $PATH | awk -F: -f app.awk REP="$1" IDX="$2" | paste -sd: } $ echo $PATH /bin:/usr/bin:/home/js/usr/bin $ path_app /baz 0 /bin:/usr/bin:/home/js/usr/bin:/baz $ path_app /baz -1 /bin:/usr/bin:/baz:/home/js/usr/bin $ path_app /baz 1 /baz:/bin:/usr/bin:/home/js/usr/bin 

Remove duplicates

This saves the first occurrences.

 { for(i = 1; i <= NF; i++) { if(!used[$i]) { print $i used[$i] = 1 } } } 

Start like this:

 echo $PATH | awk -F: -f rem_dup.awk | paste -sd: 

Check if all elements exist

The following will display an error message for all entries that do not exist in the file system and return a non-zero value.

 echo -n $PATH | xargs -d: stat -c %n 

To just check if all elements are paths and get a return code, you can also use test :

 echo -n $PATH | xargs -d: -n1 test -d 

Answer 8

Let's pretend that

 echo $PATH /usr/lib/jvm/java-1.6.0/bin:lib/jvm/java-1.6.0/bin/:/lib/jvm/java-1.6.0/bin/:/usr/lib/qt-3.3/bin:/usr/lib/ccache:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/tvnadeesh/bin 

If you want to remove / lib / jvm / java -1.6.0 / bin /, follow these steps

 export PATH=$(echo $PATH | sed 's/\/lib\/jvm\/java-1.6.0\/bin\/://g') 

sed will accept input from echo $PATH and replace / lib / jvm / java -1.6.0 / bin /: with empty

that way you can remove

Answer 9

• The PATH order does not apply.
• Handles corner cases such as empty path, space in the path gracefully
• Partial dir matching does not give false positives
• Heals the path along the head and tail of the PATH appropriately. No : garbage, etc.

Say that you have / Foo: / some / path: / some / path / dir1: / some / path / dir2: / bar and you want to replace / Some / path Then it replaces "/ some / path" correctly, but leaves "/ some / path / dir1" or "/ some / path / dir2", as you would expect.

 function __path_add(){ if [ -d "$1" ] ; then local D=":${PATH}:"; [ "${D/:$1:/:}" == "$D" ] && PATH="$PATH:$1"; PATH="${PATH/#:/}"; export PATH="${PATH/%:/}"; fi } function __path_remove(){ local D=":${PATH}:"; [ "${D/:$1:/:}" != "$D" ] && PATH="${D/:$1:/:}"; PATH="${PATH/#:/}"; export PATH="${PATH/%:/}"; } # Just for the shake of completeness function __path_replace(){ if [ -d "$2" ] ; then local D=":${PATH}:"; if [ "${D/:$1:/:}" != "$D" ] ; then PATH="${D/:$1:/:$2:}"; PATH="${PATH/#:/}"; export PATH="${PATH/%:/}"; fi fi } 

Related Posts What is the most elegant way to remove the path from the $ PATH variable in Bash?

Answer 10

The first thing to insert into my head only part of the string is to replace sed.

Example: if echo $ PATH => "/ usr / pkg / bin: / usr / bin: / bin: / usr / pkg / games: / usr / pkg / X11R6 / bin" then change "/ usr / bin" to " / usr / local / bin "can be as follows:

## prints a standard output file

## instead of the slash character ("/"), the symbol "=" is used, since this will be random, # an alternative citation character should be unlikely in PATH

## the path separator character ":" is removed and added here again, # an additional colon after the last path may be required

echo $ PATH | sed '= / usr / bin: = / usr / local / bin: ='

This solution replaces the entire path element, so it may be redundant if the new element is similar.

If the new PATHs are not dynamic, but always within some constant set, you can save them in a variable and assign them as necessary:

PATH = $ TEMP_PATH_1; # commands ...; \ n PATH = $ TEMP_PATH_2; # teams, etc .;

There cannot be what you were thinking. some of the relevant commands on bash / unix:

Pushd POPD CD ls # can be l -1A for one column; find grep which # can confirm that the file is where you think it came from; okr type

.. and all this and much more are related to PATH or directories in general. The text change part can be done in any number of ways!

Regardless of the solution chosen, there would be 4 parts:

1) select the path as it is 2) decode the path to find the part that needs changes 3) determine what changes are needed / integrate these changes 4) validate / final integration / set the variable

Answer 11

According to dj_segfault's answer , I do this in scripts that add / supplement environment variables that can be executed multiple times:

 ld_library_path=${ORACLE_HOME}/lib LD_LIBRARY_PATH=${LD_LIBRARY_PATH//${ld_library_path}?(:)/} export LD_LIBRARY_PATH=${ld_library_path}${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}} 

Using the same technique to delete, replace, or manage entries in PATH is trivial, given the pattern matching with name extension and template support for expanding shell options.

Answer 12

Now I prefer to use ruby ​​rather than awk / sed / foo, so here is my approach to fools,

 # add it to the path PATH=~/bin/:$PATH:~/bin export PATH=$(ruby -e 'puts ENV["PATH"].split(/:/).uniq.join(":")') 

create a function for reuse,

 mungepath() { export PATH=$(ruby -e 'puts ENV["PATH"].split(/:/).uniq.join(":")') } 

Hash, arrays and strings in the ruby ​​of one liner :)