Apply regex replacement globally to many files with script

Question

I want to apply some regex replacement globally to 40 Javascript files in and under the directory. I am a vim user, but doing it manually can be tedious and error prone, so I would like to automate it with a script.

I tried sed, but processing multiple lines at a time is inconvenient, especially if there is no limit to the number of lines a template can have.

I also tried this script (in one file for testing):

ex $1 <<EOF gs/,\(\_\s*[\]})]\)/\1/ EOF 

The template will remove the trailing comma in any Perl / Ruby-style list, so that "[a, b, c,]" will come out as "[a, b, c]" to satisfy Internet Explorer, which alone among browsers suffocates in such lists.

The template works beautifully in vim, but does nothing if I run it in ex, as described above in the script.

Can anyone see what I can lose?

Answer 1

You asked for a script, but you mentioned that you are a vim user. I usually do a search throughout the project and replace it inside vim, for example:

 :args **/*.js | argdo %s/,\(\_\s*[\]})]\)/\1/ge | update 

This is very similar to the solution :bufdo mentioned by another commenter, but it will use your args list and not your buflist (and therefore will not require a completely new vim session, and you will not be careful when closing buffers you don't want to touch).

• :args **/*.js - sets your arglist to store all .js files in this directory and subdirectories
• | - pipe is a vim command separator that allows us to have multiple commands on the same line
• :argdo - run the following commands for all arguments. he will swallow subsequent pipes
  • % - range representing the whole file
  • :s - replace the command that you already know about
  • :s_flags , ge - global (replace as many times as possible with a string) and suppress errors (ie "No match")
  • | - this pipe is "swallowed" with :argdo , so the following command also works once for each argument
  • :update - like :write , but only when changing the buffer

This template will obviously work for any vim command that you want to run in multiple files, so you need to keep it in mind. For example, I would like to use it to remove trailing spaces ( %s/\s\+$// ), set uniform line endings ( set ff=unix ) or file encoding ( set filencoding=utf8 ) and retab my files.

Answer 2

1) Open all files with vim:

 bash$ vim $(find . -name '*.js') 

2) Apply the substitute command to all files:

 :bufdo %s/,\(\_\s*[\]})]\)/\1/ge 

3) Save all files and close:

 :wall :q 

I think you will need to double-check your search pattern, it looks wrong. I think that where you have \_\s* , you should have \_s* .

Edit: You should also use the /ge options for the command :s... (I added these above).

Answer 3

You can automate the actions of both vi and ex by passing the +'command' argument from the command line, which allows you to use them as text filters.

In your situation, the following command should work fine:

find /path/to/dir -name '*.js' | xargs ex +'%s/,\(\_\s*[\]})]\)/\1/g' +'wq!'

Answer 4

If you are on Windows, Notepad ++ allows you to run simple regular expressions in all open files.

Find ,\s*\] and replace with ]

should work for the type of lists you are describing.

Answer 5

you can use a combination of find and sed

 find /path -type f -iname "*.js" -exec sed -i.bak 's/,[ \t]*]/]/' "{}" +;