Command line search, sed, exec

Question

I have a bunch of files in a folder, in subfolders, and I'm trying to make some kind of single line file for quick copy / paste once in a while.

Content (too long to insert here): http://pastebin.com/4aZCPbwT

I tried the following commands:

List of all files and their directories

find . -name '[!.]*'

Replace all instances of "Namespace" with "Test:

find . -name '[!.]*' -print0 | sed 's/Namespace/Test/gI' | xargs -i -0 echo '{}'

What do I need to do:

Replace the fold names as described above and copy the folders (including files) to another location. Create folders if they do not exist (they most likely will not) - BUT, there are some of them that I do not need, for example. / app since this folder exists. I could use -wholename './app' for this.

When they are copied, I need to replace some text inside each file, just as described above (the namespace with the test also happens inside the files and, of course, saves them).

Something like this I would think:

-print -exec sed -i 's/Namespace/Test/gI' {} \;

Can these 3 things be done in one layer? Replace the text in the files (Namespace <=> Test), copy the files, including their directories, using cp -p (do not want to write to folders), but renaming each directory / file as described above (Namespace <=> Test).

Thank you so much: -)

Answer 1

Besides describing how with painstaking verbosity below, this method can also be unique as it includes built-in debugging. It basically does nothing, as it is written, except compilation and saves the variable all the commands that it considers necessary to perform the requested work.

It also explicitly avoids loops as much as possible. As far as I know, apart from the recursive sed search for more than one pattern match, there is no other recursion.

And the last, it is completely null with a separator - it does not work on any character in any file name except null . I do not think you need it.

By the way, this is REALLY fast. Take a look:

 % _mvnfind() { mv -n "${1}" "${2}" && cd "${2}" > read -r SED <<SED > :;s|${3}\(.*/[^/]*${5}\)|${4}\1|;t;:;s|\(${5}.*\)${3}|\1${4}|;t;s|^[0-9]*\(.*\)${5}|\1|p > SED > find . -name "*${3}*" -printf "%d\tmv %P ${5} %P\000" | > sort -zg | sed -nz ${SED} | read -r ${6} > echo <<EOF > Prepared commands saved in variable: ${6} > To view do: printf ${6} | tr "\000" "\n" > To run do: sh <<EORUN > $(printf ${6} | tr "\000" "\n") > EORUN > EOF > } % rm -rf "${UNNECESSARY:=/any/dirs/you/dont/want/moved}" % time ( _mvnfind ${SRC=./test_tree} ${TGT=./mv_tree} \ > ${OLD=google} ${NEW=replacement_word} ${sed_sep=SsEeDd} \ > ${sh_io:=sh_io} ; printf %b\\000 "${sh_io}" | tr "\000" "\n" \ > | wc - ; echo ${sh_io} | tr "\000" "\n" | tail -n 2 ) <actual process time used:> 0.06s user 0.03s system 106% cpu 0.090 total <output from wc:> Lines Words Bytes 115 362 20691 - <output from tail:> mv .config/replacement_word-chrome-beta/Default/.../googlestars \ .config/replacement_word-chrome-beta/Default/.../replacement_wordstars 

NOTE. . The above function are likely to require the GNU version of sed and find to handle find printf and sed -z -e and :;recursive regex test;t . If they are not available to you, the functionality may be copied with minor adjustments.

This should do everything that you wanted to start from the very beginning with a very small fuss. I did fork with sed , but I also practiced some sed recursive branching methods, so here I am. This is probably like getting a haircut at a discount at the hairdresser. Here's the workflow:

• rm -rf ${UNNECESSARY}
  • I deliberately excluded any function call that could delete or destroy data of any type. You note that ./app may be objectionable. Delete it or move it to another location in advance, or, alternatively, you can build the find routine in \( -path PATTERN -exec rm -rf \{\} \) to do this programmatically, but it's all yours.
• _mvnfind "${@}"
  • Declare your arguments and call the working function. ${sh_io} especially important in that it saves a return from the function. ${sed_sep} is about to end; this is an arbitrary string used to refer to sed recursion in a function. If ${sed_sep} set to a value that could potentially be found in any of your paths or file names that act ... well, just don't let it be.
• mv -n $1 $2
  • The whole tree moves from the very beginning. This will save a lot of headache; trust me. The rest of what you want to do, renaming, is just file system metadata. If you, for example, moved it from one disk to another or to any borders of the file system, you better do it right away with one command. It is also safer. Note the -noclobber parameter set for mv ; as written, this function will not put ${SRC_DIR} , where a ${TGT_DIR} already exists.
• read -R SED <<HEREDOC
  • I found all sed commands here to save on avoiding hassles and read them in a variable that will be served in sed below. Explanation below.
• find . -name ${OLD} -printf
  
  • Let's start the find process. With find we are only looking for everything that needs to be renamed, because we have already completed all the place-to-place mv operations with the first function command. Instead of taking any direct action with find , such as calling exec , we use it instead to dynamically build the command line with -printf .
• %dir-depth :tab: 'mv '%path-to-${SRC}' '${sed_sep}'%path-again :null delimiter:'
  • After find finds the files that we need, it directly builds and prints (most of) the command that we need to process your renaming. %dir-depth , attached to the beginning of each line, will help ensure that we are not trying to rename a file or directory in a tree with a parent that has not yet been renamed. find uses all kinds of optimization methods to walk through the tree of your file system, and it is not necessary that it will return data to us in a safe order for the operation. That is why we are following ...
• sort -general-numerical -zero-delimited
  • We sort all find output based on %directory-depth so that the paths closest to $ {SRC} are processed first. This avoids possible errors using mv ing files to non-existent locations and minimizes the need for a recursive loop. (in fact, it may be difficult for you to find the loop at all)
• sed -ex :rcrs;srch|(save${sep}*til)${OLD}|\saved${SUBSTNEW}|;til ${OLD=0}
  
  • I think this is the only loop in the whole script, and it iterates over only the second %Path for each line if it contains more than one $ {OLD} value that may need to be replaced. All the other solutions that I presented included the second sed process, and although a short cycle may be undesirable, it certainly beats spawning and spins the whole process.
  • Thus, basically, sed performs a search for $ {sed_sep}, after finding it, it saves it and all the characters it encounters until it finds $ {OLD}, which then replaces $ {NEW}. Then it returns to $ {sed_sep} and again looks at $ {OLD} if it occurs more than once in a line. If it is not found, it prints the modified line before stdout (after which it catches again) and ends the loop.
  • This avoids parsing the entire line and ensures that the first half of the mv command line, which should include $ {OLD}, of course, includes it, and the second half changes as many times as necessary to erase the name $ {OLD} from the destination mv .
• sed -ex...-ex search|%dir_depth(save*)${sed_sep}|(only_saved)|out
  • Two -exec calls here occur without a second fork . In the first case, as we saw, we modify the mv command provided by the find -printf find if necessary to correctly change all the links $ {OLD} to $ {NEW}, but in order to make it so that we had to use some arbitrary reference points that should not be included in the final output. Therefore, when sed completes everything he needs to do, we advise him to erase his anchor points from the hold buffer before passing it on.

AND WE ARE NOW AROUND

read will receive a command that looks like this:

 % mv /path2/$SRC/$OLD_DIR/$OLD_FILE /same/path_w/$NEW_DIR/$NEW_FILE \000 

It will read in ${msg} as ${sh_io} , which can be examined outside the function.

Cool.

-Mike

Answer 2

I have not tested this, but I think you need to.

 find . -name '[!.]*' -print | while read line; do nfile=`echo "$line" | sed 's/Namespace/Test/gI'`; mkdir -p "`dirname $nfile`"; cp -p "$line" "$nfile"; sed -i 's/Namespace/Test/gI' "$nfile"; done