Combining two files into different folders on Linux

Question

I have two sets of folders that have files with the same file names and structure . The folder structure looks something like this:

\outputfolder\
 |---\folder1\
 |      |---file1.txt
 |      |---file2.txt
 |
 |---\folder2\
        |---file1.txt
        |---file2.txt

So what I need to do is to merge (add) all files with the same name in these folders (file1.txt with file1.txt, etc.) into another file inside the output folder. After receiving these combined files, I also need to create a tar.gz file from all of these combined files.

How can I do this in a Linux based command line environment? The folder name (folder1 and folder2, etc.) is a variable, so this needs to be provided, but the files are not needed, and it should automatically combine all the files with the same name.

In addition, these files have headers for column names, so I will need to remove this also when adding.

Answer 1

Here is the code to get you started.

topdir=outputfolder
dir1=folder1
dir2=folder2

for f in $topdir/$dir1/*.txt
do
    outf=$topdir/`basename $f .txt`-concat.txt
    cp $f $outf
    sed -e '1 d' $topdir/$dir2/`basename $f` >> $outf
done

tar czf foo.tar.gz $topdir/*-concat.txt

Edit: A part has been added that removes the header of the second file.

Answer 2

find . -name 'file1.txt' | xargs cat >file1_concat.txt

Answer 3

, 1 2:

concat_files() {
  for dir in "$@"; do
    for file in "$dir"/*; do 
      this=$(basename "$file")
      { [[ -f "$this" ]] && sed 1d "$file" || cat "$file"; } >> "$this"
    done
  done
  tar zcvf allfiles.tar.gz *
}

concat_files folder1 folder2

, .

, .

Answer 4

cat ()?

cat file1 file2 >> outputfile

, bash script, . .

.