Compressing Directory tree into multiple tar archives into a single xz

Question

I'm trying to compress a large rootdir which contains many subdir_i the folder tree looks like:

./rootdir
./rootdir/subdir_1
./rootdir/subdir_2
...

I am looking to output this to a single compressed archive but having each subdirectory in it's own tar archive:

rootdir.tar.xz  # containing:
     subdir_1.tar
     subdir_2.tar

I've tried the following :

for foo in `find rootdir -maxdepth 1 -name "subdir_*" -type d`
do
     tar --create --verbose --file=- --directory="rootdir" `basename ${foo}` 
     # in shorter form:  tar -cvf - -C rootdir subdir_i
done | xz -zc9 > rootdir.tar.xz

Which does isolate the subdirectory into an xz but a single tar archive is inside with only the last directory:

rootdir.tar.xz
     rootdir.tar # containing subdir_2/

However the size of that archive is consistent with compression of the whole rootdir tree. Any ideas why that is and how to get it to do what I want (without using intermediate archives)?

Wayne Vosberg Wayne Vosberg · Accepted Answer · 2017-05-12T16:56:16

Each time you run tar inside the loop you are creating a new archive -- which is why the final output is only the last directory.

I do not believe there is any way to do what you want without having intermediate files. You could do something like this to minimize the space required:

rm rootdir.tar
find rootdir/* -maxdepth 0 -type d | while read foo
do
    b=$(basename "$foo")
    tar --create --file "$b".tar --directory rootdir "$b"
    tar --append --file rootdir.tar "$b".tar
    rm "$b".tar
done
xc -zc9 rootdir.tar

Compressing Directory tree into multiple tar archives into a single xz

1 Answers