pipe tar extract into tar create

Question

I have a tar.gz right now, and I want to extract just a file or two from it, and pack/add those into a new tar.gz, all in one go. Of course I can just save to a temporary file and work with it, but the ABSOLUTE requirement is to do this all without having any intermediate file output, i.e. piping. In other words, what I would like is something like the following pseudo-code (obviously the syntax is incorrect)

tar -xvf first.tar.gz subdir1/file1 subdir2/file2 | tar cf - | gzip > second.tar.gz

Does anyone know the proper syntax for this? I have tried many variants, but to no avail.

I am also very open to the idea of using cpio, but again, I am stumped by how to get the syntax down properly, and from what I understand, cpio intakes only archives or filenames, not files.

Any help will be greatly appreciated.

EDIT: There is no particular filename pattern inside the tarball to extract. Given that the BSD and GNU tar can only search one pattern at a time, I'm not sure if it's even possible to use the include/exclude flags, respectively.

Answer 1

I am assuming that you are using or that you can get GNU tar.

You can use the --delete option to process one tar file to another. E.g.:

% tar cf x.tar a b c d
% tar tf x.tar
a
b
c
d
% cat x.tar | tar f - --delete b c > y.tar
% tar tf y.tar
a
d
%

Note that you can specify multiple names to delete. Then you just need to figure out how specify all the files to get rid of on the command line, instead of the files to keep.

Answer 2

If you know the filename pattern that you are going to extract, try this:

tar zcf second.tar.gz --include='filepattern' @first.tar.gz

Here is an example showing the inclusion of multiple files:

% tar cf x.tar a b c d
% tar tf x.tar
a
b
c
d
% cat x.tar | tar cf - --include='a' --include='d' @- > y.tar
% tar tf y.tar
a
d
%

Answer 3

None of the above solutions worked for me, tar complained about creating an empty archive

Instead I just used &&:

tar -xf first.tar.gz subdir1/file1 subdir2/file2 && tar -cvf second.tar --remove-files subdir1/file1 subdir2/file2 

Where --remove-files is the option to remove the files after adding to archive.

Another method I found to work is:

tar -cf second.tar `tar -tf first.tar.gz /desired/directory`

Note that keeps the entire directory context, so /desired/directory is still in the new tar.

Answer 4

When unpacking, tar normally writes the unpacked files to disk, not the output stream. You can use -O or --to-stdout to have it write files out to stdout, but there won't be a break between files or any way to know when one ends and another begins.

In addition, tar's create option can only read files from disk, not from stdin. This makes sense because of the afore mentioned problem with knowing when one file ends and another begins.

This means there is no way to do this from the command line the way you want.

However, I'm betting that you could write a perl or python script using libraries that you can get to operate strictly in-memory.