Shell script to find file and copy file from other location

Question

I'm puzzeling with a shell script.

What am I trying to achieve?

In my 'upload2website'-folder on my Mac I have numerous files that have the following AA_bestandsnaam-xx.jpg or AA_bestandsnaam - xx.jpg (Watch the spaces). xx is a filenumber, could be any number.
Below the 'upload2website'-folder there a several subfolders.
In the subfolder below upload2website there are the origins of AA_bestandnaam-xx.jpg or AA_bestandnaam - xx.jpg
If the file is in upload2website is found in one of the subfolders, then copy it to the folders uploads
It is possible that the AA is placed at the end of the file, f.i. bestandnaam-xx_AA.jpg

Example of the structure

upload2website-folder

AA_bestandsnaamA-68.jpg
AA_bestandsnaamB - 01.jpg
AA_bestandsnaamC-106.jpg
AA_bestandsnaamD-30.jpg

subfolders

upload2website/folder1/bestandsnaamA-13.jpg
upload2website/folder1/bestandsnaamA-28.jpg
upload2website/folder1/bestandsnaamA-68.jpg
upload2website/folder1/bestandsnaamA-90.jpg
upload2website/folder2/bestandsnaamB - 08.jpg
upload2website/folder2/bestandsnaamB - 09.jpg
upload2website/folder2/bestandsnaamB - 10.jpg
upload2website/folder2/bestandsnaamB - 13.jpg
upload2website/folder2/bestandsnaamB - 01.jpg
upload2website/folder3/bestandsnaamC-08.jpg
upload2website/folder3/bestandsnaamC-90.jpg
upload2website/folder3/bestandsnaamC-105.jpg
upload2website/folder3/bestandsnaamC-106.jpg
etc.

The following script I'm working on, but that doesn't work, f.i. if there are spaces in the name, than name is truncated as of the first space. I already added -iname and quotes around the variables.

#!/bin/sh
SRCDIR=upload2website
DSTDIR=uploads
PREFIX=FB_
UPLOAD=`ls "$SRCDIR"/*.jpg`
#SEARCH=`$UPLOAD | sed "/$PREFIX/s///"`
for src in $UPLOAD
do
   # echo source: "$src"
   test="`basename "$src"`"
   FBname=`basename "$src"  | sed "/$PREFIX/s///"`
   # name=`basename "$src"`
   # echo name: "$name"

   #dst=`find "$DSTDIR" -name "$FBname"`
   dst=`find "$SRCDIR" -iname "$FBname"`
   # if [ "$dst" != "" ]; then

   if [ "$dst" != "" ]; then
    echo FBname: "$FBname"
     echo cp "$dst" "$DSTDIR"
   fi
done

Can somebody please help me to get the script right?

Update: Clarify folders

the folderstructure:

/Users/gjdegraaf/Documents/upload2website <<<--- contains subfolders and origin of the image files
/Users/gjdegraaf/Documents/uploads. <<<--- should contain the copied origin of the file that matches the file without AA_

What should the execution of your script output? Which files should be copied where for the input provided? — KamilCuk
The origin of the file found in one of the subfolders below upload2website should be found and copied to uploads. — user2037412
So cp upload2website/AA_bestandsnaamA-68.jpg uploads cp upload2website/AA_bestandsnaamB - 01.jpg uploads cp upload2website/AA_bestandsnaamC-106.jpg uploads should be the output from your script? Please try not to describe, rather please show what should happen. or cp upload2website/folder1/bestandsnaamA-68.jpg uploads cp upload2website/folder2/bestandsnaamB - 01.jpg uploads cp upload2website/folder3/bestandsnaamC-106.jpg uploads? And why you describe the prefix to be AA_ yet your script has FB_? — KamilCuk
@KamilCuk the second part of your example should happen. Sorry if it was confusing. — user2037412

KamilCuk KamilCuk · Accepted Answer · 2019-12-31T11:43:25

Use $(...) instead of `. Backticks are discouraged`.
Don't for i in $(ls). Forget about ls. Never use ls in scripts. Forget it. Use find.
Don't use uppercase variables for local variables. By convention uppercase variables are exported. And you can run into clashes, for example RANDOM.

I don't fully understand where is uploads dir and how is it related to upload2website dir and how the structure really is. But maybe the folowing code snippets will help you with some ideas.

Your script could look like this:

#!/bin/sh
srcdir=upload2website
dstdir=uploads
prefix=FB_
# output a newline separated list of filenames in srcdir
find "$srcdir" -mindepth 2 -maxdepth 2 -type f -name '*.jpg' |
# read the newline separated list
while IFS= read -r src; do
   filename=$(basename "$src")
   # NOTE: here prefix is  FB_ not AA_ ....
   FBname=$(printf "%s\n" "$filename" | sed "s/$prefix//")
   # check if the file FBname exists in src folder
   if dst=$(
                    find "$srcdir" -maxdepth 1 -type f |
          grep "$FBname"
                    ); then
      echo cp "$src" "$dstdir"
   fi
done

Bu I would most probably do something like this, which needs bash:

#!/bin/bash
src=upload2website
dst=uploads
prefix=FB_

# find all files in src dir
find "$src" -maxdepth 1 -type f -printf "%f\n" |
# remove all the `AA_`
sed 's@^AA_@@; s/_AA\.jpg$//;' |
sort |
# filter from the list the files from dst
# output only paths
join -t$'\t' -11 -21 -o2.2 - <(
    # a newline separated list with two tab separated columns
    find "$src" -maxdepth 2 -mindepth 2 -type f -printf '%f\t%p\n' |
    # sort on filename
    sort -k1
) |
# copy the files
xargs -I{} echo cp {} "$dst"

I tested on repl both scripts and they outputted:

cp upload2website/folder1/bestandsnaamA-68.jpg uploads
cp upload2website/folder2/bestandsnaamB - 01.jpg uploads
cp upload2website/folder3/bestandsnaamC-106.jpg uploads

But I don't fully understand which folder is uploads and which is not.

Both scripts still assume there are no newline in the filenames. It is possible to rewrite the second code snippet to use zero terminated streams, with GNU sed -z option, find -print0, join -z, xargs -0 and sort -z. You can get around tabulation used for separator with good cut -d$'\t' -f2-, the following code uses gnu tools and bash and should be able to handle any filenames except those with the byte 0x01 in the filenames:

sep=$'\x01'
# find all files in src dir
find "$src" -maxdepth 1 -type f -printf "%f\0" |
# remove all the `AA_`
sed -z 's@^AA_@@; s/_AA\.jpg$//;' |
sort -z |
# filter from the list the files from dst
# output only paths
join -z -t"$sep" -11 -21 - <(
    # a newline separated list with two tab separated columns
    find "$src" -maxdepth 2 -mindepth 2 -type f -printf "%f$sep%p\0" |
    # sort on filename
    sort -z -t$sep -s -k1
) |
cut -z -d"$sep" -f2- |
# copy the files
xargs -0 -I{} echo cp {} "$dst"

Shell script to find file and copy file from other location

4 Answers