How can I compare two AWS Rekognition collections?

2
votes

I have two images with 40+ faces of people in it. I want to detect which faces are repeated in both images using AWS Rekognition service.

The original approach was to use the IndexFaces function of Rekognition and store all the faces of one image in one collection and the faces of the other image in another collection and then compare them using their FaceId. I thought that IndexFaces would provide a fingerprint for each face but it happens to be the FaceId is just a random identifier and not a fingerprint of the face.

I found this answer How to compare faces in a Collection to faces in a Stored Video using AWS Rekognition? but that compares all the faces in a collection with faces appearing in a video, so I would be forced to convert one of the images to a 1 second video containing the image as only frame.. which I think defeats the purpose of easy usage.

It has to be a way to compare two rekognition collections in order to check for repeated images that Im failing to find.

2
amazon-web-servicescollectionsamazon-rekognition

2 Answers

2
votes

There are two ways you could go about this:

Option 1: Use ExternalImageID

This is similar to your method.

The important part is that when a face is added to a collection, you can provide an ExternalImageID. Later, when this face is matched with an image, Amazon Rekognition will return the ExternalImageID for the face.

For example, you could store a person's name or unique identifier in the ExternalImageID.

So, your process could look like this:

  • Call DetectFaces() on image 1
  • It will return a list of FaceDetails with a bounding box for each face
  • Loop through each returned face and use the provided bounding box to call IndexFaces() for each individual face, providing an ExternalImageID each time (it could just be an incrementing number)
  • Then, call IndexFaces() on image 2
  • If it finds any faces in the collection generated from image 1, it will provide the ExternalImageID of the matching face

Option 2: Use CompareFaces()

Compares a face in the source input image with each of the 100 largest faces detected in the target input image.

This takes one input face (the largest in the source image) and compares it to all faces in the target image. Therefore, you would follow a similar process to above:

  • Call DetectFaces() on image 1
  • It will return a list of FaceDetails with a bounding box for each face
  • Loop through each returned face and use the provided bounding box to call CompareFaces() for each individual face, comparing it to image 2
  • You will be provided with a confidence level of each potentially matching face

See: Comparing Faces in Images - Amazon Rekognition

So, the second method is easier if you are just comparing two images. The first method is better if you have already stored individual faces that you wish to use again in future calls.

0
votes

Thanks to @John Rotenstein I was able to make a quick prototype using the was console:

Assuming that we have all the permissions and AWS console installed on the system and a S3 bucket called 'TestBucket' where all the images are stored, I did the following:

1.- Created a "Main Collection"

> aws rekognition create-collection --collection-id "MainCollection"

2.- Added one of the people I want to detect, I extracted the face from an individual face and ran IndexFace

> aws rekognition index-faces --image '{"S3Object":{"Bucket":"TestBucket","Name":"cristian.jpg"}}' --collection-id "MainCollection" --max-faces 100 --quality-filter "AUTO" --detection-attributes "ALL" --external-image-id "cristian.jpg"

The resulting FaceID is 'a54ef57e-7003-4721-b7e1-703d9f039da9'

3.- I added the second image to the collection:

> aws rekognition index-faces --image '{"S3Object":{"Bucket":"TestBucket","Name":"ImageContaining40plusfaces.jpg"}}' --collection-id "MainCollection" --max-faces 100 --quality-filter "AUTO" --detection-attributes "ALL" --external-image-id "ImageContaining40plusfaces.jpg"

resulted in 40+ entries like this one, showing only one for brevity:

{
  "FaceRecords": [
    {
            "FaceDetail": {
                "Confidence": 99.99859619140625, 
                "Eyeglasses": {
                    "Confidence": 54.99907684326172, 
                    "Value": false
                }, 
                "Sunglasses": {
                    "Confidence": 54.99971389770508, 
                    "Value": false
                }, 
                "Gender": {
                    "Confidence": 54.747318267822266, 
                    "Value": "Male"
                }, 
                "Landmarks": [
                    {
                        "Y": 0.311367392539978, 
                        "X": 0.1916557103395462, 
                        "Type": "eyeLeft"
                    }, 
                    {
                        "Y": 0.3120582699775696, 
                        "X": 0.20143891870975494, 
                        "Type": "eyeRight"
                    }, 
                    {
                        "Y": 0.3355730175971985, 
                        "X": 0.19253292679786682, 
                        "Type": "mouthLeft"
                    }, 
                    {
                        "Y": 0.3361922800540924, 
                        "X": 0.2005564421415329, 
                        "Type": "mouthRight"
                    }, 
                    {
                        "Y": 0.32276451587677, 
                        "X": 0.19691102206707, 
                        "Type": "nose"
                    }, 
                    {
                        "Y": 0.30642834305763245, 
                        "X": 0.1876278519630432, 
                        "Type": "leftEyeBrowLeft"
                    }, 
                    {
                        "Y": 0.3037400245666504, 
                        "X": 0.19379760324954987, 
                        "Type": "leftEyeBrowRight"
                    }, 
                    {
                        "Y": 0.3029193580150604, 
                        "X": 0.19078010320663452, 
                        "Type": "leftEyeBrowUp"
                    }, 
                    {
                        "Y": 0.3041592836380005, 
                        "X": 0.1995924860239029, 
                        "Type": "rightEyeBrowLeft"
                    }, 
                    {
                        "Y": 0.3074571192264557, 
                        "X": 0.20519918203353882, 
                        "Type": "rightEyeBrowRight"
                    }, 
                    {
                        "Y": 0.30346789956092834, 
                        "X": 0.2024637758731842, 
                        "Type": "rightEyeBrowUp"
                    }, 
                    {
                        "Y": 0.3115418553352356, 
                        "X": 0.1898096352815628, 
                        "Type": "leftEyeLeft"
                    }, 
                    {
                        "Y": 0.3118479251861572, 
                        "X": 0.1935078650712967, 
                        "Type": "leftEyeRight"
                    }, 
                    {
                        "Y": 0.31028062105178833, 
                        "X": 0.19159308075904846, 
                        "Type": "leftEyeUp"
                    }, 
                    {
                        "Y": 0.31250447034835815, 
                        "X": 0.19164365530014038, 
                        "Type": "leftEyeDown"
                    }, 
                    {
                        "Y": 0.31221893429756165, 
                        "X": 0.19937492907047272, 
                        "Type": "rightEyeLeft"
                    }, 
                    {
                        "Y": 0.3123391270637512, 
                        "X": 0.20295380055904388, 
                        "Type": "rightEyeRight"
                    }, 
                    {
                        "Y": 0.31087613105773926, 
                        "X": 0.2013435810804367, 
                        "Type": "rightEyeUp"
                    }, 
                    {
                        "Y": 0.31308478116989136, 
                        "X": 0.20125225186347961, 
                        "Type": "rightEyeDown"
                    }, 
                    {
                        "Y": 0.3264555335044861, 
                        "X": 0.19483911991119385, 
                        "Type": "noseLeft"
                    }, 
                    {
                        "Y": 0.3265785574913025, 
                        "X": 0.19839303195476532, 
                        "Type": "noseRight"
                    }, 
                    {
                        "Y": 0.3319154679775238, 
                        "X": 0.196599081158638, 
                        "Type": "mouthUp"
                    }, 
                    {
                        "Y": 0.3392537832260132, 
                        "X": 0.19649912416934967, 
                        "Type": "mouthDown"
                    }, 
                    {
                        "Y": 0.311367392539978, 
                        "X": 0.1916557103395462, 
                        "Type": "leftPupil"
                    }, 
                    {
                        "Y": 0.3120582699775696, 
                        "X": 0.20143891870975494, 
                        "Type": "rightPupil"
                    }, 
                    {
                        "Y": 0.31476160883903503, 
                        "X": 0.18458032608032227, 
                        "Type": "upperJawlineLeft"
                    }, 
                    {
                        "Y": 0.3398161828517914, 
                        "X": 0.18679481744766235, 
                        "Type": "midJawlineLeft"
                    }, 
                    {
                        "Y": 0.35216856002807617, 
                        "X": 0.19623762369155884, 
                        "Type": "chinBottom"
                    }, 
                    {
                        "Y": 0.34082692861557007, 
                        "X": 0.2045571506023407, 
                        "Type": "midJawlineRight"
                    }, 
                    {
                        "Y": 0.3160339295864105, 
                        "X": 0.20668834447860718, 
                        "Type": "upperJawlineRight"
                    }
                ], 
                "Pose": {
                    "Yaw": 4.778820514678955, 
                    "Roll": 1.7387386560440063, 
                    "Pitch": 11.82911205291748
                }, 
                "Emotions": [
                    {
                        "Confidence": 47.9405403137207, 
                        "Type": "CALM"
                    }, 
                    {
                        "Confidence": 45.432857513427734, 
                        "Type": "ANGRY"
                    }, 
                    {
                        "Confidence": 45.953487396240234, 
                        "Type": "HAPPY"
                    }, 
                    {
                        "Confidence": 45.215728759765625, 
                        "Type": "SURPRISED"
                    }, 
                    {
                        "Confidence": 50.013206481933594, 
                        "Type": "SAD"
                    }, 
                    {
                        "Confidence": 45.30225372314453, 
                        "Type": "CONFUSED"
                    }, 
                    {
                        "Confidence": 45.14192199707031, 
                        "Type": "DISGUSTED"
                    }
                ], 
                "AgeRange": {
                    "High": 43, 
                    "Low": 26
                }, 
                "EyesOpen": {
                    "Confidence": 54.95812225341797, 
                    "Value": true
                }, 
                "BoundingBox": {
                    "Width": 0.02271346002817154, 
                    "Top": 0.28692546486854553, 
                    "Left": 0.1841897815465927, 
                    "Height": 0.06893482059240341
                }, 
                "Smile": {
                    "Confidence": 53.493797302246094, 
                    "Value": false
                }, 
                "MouthOpen": {
                    "Confidence": 53.51670837402344, 
                    "Value": false
                }, 
                "Quality": {
                    "Sharpness": 53.330047607421875, 
                    "Brightness": 81.31917572021484
                }, 
                "Mustache": {
                    "Confidence": 54.971839904785156, 
                    "Value": false
                }, 
                "Beard": {
                    "Confidence": 54.136474609375, 
                    "Value": false
                }
            }, 
            "Face": {
                "BoundingBox": {
                    "Width": 0.02271346002817154, 
                    "Top": 0.28692546486854553, 
                    "Left": 0.1841897815465927, 
                    "Height": 0.06893482059240341
                }, 
                "FaceId": "570eb8a6-72b8-4381-a1a2-9112aa2b348e", 
                "ExternalImageId": "ImageContaining40plusfaces.jpg", 
                "Confidence": 99.99859619140625, 
                "ImageId": "7f09400e-2de8-3d11-af05-223f13f9ef76"
            }
        }
  ]
}

3.- Then I issued a SearchFacesById using the FaceId detected previously:

> aws rekognition search-faces --face-id "a54ef57e-7003-4721-b7e1-703d9f039da9" --collection-id "MainCollection"

and Voila! I got the face detected on the second source image as needed...

{
    "SearchedFaceId": "a54ef57e-7003-4721-b7e1-703d9f039da9", 
    "FaceModelVersion": "4.0", 
    "FaceMatches": [
        {
            "Face": {
                "BoundingBox": {
                    "Width": 0.022825799882411957, 
                    "Top": 0.31017398834228516, 
                    "Left": 0.4018920063972473, 
                    "Height": 0.06067270040512085
                }, 
                "FaceId": "bfd58e70-2bcf-403a-87da-6137c28ccbdd", 
                "ExternalImageId": "ImageContaining40plusfaces.jpg", 
                "Confidence": 100.0, 
                "ImageId": "7f09400e-2de8-3d11-af05-223f13f9ef76"
            }, 
            "Similarity": 92.36637115478516
        }
    ]
}

So now I have to do the same thing for all the other face images detected in the source image nº1 and then compare them to the ones detected from the source image nº2 using the same set of commands!