2
votes

I'm currently uploading the phones contact of my mobile users to a Firestore database.

On average a user has ~500 phone numbers, and I will be having ~1000 users.

My structure is as follow:

Firestore-root
    |
    --- users_contacts (collection)
         |
         --- uid (document)
              |
              --- contacts (subcollection)
                   |
                   --- phoneNumberOfContact (document)
                          |
                          --- phoneNumberOfContact: true

And I have another collection where I store general phone numbers that I want to compare a specific user's contacts with.

I have about 50,000 phone number in there, each as a document. This will greatly increase later, maybe to 1 million.

Firestore-root
    |
    --- db_contacts (collection)
         |
         --- phoneNumber (document)
              |
              --- phoneNumber: true

I'm trying to check the common numbers of a specific known uid and the db_contacts collection. How many numbers of a known uid exist in the db_contacts collection.

My Cloud Function will be as follow:

  1. Fetch all the Phone Numbers of a Contact
    • First I wanted to only fetch the ids of the document of a user, since the id is the phone number, hoping it would make the proccess faster. But it seems its not possible in Javascript as snapshotChanges() does not exist.
  2. Loop through the fetched contacts and check if the contact exists in db_contacts. Since I already know the reference path to check if it exists or not, this should go fast
  3. Return all the common contacts

If there was an alternative to snapshotChanges() in JavaScript my script would run much faster. Is my way of thinking correct?

What I did so far:

exports.findCommonNumbers = functions.https.onCall((data, context) => {
    return new Promise((resolve, reject) => {
        fetchCommonNumbers().then((commonNumbers) => {
            console.log(commonNumbers);
            resolve("Done");
        }).catch((err) => {
            console.log(err);
            reject("Error Occured");
        });


    });
});

async function fetchCommonNumbers() {
    var commonNumbers = [];

    let contactsReference = admin.firestore().collection("user_contacts").doc("iNaYVsDCg3PWsDu67h75xZ9v2vh1").collection("contacts");
    const dbContactReference = admin.firestore().collection('db_contacts');

    userContacts = await contactsReference.get();
    userContacts = userContacts.docs;
    for(var i in userContacts){
        var userContact = userContacts[i];
        const DocumentID = userContact.ref.id;
        //Check if Document exist 
        dbContact = await dbContactReference.doc(DocumentID).get();
        if (dbContact.exists) {
            commonNumbers.push(DocumentID);
        }
    }

    return Promise.resolve(commonNumbers);
}

The function findCommonNumbers is taking 60 seconds to execute. It has to be much faster. How can I make it faster?

1

1 Answers

4
votes

When you're looking for documents in common, you're fetching one, waiting for it to come back, fetching the next, waiting for it... I haven't used async/await before, but I'd do something like:

Promise.All(userContacts.map(userContact => {
    const DocumentID = userContact.ref.id;
    //Check if Document exists
    return dbContactReference.doc(DocumentID).get().then(dbContact => {
        if (dbContact.exists) {
            commonNumbers.push(DocumentID);
        }
    });
}));

Sorry for the code fragment and mistakes; I'm on mobile. This should request them all at once.

Edit: to return after something other than all have returned:

new Promise((resolve, reject) => {
    var returned = 0;
    userContacts.map(userContact => {
        const DocumentID = userContact.ref.id;
        //Check if Document exists
        dbContactReference.doc(DocumentID).get().then(dbContact => {
            if (dbContact.exists) {
                commonNumbers.push(DocumentID);
            }
            returned++;
            if (returned == userContact.length || commonNumbers.length >= 5) {
                resolve(commonNumbers);
            }
        });
    });
});