Firebase Firestore, query a users friend's posts

Question

I am looking create a social-media feed using Firebase. My data is structured like this:

users: {
 uid: {
   ... // details  
  }
}

friends: { 
  uid: {
    friends: { // sub collection
      fuid: {
         ... // details
     }       
    }
  }
}`

posts: { 
  postId: {
    postedBy: uid
    ... // details
  }
}

Now I am trying to get the posts from all friends of the user, limit it to the most recent 10 posts, and then create a scrolling directive that queries the next set of 10 posts so that the user doesn't have to query and load posts^N for friends^N on the page load. But I'm not really sure how to query firebase in an effective manner like this, for the user's friends and then their posts.

I have the scrolling directive working, taken from Jeff Delaney's Infinite Scrolling Lesson on AngularFirebase.com. But it only handles the posts (boats in the tutorial) collection as a whole, without selectively querying within that collection (to check if the user is a friend).

The only solution that I could think of was to query all of the user's friends posts, store that in an array, and then chunk load the results in the DOM based on the last batch of posts that were loaded. This just seems like it could be really inefficient in the long-haul if the user has 100's of friends, with 100's of posts each.

Answer 1

If I get it right, you are duplicating the post for each user in the user's friend list right? I don't think it is a good idea if your app escalates... At this time, the cost for 100k doc writes is $0,18, so:

Imagine that a user of your app have 1000 friends. When he posts anything, you are making 1000 writes in the database. imagine that you have 1000 active users like him. You have just made 1.000.000 writes now and paid $1.80.

Now even worse: you probably have on each post, a duplicated field for user displayName and a profileImageUrl. Imagine that this user has 500 posts in his history and have just changed his profile picture. You will have to update one of the fields for each post on each of his 1000 friend's feed right? You will be doing 1000 * 500 = 500.000 writes just for updating the profileImageUrl! and if the user didn't like the photo? he tries 3 new photos and now in 10 minutes you had made 2.000.000 writes in the database. This means you will be charged $3.60. It may not seems too much, but pay attention that we're talking about 1 single user in a single moment. 1000 users changing profile picture 4 times in the same day and you are paying $3,600.00. Take a look at this article: https://proandroiddev.com/working-with-firestore-building-a-simple-database-model-79a5ce2692cb#7709

Answer 2

I ended up solving this issue by leveraging Firebase Functions. I have two collections, one is called Posts and the other is called Feeds. When a user adds a post, it gets added to the Posts collection. When this happens, it triggers a Firebase Function, which then grabs the posting user's UID.

Once it has the UID, it queries another collection called Friends/UID/Friends and grabs all of their friend's UID's.

Once it has the UID's, it creates a batch add (in case the user has more than 500 friends), and then adds the post to their friend's Feeds/UID/Posts collection.

The reason I chose this route, was a number of reasons.

1. Firebase does not allow you to query with array lists (the user's friends).
2. I did not want to filter out posts from non-friends.
3. I did not want to download excessive data to the user's device.
4. I had to paginate the results in order from newest to oldest.

By using the above solution, I am now able to query the Feeds/UID/Posts/ collection, in a way that returns the next 10 results every time, without performance or data issues. The only limitation I have not been able to get around completely is it takes a few seconds to add the post to the user's personally feed, as the Function needs time to spin up. But this can be mitigated by increasing the memory allocation for that particular function.

I also do the above listed for posts that are edited and or deleted.

Answer 3

I think i have a solution for Firestore Social Feed queries. Not sure if it works but here it is;

1. A Friends collection keeps the friends UUID'S list as an array in a document. Every document in this collection is for a user. So when the user logs in we first have the friends list with a cloud function with "one read" right? All friends id's are in one document. And we also put a lastchecked time stamp to this document. Everytime we get friends array we record the date.

2. Now a cloud function can check all users posts one by one. As i understand latest IN queries allow an array up to 10 UUID's. So if user has 100 friend query will end in ten rounds. Now we have sth to serve.

3. Instead of directly serving the posts we create a collection for every user. We will put all this collected data to document but we slice it to days. Let's pretend we already have older posts in this usersfeed collection (every day as a document). So we had a last time check on our friends document. We query now -> last checked date. This way we only fetched unseen posts and sliced them daily (if they belong to more days ofcourse)

4. So while this happens on cloud function we already served the previous feed document. And when collection has new document firestore already listens and adds right? If the user scrolls down we get the previous days document. So every document will have more then one posts data as map / array.

This saves many read counts i guess.