Hey everyone,
I am currently building a React Native application for a college project. The app is a simple social network that lets you do the basic things and uses Firebase to store user data.
Like every other social network, the app will have a timeline/feed for each user and users will be able to follow each other on the platform.
Now, how would you go about building a (fairly) scalable news feed with Firebase?
My current implementation stores the timeline for each user, and whenever someone they follow posts something, the timeline field gets updated. This is obviously not scalable at all, as it would use up a lot of resources when a user has millions or thousands of followers.
Would really appreciate any kind of help here, this has been a tricky solution for me to solve :)
Top comments (9)
I wouldn't ;). Firebase isn't really well-suited to this problem. Because clients download all child properties of a given node, you want to flatten your data as much as possible. General Firebase best-practices call for de-normalization, which requires a lot of additional application logic to manage that redundant data.
If I had to use Firebase though, I'd probably use Firebase Cloud Functions to build the feeds using a fan-out on read/write approach.
It's probably worth reading up on CQRS or how to build a Twitter clone using Redis for some more ideas.
HTH!
Understood, thanks! I have looked into Redis before, but to be honest I don't want to complicate even more. I just want to implement the best possible performance using Firebase here, but my current implementation seems to be the best way to go (correct me if I'm wrong, please) -- even though it isn't performant or scalable.
Oh I wasn't suggesting adding Redis, I was just referencing an article that can give you some ideas on how you might approach the problem.
If you really want to optimize for end-user performance, I'd recommend looking at fan-out on write. Fan-out on write essentially guarantees your "feed" reads will become O(1) instead of O(n). Note that your writes, will become O(n), but if you defer the fan-out to a Cloud Function, then the end-user performance doesn't take a hit at all.
Yep, my current implementation is already in O(1) since posts from the users a user is following is stored on a their "timeline" field where posts are referenced through their IDs. Because of this when the app actually loads in all the post data on the timeline, it can be a bit slow. Any way I can cache the existing data so the app shows the cached data on initial load, and then loads in the "new" data once that's ready?
If the client needs to do an additional "Find Post By ID" call for each ID referenced in the timeline list, then the function
getMyTimelinePosts()would be at least O(n). The way around this is to "fan out" copies of the full posts into each user's timeline object. When you make that first call, everything you need for the timeline will be in the response - no additional calls are necessary. Here is another article that helps explain it.Like this.
Thank you for the help :)
Have you read the section of the Firebase docs called Structure Your Database? The examples seem very relevant to what you're trying to do (especially the last one).
In your case, rather than storing a separate timeline for each user that you have to update manually, you might be better off storing a list of which users they're following, and then fetching posts based on that.
This will result in data duplication + updating like/RT values would be cumbersome. Unless I store posts separately, which would result in further increase in the time required to load the timeline
Great article! Nowadays you can simply use an existing app template (e.g. this Facebook Clone app is written in Swift) to build a fully functional social network.