Facebook System Design Interview: Design an Analytics Platform (Metrics & Logging)

Watch our mock Facebook system design interview. Neamah asks Hozefa (Facebook, Wealthfront EM) a system design question on building a metrics and logging service.

Chapters -
00:00:00 - Introduction
00:00:33 - Clarifying questions
00:01:52 - Requirements
00:07:34 - API
00:10:12 - Design
00:21:02 - Interview analysis

Watch more videos here:

ABOUT US:
Did you enjoy this interview question and answer? Want to land your dream career? Exponent is an online community, course, and coaching platform to help you ace your upcoming interview. Exponent has helped people land their dream careers at companies like Google, Microsoft, Amazon, and high-growth startups. Exponent is currently licensed by Stanford, Yale, UW, and others.

Our courses include interview lessons, questions, and complete answers with video walkthroughs. Access hours of real interview videos, where we analyze what went right or wrong, and our 1000+ community of expert coaches and industry professionals, to help you get your dream job and more!

#systemdesign #facebook #software #engineeringmanagement #tech #entrepreneurship #exponent #tpm

[yuanzhang]

for the system needs to be in near real time, this is a clarify question to ask the interviewer instead of assuming.

[rituraj]

I have a suggestion for all videos under this platform
First they are super helpful when it comes how to carry out the whole design..like how to estimate or begin with
But all of them lack cross questions from interviewee side..I mean in real life we can be bombarded with minute detail level questions
Like in this video...how is data enrichment working or how come we are making data collection to be configurable without having whole business use cases

Bottomline : make it more tougher :)

[dicksonchibuzor]

Heard the interviewer made mention as part of the requirements that system could scale up to a billion users meaning events could be at least double of that (depending on the metrics that want to be tracked). I think in that case maybe a nosql wouldn't be the best persistent data storage decision. Maybe an OLAP kind of database (like clickhouse) should be used. This will definitely have a drastic positive impact in the query time for both visualization (retrieving) and inserting of events and will also help in creating way faster aggregates.

Also another improvement that can be made with the design is maybe the queue can come after the validation/scrubbing service and not before. It could help save some space in the queues and not have them overwhelmed because only validated data will get into the queue and invalidated ones are discarded . Only when I see the queue should come before is if we are validating a very large batch of payload at a time then maybe we can stick with this design because validation might take some time for extremely large batches.

[mrsiddhu]

Great video - Kudos to the interviewer for making the environment so comfortable. Few things:
1. I could not understand the choice of data base. Ideally this should be a combination of TimeSeries DB + Data warehouse?
2. Certain key components/Aspects like rule engine ( for acting on events), Notification systems ( for notifying the interested subscribers) were missing?
3. 90 days retention is a very less SLA. Down-sampling the data for lowering the volume and storing it for long term could have been discussed.
4. I thought that the interviewer wanted to go beyond just visualization - To automated actions ( alarms etc.) and analytics too.

[heterodyned]

Metrics and logging sound like two “separate” design questions 🤷🏻‍♂️

[sivaranjansahu]

Great video. Very natural and realistic, not like the rehearsed and phony ones like many other videos on YT.

[seenu]

Not satisfied with the discussion. Scope of the question is not clear. Are we building a system for analytics computed out of logging data or are we building a system that has logging and analytics as separate components?

Interviewee could have discussed about:

1. Grain of data that is sent by logging system. Is it individual events or aggregated counts?
2. Database design that is optimized for analyzing time-series data
3. Could have expanded machine-generated events and user-generated events and have different treatments on those datasets down the line.

[saip]

Question to people proficient in designing backend systems - is this a good example of an interview or the design? I personally found this to be crossing out checkboxes in an interview. There isn't enough trade off discussions or building towards a solution. This seems like an inconsistent brain dump of a known solution.

[schan]

This interview is too short so there is not enough time to talk about some of the details. The design interview should be at least 40 minutes. The candidate only had 21 minutes. There's not enough time to do deep dive and it seemed rushed. There is not time to talk about scaling the individual components. Sampling is not scaling. The interview should be longer so that the candidate can talk about how many servers are needed, how much disk space required for X number of years/month, how many requests can be served per second, etc.

The requirements list is too short. I feel we didn't spend enough time on the requirements.

How would you determine the level of the candidate based on this interview performance?

[bentchow]

Introducing a Data Catalog would really help with managing PII and auditing where and how sensitive data is being used through data lineage.

[psychoprincess]

Small correction at 6:00:
For money/banking system, consistency should be more prioritized over availability.

[RM-bgcd]

Event Sourcing and projections for visualization would have been amazing here

[OneMillionDollars-tuur]

I am a system design interviewer and a hiring manager and I will probably give him a NO.

[RituRaj-rw]

Its surprising that there was no discussion on OLAP storage solutions, since we will be analysing these metrics as end product

[rohitparthasarathy]

I think for time series data we should be using RDBMS with Sharding or even better have the graph being generated from In memory DB.

[riit]

Feedback:
1. should have talked more details about data storage and how the storage would support faster queries. Some sample queries as example must be shown and these queries are served.
2. No mention of how logs are stored and indexed for faster search.
3. Didn't justify the usage of queue?

[sagarchoudhury]

I think this interview will not fly. lots of flaws

[amazingabhay]

whats the process of archiving looks like ? how/who gonna move data from main db to archive db and what would happen to precomputed visualisation data ?

[KrishnaG]

This could be a case for Kafka for message processing queue with event driven API in mind..

[CommanderShepard]

dear team, please provide the name of the tool that the user is using for drawing the architecture