Meta (Facebook) Machine Learning Mock Interview: Illegal Items Detection

Today Zarrar talks us through this question asked by Facebook about how to use Machine Learning to flag illegal items posted on a marketplace.

More from Jay:

Related Links:

[iqjayfeng]

If you enjoyed discussing this machine learning question with us, explore more on our website!

[AlexXPandian]

This guy has mastered the art of how to talk for 20 minutes something that can be explained to a technologist in 2 minutes, and that my friends is what a system design interview is all about. You have to talk about every detail no matter how boring/mundane it is to you or how obvious you might think it is.

[anasal-tirawi]

Typical end to end ML Question:

Understate the problem, Data collection, Feature Engineering, Building Model, Train Model, Evaluate Performance ( Confusion Matrix: Precision ± Recall), Deploy Model, Rebuild Model if needed

[julianmartindelfiore]

I feel this video is a fantastic resource, not only the explanation was great and very insightful, but I think you also made the right questions, going for the extra-mile of the explanation/analysis...thank you for sharing!

[ploughable]

2 points that I would added for the end questions:
1. in order to overcome the coded firearm words -> use tranformers models like BERT as you can catch the meaning by the embeddings (ie: cosine similarity) and filter the best ratings
2. Computer Vision on the images can be used as additional inference if the F1 score is low, but not always as this type of inference is more expensive

[umamiplaygroundnyc]

Wow this guy is good. I really like how he start from model framework with baseline model, point out the reasoning and key considerations - and we can evolve from there to more complicated model just by all similar reasoning

[ziruiwang]

The first tree based model Zarrar was talking about is probably AdaBoost, where the weight of those misclassified instances will be enlarged for the next tree.

[iqjayfeng]

Thanks for tuning in! If you're interested in learning more about machine learning, be sure to check out our machine learning course. It's designed to help you master the key concepts and skills needed to excel in machine-learning roles.

[sunny]

Should've mentioned that people try to disguise the actual product description using proxy words.

Also, to include image analysis or not, I'd draw multiple samples and train models in A/B setting. Then run a t-test to see if the mean prediction metric is significantly different or not.

[prak]

Sorry, where did you discuss the label generation part? There are multiple ways to generate labels with pros and cons:
1. user feedback: Automatic, lot of data but noisy.
2. Manual annotation: accurate labels but not scalable. Very high proportion of examples would be tagged as negative.
3. Bootstrap: Train a simple model and sample more examples based on model scores to get a higher proportion of positive examples.
4. Hybrid: Manually annotate examples marked as "X" by users where "X" can be tags like "illegal", "offsensive", etc.

[sallespadua]

Amazing! As a point to improve even more, I’d add as finishing touch fine-tuning the model with adversarial examples.

[junweima]

It's also possible to use re-ranking or bagging approaches to combine xgboost model and vision/nlp model, which would most likely improve performance

[KS-dfcp]

I would have suggested CNN as an alternative approach but ya agree. The listing is not only about an image but also text. Edge case where they have different text and different images then that won't get captured. Thank you.

[_seeker]

Re; whether or not to do CV on images - shouldn't one do error analysis to check if text and other features lacked the predictive power and the signal was elsewhere (aka images) which is why we should invest in extracting signals from images; as opposed to building a giant model with all features and doing ablations to understand feature class importance. Latter seems quite expensive?

[dkshmeeks]

Great video. I find all the quick cuts to be a bit disorienting though.

[being.jajabor]

This is a fantastic video for giving an idea for an ML system design interview ! Thanks for making this.

[jamessukanto]

Hello. I think this was super helpful overall. I'm a little confused when he describes Gradient Boosting. For each successor tree, we should set new target labels for training errors in the predecessor, no? (and leave the weights alone)

[mdaniels]

You wan a system that overfits and hits lots of false positives, as false negatives can be catastrophic, legally, for the reputation of the business, could even lead to regulatory action and media scrutiny, killing sales, market cap, etc.. You then have agents go through the false positives and efficiently decide if they are truly false positive or not. This data can also help train the model. The cost of hiring people to go through and check is much lower than losing 5% of market cap due to negative press.

[Gerald-izmv]

what does the following mean? TF-IDF: "We scale the values of each word based of each frequency in different postings"?

[claude]

@iqjayfeng I think Zarrar mistakenly mixed up False Pos and False Neg around 2:00 mark. It would be ok if customer service received False Neg (model pred True but its really False) not False Pos