Interview Prep #1 — Facebook Event clicks increased by 10%, how would you investigate?

Facebook Data Scientist Interview Question. Source: Glassdoor

My Answer:

Ask Clarifying Questions:

INTERVIEWEE: Before we dive into details of the analysis, I want to make sure my understanding of the problem is correct — Facebook searches return multiple types of results: Post, Link, Group, etc. For this problem, we noticed that number of users who click on Event on search results page increased by 10% week-over-week. Is that right?

Explain my overall approach:

INTERVIEWEE: Got it. The way I want to tackle this problem is as follows: I will first try some segmentation(e.g. by language/region/platform) to narrow down the scope of the problem; next, I will dive deeper into the root causes of this sudden increase, and determine whether it is a good or bad thing for each case scenario. Does it sound like a good approach to you?

Possible Segmentations:

  • Geography: Did the increase happen in any particular region?

Construct a MECE framework:

INTERVIEWEE: For the next step, I will analyze both internal & external factors that could possibly cause the increase in clicks on Event, as well as which cases would be good/bad to us. The root cause is either internal or external, so with this framework, we can make sure we don’t miss any notable cause.

Walk through probable causes:

  • Data Accuracy: I want to first make sure we collected the right data. To verify that, I will check metrics that should correspond to clicks on Event on search result page. For example, if more users are clicking on Event, # of active users of Event feature should increase too. If there is no notable change in these metrics, it could imply there is a data collection error and we should talk with the engineers.
  • Feature Launch, UI/Algorithm Change: After confirming the data we collect is correct, I will check if our search team recently launched something that could suddenly change user behavior. If so, I will then look at the tests they did prior to launch and check if test result corresponds to what we just observed.

Decide it’s good or bad:

INTERVIEWER: You talked about different causes that could lead to this increase — how do you determine if it’s a good or bad thing?

Test new ranking algorithm:

INTERVIEWER: Good. Here’s a natural extension of the original problem — The Events team wants to up-rank Events such that they show up higher in Search. How would you determine this is a good thing or a bad thing?

Design test metrics:

INTERVIEWEE: So our business hypothesis is that we expect that if we up-rank Events, more users will click on Events. Therefore, primary metric of our experiment is # of users click on Event. In terms of secondary metrics, we can also track # of downstream actions on Event like ‘Interested’, ‘Going’, ‘Invite’ to understand how up-ranking impacts engagement with Event. If only more users are viewing Event but they’re not creating more meaningful social interactions, maybe it’s worth to launch the new algorithm.

Ricky Zhang

M.S. Data Science @ USFCA. Graduating in August.

  • Hiring Managers: if you have Data Scientist, Analytics / Product Analyst / Business Intelligence Analyst opening on your team, let’s connect on LinkedIn or send me an email at rzhang67@usfca.edu . I appreciate your help!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Ricky Zhang

Ricky Zhang

74 Followers

Data Scientist @ Twitch. M.S. Data Science @ USFCA. Sharing Data Science Case Study Interview Preps.