In tackling the machine learning and modeling case study interview, we should utilize a basic framework to craft our solution in a way that satisfies most of the requirements from an interviewer.

Notice that these steps are very similar to product and business case studies. Notably, the steps that are taken before diving into a problem are consistent across the board when it comes down to breaking down problems and applying solutions. The only difference here is that we have to be cognizant of how we can apply machine learning and dive a bit deeper into the technical details.

1. Clarification

The most common mistake that most people make when facing a modeling case study question is to underestimate the scope of the problem. For example, many inexperienced candidates will dive right into selecting a model.

Let’s take an example question asked by Uber: How would you build a model to give an estimated time of arrival for selecting a ride?

Bad Response

I would build a regression model and use features like time of day, passenger location, and driver distance. I would select a neural network architecture that would work well….

Good Response

1. What’s the scope of this model? Is it for a particular city or location, across the entire US, or for the entire world?
2. Additionally, how many ride ETAs would we have to serve per minute?
3. Lastly, how much does accuracy matter? Do riders have a propensity to cancel their rides if the estimated time is much longer than the actual estimated time after matching?

Notice how all of the initial responses were questions. This is a recurring theme in all case study questions, as asking questions in the beginning is the best signal that shows a candidate can think holistically about a problem beyond a narrow scope.

When you do not ask questions, it’s like you are playing a game of chess blindfolded against the interviewer. The information and expectation disadvantage is tremendous. The ultimate goal is to then extract the most useful information in the least number of questions to gain a foothold into the prioritization of each problem.

We have to understand a few things right off the bat when presented with a case study.

• Why would we want to build this model in the first place?
• What is the model is being used for?
• How would the model be integrated into the product?
• How does the model affect the business?
• How much do we care about model accuracy?
  

Understanding the original idea behind the question is imperative before jumping into technical details. If we ask questions like these starting out, we can get a greater understanding from the interviewer on what their expectations of this modeling scenario will look like, and can begin to implement some of their vision.

2. Make Assumptions

At some point, we will have to stop asking questions and begin making assumptions for ourselves. We want to strike a delicate balance between pestering an interviewer and coming up with ideas on our own that can help lead the discussion.

Let’s pretend the interviewer has framed the problem after our questions. We can now make some assumptions given that the interviewer has not corrected us.

Candidate Response

The model should be used across the entire United States and should be able to serve any user that wants a ride at any given time. I am assuming that the model would serve a prediction when a rider opens the Uber app and then immediately sees a ride estimated ETA if the rider were to request a ride.

3. Define Priorities

The third step we need to complete before diving into the technical implementation is defining priorities. Our job should be to confirm that the goal of the model is aligned with the goals of the business and to raise any considerations that might need to be expressed.

For example:

There is the consideration that the lower the ETA our model provides, the higher the propensity of a user to request a ride at that very moment. However, I’ll assume that our goal is to address complete accuracy of the ETA and let another ensemble of models address what should be done to increase ride request rates.

Lastly, I want to dive into the specifics of how this model could be built. I am going to assume that there is already an existing ETA model that serves predictions currently in production. I’m also assuming we have location data of both the user and drivers at any point in time while they are active on the app, and we are saving the actual time of arrival data, which is computed by end time minus start time. End time is defined by the time that the driver arrives to pick up the user and start time is defined by when the user presses the request button.

In this scenario, our priorities have been pretty well defined from our assumption and clarification points above. Our only added points are that we want to make sure:

• Whether we are building a new model or improving upon an existing one.
• Whether the user behavior based on the effect of ETAs is out of scope or not.
• What exists in our dataset to build this model.

4. Go through the modeling lifecycle

This last step is actually the meat of the solution. Now, it’s time for us to jump into more of the nitty-gritty of what’s required to build this model and analyze it.