Sensitivity Analysis

Due: Nov 18 by 11:59pm

Weight: This assignment is worth 3% of your final grade.

Purpose: The purpose of this assignment is to introduce the concept of preference heterogeneity - that is, not everyone has the same preferences. We will see two ways to account for this: 1) specify preference parameters as distributions, and 2) estimate fixed parameter models on different sub-groups in your data.

Assessment: This assignment is graded using a check system:

  • ✔+ (110%): Responses shows phenomenal thought and engagement with the course content. I will not assign these often.
  • ✔ (100%): Responses are thoughtful, well-written, and show engagement with the course content. This is the expected level of performance.
  • ✔− (50%): Responses are hastily composed, too short, and/or only cursorily engages with the course content. This grade signals that you need to improve next time. I will hopefully not assign these often.

Notice that this is essentially a pass/fail system. I’m not grading your writing ability and I’m not counting the number of words you write - I’m looking for thoughtful engagement. One or two sentences is not enough. Write at least a paragraph and show me that you did the readings assigned.

1. Get Organized

Follow these instructions:

  1. Download and edit this template.
  2. Unzip the template folder. Make sure you actually unzip it! (in Windows, right-click it and use “extract all”)
  3. Open the .Rproj file to open RStudio.
  4. Inside RStudio, open the hw11.qmd file, take notes, and write some example code as you go through the readings / exercises below.

2. Readings

This is your last homework! Okay, well not really - your homework next week will be a take home exam. But don’t worry, we will spend our time in class next week going over the entire class, so there will be time to get answers to any questions you have.

For this week, the only thing you will look over is the concept of a sensitivity analysis. Last week in class we covered how to use our estimated logit model to conduct a market simulation, which gave us predicted market shares for different products. Naturally, one of the key things we might want to know is how sensitive our predicted market shares are to changes in our attributes.

To understand how we calculate such sensitivities, take a look at the logitr-cars repo. Download the repo using this link, then open the R project file to open RStudio. Then in RStudio, open the following two files:

  • 9.1-compute-sensitivity.R
  • 9.2-plot-sensitivity.R

Your job this week is to go through each of these files line-by-line to make sure you understand what is being done. (It might also help to look back at some of the earlier files we covered as a refresher, such as 3.1 on estimating models or 5.1 on uncertainty).

Let’s start with the first file, 9.1. In this file, we use the predict() function to compute market shares for different markets.

In the first sensitivity analysis, we define a baseline market, then we create repeated copies of this market but with incremental changes to the price of alternative 2. When we use predict(), we get predicted probabilities for each market, which we can then use to see how the predicted probability for alternative 2 changes with price.

In the second sensitivity analysis, we also start with repeated copies of our baseline market, but then we substitute in a series of different cases where we change the values of other attributes. Each case has a “low” and “high” value, where “high” means they result in expected higher market shares, and “low” means the opposite. Again, we use predict() to obtain predicted market shares for each scenario.

At the bottom of the script, we save the data frames.

In 9.2, we use the saved results to visualize the results computed in 9.1. This code shows us how to create line plots of the predicted market share versus price as well as revenue versus price. The third chart is a “tornado” plot, which shows us the predicted market share for each of the “high” and “low” conditions in 9.1.

These charts – sensitivity to price, and sensitivity to changes in other attributes – will be important charts to include in your final analysis report. You can use these examples to help you create similar charts for your project.

Once you have gone through each file, consider what might make a good “baseline” market for your project, then write your reflection.

3. Reflect

Reflect on what you’ve learned while going through these exercises. Is there anything that jumped out at you? Anything you found particularly interesting or confusing? Write at least a paragraph in your hw11.qmd file, and include at least one question. The teaching team will review the questions we get and will try to answer them either in Slack or in class.

If you’re unsure where to start with a reflection, try filling out this template:

“I used to think ______, now I think ______ 🤔”

4. Submit

To submit your assignment, follow these instructions:

  1. Render your .qmd file by either clicking the “Render” button in RStudio or running the command quarto::quarto_render("hw11.qmd") command.
  2. Open the rendered html file and make sure it looks good! Is all the formatting as you expected?
  3. Create a zip file of all the files in your R project folder for this assignment and submit it on the corresponding assignment submission on Blackboard.