Exam Review & Final Analysis Overview

.leftcol30[
<center>
<img src="https://github.com/emse-madd-gwu/emse-madd-gwu.github.io/raw/master/images/madd_hex_sticker.png" width=250>
</center>
]

### <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M496 128v16a8 8 0 0 1-8 8h-24v12c0 6.627-5.373 12-12 12H60c-6.627 0-12-5.373-12-12v-12H24a8 8 0 0 1-8-8v-16a8 8 0 0 1 4.941-7.392l232-88a7.996 7.996 0 0 1 6.118 0l232 88A8 8 0 0 1 496 128zm-24 304H40c-13.255 0-24 10.745-24 24v16a8 8 0 0 0 8 8h464a8 8 0 0 0 8-8v-16c0-13.255-10.745-24-24-24zM96 192v192H60c-6.627 0-12 5.373-12 12v20h416v-20c0-6.627-5.373-12-12-12h-36V192h-64v192h-64V192h-64v192h-64V192H96z"/></svg> EMSE 6035: Marketing Analytics for Design Decisions
### <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M224 256c70.7 0 128-57.3 128-128S294.7 0 224 0 96 57.3 96 128s57.3 128 128 128zm89.6 32h-16.7c-22.2 10.2-46.9 16-72.9 16s-50.6-5.8-72.9-16h-16.7C60.2 288 0 348.2 0 422.4V464c0 26.5 21.5 48 48 48h352c26.5 0 48-21.5 48-48v-41.6c0-74.2-60.2-134.4-134.4-134.4z"/></svg> John Paul Helveston
### <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M0 464c0 26.5 21.5 48 48 48h352c26.5 0 48-21.5 48-48V192H0v272zm320-196c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12h-40c-6.6 0-12-5.4-12-12v-40zm0 128c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12h-40c-6.6 0-12-5.4-12-12v-40zM192 268c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12h-40c-6.6 0-12-5.4-12-12v-40zm0 128c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12h-40c-6.6 0-12-5.4-12-12v-40zM64 268c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12H76c-6.6 0-12-5.4-12-12v-40zm0 128c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12H76c-6.6 0-12-5.4-12-12v-40zM400 64h-48V16c0-8.8-7.2-16-16-16h-32c-8.8 0-16 7.2-16 16v48H160V16c0-8.8-7.2-16-16-16h-32c-8.8 0-16 7.2-16 16v48H48C21.5 64 0 85.5 0 112v48h448v-48c0-26.5-21.5-48-48-48z"/></svg> December 01, 2021
]

---

# Week 14: .fancy[Exam Review & Final Analysis Overview]

### 1. Final Report & Presentation
### 2. Sensitivity Analysis

### BREAK

### 3. Exam Review

---

# Week 14: .fancy[Exam Review & Final Analysis Overview]

### 1. .orange[Final Report & Presentation]
### 2. Sensitivity Analysis

### BREAK

### 3. Exam Review

---

## 1. Clean data 
## 2. Modeling

- Simple logit
- Mixed logit
- One sub-group model

## 3. Analysis

- WTP for key features
- Market simulation
- Sensitivity analysis
]

## 1. Introduction
## 2. Survey Design
## 3. Data Analysis
## 4. Results (plots / text)
## 5. Recommendations
]

---

# Final Presentation

## - In class, 12/15

## - 10 minutes (strict)

## - External Panel of Reviewers

## - Slides due on Blackboard by midnight on 12/14

---

# How to design good slides

---

# Hitchcock's rule

---

## .center[Hitchcock's rule]

> # The size of any object in your frame should be proportional to its importance to the story at that moment

## .center[[Watch this example](https://youtu.be/E1LzhiCcOY0?t=174)]

---

## .center[Hitchcock's rule]

> # The size of any object in your ~~frame~~ **slide** should be proportional to its importance to the story at that moment

---

...and finally you will read this

<br><br>
<div style='font-size: 70pt; text-align: center;'>You will read this first</div>

---

# .font120[Main point at top and use a big font!]

</br>
</br>

## (see Stephanie Evergreen's blog post ["So What?"](https://stephanieevergreen.com/so-what/))

---

# Except for Tesla, EV adoption in the U.S. is **flat**

---

# Tesla's Model 3 is a Game Changer for EVs

---

<p style="font-size:40pt;">> 40pt font for titles</p>
<p style="font-size:24pt;">> 24pt font for all other text</p>

(Exception: footer text can be small)

<div class="footer-small"><span>Footer text</span></div>

---

# Avoid fonts like

</br>
<div style='font-family: "Comic Sans MS"; font-size: 40pt'>Comic Sans</div>
</br>
<div style='font-family: "Papyrus"; font-size: 40pt'>Papyrus</div>
</br>
</br>

# They make your work look amateurish

---

# Consider using a light-colored background<br>(tan / gray)

---

# Use high contrast between font and background color

</br>

.leftcol[
<center>
<div style="background-color: #FFF; padding: 50px; margin: 0px; width: 300px; font-size: 28pt;">
Dark text on a light background works well
</div>
</center>
]

.rightcol[
<center>
<div style="background-color: #000; color: #FFF; padding: 50px; margin: 0px; width: 300px; font-size: 28pt;">
Light text on a dark background also works well
</div>
</center>
]

---

# Use high contrast between font and background color

</br>

.leftcol[
<center>
<div style="background-color: #FFF; color: #ffef01; padding: 50px; margin: 0px; width: 350px; font-size: 28pt;">
Yellow text on a white background is horrible
</center>
]

.rightcol[
<center>
<div style="background-color: #000; color: #0000ff; padding: 50px; margin: 0px; width: 350px; font-size: 28pt;">
Blue text on a black background is horrible
</center>
]

---

# 1 slide, 1 idea

## Break up main points into multiple slides

---

# Number your slides!

---

---

</br></br></br></br></br></br>

# Example of an acceptable slide footer

# </br></br></br>&darr;

<div class="footer-small"><span>Data source: http://somesourceofdata.com &emsp;&emsp; © John Paul Helveston, GWU, Apr. 2021</span></div>

---

# If you are in person, consider using handouts<br>(1-2 pages)

???

1. They can preview your results
2. They're more likely to remember your main points

---

# How to design good slides

- **Hitchcock's rule**: The size of any object on your slide should be proportional to its importance to the story at that moment
- **Slide titles**: A single statement about what slide means (in big font!)
- **Use large font sizes** (>40 titles, >24 text)
- Consider using a **light-colored background** (tan / gray) 
- Use **high contrast** between font and background color
- **Don't use silly fonts** like Comic Sans, Papyrus, etc.
- **1 slide, 1 idea**: Break up main points into multiple slides
- **Slide numbers**: bottom-left or bottom-right
- **Remove "chart junk"**: logos, etc. (exception: small footers)
- **Consider using handouts**

- **Don't pack the slide with bullet lists** (see what I did there?)

---

# Week 14: .fancy[Exam Review & Final Analysis Overview]

### 1. Final Report & Presentation
### 2. .orange[Sensitivity Analysis]

### BREAK

### 3. Exam Review

---

.leftcol[
### .center[**Market share** sensitivity to price]
<center>
<img src="images/share_price_plot.png" width=100%>
</center>
]

.rightcol[
### .center[**Revenue** sensitivity to price]
<center>
<img src="images/rev_price_plot.png" width=100%>
</center>
`$$R = Q*P$$`
]

---

.leftcol[
### .center[**Market share** sensitivity to price]
<center>
<img src="images/share_price_plot.png" width=100%>
</center>
]

- Solid line reflects _interpolation_ (attribute range in survey)
- Dashed line reflects _extrapolation_ (beyond attribute range in survey)
- Ribbon reflects _parameter uncertainty_
]

---

## .center[Market share sensitivity to all attributes]

---

.leftcol[
### .center[Market share sensitivity to all attributes]
<center>
<img src="images/tornado_plot.png" width=100%>
</center>
]

- Middle point reflects baseline market share: 
    - **Price**: $25,000 
    - **Fuel Economy**: 100 mpg 
    - **0-60 mph Accel. time**: 6 sec
    
- Boundaries on each attribute should reflect max feasible attribute bounds
]

---

# .center[Sensitivity analyses]

<br>

## 1. Open `logitr-cars`

## 2. Open `code/9.1-compute-sensitivity.R`
## 3. Open `code/9.2-plot-sensitivity.R`

---

# .fancy[Break]

---

# Week 14: .fancy[Exam Review & Final Analysis Overview]

### 1. Final Report & Presentation
### 2. Sensitivity Analysis

### BREAK

### 3. .orange[Exam Review]

---

- Data wrangling in R 
- Utility models 
- Maximum likelihood estimation
- Optimization
- Uncertainty 
- Design of experiment
- WTP 
- Market simulations
- Sub-group models
- Using R for all of the above<br>(e.g., estimating models wiht `logitr`)
]

- formr.org
- Mixed logit
]

---

# Data wrangling in R

---

# Steps to importing external data files

## 1. Create a path to the data

```r
library(here)
*path_to_data <- here('data', 'data.csv')
path_to_data
```

```
#> [1] "/Users/jhelvy/gh/0MADD/2021-Fall/class/14-review/data/data.csv"
```

## 2. Import the data

```r
library(tidyverse)
*data <- read_csv(path_to_data)
```

---

# Steps to importing external data files

```r
library(tidyverse)

data <- read_csv(here::here('data', 'data.csv'))
```

---

# .center[The main `dplyr` "verbs"]

<br>

"Verb"        | What it does
--------------|--------------------
`select()`    | Select columns by name
`filter()`    | Keep rows that match criteria
`arrange()`   | Sort rows based on column(s)
`mutate()`    | Create new columns

---

# Example data frame

```r
beatles <- tibble(
    firstName   = c("John", "Paul", "Ringo", "George"),
    lastName    = c("Lennon", "McCartney", "Starr", "Harrison"),
    instrument  = c("guitar", "bass", "drums", "guitar"),
    yearOfBirth = c(1940, 1942, 1940, 1943),
    deceased    = c(TRUE, FALSE, FALSE, TRUE)
)

beatles
```

```
#> # A tibble: 4 × 5
#>   firstName lastName  instrument yearOfBirth deceased
#>   <chr>     <chr>     <chr>            <dbl> <lgl>   
#> 1 John      Lennon    guitar            1940 TRUE    
#> 2 Paul      McCartney bass              1942 FALSE   
#> 3 Ringo     Starr     drums             1940 FALSE   
#> 4 George    Harrison  guitar            1943 TRUE
```

---

# `filter()` and `select()`:

Get the **first & last name** of members born after 1941 & are still living

```r
beatles %>% 
  filter(yearOfBirth > 1941, deceased == FALSE) %>% 
  select(firstName, lastName)
```

```
#> # A tibble: 1 × 2
#>   firstName lastName 
#>   <chr>     <chr>    
#> 1 Paul      McCartney
```

---

# Create new variables with `mutate()`

Use the `yearOfBirth` variable to compute the age of each band member

```r
beatles %>%
    mutate(age = 2021 - yearOfBirth) %>% 
    arrange(age)
```

```
#> # A tibble: 4 × 6
#>   firstName lastName  instrument yearOfBirth deceased   age
#>   <chr>     <chr>     <chr>            <dbl> <lgl>    <dbl>
#> 1 George    Harrison  guitar            1943 TRUE        78
#> 2 Paul      McCartney bass              1942 FALSE       79
#> 3 John      Lennon    guitar            1940 TRUE        81
#> 4 Ringo     Starr     drums             1940 FALSE       81
```

---

# .center[Handling if/else conditions]

### .center[`ifelse(<condition>, <if TRUE>, <else>)`]

```r
beatles %>%
    mutate(playsGuitar = ifelse(instrument == "guitar", TRUE, FALSE))
```

```
#> # A tibble: 4 × 6
#>   firstName lastName  instrument yearOfBirth deceased playsGuitar
#>   <chr>     <chr>     <chr>            <dbl> <lgl>    <lgl>      
#> 1 John      Lennon    guitar            1940 TRUE     TRUE       
#> 2 Paul      McCartney bass              1942 FALSE    FALSE      
#> 3 Ringo     Starr     drums             1940 FALSE    FALSE      
#> 4 George    Harrison  guitar            1943 TRUE     TRUE
```

---

# Utility models

---

# Random utility model

<br>

## The utility for alternative `$j$` is
# `$$\tilde{u}_j = v_j + \tilde{\varepsilon}_j$$`

## `$v_j$` = Things we observe (non-random variables)
## `$\tilde{\varepsilon}_j$` = Things we _don't_ observe (random variable)

---

## **Logit model**: Assume that `$\tilde{\varepsilon}_j$` ~ [Gumbel Distribution](https://en.wikipedia.org/wiki/Gumbel_distribution)

<center>
<img src="images/utility.png" width=450>
</center>
]

# `$$P_j = \frac{e^{v_j}}{\sum_k{e^{v_k}}}$$`
]

---

#.center[Notation Convention]

## Continuous: `$x_j$`

## `$$u_j = \beta_1 x_{j}^{\mathrm{price}} + \dots$$`

```
#>   price
#> 1     1
#> 2     2
#> 3     3
```
]

## Discrete: `$\delta_j$`

## `$$u_j = \beta_1 \delta_{j}^{\mathrm{ford}} + \beta_2 \delta_{j}^{\mathrm{gm}} \dots$$`

```
#>   brand brand_BMW brand_Ford brand_GM
#> 1  Ford         0          1        0
#> 2    GM         0          0        1
#> 3   BMW         1          0        0
```
]

---

# .center[Dummy-coded variables]

```r
data <- data.frame(
    brand = c("Ford", "GM", "BMW"))

data
```

```
#>   brand
#> 1  Ford
#> 2    GM
#> 3   BMW
```
]

```r
library(fastDummies)

dummy_cols(data, "brand")
```

```
#>   brand brand_BMW brand_Ford brand_GM
#> 1  Ford         0          1        0
#> 2    GM         0          0        1
#> 3   BMW         1          0        0
```
]

---

`$v_j = \beta_1 x^\mathrm{price}$`

]

```r
model <- logitr(
    data   = data,
    choice = "choice",
    obsID  = "obsID",
    pars   = "price"
)
```

<br>

Coef. | Interpretation
------|------------------
β1 | how utility changes with increasing _price_

]

`$v_j = \beta_1 \delta_{j}^{\mathrm{ford}} + \beta_2 \delta_{j}^{\mathrm{gm}}$`
]

```r
model <- logitr(
    data   = data,
    choice = "choice",
    obsID  = "obsID",
    pars   = c("brand_Ford", "brand_GM")
)
```

Coef. | Interpretation
------|------------------
β1 | utility for _Ford_ relative to _BMW_
β2 | utility for _GM_ relative to _BMW_

]

---

# .center[Estimating utility models]

<br>

## 2. Open `code/3.1-model-mnl.R`
]

---

# `mnl_dummy`

All discrete (dummy-code) variables

```r
pars = c(
  "price_20", "price_25",
  "fuelEconomy_25", "fuelEconomy_30",
  "accelTime_7", "accelTime_8",
  "powertrain_Electric")
```

Reference Levels:

- Price: 15
- Fuel Economy: 20
- Accel. Time: 6
- Powertrain: "Gasoline"
]

All continuous (linear), except for `powertrain_Electric`

```r
pars = c(
  'price', 'fuelEconomy', 'accelTime', 
  'powertrain_Electric')
```

Reference Levels:

- Powertrain: "Gasoline"
]

---

# Practice Question 1

Let's say our utility function is:

.font80[$$v_j = \beta_1 x_j^{\mathrm{price}} + \beta_2 x_j^{\mathrm{cacao}} + \beta_3 \delta_j^{\mathrm{hershey}} + \beta_4 \delta_j^{\mathrm{lindt}}$$]

And we estimate the following coefficients:

Parameter | Coefficient 
----------|-----------
`$\beta_1$` | -0.1
`$\beta_2$` | 0.1
`$\beta_3$` | -2.0
`$\beta_4$` | -0.1
]

<table class="table table-hover table-condensed" style="width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> Attribute </th>
   <th style="text-align:left;"> Bar 1 </th>
   <th style="text-align:left;"> Bar 2 </th>
   <th style="text-align:left;"> Bar 3 </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;"> Price </td>
   <td style="text-align:left;"> $1.20 </td>
   <td style="text-align:left;"> $1.50 </td>
   <td style="text-align:left;"> $3.00 </td>
  </tr>
  <tr>
   <td style="text-align:left;"> % Cacao </td>
   <td style="text-align:left;"> 10% </td>
   <td style="text-align:left;"> 60% </td>
   <td style="text-align:left;"> 80% </td>
  </tr>
  <tr>
   <td style="text-align:left;"> Brand </td>
   <td style="text-align:left;"> Hershey </td>
   <td style="text-align:left;"> Lindt </td>
   <td style="text-align:left;"> Ghirardelli </td>
  </tr>
</tbody>
</table>
]

---

# Maximum likelihood estimation

---

# Maximum likelihood estimation

---

## .center[Computing the likelihood]

`$f(x)$`: probability of observing `$x$`
]

---

## .center[Computing the likelihood]

`$f(x)$`: probability of observing `$x$`

`$\mathcal{L}(\theta | x)$`: probability that `$\theta$` are the true parameters, given that observed `$x$`

`$\mathcal{L}(\theta | x) = f(x_1) f(x_2) \dots f(x_n)$`

Log-likelihood converts multiplication to summation:

`$\ln \mathcal{L}(\theta | x) = \ln f(x_1) + \ln f(x_2) \dots \ln f(x_n)$`
]

---

# Practice Question 2

**Observations** - Height of students (inches):

```
#>  [1] 65 69 66 67 68 72 68 69 63 70
```

a) Let's say we know that the height of students, `$\tilde{x}$`, in a classroom follows a normal distribution. A professor obtains the above height measurements students in her classroom. What is the log-likelihood that `$\tilde{x} \sim \mathcal{N} (68, 4)$`? In other words, compute `$\ln \mathcal{L} (\mu = 68, \sigma = 4)$`.

b) Compute the log-likelihood function using the same standard deviation `$(\sigma = 4)$` but with the following different values for the mean, `$\mu: 66, 67, 68, 69, 70$`. How do the results compare? Which value for `$\mu$` produces the highest log-likelihood?

---

# Optimization

---

## Optimality conditions

---

---

# Uncertainty

---

---

## The _curvature_ of the log-likelihood function is<br>inversely related to the hessian

---

## The _curvature_ of the log-likelihood function is<br>inversely related to the hessian

---

### Usually report parameter uncertainty ("standard errors") with `$\sigma$` values

---

## .center[Two approaches for obtaining confidence interval]

## Using Standard Errors

1. Get coefficients, `beta`
2. Get covariance matrix, `covariance` 
3. `se <- sqrt(diag(covariance))`
4. `coef_ci <- c(beta - 2*se, beta + 2*se)`

## Using Simulated Draws

1. Get coefficients, `beta`
2. Get covariance matrix, `covariance`  
3. `draws <- as.data.frame(MASS::mvrnorm(10^5, beta, covariance))`
4. `coef_ci <- maddTools::ci(draws, ci = 0.95)`

---

```r
# 1. Get coefficients
beta <- c(
    price = -0.7, mpg = 0.1, elec = -4.0)

# 2. Get covariance matrix
hessian <- matrix(c(
    -6000,   50,   60,
       50, -700,   50,
       60,   50, -300),
    ncol = 3, byrow = TRUE)

covariance <- -1*solve(hessian)
```
]

```r
beta <- coef(model)
covariance <- vcov(model)
```
]

---

# Practice Question 3

Suppose we estimate the following utility model describing preferences for cars:

$$
u_j = \alpha p_j + \beta_1 x_j^{mpg} + \beta_2 x_j^{elec} + \varepsilon_j
$$

Compute a 95% confidence interval around the coefficients using:

a) Standard errors 
b) Simulated draws

]

Parameter | Coefficient
----------|------------
`$\alpha$` | -0.7
`$\beta_1$` | 0.1
`$\beta_2$` | -0.4

Hessian:

$$
`\begin{bmatrix}
-6000 & 50 & 60
\\ 
50 & -700 & 50
\\
60 & 50 & -300
\end{bmatrix}`
$$
]

---

# Design of experiment

---

# .center[Wine Pairings Example]

meat | wine
-----|------
fish | white 
fish | red
steak | white 
steak | red

]

## Main Effects

1. **Fish** or **Steak**?
2. **Red** or **White** wine?

## Interaction Effects

1. **Red** or **White** wine _with **Steak**_?
2. **Red** or **White** wine _with **Fish**_?
]

---

## "D-optimal" designs maximize **main** effect information<br>but confound **interaction** effect information

## `$$D = \left( \frac{|\boldsymbol{I}(\boldsymbol{\beta})|}{n^p} \right)^{1/p}$$`

where `$p$` is the number of coefficients in the model and `$n$` is the total sample size

---

# WTP

---

## Willingness to Pay (WTP)

<br>

## `$$\tilde{u}_j = \alpha p_j + \boldsymbol{\beta} x_j + \tilde{\varepsilon_j}$$`

<br>

## `$$\boldsymbol{\omega} = \frac{\boldsymbol{\beta}}{-\alpha}$$`

---

# .center[Computing WTP with draws]

## `$$\hat{\boldsymbol{\omega}} = \frac{\hat{\boldsymbol{\beta}}}{-\hat{\alpha}}$$`

```r
draws_other <- draws[,2:ncol(draws)]
draws_price <- draws[,1]
draws_wtp <- draws_other / (-1*draws_price)
head(draws_wtp)
```

```
#>            [,1]      [,2]
#> [1,] 0.10069956 -5.845472
#> [2,] 0.14959437 -5.808602
#> [3,] 0.20743359 -5.874524
#> [4,] 0.32113064 -5.655103
#> [5,] 0.20449059 -5.723632
#> [6,] 0.02892836 -5.916094
```
]

```r
maddTools::ci(draws_wtp)
```

```
#>         mean       lower      upper
#> 1  0.1426251  0.03758061  0.2494388
#> 2 -5.7193063 -5.97968329 -5.4715373
```
]

---

## Willingness to Pay (WTP)

## `$$\tilde{u}_j = \alpha p_j + \boldsymbol{\beta} x_j + \tilde{\varepsilon_j}$$`
]

## `$$\boldsymbol{\omega} = \frac{\boldsymbol{\beta}}{-\alpha}$$`
## `$$\lambda = - \alpha$$`
## `$$\tilde{u}_j = \lambda (\boldsymbol{\omega} x_j - p_j) + \tilde{\varepsilon_j}$$`
]

---

# WTP space models have non-convex<br>log-likelihood functions!

<br>

# **Use multi-start loop with<br>random starting points**

---

# Market simulations

---

# .center[Simulate Market Shares]

## 1. Define a market, `$X$`

## 2. Compute shares:

## `$$\hat{P}_j = \frac{e^{\hat{\boldsymbol{\beta}}'\boldsymbol{X}_j}}{\sum_{k=1}^J e^{\hat{\boldsymbol{\beta}}'\boldsymbol{X}_k}}$$`

---

# .center[Simulate Market Shares]

---

# .center[Simulate Market Shares]

```r
X %*% beta
```
]

---

# .center[Simulating Market Shares **with Uncertainty**]

Rely on the `predict()` function to compute shares with uncertainty.

Internally, it:

1. Takes draws of `$\boldsymbol{\beta}$`
2. Computes `$P_j$` for each draw 
3. Returns mean and confidence interval computed from draws

---

# Review the `logitr-cars` examples

---

## Your Turn

### As a team:

.leftcol80[.font120[
- Read in and clean your final data. 
- Estimate a baseline model. 
- Set your baseline market simulation case. 
- Compute sensitivities to price and other attributes. 
]]