DOE & Power Analysis

.leftcol30[
<center>
<img src="https://github.com/emse-madd-gwu/emse-madd-gwu.github.io/raw/master/images/madd_hex_sticker.png" width=250>
</center>
]

### <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M496 128v16a8 8 0 0 1-8 8h-24v12c0 6.627-5.373 12-12 12H60c-6.627 0-12-5.373-12-12v-12H24a8 8 0 0 1-8-8v-16a8 8 0 0 1 4.941-7.392l232-88a7.996 7.996 0 0 1 6.118 0l232 88A8 8 0 0 1 496 128zm-24 304H40c-13.255 0-24 10.745-24 24v16a8 8 0 0 0 8 8h464a8 8 0 0 0 8-8v-16c0-13.255-10.745-24-24-24zM96 192v192H60c-6.627 0-12 5.373-12 12v20h416v-20c0-6.627-5.373-12-12-12h-36V192h-64v192h-64V192h-64v192h-64V192H96z"/></svg> EMSE 6035: Marketing Analytics for Design Decisions
### <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M224 256c70.7 0 128-57.3 128-128S294.7 0 224 0 96 57.3 96 128s57.3 128 128 128zm89.6 32h-16.7c-22.2 10.2-46.9 16-72.9 16s-50.6-5.8-72.9-16h-16.7C60.2 288 0 348.2 0 422.4V464c0 26.5 21.5 48 48 48h352c26.5 0 48-21.5 48-48v-41.6c0-74.2-60.2-134.4-134.4-134.4z"/></svg> John Paul Helveston
### <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M0 464c0 26.5 21.5 48 48 48h352c26.5 0 48-21.5 48-48V192H0v272zm320-196c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12h-40c-6.6 0-12-5.4-12-12v-40zm0 128c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12h-40c-6.6 0-12-5.4-12-12v-40zM192 268c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12h-40c-6.6 0-12-5.4-12-12v-40zm0 128c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12h-40c-6.6 0-12-5.4-12-12v-40zM64 268c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12H76c-6.6 0-12-5.4-12-12v-40zm0 128c0-6.6 5.4-12 12-12h40c6.6 0 12 5.4 12 12v40c0 6.6-5.4 12-12 12H76c-6.6 0-12-5.4-12-12v-40zM400 64h-48V16c0-8.8-7.2-16-16-16h-32c-8.8 0-16 7.2-16 16v48H160V16c0-8.8-7.2-16-16-16h-32c-8.8 0-16 7.2-16 16v48H48C21.5 64 0 85.5 0 112v48h448v-48c0-26.5-21.5-48-48-48z"/></svg> November 03, 2021
]

---

# Quiz 4

---

# Week 10: .fancy[DOE & Power Analysis]

### 1. Design of Experiment

### BREAK

### 2. Design Efficiency
### 3. Power Analysis
### 4. Interactions

---

# Week 10: .fancy[DOE & Power Analysis]

### 1. .orange[Design of Experiment]

### BREAK

### 2. Design Efficiency
### 3. Power Analysis
### 4. Interactions

---

# Before we start, re-install {conjointTools}

```r
remotes::install_github("jhelvy/conjointTools")
```
]

---

# Main & Interaction Effects

---

# .center[Full design space for 3 effects: A, B, C]

---

# .center[Full design space for 3 effects: A, B, C]

## A: Electric? (Yes+ or No-)
## B: Warranty? (Yes+ or No-)
## C: Ford? (Yes+ or No-)
]

---

## Main Effects

$$
\left( \frac{A + AB + AC + ABC}{4}\right) - 
$$

$$
\left( \frac{I + B + C + BC}{4}\right)
$$
]

---

## Interaction Effects

$$
\frac{1}{2}\left[ \left( \frac{AB + ABC}{2}\right) - \left( \frac{B + BC}{2}\right) \right] - 
$$

$$
\frac{1}{2}\left[ \left( \frac{A + AC}{2}\right) - \left( \frac{I + C}{2}\right) \right]
$$
]

---

# .center[Example: Wine Pairings]

meat | wine
-----|------
fish | white 
fish | red
steak | white 
steak | red

]

## Main Effects

1. **Fish** or **Steak**?
2. **Red** or **White** wine?

## Interaction Effects

1. **Red** or **White** wine _with **Steak**_?
2. **Red** or **White** wine _with **Fish**_?
]

---

# Open `winePairings.Rmd`

---

# Fractional vs Full Factorial Designs

---

## .center[Full Factorial Design]

## A: Electric? (Yes+ or No-)
## B: Warranty? (Yes+ or No-)
## C: Ford? (Yes+ or No-)
]

```r
library(conjointTools)

levels <- list(
    electric = c(1, 0),
    warranty = c(1, 0),
    ford     = c(1, 0)
)

doe <- makeDoe(levels)
recodeDoe(doe, levels)
```

```
#>   electric warranty ford
#> 1        1        1    1
#> 2        0        1    1
#> 3        1        0    1
#> 4        0        0    1
#> 5        1        1    0
#> 6        0        1    0
#> 7        1        0    0
#> 8        0        0    0
```
]

---

## .center[Full Factorial Design]

All levels appear an equal number of times.

## Orthogonal?

All pairs of levels appear together an equal number of times.
]

```r
library(conjointTools)

levels <- list(
    electric = c(1, 0),
    warranty = c(1, 0),
    ford     = c(1, 0)
)

doe <- makeDoe(levels)
doe <- recodeDoe(doe, levels)
doe
```

---

## .center[Fractional Factorial Design]

All levels appear an equal number of times.

## Orthogonal?

All pairs of levels appear together an equal number of times.
]

```r
doe[c(1, 3, 5, 6),]
```

```
#>   electric warranty ford
#> 1        1        1    1
#> 3        1        0    1
#> 5        1        1    0
#> 6        0        1    0
```
]

---

# Comparing Full and Fractional Factorial Designs

# Open `cars.Rmd`

---

# Practice Question 1

a | b | c | Effect
--|---|---|-------
+ | - | - | A
- | + | - | B 
+ | - | + | AC 
- | + | + | BC

]

a) Is the design balanced? Is is orthogonal?

b) Write out the equation to compute the main effect for a, b, and c.

c) Are any main effects confounded? If so, what are they confounded with?
]

---

# .fancy[Break]

---

# Week 10: .fancy[DOE & Power Analysis]

### 1. Design of Experiment

### BREAK

### 2. .orange[Design Efficiency]
### 3. Power Analysis
### 4. Interactions

---

## We want to find `$\boldsymbol{\beta}$` by maximizing the log-likelihood

---

### Covariance of `$\boldsymbol{\beta}$` inversely related to matrix of 2nd derivatives

### Negative of the hessian evaluated at the MLE<br>solution is the **"Observed Information Matrix"**

## `$$\boldsymbol{I}(\boldsymbol{\beta}) = - \nabla_{\boldsymbol{\beta}}^2 \ln (\mathcal{L})$$`

---

## "D-optimal" designs attempt to maximize the<br>"D-efficiency" of a design

## `$$D = \left( \frac{|\boldsymbol{I}(\boldsymbol{\beta})|}{n^p} \right)^{1/p}$$`

where `$p$` is the number of coefficients in the model and `$n$` is the total sample size

# D ranges from 0 to 1
# Designs are _more_ orthogonal as D --> 1

---

# Finding Efficient Designs

# Open `efficiency.Rmd`

---

## Your Turn

.leftcol80[
1. Individually, create a fractional factorial design of experiment for your team project. Are you able to identify a high D-efficient design with fewer trials than a full factorial design. Can you find a _balanced_ design that is also efficient?

2. Compare your results with your teammates.

3. As a team, consider whether there are any restrictions you should make on your design and examine the impact (if any) those restrictions have on your design efficiency.
]

---

# Week 10: .fancy[DOE & Power Analysis]

### 1. Design of Experiment

### BREAK

### 2. Design Efficiency
### 3. .orange[Power Analysis]
### 4. Interactions

---

# How many respondents do I need?

---

# How many respondents do I need<br>_to get X level of precision on `$\boldsymbol{\beta}$`_?

---

# Standard errors are inversely related to `$\sqrt{N}$`

```r
n <- seq(100)
se <- 1/sqrt(n)
plot(n, se, type = "l")
```

Standard errors also decrease with:

- Fewer attributes
- Fewer levels in each categorical attribute
- More questions per respondent
]

---

## Using {conjointTools}, we can run simulations to determine the necessary sample size for a specific model

# Open `powerAnalysis.Rmd`

---

## Your Turn

1. Using your design of experiment you just created in the last practice, conduct a power analysis to determine the necessary sample size to achieve a 0.05 significance level on your parameter estimates.

2. Compare your results with your teammates.
]

---

# Week 10: .fancy[DOE & Power Analysis]

### 1. Design of Experiment

### BREAK

### 2. Design Efficiency
### 3. Power Analysis
### 4. .orange[Interactions]

---

# Open `powerAnalysis_interactions.Rmd`