DOE & Power Analysis

]

# Week 10: .fancy[DOE & Power Analysis]

### <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M243.4 2.6l-224 96c-14 6-21.8 21-18.7 35.8S16.8 160 32 160v8c0 13.3 10.7 24 24 24H456c13.3 0 24-10.7 24-24v-8c15.2 0 28.3-10.7 31.3-25.6s-4.8-29.9-18.7-35.8l-224-96c-8.1-3.4-17.2-3.4-25.2 0zM128 224H64V420.3c-.6 .3-1.2 .7-1.8 1.1l-48 32c-11.7 7.8-17 22.4-12.9 35.9S17.9 512 32 512H480c14.1 0 26.5-9.2 30.6-22.7s-1.1-28.1-12.9-35.9l-48-32c-.6-.4-1.2-.7-1.8-1.1V224H384V416H344V224H280V416H232V224H168V416H128V224zm128-96c-17.7 0-32-14.3-32-32s14.3-32 32-32s32 14.3 32 32s-14.3 32-32 32z"/></svg> EMSE 6035: Marketing Analytics for Design Decisions
### <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M272 304h-96C78.8 304 0 382.8 0 480c0 17.67 14.33 32 32 32h384c17.67 0 32-14.33 32-32C448 382.8 369.2 304 272 304zM48.99 464C56.89 400.9 110.8 352 176 352h96c65.16 0 119.1 48.95 127 112H48.99zM224 256c70.69 0 128-57.31 128-128c0-70.69-57.31-128-128-128S96 57.31 96 128C96 198.7 153.3 256 224 256zM224 48c44.11 0 80 35.89 80 80c0 44.11-35.89 80-80 80S144 172.1 144 128C144 83.89 179.9 48 224 48z"/></svg> John Paul Helveston
### <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M152 64H296V24C296 10.75 306.7 0 320 0C333.3 0 344 10.75 344 24V64H384C419.3 64 448 92.65 448 128V448C448 483.3 419.3 512 384 512H64C28.65 512 0 483.3 0 448V128C0 92.65 28.65 64 64 64H104V24C104 10.75 114.7 0 128 0C141.3 0 152 10.75 152 24V64zM48 448C48 456.8 55.16 464 64 464H384C392.8 464 400 456.8 400 448V192H48V448z"/></svg> November 02, 2022

]

---

# Before we start, re-install {cbcTools}

```r
remotes::install_github("jhelvy/cbcTools")
```

]

---

# Week 10: .fancy[DOE & Power Analysis]

### 1. Design of Experiment
### 2. Design Efficiency
### 3. Power Analysis

---

# Week 10: .fancy[DOE & Power Analysis]

### 1. .orange[Design of Experiment]
### 2. Design Efficiency
### 3. Power Analysis

---

# Main & Interaction Effects

---

background-color: #EEEDEE

# .center[Full design space for 3 effects: A, B, C]

---

background-color: #EEEDEE

# .center[Full design space for 3 effects: A, B, C]

# Example: _Cars_

## A: Electric? (Yes+ or No-)
## B: Warranty? (Yes+ or No-)
## C: Ford? (Yes+ or No-)

]

]

---

background-color: #EEEDEE
class: center

## Main Effects

$$
ME(a) = 
$$

$$
\left( \frac{A + AB + AC + ABC}{4}\right) - 
$$

$$
\left( \frac{I + B + C + BC}{4}\right)
$$

<br>

(A: Electric? Yes+ or No-)

]

]

---

background-color: #EEEDEE
class: center

## Interaction Effects

$$
INT(ab) = 
$$

$$
\frac{1}{2}\left[ \left( \frac{AB + ABC}{2}\right) - \left( \frac{B + BC}{2}\right) \right] - 
$$

$$
\frac{1}{2}\left[ \left( \frac{A + AC}{2}\right) - \left( \frac{I + C}{2}\right) \right]
$$

]

]

---

# .center[Example: Wine Pairings]

meat | wine
-----|------
fish | white 
fish | red
steak | white 
steak | red

]

## Main Effects

1. `meat`: **Fish** or **Steak**?
2. `wine`: **Red** or **White**?

]

---

# .center[Example: Wine Pairings]

meat | wine
-----|------
fish | white
fish | red
steak | white
steak | red

]

## Main Effects

1. `meat`: **Fish** or **Steak**?
2. `wine`: **Red** or **White**?

## Interaction Effects

1. `meat*wine`: **Red** or **White** wine _with **Steak**_?
2. `meat*wine`: **Red** or **White** wine _with **Fish**_?

]

---

# Open `interactions.Rmd`

---

# Fractional vs Full Factorial Designs

---

## .center[Full Factorial Design]

## Example: _Cars_

## A: Electric? (Yes+ or No-)
## B: Warranty? (Yes+ or No-)
## C: Ford? (Yes+ or No-)

]

```r
library(cbcTools)

profiles <- cbc_profiles(
    electric = c(1, 0),
    warranty = c(1, 0),
    ford     = c(1, 0)
)

profiles
```

```
#>   profileID electric warranty ford
#> 1         1        1        1    1
#> 2         2        0        1    1
#> 3         3        1        0    1
#> 4         4        0        0    1
#> 5         5        1        1    0
#> 6         6        0        1    0
#> 7         7        1        0    0
#> 8         8        0        0    0
```

]

---

## .center[Full Factorial Design]

## Balanced?

All levels appear an equal number of times.

## Orthogonal?

All pairs of levels appear together an equal number of times.

]

```r
library(cbcTools)

profiles <- cbc_profiles(
    electric = c(1, 0),
    warranty = c(1, 0),
    ford     = c(1, 0)
)

profiles
```

]

---

## .center[Fractional Factorial Design]

## Balanced?

All levels appear an equal number of times.

## Orthogonal?

All pairs of levels appear together an equal number of times.

]

```r
profiles[c(1, 3, 5, 6),]
```

```
#>   profileID electric warranty ford
#> 1         1        1        1    1
#> 3         3        1        0    1
#> 5         5        1        1    0
#> 6         6        0        1    0
```

]

---

# Comparing Full and Fractional Factorial Designs

# Open `balance-orthogonality.Rmd`

---

# Practice Question 1

Consider the following experiment design

a | b | c | Effect
--|---|---|-------
+ | - | - | A
- | + | - | B 
+ | - | + | AC 
- | + | + | BC

]

a) Is the design balanced? Is is orthogonal?

b) Write out the equation to compute the main effect for a, b, and c.

c) Are any main effects confounded? If so, what are they confounded with?

]

---

# Week 10: .fancy[DOE & Power Analysis]

### 1. Design of Experiment
### 2. .orange[Design Efficiency]
### 3. Power Analysis

---

# .center[A simple conjoint experiment about _cars_]

Attribute | Levels
----------|----------
Brand     | GM, BMW, Ferrari
Price     | $20k, $40k, $100k

```
Attribute counts:

brand:
  GM   BMW  Ferrari
  10    11    6

price:

20k  40k 100k
  9    9   9
```

]

```
Pairwise attribute counts:

brand & price:

20k 40k 100k
  GM        3   0    7
  BMW       4   5    2
  Ferrari   2   4    0
```

]

---

# .center[A simple conjoint experiment about _cars_]

Attribute | Levels
----------|----------
Brand     | GM, BMW, Ferrari
Price     | $20k, $40k, $100k

```
Attribute counts:

brand:
  GM    BMW   Ferrari
  92    80     98

price:

20k  40k 100k
  91   84   95
```

]

```
Pairwise attribute counts:

brand & price:

20k 40k 100k
  GM      31  31  30
  BMW     25  25  30
  Ferrari 35  28  35
```

]

---

# .center[Bayesian D-efficient designs]

### .center[Maximize information on "Main Effects" according to priors]

Attribute | Levels | Prior
----------|-------------------|----------
Brand     | GM, BMW, Ferrari  | 0, 1, 2
Price     | $20k, $40k, $100k | 0, -1, -4

`$$v_j = 1 \delta^{\mathrm{BMW}} + 2 \delta^{\mathrm{Ferrari}} -1 \delta^{\mathrm{40k}} -4 \delta^{\mathrm{100k}}$$`

---

# .center[Bayesian D-efficient designs]

### .center[Maximize information on "Main Effects" according to priors]

Attribute | Levels | Prior
----------|-------------------|----------
Brand     | GM, BMW, Ferrari  | 0, 1, 2
Price     | $20k, $40k, $100k | 0, -1, -4

```
Attribute counts:

brand:
  GM    BMW   Ferrari
  93    90     86

price:

20k  40k 100k
  97   93   78
```

]

```
Pairwise attribute counts:

brand & price:

20k 40k 100k
  GM      52  41  0
  BMW     30  30  30
  Ferrari 15  22  49
```

]

---

### Negative of the hessian evaluated at a set of parameters is called the **"Information Matrix"**

## `$$\boldsymbol{I}(\boldsymbol{\beta}) = - \nabla_{\boldsymbol{\beta}}^2 \ln (\mathcal{L})$$`

---

## "D-optimal" designs attempt to minimize the<br>"D-error" of a design

## `$$D = |\boldsymbol{I}(\boldsymbol{\beta})| ^{-1/p}$$`

where `$p$` is the number of coefficients in the model

---

# Finding Efficient Designs

# Open `d-efficiency.Rmd`

---

## Your Turn

1. Individually, create a Bayesian D-efficient fractional factorial survey design. Inspect the attribute balance and overlap.

2. Compare your results with your teammates.

---

# Quiz 4

### Link is in the #class channel

]

]

---

# Week 10: .fancy[DOE & Power Analysis]

### 1. Design of Experiment
### 2. Design Efficiency
### 3. .orange[Power Analysis]

---

# How many respondents do I need?

---

# How many respondents do I need<br>_to get X level of precision on `$\boldsymbol{\beta}$`_?

---

# Standard errors are inversely related to `$\sqrt{N}$`

```r
n <- seq(100)
se <- 1/sqrt(n)
plot(n, se, type = "l")
```

Standard errors also decrease with:

- Fewer attributes
- Fewer levels in each categorical attribute
- More questions per respondent

]

]

---

## Using {cbcTools}, we can run simulations to determine the necessary sample size for a specific model

# Open `powerAnalysis.Rmd`

---

## Your Turn

Individually:

1. Using the survey design you created in the last practice, conduct a power analysis to determine the necessary sample size to achieve a 0.05 significance level on your parameter estimates.

2. Compare your results with your teammates.

]