class: middle, inverse .leftcol30[ <center> <img src="https://madd.seas.gwu.edu/images/logo.png" width=250> </center> ] .rightcol70[ # Week 10: .fancy[DOE & Power Analysis] ### <svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M243.4 2.6l-224 96c-14 6-21.8 21-18.7 35.8S16.8 160 32 160v8c0 13.3 10.7 24 24 24H456c13.3 0 24-10.7 24-24v-8c15.2 0 28.3-10.7 31.3-25.6s-4.8-29.9-18.7-35.8l-224-96c-8-3.4-17.2-3.4-25.2 0zM128 224H64V420.3c-.6 .3-1.2 .7-1.8 1.1l-48 32c-11.7 7.8-17 22.4-12.9 35.9S17.9 512 32 512H480c14.1 0 26.5-9.2 30.6-22.7s-1.1-28.1-12.9-35.9l-48-32c-.6-.4-1.2-.7-1.8-1.1V224H384V416H344V224H280V416H232V224H168V416H128V224zM256 64a32 32 0 1 1 0 64 32 32 0 1 1 0-64z"/></svg> EMSE 6035: Marketing Analytics for Design Decisions ### <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M304 128a80 80 0 1 0 -160 0 80 80 0 1 0 160 0zM96 128a128 128 0 1 1 256 0A128 128 0 1 1 96 128zM49.3 464H398.7c-8.9-63.3-63.3-112-129-112H178.3c-65.7 0-120.1 48.7-129 112zM0 482.3C0 383.8 79.8 304 178.3 304h91.4C368.2 304 448 383.8 448 482.3c0 16.4-13.3 29.7-29.7 29.7H29.7C13.3 512 0 498.7 0 482.3z"/></svg> John Paul Helveston ### <svg aria-hidden="true" role="img" viewBox="0 0 448 512" style="height:1em;width:0.88em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:white;overflow:visible;position:relative;"><path d="M152 24c0-13.3-10.7-24-24-24s-24 10.7-24 24V64H64C28.7 64 0 92.7 0 128v16 48V448c0 35.3 28.7 64 64 64H384c35.3 0 64-28.7 64-64V192 144 128c0-35.3-28.7-64-64-64H344V24c0-13.3-10.7-24-24-24s-24 10.7-24 24V64H152V24zM48 192H400V448c0 8.8-7.2 16-16 16H64c-8.8 0-16-7.2-16-16V192z"/></svg> November 01, 2023 ] --- class: center, middle # Before we start, re-install {cbcTools} .code100[ ```r remotes::install_github("jhelvy/cbcTools") ``` ] --- class: inverse, middle # Week 10: .fancy[DOE & Power Analysis] ### 1. Design of Experiment ### 2. Design Efficiency ### 3. Power Analysis --- class: inverse, middle # Week 10: .fancy[DOE & Power Analysis] ### 1. .orange[Design of Experiment] ### 2. Design Efficiency ### 3. Power Analysis --- class: inverse, middle, center # Main & Interaction Effects --- background-color: #EEEDEE # .center[Full design space for 3 effects: A, B, C] <center> <img src="images/doe_cube.png" width=600> </center> --- background-color: #EEEDEE # .center[Full design space for 3 effects: A, B, C] .leftcol[ # Example: _Cars_ ## A: Electric? (Yes+ or No-) ## B: Warranty? (Yes+ or No-) ## C: Ford? (Yes+ or No-) ] .rightcol[ <center> <img src="images/doe_cube.png" width=100%> </center> ] --- background-color: #EEEDEE class: center ## Main Effects .leftcol40[ $$ ME(a) = $$ $$ \left( \frac{A + AB + AC + ABC}{4}\right) - $$ $$ \left( \frac{I + B + C + BC}{4}\right) $$ <br> (A: Electric? Yes+ or No-) ] .rightcol60[ <center> <img src="images/main_a.png" width=550> </center> ] --- background-color: #EEEDEE class: center ## Interaction Effects .leftcol[ $$ INT(ab) = $$ $$ \frac{1}{2}\left[ \left( \frac{AB + ABC}{2}\right) - \left( \frac{B + BC}{2}\right) \right] - $$ $$ \frac{1}{2}\left[ \left( \frac{A + AC}{2}\right) - \left( \frac{I + C}{2}\right) \right] $$ ] .rightcol[ <center> <img src="images/int_ab.png" width=550> </center> ] --- # .center[Example: Wine Pairings] .leftcol40[ meat | wine -----|------ fish | white fish | red steak | white steak | red ] -- .rightcol60[ ## Main Effects 1. `meat`: **Fish** or **Steak**? 2. `wine`: **Red** or **White**? ] --- # .center[Example: Wine Pairings] .leftcol40[ meat | wine -----|------ fish | white fish | red steak | white steak | red ] .rightcol60[ ## Main Effects 1. `meat`: **Fish** or **Steak**? 2. `wine`: **Red** or **White**? ## Interaction Effects 1. `meat*wine`: **Red** or **White** wine _with **Steak**_? 2. `meat*wine`: **Red** or **White** wine _with **Fish**_? ] --- class: center, middle # Open `interactions.qmd` --- class: inverse, middle, center # Fractional vs Full Factorial Designs --- ## .center[Full Factorial Design] .leftcol[ ## Example: _Cars_ ## A: Electric? (Yes+ or No-) ## B: Warranty? (Yes+ or No-) ## C: Ford? (Yes+ or No-) ] .rightcol[ ```r library(cbcTools) profiles <- cbc_profiles( electric = c(1, 0), warranty = c(1, 0), ford = c(1, 0) ) profiles ``` ``` #> profileID electric warranty ford #> 1 1 1 1 1 #> 2 2 0 1 1 #> 3 3 1 0 1 #> 4 4 0 0 1 #> 5 5 1 1 0 #> 6 6 0 1 0 #> 7 7 1 0 0 #> 8 8 0 0 0 ``` ] --- ## .center[Full Factorial Design] .leftcol[ ## Balanced? All levels appear an equal number of times. ## Orthogonal? All pairs of levels appear together an equal number of times. ] .rightcol[ ```r library(cbcTools) profiles <- cbc_profiles( electric = c(1, 0), warranty = c(1, 0), ford = c(1, 0) ) profiles ``` ``` #> profileID electric warranty ford #> 1 1 1 1 1 #> 2 2 0 1 1 #> 3 3 1 0 1 #> 4 4 0 0 1 #> 5 5 1 1 0 #> 6 6 0 1 0 #> 7 7 1 0 0 #> 8 8 0 0 0 ``` ] --- ## .center[Fractional Factorial Design] .leftcol[ ## Balanced? All levels appear an equal number of times. ## Orthogonal? All pairs of levels appear together an equal number of times. ] .rightcol[ ```r profiles[c(1, 3, 5, 6),] ``` ``` #> profileID electric warranty ford #> 1 1 1 1 1 #> 3 3 1 0 1 #> 5 5 1 1 0 #> 6 6 0 1 0 ``` ] --- class: center, middle # Comparing Full and Fractional Factorial Designs # Open `balance-orthogonality.qmd` --- class: inverse # Practice Question 1 .leftcol[ Consider the following experiment design a | b | c | Effect --|---|---|------- + | - | - | A - | + | - | B + | - | + | AC - | + | + | BC ] .rightcol[ a) Is the design balanced? Is is orthogonal? b) Write out the equation to compute the main effect for a, b, and c. c) Are any main effects confounded? If so, what are they confounded with? ] --- class: inverse, middle # Week 10: .fancy[DOE & Power Analysis] ### 1. Design of Experiment ### 2. .orange[Design Efficiency] ### 3. Power Analysis --- # .center[A simple conjoint experiment about _cars_] Attribute | Levels ----------|---------- Brand | GM, BMW, Ferrari Price | $20k, $40k, $100k .center[**Design: .red[9] choice sets, .blue[3] alternatives each**] -- .leftcol[ ``` Attribute counts: brand: GM BMW Ferrari 10 11 6 price: 20k 40k 100k 9 9 9 ``` ] -- .rightcol[ ``` Pairwise attribute counts: brand & price: 20k 40k 100k GM 3 0 7 BMW 4 5 2 Ferrari 2 4 0 ``` ] --- # .center[A simple conjoint experiment about _cars_] Attribute | Levels ----------|---------- Brand | GM, BMW, Ferrari Price | $20k, $40k, $100k .center[**Design: .red[90] choice sets, .blue[3] alternatives each**] -- .leftcol[ ``` Attribute counts: brand: GM BMW Ferrari 92 80 98 price: 20k 40k 100k 91 84 95 ``` ] -- .rightcol[ ``` Pairwise attribute counts: brand & price: 20k 40k 100k GM 31 31 30 BMW 25 25 30 Ferrari 35 28 35 ``` ] --- # .center[Bayesian D-efficient designs] ### .center[Maximize information on "Main Effects" according to priors] -- Attribute | Levels | Prior ----------|-------------------|---------- Brand | GM, BMW, Ferrari | 0, 1, 2 Price | $20k, $40k, $100k | 0, -1, -4 `$$v_j = 1 \delta^{\mathrm{BMW}} + 2 \delta^{\mathrm{Ferrari}} -1 \delta^{\mathrm{40k}} -4 \delta^{\mathrm{100k}}$$` --- # .center[Bayesian D-efficient designs] ### .center[Maximize information on "Main Effects" according to priors] Attribute | Levels | Prior ----------|-------------------|---------- Brand | GM, BMW, Ferrari | 0, 1, 2 Price | $20k, $40k, $100k | 0, -1, -4 .leftcol[ ``` Attribute counts: brand: GM BMW Ferrari 93 90 86 price: 20k 40k 100k 97 93 78 ``` ] -- .rightcol[ ``` Pairwise attribute counts: brand & price: 20k 40k 100k GM 52 41 0 BMW 30 30 30 Ferrari 15 22 49 ``` ] --- class: center ### Negative of the hessian evaluated at a set of parameters is called the **"Information Matrix"** ## `$$\boldsymbol{I}(\boldsymbol{\beta}) = - \nabla_{\boldsymbol{\beta}}^2 \ln (\mathcal{L})$$` --- class: center ## "D-optimal" designs attempt to minimize the<br>"D-error" of a design ## `$$D = |\boldsymbol{I}(\boldsymbol{\beta})| ^{-1/p}$$` where `\(p\)` is the number of coefficients in the model --- class: center, middle # Finding Efficient Designs # Open `design-efficiency.qmd` --- class: inverse <div class="countdown" id="timer_a5e2e679" data-warn-when="15" data-update-every="1" tabindex="0" style="top:0;right:0;font-size:2em;"> <div class="countdown-controls"><button class="countdown-bump-down">−</button><button class="countdown-bump-up">+</button></div> <code class="countdown-time"><span class="countdown-digits minutes">20</span><span class="countdown-digits colon">:</span><span class="countdown-digits seconds">00</span></code> </div> ## Your Turn 1. Individually, create a Bayesian D-efficient fractional factorial survey design. Inspect the attribute balance and overlap. 2. Compare your results with your teammates. --- class: inverse # Quiz 4 <div class="countdown" id="timer_fae01404" data-warn-when="30" data-update-every="1" tabindex="0" style="bottom:0;left:0;font-size:4em;"> <div class="countdown-controls"><button class="countdown-bump-down">−</button><button class="countdown-bump-up">+</button></div> <code class="countdown-time"><span class="countdown-digits minutes">10</span><span class="countdown-digits colon">:</span><span class="countdown-digits seconds">00</span></code> </div> .leftcol[ ### Link is in the #class channel ] .rightcol[ <center> <img src="https://github.com/emse-p4a-gwu/2022-Spring/raw/main/images/quiz_doge.png" width="400"> </center> ] --- class: inverse, middle # Week 10: .fancy[DOE & Power Analysis] ### 1. Design of Experiment ### 2. Design Efficiency ### 3. .orange[Power Analysis] --- class: center, middle # How many respondents do I need? --- class: center, middle # How many respondents do I need<br>_to get X level of precision on `\(\boldsymbol{\beta}\)`_? --- # Standard errors are inversely related to `\(\sqrt{N}\)` .leftcol[ ```r n <- seq(100) se <- 1/sqrt(n) plot(n, se, type = "l") ``` Standard errors also decrease with: - Fewer attributes - Fewer levels in each categorical attribute - More questions per respondent ] .rightcol[ <img src="figs/unnamed-chunk-11-1.png" width="432" /> ] --- class: center, middle ## Using {cbcTools}, we can run simulations to determine the necessary sample size for a specific model # Open `powerAnalysis.qmd` --- class: inverse <div class="countdown" id="timer_c3183e99" data-warn-when="15" data-update-every="1" tabindex="0" style="top:0;right:0;font-size:2em;"> <div class="countdown-controls"><button class="countdown-bump-down">−</button><button class="countdown-bump-up">+</button></div> <code class="countdown-time"><span class="countdown-digits minutes">20</span><span class="countdown-digits colon">:</span><span class="countdown-digits seconds">00</span></code> </div> ## Your Turn .leftcol80[ Individually: 1. Using the survey design you created in the last practice, conduct a power analysis to determine the necessary sample size to achieve a 0.05 significance level on your parameter estimates. 2. Compare your results with your teammates. ]