This worksheet builds on three previous worksheets: within-subject differences, understanding interactions, and factorial differences. We therefore assume you are familiar with the concepts and commands in those worksheets. If that is not the case, read (or re-read) those worksheets before proceeding.

In the factorial differences worksheet, we did a two-factor Bayesian ANOVA where one factor was within-subject and the other was between-subjects. In this worksheet, we’ll use the same data frame to illustrate another couple of two-factor ANOVAs.

Return to the project that you used for the factorial differences worksheet, which you named **rminr-data**.

Ensure you have the files required for this worksheet, by asking git to “`pull`

” the repository. Select the `Git`

tab, which is located in the row of tabs which includes the `Environment`

tab. Click the `Pull`

button with a downward pointing arrow. A window will open showing the files which have been pulled from the repository. Close the `Git pull`

window.

Now, open a new blank script and save it as **anova4.R**. Put all commands in that script, and save the script regularly.

First, you’ll need to load some standard packages, load the data we used in the previous worksheet, and set the appropriate columns up as factors.

**Enter these commands into your script, and run them:**

```
rm(list = ls())
library(tidyverse)
library(BayesFactor, quietly = TRUE)
words <- read_csv("wordnaming2.csv")
words$sex <- factor(words$sex)
words$subj <- factor(words$subj)
words$medit <- factor(words$medit)
words$congru <- factor(words$congru)
words$block <- factor(words$block)
```

We can use `anovaBF`

to do a factorial Bayesian ANOVA where both factors are between-subjects. It works exactly the same way as before. For example, we can do a factorial Bayesian ANOVA for `sex`

and `medit`

like this:

First we create a subject-level summary of the data - this involves averaging across the 90 data points that contribute to the reaction time (RT) measure for each participant. We do this using the now-familiar `group_by`

and `summarise`

commands.

**Enter these commands into your script, and run them:**

`wordsum <- words %>% group_by(subj, medit, sex) %>% summarise(rt = mean(rt))`

``summarise()` has grouped output by 'subj', 'medit'. You can override using the `.groups` argument.`

Now we run the Bayesian ANOVA, as follows; **enter these commands into your script, and run them:**

```
bf2bs <- anovaBF(
formula = rt ~ medit*sex,
data = data.frame(wordsum))
```

Note that, unlike the Bayesian ANOVAs we previously did that included a within-subjects factor, we do not include `+ subj`

or `whichRandom = 'subj'`

in this entirely between-subjects ANOVA. This is because we have created a subject-level summary of the data in which each participant only contributes one data point.

The output is interpreted the same way as before; **enter this command into your script, and run it:**

`bf2bs`

```
Bayes factor analysis
--------------
[1] medit : 718.3828 ±0.01%
[2] sex : 1.446498 ±0%
[3] medit + sex : 1177.772 ±3.16%
[4] medit + sex + medit:sex : 82.06562 ±1.94%
Against denominator:
Intercept only
---
Bayes factor type: BFlinearModel, JZS
```

`[1] medit`

This is the **main effect** of training type, `medit`

. More specifically, it is a test of the hypothesis that `medit`

, affects `rt`

.

The Bayes Factor for this hypothesis is around 718, which is strong evidence that there is a main effect of meditation training. If you’ve got a particularly good memory, you’ll have noticed this isn’t quite the same Bayes Factor as we saw for the main effect of meditation in this analysis of the same data. As with many forms of multi-factor ANOVA (including traditional, p-value, ones) the evidence terms you get depend somewhat on the formula you specify. Here, you included `sex`

but not `congru`

as factors; in the previous analysis, it was the other way around. This can affect the results you get.

If the type of analysis you choose to conduct affects the central conclusions of your study, then that is normally an indication that you need to replicate your result with a larger sample. Conclusions should be robust to variations in analysis technique.

`[2] sex`

This is the **main effect** of participant sex. More specifically, it is the hypothesis that `sex`

affects `rt`

. This is compared against the hypothesis that there is no effect of sex.

The Bayes Factor is about 1.4, meaning the evidence for a main effect of sex is inconclusive.

As before, `anovaBF`

does not directly give us a Bayes Factor for the interaction of the two main effects. Instead, it gives us two Bayes Factors for things that we can use to work out the interaction BF. These are:

`medit + sex`

This the hypothesis that there is a main effect of both factors. There is no assumption that the two main effects are of the same size. This ‘main effects’ hypothesis is compared against the null hypothesis, i.e. that neither `medit`

nor `sex`

affect `rt`

.

The BF for this hypothesis is large (about 1200). We’d expect this, given that there was substantial evidence for `medit`

alone.

`medit + sex + medit:sex`

This is the Bayes Factor for the hypothesis that there are main effects for both factors (`medit + sex`

) *and* that the two factors interact (`+ medit:sex`

). This is again compared against the null hypothesis that neither `medit`

nor `sex`

have any effect.

The BF for this hypothesis is also large (about 85). We’d expect this, because there was substantial evidence for the ‘main effects’ hypothesis `medit + congru`

. However, the BF is considerably lower than for the `main effects`

hypothesis.

The Bayes Factor for the interaction is given, as before, by **entering this command into your script, and running it:**

`bf2bs[4] / bf2bs[3]`

```
Bayes factor analysis
--------------
[1] medit + sex + medit:sex : 0.06967872 ±3.71%
Against denominator:
rt ~ medit + sex
---
Bayes factor type: BFlinearModel, JZS
```

This gives us a Bayes Factor for the interaction around 0.07. So, there’s substantial evidence for the *absence* of an interaction.

We can also use `anovaBF`

to look at two within-subjects factors. Again, this works just the same way as before. For example, we can do a factorial Bayesian ANOVA for `congru`

and `block`

like this:

First, we create the appropriate subject-level summary for our analysis.

**Enter these commands into your script, and run them:**

```
wordsum2 <- words %>%
group_by(subj, congru,block) %>%
summarise(rt = mean(rt))
```

``summarise()` has grouped output by 'subj', 'congru'. You can override using the `.groups` argument.`

Next, for the purposes of this worksheet, we’re also going to reduce the sample size from 420 participants to 60 participants. This is something you’d never do in a real analysis, but here we need to address the issue that within-subject Bayesian ANOVAs can take a very long time to run when we have, as in this case, several hundred participants (in this case, it would take around 15 minutes). The correct approach when analysing your own data is, of course, to just wait for the analysis to finish.

**Enter this command into your script, and run it:**

`wordsum2 <- wordsum2 %>% filter(as.numeric(subj) < 61)`

Now we run the Bayesian analysis as before.

**Enter these commands into your script, and run them:**

```
bf2ws <- anovaBF(
formula = rt ~ congru*block + subj,
data = data.frame(wordsum2),
whichRandom = 'subj')
```

The output is interpreted the same way as before; **enter this command into your script, and run it:**

`bf2ws`

```
Bayes factor analysis
--------------
[1] congru + subj : 3844.792 ±1.1%
[2] block + subj : 0.03422314 ±1.46%
[3] congru + block + subj : 138.0297 ±1.65%
[4] congru + block + congru:block + subj : 1.559342 ±2.25%
Against denominator:
rt ~ subj
---
Bayes factor type: BFlinearModel, JZS
```

`[1] congru`

The is the **main effect** of congruence. More specifically, it is the hypothesis that `congru`

affects `rt`

. This is compared against the hypothesis that there is no effect of congruence.

The Bayes Factor is very large (about 3,800), meaning the evidence for a main effect of congruence is conclusive.

`[2] block`

This is the **main effect** of test `block`

. More specifically, it is a test of the hypothesis that `block`

, affects `rt`

.

The Bayes Factor for this hypothesis is around 0.04, which is strong evidence for the *absence* of an effect of block.

As before, `anovaBF`

does not directly give us a Bayes Factor for the interaction of the two main effects. Instead, it gives us two Bayes Factors for things that we can use to work out the interaction BF. These are:

`congru + block`

This the hypothesis that there is a main effect of both factors. There is no assumption that the two main effects are of the same size. This ‘main effects’ hypothesis is compared against the null hypothesis, i.e. that neither `congru`

nor `block`

affect `rt`

.

The BF for this hypothesis is large - about 135. We’d expect this, given that there was substantial evidence for `congru`

alone.

`congru + block + congru:block`

This is the Bayes Factor for the hypothesis that there are main effects for both factors (`congru + block`

) *and* that the two factors interact (`+ congru:block`

). This is again compared against the null hypothesis that neither `congru`

nor `block`

have any effect.

The BF for this hypothesis is considerably lower than for the `main effects`

hypothesis.

The Bayes Factor for the interaction is given, as before. **Enter this command into your script, and run it:**

`bf2ws[4] / bf2ws[3]`

```
Bayes factor analysis
--------------
[1] congru + block + congru:block + subj : 0.01129715 ±2.8%
Against denominator:
rt ~ congru + block + subj
---
Bayes factor type: BFlinearModel, JZS
```

This gives us a Bayes Factor for the interaction is around 0.01. So, there’s substantial evidence for the *absence* of an interaction.

It is also possible to use `anovaBF`

to analyse a single factor, between-subjects design. Where that factor has two levels, you could alternatively use ttestBF. One of the useful things about `anovaBF`

is that, unlike `ttestBF`

, you can use it in a one-factor design that has more than two levels. For example: (**enter these commands into your script, and run them:**)

```
wordsum3 <- words %>%
group_by(subj, medit) %>%
summarise(rt = mean(rt))
```

``summarise()` has grouped output by 'subj'. You can override using the `.groups` argument.`

```
bf1bs <- anovaBF(
formula = rt ~ medit,
data = data.frame(wordsum3))
bf1bs
```

```
Bayes factor analysis
--------------
[1] medit : 718.3828 ±0.01%
Against denominator:
Intercept only
---
Bayes factor type: BFlinearModel, JZS
```

Although in principle it is possible to run Bayesian ANOVAs with more than two factors, they can take hours or days to run with realistically large data sets. They also produce output that is considerably harder to interpret than the output of two-factor analyses. For these reasons, we only look at up to two-factor Bayesian ANOVA in these worksheets. If you need to conduct an ANOVA with more than two factors, consider using traditional ANOVA techniques (bearing in mind, of course, the interpretative difficulties surrounding p values - and that *any* kind of analysis with more than two factors is often going to be pretty difficult to interpret).

The data for this first exercise comes from a between-subjects experiment with two factors. Start by loading the data and removing any missing data (`NA`

), using thd following command.

**Enter these commands into your script, and run them:**

`cf_raw <- read_csv('going-further/counterfactual.csv') %>% drop_na()`

The first few rows of data look like this:

subj | perspective | predictability | blame |
---|---|---|---|

136 | first | Yes | 6 |

137 | first | No | 6 |

138 | first | Yes | 4 |

139 | first | Yes | 6 |

140 | first | Yes | 4 |

141 | first | Yes | 4 |

In this experiment, participants read a scenario describing some events leading up to an accident. The scenario was written from either a first-person or a third-person perspective. Next, they generated some counterfactual thoughts in relation to the scenario. For example, “If only I hadn’t been looking at my phone, then …”. These thoughts were coded as either predictive, or not predictive of whether the accident occurred. Participants rated how much the protagonist was to blame for the accident, on a scale of 1 (strongly disagree) to 7 (strongly agree).

It’s crucial to look at your data before analysing it. Start by calculating the mean `blame`

score in each of the four conditions of this experiment. You can do this using the `group_by`

and `summarise`

commands that we have covered in many previous worksheets; for example, in the group differences worksheet.

If you’ve done it correctly, your means should look like this:

```
# A tibble: 4 x 3
# Groups: perspective [2]
perspective predictability blame
<fct> <fct> <dbl>
1 first No 4.89
2 first Yes 4.70
3 third No 4.06
4 third Yes 4.38
```

Next, plot those means in a line graph - this helps you visualize the patterns in these data. Plotting of line graphs was covered in the understanding interactions worksheet. Your graph should look like this:

Now, do your Bayesian ANOVA. Set each of the independent variables as a `factor`

and then use `anovaBF`

to conduct the Bayesian ANOVA. There is only one observation for each participant in this dataset, so don’t include `subj`

or `whichRandom=subj`

.

Your results should look similar to this:

```
Bayes factor analysis
--------------
[1] perspective : 3.252175 ±0%
[2] predictability : 0.2035932 ±0.01%
[3] perspective + predictability : 0.6595293 ±0.98%
[4] perspective + predictability + perspective:predictability : 0.3121951 ±5.81%
Against denominator:
Intercept only
---
Bayes factor type: BFlinearModel, JZS
```

```
Bayes factor analysis
--------------
[1] perspective + predictability + perspective:predictability : 0.4733605 ±5.89%
Against denominator:
blame ~ perspective + predictability
---
Bayes factor type: BFlinearModel, JZS
```

**Write a short paragraph explaining what, on the basis of your analysis, you can conclude from these data. Copy this paragraph, and the code you used to do your analysis, into PsycEL.**

The data for this second exercise come from a within-subjects experiment with two factors. Start by loading the data, selecting the columns and rows we’ll need for this analysis, and removing missing data (`NA`

).

**Enter these commands into your script, and run them:**

```
raw <- read_csv('case-studies/chris-mitchell/priming-faces.csv') %>%
filter(Running == "Test") %>%
select(Subject, Congruency, Load, RT) %>%
drop_na()
```

The first few rows of data look like this:

Subject | Congruency | Load | RT |
---|---|---|---|

1 | Congruent | NoLoad | 684 |

1 | Congruent | Load | 371 |

1 | Incongruent | NoLoad | 907 |

1 | Incongruent | Load | 659 |

1 | Congruent | NoLoad | 453 |

1 | Congruent | Load | 969 |

Participants were trained to associate two screen colours with pictures of two different food rewards. At test, they saw pairs of screen colours and food pictures. Their reaction time (RT) in milliseconds was measured in cases where the pairs matched the associations they’d previously learned (congruent trials), and in cases where the pairs did not match what they had previously learned (incongruent trials). The participants experienced this test both with, and without, a secondary task. In the secondary task, the participant had to verbally rate how much they liked pictures of faces. As all participants completed all conditions, we have a fully within-subjects design, with factors congruency (congruent, incongruent) and load (load, no load).

Start by calculating the mean RT for each of the four conditions, and plotting these in a line graph. You can use the commands from Exercise 1, slightly modified, to do this. If you get it right, your output should look like this:

```
# A tibble: 4 x 3
# Groups: Congruency [2]
Congruency Load RT
<chr> <chr> <dbl>
1 Congruent Load 672.
2 Congruent NoLoad 575.
3 Incongruent Load 665.
4 Incongruent NoLoad 616.
```

Next, calculate Bayes Factors for the main effects and the interaction. In order to do this, you will first have to create a subject-level summary of your data - you can then pass this your your Bayesian ANOVA.

Your results should look similar to this:

```
Bayes factor analysis
--------------
[1] Congruency + Subject : 0.3064788 ±1.54%
[2] Load + Subject : 19255.45 ±0.98%
[3] Congruency + Load + Subject : 6385.015 ±1.4%
[4] Congruency + Load + Congruency:Load + Subject : 2148.05 ±2.65%
Against denominator:
RT ~ Subject
---
Bayes factor type: BFlinearModel, JZS
```

```
Bayes factor analysis
--------------
[1] Congruency + Load + Congruency:Load + Subject : 0.3364206 ±3%
Against denominator:
RT ~ Congruency + Load + Subject
---
Bayes factor type: BFlinearModel, JZS
```

**Write a short paragraph explaining what, on the basis of your analysis, you can conclude from these data. Copy this paragraph, and the code you used to do your analysis, into PsycEL.**

This material is distributed under a Creative Commons licence. CC-BY-SA 4.0.