This tutorial covers how to prepare data for, perform, and report t-tests in R. Before you start this tutorial, you should make sure you review the relevant lecture, as this tutorial assumes you already know what t-tests are and what they’re for.

Writing Lab Reports

If you are writing your lab report on the Red study (Elliot et al., 2010), follow the steps of this tutorial to analyse and report your results.

Setting Up

All you need is this tutorial and RStudio. Remember that you can easily switch between windows with the Alt + ↹ Tab (Windows) and ⌘ Command + ⇥ Tab (Mac OS) shortcuts.

Task 1

Create a new week_07 project and open it in RStudio. Then, create two new folders in the new week_07 folder: r_docs and data. Finally, open a new R Markdown file and save it in the r_docs folder. Since we will be practising reporting (writing up results), we will need R Markdown later on. For the tasks, get into the habit of creating new code chunks as you go.

Remember, you can add new code chunks by:

Using the RStudio toolbar: Click Code > Insert Chunk
Using a keyboard shortcut: the default is Ctrl + Alt + I (Windows) or ⌘ Command + Alt + I (MacOS), but you can change this under Tools > Modify Keyboard Shortcuts…
Typing it out: ```{r}, press Enter, then ``` again.
Copy and pasting a code chunk you already have (but be careful of duplicated chunk names!)

Task 2

Add and run the command to load the tidyverse package in the setup code chunk.

library(tidyverse)

Exploring the Data

Our experiment today will be based on data from a study by Mealor et al. (2016) on synæsthesia.¹

Task 3

Read in the data hosted at the link below and save it in a new object.

Link: https://and.netlify.app/docs/syn_data.csv

syn_data <- readr::read_csv("https://and.netlify.app/docs/syn_data.csv")

Task 4

Have a poke around this dataset to get familiar with it.

Task 4.1

How many participants were there in this experiment?

nrow(syn_data)

[1] 1211

Task 4.2

How many variables are there?

ncol(syn_data)

[1] 12

Task 4.3

What are the variables called?

names(syn_data)

 [1] "GraphCol"         "SeqSpace"         "Imagery"         
 [4] "TechnicalSpatial" "Language"         "Organisation"    
 [7] "Global"           "Systemise"        "gender"          
[10] "GCSscore"         "Syn"              "id_code"

Composite Scores

For today’s analysis, we will investigate differences between synæsthetes and non-synæsthetes in overall imagery ability, across both visual and technical/spatial domains. Before we carry on with our analysis, let’s create a composite score that represents the mean of each participant’s score on the Imagery and TechnicalSpatial variables.

Task 5

Create the OverallImg composite score.

Think about what the mean/average is and how it’s calculated

This one’s only a little bit tricky, but it’s nothing you couldn’t do.

Let’s first see what doesn’t work. Perhaps it would seem intuitive to use the mean() function inside of mutate() but look at what happens when we try that:

syn_data %>% 
  mutate(OverallImg = mean(c(Imagery, TechnicalSpatial)))

As you can see (if you click through the table to see the last column in the data), all values of our new variable are the same. Every single value is just the mean of the two columns and not the mean of the two variables per each participant. A different approach is required…

All you need to do is realise that what we want is to calculate the average of two things: someone’s score on the Imagery variable and their score on the TechnicalSpatial variable. How do we do that? Well, we can simply create a new variable in which we add these two together and divide them by 2.

syn_data <- syn_data %>%
  mutate(OverallImg = (Imagery + TechnicalSpatial) / 2)

If you run this code, you should see a new variable in your dataset with a different number for each person.

Study Design

As we discussed in the lecture, an independent samples t-test compares the means of two groups, essentially testing the null hypothesis that the two sample means are drawn from the same population. In this case, our two sample means are from groups of people with synæsthesia and people without.

Task 6

Stop and think about this design before we go on.

Task 6.1

What is the null hypothesis for this experiment? What pattern would we expect to see in the data if the null hypothesis were true? Write down your thoughts in your Markdown.

Conceptually speaking, the null hypothesis in this case states that there is no relationship between the presence or absence of synæsthesia and a person’s imagery.

A statistical null hypothesis rephrases this in terms of equality of means of the imagery measure across the two groups of people: \[H_0: \mu_{I\_syn} = \mu_{I\_gen}\] (The \(I\_syn\) and \(I\_gen\) subscripts are just made up. There is no “official” notation.)

The important thing to understand that the hypothesis is concerned with population parameters and that even if it’s true, we might still expect some differences in the sample.

If the null is true, however, we would expect the measured differences to be close to zero and equally likely positive and negative.

Task 6.2

What is the alternative hypothesis for this experiment? What pattern would we expect to see in the data if the alternative hypothesis were true? Write down your thoughts in your Markdown.

The alternative hypothesis is simply a negation of the null hypothesis, and so if the null predicts no effect, the alternative hypothesis predicts some effect. In other words, the hypothesis claims that there is indeed a relationship between the presence or absence of synæsthesia and one’s imagery.

A two-tailed statistical alternative hypothesis could be stated as: \[H_0: \mu_{I\_syn} \ne \mu_{I\_gen}\]

As you can see, this hypothesis makes no predictions about the direction of the relationship.

If it is true, we would expect to observe more pronounced differences between the means of the two groups than if the null is true.

Task 6.3

What do you think we will find? Do you think that there will be a difference in overall imagery ability? Write down your thoughts in your Markdown.

You can write whatever you like here, although do write something. It’s very important to think about - and document! - what you are expecting to find before you dive into your data.

Descriptives and Visualisations

Now that we have our predictions nailed down, let’s have a look at the data!

Task 7

Create a summary tibble for the OverallImg variable, split up by Syn and containing:

the number N in each group
the mean and standard deviation of OverallImg
the standard error of OverallImg

Save the resulting tibble to a new object; I’m calling mine img_summary.

We made a table just like this in the Week 2 Practical!

img_summary <- syn_data %>% 
  dplyr::group_by(Syn) %>% 
  dplyr::summarise(
    n = dplyr::n(),
    mean_img = mean(OverallImg),
    sd_img = sd(OverallImg),
    se_img = sd_img/sqrt(n)
  )

If we have a look at the descriptives, we can immediately notice a few things: a difference in N, similar (but not quite the same!) means, and a very similar SD but quite different SE.

Task 8

Using the tibble we’ve just created, let’s build a magnificent means plot to represent the count data we have. We’ll start with a basic plot and build it up from there to create a publication-worthy, lab-report-quality graph. In the end, it will look like this:

Task 8.1

Start by creating a basic means plot, and adding a theme to spruce it up.

Again, we already did means plots in the Week 2 Practical.

For a theme, there are a lot pre-loaded in R. Check out theme_minimal() and theme_bw for clean, simple themes. Naturally, you can also install extra themes. I like theme_cowplot(), but you’ll have to install the cowplot package to use it. Any of these are fine - choose one you like!

For the simple means plot, we just need to tell R what data we want to use and then the kind of figure to draw. In this case, we’re using the data from our summary tibble.

img_summary %>%
  # use variable names from the img_summary tibble
  ggplot(aes(x = Syn, y = mean_img)) +
    geom_point() + # draw the points
  cowplot::theme_cowplot()

Hmm - those two dots floating in space aren’t very informative. It sure would be useful to have some error bars!

Task 8.2

Let’s now add error bars representing ±2 SEM (standard errors of the mean) to our points to make this an errorbar chart.

We also did error bars in the Week 2 Practical! What a strange coincidence 😉

Use the ymin= and ymax= arguments of geom_errorbar() to add ±2×SEM errorbars.

img_summary %>%
  # use variable names from the img_summary tibble
  ggplot(aes(x = Syn, y = mean_img)) +
    geom_point() + # draw the points
  geom_errorbar(aes(ymin = mean_img - 2*se_img,
                    ymax = mean_img + 2*se_img),
                width = .1) +
  cowplot::theme_cowplot()

Adding 95% CIs

Errorbars in published plots often represent the 95% confidence interval around the point-estimate. Naturally, you can to this relatively easily with ggplot2. There are two main approaches to this task: either you prepare a summary tibble with a column representing the CIs to then feed to ggplot(), just like we just did with errorbars, or you use the stat_summary() function instead of our two geom_ functions.

stat_summary allow you to tell ggplot() how to summarise the raw data before plotting and so, instead of a summary table, we will be using the raw dataset:

syn_data %>% 
  ggplot(aes(x = Syn, y = OverallImg)) + # raw data
  # use summary with the mean function to add points
  stat_summary(geom = "point", fun = mean) +
  cowplot::theme_cowplot()

Now that we have our points, we can add another layer of stat_summary(), this time with the "errorbar" geom and a different summary function, mean_cl_boot(), to give us the 95% CIs:

syn_data %>% 
  ggplot(aes(x = Syn, y = OverallImg)) +
  stat_summary(geom = "point", fun = mean) +
  # add 95% CI errorbars
  stat_summary(geom = "errorbar", fun.data = mean_cl_boot,
               width = .1) +
  cowplot::theme_cowplot()

You can style this plot further in exactly the same way as you would any other.

Task 8.3

Style the axes of this plot to make them clear and easy to read.

scale_x_discrete() to change the name and labels on the x-axis
scale_y_continuous()to change the name on the y-axis

There are lots of ways to change the names of scales, including labs(). I like doing them separately with these scale_ functions because you can also use them to change other things, like what numbers appear on the y-axis, and what the labels are for categories. Ultimately you should build your plots in a way that makes sense to you, and that ends up looking the way you want it.

For example, in the solution I’ve also added another argument to scale_y_continuous, namely limits. This changes the minimum and maximum values on the y axis. This is optional, but you can try messing about with these two arguments to see what happens if you want to get a feel for how it works.

Use the help documentation if you’re not sure how to use these functions!

img_summary %>% 
  # use variable names from the img_summary tibble
  ggplot(aes(x = Syn, y = mean_img)) +
  geom_point() + # draw the points
  geom_errorbar(aes(ymin = mean_img - 2*se_img,
                    ymax = mean_img + 2*se_img),
                width = .1) +
  scale_x_discrete(name = "Presence of synaesthesia") +
  scale_y_continuous(name = "Mean Overall Imagery Score",
                     limits = c(0,5)) +
  cowplot::theme_cowplot()

Task 8.4

Optionally, you can style the points as well. Let’s make them larger, turn them into diamond shapes, and give them a more interesting fill colour.

To add colours, you can use either named colours that R knows, or hexidecimal colour codes.

R recognises a lot of colour names. Just put the name you want in “quotes” to use it.

If you want a very precise shade that isn’t on the list of coloured names, you can also use any colour that your heart desires using the colour’s hexidecimal code. The hex code is a series of 6 letters and numbers that uniquely specify any colour in the RGB colour gamut. You can try typing random ones, or use this handy colour selector to get the exact shade you want.

If you wanted the Analysing Data theme colours, you could use darkcyan or #009Fa7 for teal, and #52006F or purple4 for the purple.

img_summary %>%
  # use variable names from the img_summary tibble
  ggplot(aes(x = Syn, y = mean_img)) +
  geom_errorbar(aes(ymin = mean_img - 2*se_img,
                    ymax = mean_img + 2*se_img),
                width = .1, colour = "#52006F") +
  geom_point(size = 3, pch = 23, fill = "#009Fa7", colour = "#52006F") +
  scale_x_discrete(name = "Presence of synaesthesia") +
  scale_y_continuous(name = "Mean Overall Imagery Score",
                     limits = c(0,5)) +
  cowplot::theme_cowplot()

Now, as stylish and attractive as that Analysing Data colour palette is, it isn’t really suitable for formal reporting. Let’s tone it down a bit.

Task 8.5

Choose a more formal colour for the points if you didn’t already.

img_summary %>% 
  ggplot(aes(x = Syn, y = mean_img)) +
  geom_errorbar(aes(ymin = mean_img - 2*se_img,
                    ymax = mean_img + 2*se_img),
                width = .1) +
  geom_point(size= 3, pch = 23, fill = "grey") +
  scale_x_discrete(name = "Presence of synaesthesia") +
  scale_y_continuous(name = "Mean Overall Imagery Score",
                     limits = c(0,5)) +
  cowplot::theme_cowplot()

Now that’s looking good!

Choosing Axis Limits

Deciding how to present your data is always a compromise. In the plot above, the y-axis spans the entire possible range of the OverallImg variable, which is great, but it makes the points appear very near each other and the errorbar span tiny. Restricting the y-axis limits to, for instance, 3-4 truncates the axis, which is not that great but, on the flip side, gives us more of a visual appreciation for the differences between the group means:

img_summary %>% 
  ggplot(aes(x = Syn, y = mean_img)) +
  geom_errorbar(aes(ymin = mean_img - 2*se_img,
                    ymax = mean_img + 2*se_img),
                width = .1) +
  geom_point(size= 3, pch = 23, fill = "grey") +
  scale_x_discrete(name = "Presence of synaesthesia") +
  scale_y_continuous(name = "Mean Overall Imagery Score",
                     limits = c(3,4)) +
  cowplot::theme_cowplot()

Ultimately, it’s up to you which one you pick and both choices are justifiable. In fact, Jennifer prefers the full range axis, while Milan would go for the highlighted differences…

The important thing to understand is that you are the one making a decision and you should have the tools to implement it.

Lab Report Plots

For the Red lab reports, you must include a means plot of your results. You don’t have to only use grey, or any particular theme, but you should choose colour and styling that look like something you might see in a journal article.

Task 9

How can you interpret the means in this plot? Write down your thoughts in your Markdown.

In our sample, synæstheses have a higher mean imagery score than non-synæstheses. However, it is not possible to tell just by looking at the points whether or not this difference is statistically significant!

Task 10

How can you interpret the error bars in this plot? Write down your thoughts in your Markdown.

Our errorbars represent ±2×SEM. Given what we know about the sampling distribution of the mean and the standard error, we can tell that, if we were to collect many samples, a little over 95% of sample mean imagery scores in these groups would fall within these errorbars.

From this, it follows that, if the respective errorbars of two groups do not overlap, the difference between them is statistically significant.

Now that we’ve had a look at our data, we can move on to actual statistical testing!

The t-test

Running the Analysis

So, we’ve made our predictions, and we’ve had a look at the data. Now we’re ready to conduct our t-test.

Task 11

Conduct and report the independent samples t-test.

Task 11.1

Look up the help documentation for t.test(). How will you write the formula to specify your analysis?

First, pull up the help documentation:

?t.test()

Under “Arguments”, the entry for formula tells us that this should be “a formula of the form lhs ~ rhs where lhs is a numeric variable giving the data values and rhs either 1 for a one-sample or paired test or a factor with two levels giving the corresponding groups.”

So, the first element of the formula should be a numeric variable with the data values. In our case, this will be our OverallImg variable, which contains the numeric imagery scores.

The second element should be our grouping variable. Here, this is the variable that contains a code for whether someone is a synaesthete or not: in our dataset, Syn.

Putting it all together, we can write the formula as: OverallImg ~ Syn.

Task 11.2

Use the t.test() function to perform an independent samples t-test on the syn_data data to find out whether there is a difference in OverallImg between Syn groups. Your output should look like the one below.

Use the outcome ~ predictor formula, with the alternative = "two.sided" and var.equal = TRUE options.

syn_data %>% 
  t.test(OverallImg ~ Syn,
         data = .,
         alternative = "two.sided",
         var.equal = TRUE)


    Two Sample t-test

data:  OverallImg by Syn
t = -8.214, df = 1209, p-value = 5.448e-16
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -0.4163794 -0.2558225
sample estimates:
 mean in group No mean in group Yes 
         3.253940          3.590041

Pretty painless, eh?

Reporting the Results

Now that we’ve got our test result, let’s report it in APA style. This takes the general form:

estimate_name(degrees_of_freedom) = estimate_value, p = exact_p, M_diff = difference_in_means, 95% CI = [CI_lower, CI_upper]

Task 12

Use the general format above and the values to type out the result of the t-test.

We start with the general form:

estimate_name(degrees_of_freedom) = estimate_value, p =_p, M_diff = difference_in_means, 95% CI [CI_lower, CI_upper]

Now we need each of these values from the output:

estimate_name: t
degrees_of_freedom: 1209
estimate_value: −8.21
p: < .001
difference_in_means: −0.34
CI_lower: −0.42
CI_upper: −0.26

So, our reporting should look like: t(1209) = −8.21, p < .001, M_diff = −0.34, 95% CI [−0.42, −0.26]

What is M_diff?

Everything we’ve reported above we’ve seen before, except for M_diff. This represents the differences in the means of the two groups. This is a key element of the t-test, but it’s also important because the 95% CIs are an interval estimate around that mean difference. Without reporting M_diff, the CIs are quite hard to interpret!

We should also describe in words what this result means. We essentially include the statistical result as a citation to give evidence for our claim.

Task 12.1

Have a go writing out a report of this statistical analysis.

You should definitely write out your own report in your own words, but here’s an example:

“An independent samples t-test revealed a significant difference in means between the two groups, t(1209) = −8.21, p < .001, M_diff = −0.34, 95% CI [−0.42, −0.26].”

That’s looking pretty slick! This is exactly the sort of thing you would expect to see in a journal article - or a lab report 😉

You should never have any raw (unformatted) code or output in a report! Always report results in the text, in a figure, or in a formatted table.

Task 12.2

Save your test output to a new object, syn_t_test, to refer to it later.

syn_t_test <- syn_data %>% 
  t.test(OverallImg ~ Syn,
         data = .,
         alternative = "two.sided",
         var.equal = TRUE)

Putting It All Together

Task 13

Put all of this together into one report of your findings. Include both descriptives (M and SD in both groups) as well as the results of your statistical test.

You should write this in your own words, but here’s an example:

“Synæsthetes had a higher mean imagery scores (M = 3.59, SD = 0.41) than non-synæsthetes (M = 3.25, SD = 0.43). This difference was statistically significant according to the independent samples t-test, t(1209) = −8.21, p < .001, M_diff = −0.34, 95% CI [−0.42, −0.26]. We thus found support for the alternative hypothesis and reject the null hypothesis of no differences.”

Together with the graph we already made, this would make a great report!

Recap

Well done conducting your t-test.

In sum, we’ve covered:

How to create report-worthy means plots
How to run the independent samples t-test, read the output, and store it in an object
How to report and interpret the statistical results of these tests in real-world terms

Remember, if you get stuck or you have questions, post them on Piazza, or bring them to #StatsChats or to drop-ins.

Good job!

That’s all for today. See you soon!

Jennifer hates spelling the word with the æ ligature but hey, I have the keys to the website -Milan↩︎

t-test

Setting Up

Task 1

Task 2

Exploring the Data

Task 3

Task 4

Task 4.1

Task 4.2

Task 4.3

Composite Scores

Task 5

Study Design

Task 6

Task 6.1

Task 6.2

Task 6.3

Descriptives and Visualisations

Task 7

Task 8

Task 8.1

Task 8.2

Adding 95% CIs

Task 8.3

Task 8.4

Task 8.5

Choosing Axis Limits

Task 9

Task 10

The t-test

Running the Analysis

Task 11

Task 11.1

Task 11.2

Reporting the Results

Task 12

What is Mdiff?

Task 12.1

Task 12.2

Putting It All Together

Task 13

Recap

What is M_diff?