Analysis Plans:

class: center, middle, inverse, title-slide

.title[
# Analysis Plans:
]
.subtitle[
## <i>NeveR gonna let you down</i>
]
.author[
### Dr Danielle Evans
]
.date[
### 27 Oct 2023
]

---

.remark-inline-code {
  font-size: var(--code-inline-font-size);
  color: #428aff;
  padding: 2px;
}

strong {
  font-weight: bold;
  color: black;
}

.remark-slide-number {
  color: black;
  opacity: 1;
  font-size: 0.9rem;
}

.hljs-github .hljs-string, .hljs-github .hljs-doctag {
    color: #333;
}

.hljs-github .hljs-literal {
    color: #333;
}

.hiddenFrame{
    height:1px;
    width:1px;
    opacity: 0;
    }
</style>

## Overview

- **Why plan my analyses?**

+ As best practice in research and data analysis
  + To dodge bad decisions & help your future self 🥰
  + To avoid <a class='glossary' title='Dissertation disasteRs that could have been avoided with a little bit of planning!'>HorroR stoRies!</a>

- **How to plan my analyses?**

+ Defining your research question & formulating hypotheses
  + Operationalising variables
  + Mapping research questions to data & identifying appropriate analyses

- **Workshop activity**

- **Post questions/comments on the [Padlet](https://uofsussex.padlet.org/de84/dissertation-workshop-analysis-plans-lgocnxyr5i8t2hd6), password: diss**

<div class="gloss" style="font-size:90%">
<p><b>Top Tip!</b> Hover over <b style='color:#a497f0'>keywords</b> for definitions! Go on, try it now, I dare you...</p>
</div>

---

## So what is an ✨Analysis Plan✨?

- An outline

- A recipe

- A plan of action

- A detailed and structured document that outlines the methods, procedures, and steps you will follow to analyse data to answer your research question(s)

+ AKA a roadmap for how you intend to approach your data analysis in a systematic and organised manner

- Essentially, your **best friend** when it comes to conducting your analysis!

---

## But, why???

- It's best practice within science and research to always plan your analyses before you start <a class='glossary' title='the systematic process of gathering information, observations or measurements in quantitative or qualitative research'>data collection</a>

- But even if you're using secondary data sources, you should still plan your analyses!

- Planning in advance helps you to avoid making bad design, analytical, or statistical decisions:

+ No <a class='glossary' title='selective reporting of significant p-values'>p-hacking</a>
  
  + & no <a class='glossary' title='‘Hypothesising After the Results are Known’ - presenting a hypothesis that was made after data were collected as though it were made prior to data collection/analysis'>HARKing</a>

- A lot of the (rash) decisions you make will seem sensible at the time and can come back to haunt you later! 👻

- & the beauty of R is that if you plan your analysis in advance, you could even write all the code you need to prep and analyse your data before you even have any!

---

## But, why??? (pt 2)

- Your ethics application will include various questions that require decision-making - completing an analysis plan will often mean you've already made those decisions in advance!

- Helps you avoid <a class='glossary' title='delaying or putting off conducting your analyses until the last minute - aka, a bad idea!'>analysis procrastination</a>

- When you then go to analyse your data in **R**, it'll be 100x easier if you've made a plan that you can follow like a recipe!

- Your analysis plan is never gonna give you up, its never gonna let you down, its never gonna run around and desert you:

<br>

> "The analysis plan was really helpful as it made me realise I didn’t have clear research questions and hypotheses (lol) this made me arrange a meeting with my supervisor and we could go through the sheet together which was really helpful. Even more detail would have been great. I referred to it a lot during analysis"
>
><i>.right[-- Former Student]<i>

---

## HorroR StoRies!

- Careful planning <b><u>now</u></b> can help you avoid any disasters later on!

- Just trust me on this one OK...

- Planning now will help you catch issues with your design that would otherwise lead to a new ethics application, or losing very valuable data

- Things like <a class='glossary' title='a technique used to deal with order effects when using repeated measures designs'>counterbalancing</a>, <a class='glossary' title='the process of randomly assigning participants to different conditions'>randomisation</a>, how to <a class='glossary' title='the process by which a researcher defines how a concept is measured, observed, or manipulated within a particular study'>operationalise</a> your variables, deciding *what* to measure, setting up the  conditions of repeated/mixed designs correctly
  
- HorroR stoRies are very rare, but the ones I've seen are always preventable design issues and not **R**

- And if that's not enough incentive, an analysis plan must be completed to access the **R Help Desk** next term!

---

## Fine, I'm convinced, now what?

- Download & open the [analysis plan document](https://canvas.sussex.ac.uk/courses/27521/files/4307431/download?download_frd=1)!

- Now, the first thing we need to do is identify our <a class='glossary' title='question(s) that a study or research project aims to answer'>research question</a>

--
<br>

**With the people around you, take turns to**:

- Briefly summarise the background motivation of your study

- Clearly state the research question your study will answer

- Briefly discuss and ask/answer questions about the study

- Write down your research question in 1-2 crystal clear sentences in the first box in the doc!

---

## Hypotheses

- Once we have defined our research question(s), we can start to form hypotheses!

- Remember, we want to avoid <a class='glossary' title='‘Hypothesising After the Results are Known’ - presenting a hypothesis that was made after data were collected as though it were made prior to data collection/analysis'>HARKing</a> - write your plan and stick to it!

**With the people around you, take turns to**:

- Clearly state the hypotheses your study will test

- Discuss what your results will look like for your hypotheses to be supported

- Write down specific, testable hypotheses in your analysis plan!

---

## Design

- Is your design <a class='glossary' title='a form of research in which you naturally observe the relationships between variables. Correlational studies are non-experimental, which means that the experimenter does not manipulate or interfere with any of the variables'>correlational</a> or is it <a class='glossary' title='a form of research in which at least one variable is systematically manipulated to see the effect on an outcome variable'>experimental</a>?

- How will you collect your data? (i.e., surveys, secondary data sources, lab experiments)
  
- Is it an <a class='glossary' title='different entities/participants in different conditions'>independent</a>, <a class='glossary' title='the same entities/participants in all conditions'>repeated measures</a>, or <a class='glossary' title='a design with 2+ predictors where at least one predictor has different participants in different conditions, and at least one predictor has the same participants across all conditions'>mixed design</a>?

- How many groups or conditions will you have?
  
  - How will participants be assigned to conditions?

- Will you control for any variables? Are there any potential <a class='glossary' title='a variable other than the predictor variable(s) that potentially affects the outcome variable'>confounds</a>?

**Complete the design & confound Qs on the plan!**

---

## Measurement

- What concepts are you actually measuring? & How? Are they categorical or continuous?

- RTs, test scores, questionnaire responses, scales, conditions/groups/levels etc
  
  + How are the conditions/groups/levels defined?
  
  + For continuous variables, what does a high and low score represent? What's the possible range? What units are they measured in?

**With the people around you, take turns to**:

- State what variables you'll be measuring as your **primary** interest, discuss exactly how you'll measure each one

- Write it on your plan - the more detail the better!!

???

tip: categorical data can be tricky to work with sometimes, if you can measure it continuously, do so! then you can transform it to be categorical later if you need to, but you cant do it the other way round!

---

## Data Manipulation

- Will you need to wrangle or transform your data in any way?

+ Do you need to categorise people into groups depending on their responses/scores?

+ Do you need to create composite scores for your measures? Will these be means or totals?
  
  + Have you looked at the scoring guidelines for any pre-existing measures you're using to find out?

**With the people around you, take turns to**: 
  
- Discuss it & write it down!

---

## Participants

- Who will they be? Will you have any specific inclusion/exclusion criteria?

- Will you adjust for any participant characteristics? How will you identify real vs inattentive responses, or mistakes?

- What information will you need to collect to produce informative participant summaries in your method section?

**With the people around you, take turns to**:

- State your participants and inclusion/exclusion criteria

- Discuss what an informative participant summary would look like for your study

- Write it on your plan!

---

## Thinking ahead to analyses

- It's important to think about how your data map onto your research questions, otherwise you'll end up in a muddle!

- So, to answer your research question(s), what do you need to investigate? What relationships or effects are of interest? Will you make any comparisons between groups?

- What analyses are appropriate?

**Take some time to think, & then write it down**:

---

## What's next?

- Fill out the rest of the analysis plan!

- Make as many decisions as possible <b><u>before</u></b> you've seen the data

- Keep thinking about how your data map onto your research questions
  
  - The process of going from random, messy numbers to answering your research question depends on you!

- There are some questions in the plan that you might not immediately know the answer to - and that's OK!!!

- Your **supervisor** can help you to complete it (but I can't.. 😥)

+ A lot of the decisions you make will be specific to *your* project, so discuss your analysis options with your supervisor and get their help!

+ They can help you avoid <a class='glossary' title='Dissertation disasteRs that could have been avoided with a little bit of planning!'>horroR stoRies!</a>
  
  
---

## What's next?

- When it comes to using **R**, we want to pick our battles and make it easier for ourselves

- Putting in the work <b><u>now</u></b> will hugely benefit your future self!

- You could even make your own <a class='glossary' title='A codebook provides information on the structure, contents, and layout of a data file, i.e., what variables exist, the data type, and expected values'>codebook</a> to make it easier when working with your data!

- But as always...

<br>

<h2>Don't panic, you got this!! </h>

<p>

</p>

</center>

---

## *That's all - happy planning!* 😎

<br>

</a>

</center>

.center[

[Give session feedback here!](https://forms.gle/ZyXAB7kZzUUyct9n6) 😀

]