DS Skills Lab 04

.title[
# DS Skills Lab 04
]
.subtitle[
## <i>Show me the receipts!</i>
]
.author[
### Dr Danielle Evans
]
.date[
### 27 Oct 2023
]

---

a.glossary {
  font-weight: bold;
  color: #a497f0;
  cursor: help;
  position: relative;
}

.remark-inline-code {
  font-size: var(--code-inline-font-size);
  color: #428aff;
  padding: 2px;
}

strong {
  font-weight: bold;
  color: black;
}

.remark-slide-number {
  color: black;
  opacity: 1;
  font-size: 0.9rem;
}

.hljs-github .hljs-string, .hljs-github .hljs-doctag {
    color: #333;
}

.hljs-github .hljs-literal {
    color: #333;
}

.hiddenFrame{
    height:1px;
    width:1px;
    opacity: 0;
    }
</style>

.tooltip {
  position: relative;
  display: inline-block;
}

.tooltip .tooltiptext {
  visibility: hidden;
  width: 130px;
  background-color: #c8c0f6;
  color: #fff;
  text-align: center;
  border-radius: 6px;
  padding: 5px 0;
  position: absolute;
  z-index: 1;
  bottom: 35%;
  left: 120%;
  margin-left: -60px;
  opacity: 0;
  transition: opacity 0.3s;
}

.tooltip .tooltiptext::after {
  content: "";
  position: absolute;
  top: 50%;
  right: 100%; /* To the left of the tooltip */
  margin-top: -5px;
  border-width: 5px;
  border-style: solid;
  border-color: transparent #c8c0f6 transparent transparent;
}

.tooltip:hover .tooltiptext {
  visibility: visible;
  opacity: 1;
}

</style>

## Overview

- **Making Claims & Providing Evidence**

- **Reporting Statistical Values**

- **Reporting Model Fit**

- **Justified Decisions**

- **Reporting Assumption Checks**

- **Reporting Model Parameters**

- **KahootR**

- **Post questions/comments/errors/cool things on the [Padlet](https://uofsussex.padlet.org/de84/ds-skills-lab-week-04-35wc1d3y9ical2yh), password: ds**

---

## Background

- There are multiple <a class='glossary' title='refers to the flexibility often inherent in the scientific process, from hypothesis generation, designing and conducting a research study to processing the data and analyzing as well as interpreting and reporting results. Due to a lack of precisely defined theories and/or empirical evidence, multiple decisions are often equally justifiable'>researcher degrees of freedom</a> when it comes to conducting research and statistical analyses

- Essentially, this means that the **choice** of the analysis, interpretation, and final model is up to you as the analyst!

- So, you must **decide** which is the final/best model for your data for each analysis and **argue**/**defend** your case in your results section

---

## Making Claims

- What do each of these statements need?

+ "People who have synaesthesia are more creative than people who don't have synaesthesia."

+ "There are only two types of people in this world: those who can extrapolate from incomplete data."

+ "The second, more complex model was a better model than the first model with only one predictor."

---

## Making Claims

- What do each of these statements need? <u style="color:#ed2791;"><b style="color:#ed2791;">EVIDENCE!</b></u>

+ "People who have synaesthesia are more creative than people who don't have synaesthesia."

+ "There are only two types of people in this world: those who can extrapolate from incomplete data."

+ "The second, more complex model was a better model than the first model with only one predictor."

---

## But How?

---

## Forms of Evidence

- Use your in-text statistical results as if they were **citations**!

- The **statistics** you report in each sentence should provide **evidence** for the **claim** you make in that sentence, and the sentence should be a complete thought without the statistics:

- <b style = "color: red";>NO</b>: "The means were (mean = 135.30…"

- <b style = "color: green";>YES</b>: "The means indicated that the synaesthete group (<i>M</i> = 135.30, <i>SD</i> = 12.06) had a higher score for creativity than the non-synaesthete group (<i>M</i> = 105.78, <i>SD</i> = 32.90)."

---

## APA Reporting

- When reporting numbers to provide evidence for a claim you're making, keep in mind:

- Test statistics like _t_, _F_, and _p_ are italicised

- Always round to 2 decimal places, except for _p_ which should be rounded to three

- Look up the correct reporting format if you're not sure!

---

## Reporting Results

- When reporting our overall model, we need to build a **narrative** clearly describing each step (in full sentences), with **evidence** to back up any claims, and **justified decisions** throughout:

- Specifically, we need to include:

+ What models we've fit to our data (& how well they fit)
  
  + What model was better
  
  + The assumptions you checked, and the outcome of those checks
  
  + The main results from the better model

---

## What Models*

- First we need to describe what models we've fit to our data:

- <b style = "color: red";>NO</b>: "The model being constructed is a dual-predictor multiple regression with OLS estimation. Predictors will be considered to be significant if the probability p of finding the b values by chance is less than 0.05. The two first predictor was beauty and had p = 3.13e-05. The second p-value was 0.00165 for nativeno."

- <b style = "color: green";>YES</b>: "The first model investigated instructor beauty ratings as a predictor of teaching evaluations, and showed satisfactory model fit; (<i>R</i><sup>2</sup> = .04, _F_(1, 461) = 17.08, _p_ < .001). The second model investigated instructor beauty ratings and native speakers of English as predictors of teaching evaluations (<i>R</i><sup>2</sup> = .06, _F_(2, 460) = 13.72, _p_ < .001), showing significantly better model fit compared to the first model (<i>R</i><sup>2</sup><sub>change</sub> = .02, _F_(1, 460) = 10.02, _p_ = .002)."

<br>

<h4 style = "color: #ed2791"> Don't directly copy this in your assessments otherwise you'll be flagged for academic misconduct!! Use it to guide you.. </h4>

]

---

## Making Justified Decisions*

- After fitting our models, we want to check our assumptions to see if our model was biased, and decide on which final model to report

- But there are fewer standardised formats for assumptions checks reporting

- You should aim to explain the process of your decision-making clearly, step by step

- So let's look at the assumptions first, and then think about what we did to check our model, and then let's add in the evidence...

---

## First, What Steps Did We Take?

1. We checked some plots of linearity, heteroscedasticity, normality of residuals, & influential cases

2. We checked outliers using standardised residuals

3. We fit robust models as a sensitivity check to examine the pattern of results compared to our original model

4. We decided which model to report

---

## Now Show The Receipts!

---

## Providing Evidence

- "Non-linearity and heteroscedasticity were checked using a scatterplot of the predicted values against the residuals. The plot showed no issues with non-linearity or heteroscedasticity in the data."

- "A Q-Q plot of the standardized residuals indicated the residuals were normally distributed."

- "Influential cases were checked using Cook’s distance with all values below 0.05 suggesting there were no influential cases in our data. The standardised residuals were inspected for outliers; all cases fell within the expected ranges."

- "Robust models were fit to the data as a sensitivity check, and showed the same pattern of results as the original model."

- "Therefore, the final model reported is the unadjusted model predicting teaching evaluation scores from instructor beauty ratings and whether the instructor is a native english speaker."

<h4 style = "color: #ed2791"> Again, don't directly copy this in your assessments otherwise you'll be flagged for academic misconduct!! Use it to guide you.. </h4>

]

---

## That's BetteR!

---

## Reporting our Model*

- We should report the effect of **each** predictor in full, with **statistics** and a **plain language summary**, and should compare the **standardised betas** to evidence which predictor had a stronger relationship with our outcome, e.g.,

+ "In the final model, instructors' beauty ratings significantly predicted their teaching evaluation scores (_b_ = 0.13, SE(_b_) = 0.03, _t_ = 4.21, _p_ < .001, 95% CI [0.07, 0.20])"

--
  
  + "The findings suggest that as instructors’ beauty scores increase by one point on a scale from one to five, their teaching evaluations increase by 0.13 points (scale: 1-10)."

--
  
  + "The standardised estimates for instructor beauty and native English speaker suggest that whether the instructor is a native English speaker is a stronger predictor (<i>B</i> = -0.60, SE(_B_) = 0.19, 95% CI [-0.97, -0.23]) of teaching evaluations compared to instructor beauty (<i>B</i> = 0.19, SE(_B_) = 0.05, 95% CI [ 0.10,  0.28])"

- We can also include an APA style table of our results...

---

<div class="tooltip"><a href="https://www.youtube.com/watch?v=HlBYdiXdUa8" style="color:#ed2791;"><h2>Common Mistakes 🙅‍♀️</a></h2><span class="tooltiptext">Click me! 🐣</span></div>

- Don't define any statistical concepts you would find in a statistics/psychology textbook

- <b style = "color: red";>NO</b>: "The p-value is the probability of finding this result…"

- You should definitely explain the decisions & results specific to your study

- <b style = "color: green";>YES</b>: "Using an alpha level of .05, there was a significant relationship between…"

- You can assume that your audience are interested in & somewhat familiar with the field of research & know analysis techniques/stats terms

- But they do not know any of the details of the study you have conducted and analysed & have no idea what your data look like:

- **Don't EVER refer to measures as the variable name as it appears in R!!!!**

- If you're not sure whether you've given enough evidence or clearly justified a decision, for each claim you make just ask yourself...

---

## The GuRu

---

## Final RemindRs!

-   Submit the easteR eggs you've found today on [Canvas](https://canvas.sussex.ac.uk/courses/26318/quizzes/41382)

-   Nominate your peers for the [savioR award](https://canvas.sussex.ac.uk/courses/26318/quizzes/39822)

-   Have a go at the [Cloud ChallengRs](https://posit.cloud/spaces/392709/content/lists/6597?sort=created_time_desc)

-   Design our new Sussex Psychology [Hex Sticker](https://canvas.sussex.ac.uk/courses/26318/quizzes/41392)

-   Give session feedback [here](https://forms.gle/ZyXAB7kZzUUyct9n6)

-   Add songs to our [Spotify Kahoot Karaoke playlist](https://open.spotify.com/playlist/4N8OglRk1zADpe3Wmi3W8Z?si=cf5d338446e74871&pt=eab8825a5ef6c5d88c41430bec43ab4c)! 💃

---

## KahootR!

### Kahoot pin: .can-edit.black[  ]

]