If you are here, you have likely used Julius to conduct descriptive statistics and explore relationships within your datasets. You might have already tested for normality and homogeneity of variances. If you have not, please refer to Guide: Descriptive Statistics & How to Prompt for and run them on Julius and Guide: Parametric or Non-parametric .
This guide focuses on t-tests (parametric tests) that can be conducted on Julius. For simplicity, we are assuming that we have already confirmed the dataset meets normality and homogeneity of variances assumptions. Let’s delve into these tests
Students T-Test (Independent T-Test)
The independent t-test is used to determine if there are statistically significant differences between the means of two independent groups (groups that do not influence one another).
Scenario: Researchers aim to assess whether there is a significant difference in exam scores between two different teaching methods. They randomly assigned 20 students to Group A (hands on approach to teaching), and another 20 to Group B (independent study). Both groups take the same test after the teaching period. Here are the results:
| group_a | group_b |
|---|---|
| 75 | 78 |
| 80 | 82 |
| 85 | 87 |
| 90 | 91 |
| 92 | 93 |
| 78 | 90 |
| 83 | 85 |
| 87 | 89 |
| 79 | 81 |
| 81 | 83 |
| 82 | 84 |
| 84 | 86 |
| 86 | 88 |
| 88 | 90 |
| 91 | 92 |
| 77 | 79 |
| 76 | 77 |
| 89 | 90 |
| 85 | 86 |
| 83 | 84 |
Objective: Determine if there is a statistically significant difference between the mean test scores of Group A and Group B.
Question 1a: Is there a statistically significant difference between the mean test scores of Group A and Group B?
Prompt 1a: Perform an independent samples t-test to compare the means of Group A and Group B, please.
The t-test results show no statistically significant difference between Group A and Group B. However, let’s explore the effect size of this data regardless using Cohen’s d.
Effect size quantifies the magnitude of the difference between two groups, with 0.2 being small effect, 0.5 being medium, and 0.8 being large effect.
Question 1b: What is the effect size for the comparison between Group A and Group B?
Prompt 1b: Calculate Cohen’s d for the comparison between Group A and Group B, please.
Julius has calculated the effect size, indicating that Group A’s mean is slightly lower than Group B’s.
To gain a clearer understanding of the data characteristics, we can visualize the results.
Question 1c: What does the data look like?
Prompt 1c: Provide a visualization of the data so that we can see the slight differences between Groups A and B?
Julius provides a boxplot, which is effectively for showing data distributions. The boxplot reveals the central tendency and variability of the data, with the ‘whiskers’ representing the spread of data. The shaded area is referred to as the “interquartile range” (IQR), with this example showing that ~25% of the data falling below the median (lower quartile), and ~25% of the data falling above the median line (upper quartile).
Paired Samples T-Test
The paired samples t-test is used when there are two sets of measurements from the same individual under different conditions. This test is appropriate for:
- Comparing measurements before and after treatment.
- Assessing effects of two different treatments on the same individuals.
- Any paired comparisons where measurements are not independent.
Situation: A pharmaceutical company is evaluating a new medication’s effectiveness in lowering blood pressure within 8 hours. They recruit 20 people who are diagnosed with hypertension. They measure their blood pressure levels before administering the medication, then wait 8 hours and remeasure their blood pressure. Below is the data they collected:
| student | before_score | after_score |
|---|---|---|
| 1 | 140 | 130 |
| 2 | 150 | 140 |
| 3 | 160 | 150 |
| 4 | 155 | 145 |
| 5 | 165 | 155 |
| 6 | 148 | 138 |
| 7 | 152 | 142 |
| 8 | 158 | 148 |
| 9 | 145 | 135 |
| 10 | 147 | 137 |
| 11 | 155 | 145 |
| 12 | 160 | 150 |
| 13 | 157 | 147 |
| 14 | 150 | 153 |
| 15 | 143 | 140 |
| 16 | 155 | 133 |
| 17 | 158 | 145 |
| 18 | 162 | 148 |
| 19 | 156 | 152 |
| 20 | 148 | 146 |
Objective: Determine if there is a significant difference between the blood pressure readings before and after.
Question 2a: Is there a significant difference between the before and after blood pressure scores?
Prompt 2a: Run a paired samples t-test comparing the before_score and after_score to see if there is any difference between the groups?
Julius performed the paired t-test and reports the results (t(38) = 8.267, p ≤ 0.001). We will now visualize the data to better understand the impact.
Now I’ll ask for a summary of the paired samples t-test result to get an overview of the findings:
Prompt 2b: Provide a summary of the paired samples t-test results please?
Julius provides a detailed summary of the test results, including the breakdown of the statistical analysis. To quantify the magnitude of the difference, we can calculate Cohen’s d.
Cohen’s d = -1.849, indicating a large negative effect size, suggesting that the medication substantially reduced blood pressure. The negative value indicates the direction of the effect, meaning that there is a decrease in scores from the before_score to after_score.
Finally, let’s look at the variability of the responses amongst patients by using a change score analysis. This analysis examines the distribution of the change scores (difference between the before and after scores) to better understand how each participant varied in their response to the drug. It can help us determine if the drug was consistent in the effect it had amongst all participants.
Question 2c: What is the variability of responses between participants?
Prompt 2c: Perform a change score analysis to further understand the variability of the response to the drug.
Now we understand a lot more about the the average change score and some of the variability each participant experienced! Feel free to play around with the dataset to explore other options Julius has.
Keywords: AI statistics, AI statistical analysis, GPT, parametric tests, students t-test, independent t-test, paired samples t-test, descriptive statistics, cohen’s d, change score analysis