Study overview
Summary of conditions, sample, and headline findings. All statistics from official analysis CSV.
Participants
90
High school, ages 14–18
Implicitness groups
3
Low (25) · Moderate (37) · High (28)
Adaptability groups
2
No (52) · Yes (38)
Overall completion
46.7%
42 of 90 players finished
Sig. findings (p<.05)
2
Engagement & scaffolding (adaptability)
Marginal (p<.10)
3
Enjoyment adapt, completion impl, enjoyment interaction
Outliers removed
7–6
Deaths/time (N→83); Stars (N→84); BPS (N→87)
Interaction sig.
0
Enjoyment marginal p=0.063 †
Completion rate by implicitness
Low implicitness players completed at twice the rate of high implicitness players
χ²=5.796, p=0.055 †
Survey composites by adaptability
No-adaptability players scored higher on every composite measure
Engagement p=0.039 *Scaffolding p=0.029 *Enjoyment p=0.077 †
* p < 0.05 † p < 0.10 ns = not significant · Stars = sum of Level 4–7 boolean columns (max=4, N=84, 6 outliers removed) · Welch t-tests used for adaptability comparisons
Survey composites
Three composite measures (Likert 1–5). No outliers detected in any survey variable (N=90 for all survey outcomes).
By implicitness level — one-way ANOVA
Low (n=25)
Moderate (n=37)
High (n=28)
Enjoyment composite
Enjoyed · Want to continue · Play again
F(2,87)=1.663, p=0.196 ns
Engagement composite
Felt engaged · Paid attention
F(2,87)=2.055, p=0.134 ns
Scaffolding composite
Easy to figure out · Challenge · Levels helped/prepared · Confident
F(2,87)=1.562, p=0.216 ns
By adaptability condition — Welch t-test
No adaptability (n=52)
With adaptability (n=38)
Enjoyment composite
No (3.54) · Yes (3.10)
t(77.7)=1.792, p=0.077 †
Engagement composite
No (3.87) · Yes (3.41)
t(72.6)=2.107, p=0.039 *
Scaffolding composite
No (3.85) · Yes (3.38)
t(75.3)=2.222, p=0.029 *
All Likert items rated 1–5. Composites are row-wise means. Welch t-tests do not assume equal variances.
All 10 survey items
Individual item means (Likert 1–5). Items 1–5 contribute to Enjoyment/Engagement composites; items 6–10 to the Scaffolding composite.
All items by implicitness
Low consistently scores highest; high consistently lowest on nearly every item
LowModerateHigh
All items by adaptability
No-adaptability scores higher on every single item without exception
No adaptabilityWith adaptability
Performance metrics
Stars (corrected, N=84, 6 outliers removed), deaths, total time, and BPS — before and after outlier removal where applicable.
Stars earned — sum of L4–L7 boolean columns (max=4 · N=84, 6 outliers removed)
LowModerateHigh
No adapt.With adapt.
Stars by implicitness (N=84)
Low (0.88) · Moderate (1.30) · High (1.14)
F(2,81)=1.251, p=0.292 ns
Stars by adaptability (N=84)
No (1.27) · Yes (0.95)
t(81.4)=0.562, p=0.576 ns
Total deaths — before (N=90) vs. after outlier removal (N=83, 7 removed)
Deaths by implicitness — before / after
Moderate's apparent high-death pattern collapses entirely after outlier removal
Before: p=0.119 nsAfter: F(2,80)=0.498, p=0.610 ns
Deaths by adaptability — before / after
No (11.3→6.9) · Yes (10.5→8.6) — direction reverses after cleaning
Before: p=0.792 nsAfter: t(63.9)=−0.869, p=0.388 ns
Total time (seconds) — before (N=90) vs. after outlier removal (N=83, 7 removed)
Time by implicitness — before / after
Extreme outliers inflated means by 200–300s; gap narrows substantially after cleaning
Before: p=0.512 nsAfter: F(2,80)=0.999, p=0.373 ns
Time by adaptability — before / after
No (418→248s) · Yes (509→224s)
Before: p=0.721 nsAfter: t(79.2)=0.687, p=0.494 ns
Average button presses per second — before (N=90) vs. after (N=87, 3 removed)
BPS by implicitness — before / after
Near-identical across all groups; no meaningful variation
Before: p=0.899 nsAfter: F(2,84)=0.024, p=0.976 ns
BPS by adaptability — before / after
No (1.86→1.81) · Yes (1.81→1.69)
Before: p=0.793 nsAfter: t(72.6)=0.721, p=0.473 ns
Before/after comparisons use darker bars for raw data, lighter bars for cleaned data. Welch t-tests used for all adaptability comparisons.
Per-level analysis
Time and BPS broken down by level. Outliers removed independently per level (IQR × 1.5). L6 and L7 significance tests from official analysis CSV; L1–L5 from preliminary analysis. N shrinks at later levels.
Time per level (seconds · cleaned per level)
Low
Moderate
High
No adapt.
With adapt.
Time per level by implicitness
No significant implicitness effect on time at any level per official analysis
L6: F(2,57)=1.629, p=0.205 ns (N=60)
L7: F(2,37)=0.002, p=0.998 ns (N=40)
Time per level by adaptability
No adaptability effect on time at any level
L6: t(57.8)=1.189, p=0.239 ns
L7: t(37.1)=−0.853, p=0.399 ns
BPS per level (button presses ÷ time · cleaned per level)
BPS per level by implicitness
L1 dramatically lower (~0.2) as introductory level; stabilises from L2 onward
L6: F(2,59)=0.845, p=0.435 ns (N=62)
L7: F(2,38)=0.020, p=0.980 ns (N=41)
BPS per level by adaptability
No adaptability effect on input rate at any level
L6: t(40)=−0.543, p=0.590 ns
L7: t(25)=1.466, p=0.155 ns
Players with data per level
N per level (before per-level outlier removal)
Attrition reflects players who did not reach or complete later levels
N per level: L1=90, L2=83, L3=80, L4=77, L5=72, L6=65, L7=44. Official analysis covers L6 and L7 only. BPS = button presses per second.
Game completion
Whether players completed the game (yes/no) across conditions. Chi-square tests used for categorical outcome.
Overall completion
46.7%
42 of 90
Low implicitness
60.0%
15 of 25 — highest
High implicitness
28.6%
8 of 28 — lowest
High impl. + adapt.
16.7%
2 of 12 — worst subgroup
Completion by implicitness (stacked %)
Low (60%) · Moderate (51%) · High (29%)
χ²(2)=5.796, p=0.055 †
Completion by adaptability (stacked %)
No (52%) · Yes (40%)
χ²(1)=0.913, p=0.339 ns
Completion rate — implicitness × adaptability
Adaptability helps slightly at low implicitness but severely hurts at high implicitness (16.7%)
No adaptability
With adaptability
Two-way interaction: F(2,84)=0.799, p=0.453 ns
Chi-square test for independence. Subgroup Ns: High/No=16, High/Yes=12, Low/No=16, Low/Yes=9, Moderate/No=20, Moderate/Yes=17.
Two-way ANOVA interactions
Interaction terms (Implicitness × Adaptability) for all metrics from official analysis CSV. Only enjoyment showed a marginally significant interaction.
Two-way ANOVA summary — all metrics
Adaptability main effects and interaction F-statistics. Implicitness main effects from one-way ANOVAs (Section 1).
| Metric | N | Adaptability (main) | Interaction (Impl × Adapt) |
|---|---|---|---|
| Enjoyment | 90 | F(1,84)=3.478, p=0.066 † | F(2,84)=2.855, p=0.063 † |
| Engagement | 90 | F(1,84)=4.430, p=0.038 * | F(2,84)=0.815, p=0.446 ns |
| Scaffolding | 90 | F(1,84)=4.646, p=0.034 * | F(2,84)=1.015, p=0.367 ns |
| Stars (corrected) | 84 | F(1,78)=0.575, p=0.451 ns | F(2,78)=0.706, p=0.497 ns |
| Total deaths (cleaned) | 83 | F(1,77)=0.936, p=0.336 ns | F(2,77)=0.493, p=0.613 ns |
| Total time (cleaned) | 83 | F(1,77)=0.320, p=0.573 ns | F(2,77)=0.034, p=0.967 ns |
| BPS (cleaned) | 87 | F(1,81)=0.500, p=0.482 ns | F(2,81)=0.680, p=0.510 ns |
| Completion | 90 | F(1,84)=1.244, p=0.268 ns | F(2,84)=0.799, p=0.453 ns |
| L6 Time | 60 | F(1,54)=0.707, p=0.404 ns | F(2,54)=1.449, p=0.244 ns |
| L6 BPS | 62 | F(1,56)=0.216, p=0.644 ns | F(2,56)=0.205, p=0.816 ns |
| L7 Time | 40 | F(1,34)=0.614, p=0.439 ns | F(2,34)=1.680, p=0.202 ns |
| L7 BPS | 41 | F(1,35)=2.323, p=0.137 ns | F(2,35)=0.600, p=0.554 ns |
Enjoyment — the only marginal interaction (p=0.063†)
No adaptability
With adaptability
Enjoyment cell means (grouped bar)
High/no=2.85 · High/yes=3.25 · Low/no=3.75 · Low/yes=2.96 · Mod/no=3.92 · Mod/yes=3.06
Interaction F(2,84)=2.855, p=0.063 †
Enjoyment interaction plot (line)
Lines cross at high implicitness — disordinal interaction: adaptability reverses direction
Engagement cell means
Near-parallel lines — additive effects only (p=0.446 ns)
Scaffolding cell means
Near-parallel lines — additive effects only (p=0.367 ns)
Two-way ANOVA values from official analysis CSV (S3-Interaction). Implicitness main effects not included in CSV — see one-way ANOVA section.
Demographic analysis
Effects of gender, grade level, and age on learning outcomes. Note: the CSV does not include demographic tests — values below are from preliminary analysis and should be interpreted with caution.
Male
49
54.4% of sample
Female
35
38.9% of sample
Ages
14–18
Modal age = 16 (n=30)
Grades
9–12
Modal grade = 10 (n=30)
Gender — engagement, scaffolding, and completion
Male (n=49)
Female (n=35)
Other (n=6)
Enjoyment by gender
Male (3.56) · Female (3.07) · Other (3.33)
ANOVA p=0.162; Male vs. Female p=0.063 †
Engagement by gender
Male (3.91) · Female (3.33) · Other (3.75)
ANOVA p=0.033 *; Male vs. Female p=0.010 *
Scaffolding by gender
Male (3.90) · Female (3.28) · Other (3.77)
ANOVA p=0.014 *; Male vs. Female p=0.004 *
Completion rate by gender
Male (61.2%) · Female (22.9%) · Other (66.7%)
ANOVA p=0.001 *; Male vs. Female χ²=10.632, p=0.001 *
Stars earned by gender
Male (1.29) · Female (1.00) · Other (0.67)
ANOVA p=0.208 ns
Grade level — significant differences in deaths and time (cleaned, N=83)
Grade 9 (n=18)
Grade 10 (n=30)
Grade 11 (n=26)
Grade 12 (n=16)
Total deaths by grade (cleaned, N=83)
Grade 12 died substantially more than grades 10 and 11
ANOVA p=0.019 *
Gr 10 vs 12: p=0.003 *
Gr 11 vs 12: p=0.019 *
Total time by grade (cleaned, N=83)
Grade 12 spent significantly more time than grades 10 and 11
ANOVA p=0.039 *
Gr 10 vs 12: p=0.007 *
Gr 11 vs 12: p=0.034 *
Enjoyment by grade
Gr 9 (3.52) · 10 (3.29) · 11 (3.30) · 12 (3.38)
ANOVA p=0.916 ns
Engagement by grade
Gr 9 (3.89) · 10 (3.50) · 11 (3.62) · 12 (3.84)
ANOVA p=0.534 ns
Completion rate by grade
Gr 9 (61%) · 10 (33%) · 11 (54%) · 12 (44%)
ANOVA p=0.243 ns
Age — marginal effect on deaths (cleaned); no effect on subjective outcomes
Total deaths by age (cleaned, N=83)
Ages 15–16 died fewest; ages 14, 17, 18 died more
ANOVA p=0.044 *
Age 14 vs 16: p=0.046 *
Age 16 vs 17: p=0.024 *
Enjoyment & engagement by age
No consistent trend with age for subjective outcomes
Enjoyment p=0.592 ns
Engagement p=0.621 ns
Demographic tests not included in official analysis CSV — values from preliminary analysis only. "Other" gender combines Nonbinary, Transgender Woman, Prefer not to say, and two informal responses (n=6). Interpret with caution due to small n. Gaming experience not collected; gender differences in completion likely confounded by prior gaming background.