[workshop-sim] Workshop Simulation Report — 2026-08-27 (Run #49, 1000×Monte Carlo)

[github-actions]

Overview

• Date: 2026-08-27
• Students simulated: 46 × 1000 Monte Carlo runs
• Workshop steps available: 30/30
• Overall success rate: 0% (95% Monte Carlo interval: 0%–0.008%)
• Highest-dropout step: 08-run-your-workflow (100% conditional dropout among 14,962 at-risk runs; 95% Monte Carlo interval: 99.98%–100%)
• Lowest curriculum quality step: 04-github-actions-intro.md (overall score 5.42/10)
• Learning KPI index: 2.91/10 (active_learning 4.25 · checkpoint_quality 0.00 · scaffolding 5.00)
• Model: unversioned / 2026-07-assumption-model-v2 (parameter hash n/a)
• Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty

Part Summary

Part	Files	Mean Score	Std Dev
Part 1 — core path (lessons 00–14)	15	6.69 / 10.0	±1.74
Part 2 — advanced (lessons 15+)	15	6.10 / 10.0	±0.36
Overall corpus	30	6.39 / 10.0	±1.13

No pages are classified as other.

Critical Findings

1. Step 8 (08-run-your-workflow) is a terminal blocker in Part 1: 100% of learners who reach it fail due to model-access-not-configured (14,962 failures). Billing configuration is delegated entirely to side-quest-07d-billing-paths.md — learners who skip or partially complete that side quest arrive at Step 8 with a mismatched lock file.
2. Step 7 compound dropout (33.8%): copilot-access-missing (3,467) and workflow-authoring-friction (4,185) both fire heavily. The billing side-quest reference in Step 7d has no inline verification gate, so beginner/github-basic learners skip it.
3. Learning KPI index is critically low at 2.91/10: checkpoint_quality scores 0.0 across all 30 steps. Learners who stay in the workshop are building some skills (AL=4.25, SC=5.0) but receive no rubric-confirmable evidence-based practice confirmation at any step.
4. Most urgent repair is in Part 1: Closing the billing-configuration state gap between Steps 7, 7d, and 8 is the single highest-impact change. All other improvements are secondary until Step 8 allows at least some learners through.

Top Repairs to Prioritize

Note: some student dropout is expected and acceptable. Repairs must maintain or improve the learning KPI index — do not lower the cognitive bar or remove practice to chase headline completion numbers.

1. Add inline billing-verification gate to Step 7d checkpoint — require explicit confirmation of billing method, permission match, recompile, and both files committed to main before advancing (completion impact: ↑ · learning KPI impact: ↑)
2. Add checkpoint_quality items to high-dropout learning steps — add at least one self-check or active-recall prompt per learning step in Steps 4, 5, 7, 8 to lift KPI index from 2.91 toward 5.0 (completion impact: ↔ · learning KPI impact: ↑)
3. Strengthen Step 5 concept scaffolding — add a concrete one-sentence definition callout and one worked deployment example in 05-agentic-workflows-intro.md to reduce 16.8% concept-gap dropout (completion impact: ↑ · learning KPI impact: ↑)

Dropout by step

Step	At-risk runs	Dropout rate	95% CI	Failure mode	Top reason
08-run-your-workflow (P1)	14,962	100.00%	[99.98%–100%]	Access barrier	Model/billing configuration mismatch: lock file not compiled with chosen billing method
07-first-workflow (P1)	22,614	33.84%	[33.22%–34.46%]	Access barrier	Workflow authoring friction + Copilot access not confirmed before proceeding
05-agentic-intro (P1)	37,096	16.75%	[16.38%–17.14%]	Learning barrier	Agentic concept overload for beginners without concrete anchoring examples
04-actions-intro (P1)	44,157	15.99%	[15.65%–16.34%]	Learning barrier	Concept overload: 25 new concepts introduced with no inline practice or checkpoint
05c-agentic-practice (P1)	30,881	12.89%	[12.52%–13.27%]	Learning barrier	Agentic classification gap — learners cannot distinguish agentic from non-agentic patterns
05b-agentic-security (P1)	26,901	10.07%	[9.71%–10.43%]	Learning barrier	Security concepts introduced without hands-on self-assessment
06-install-gh-aw (P1)	24,193	6.53%	[6.22%–6.84%]	Access barrier	Extension install friction on varied environments
02-setup (P1)	46,000	4.01%	[3.83%–4.19%]	Access barrier	Codespace setup friction
00-welcome (P1)	46,000	0.00%	[0%–0.01%]	—	None
01-prerequisites (P1)	46,000	0.00%	[0%–0.01%]	—	None

Curriculum quality and learning KPIs

Step	Overall	AL	CQ	SC	KPI	Weakest dim	Repair priority
00-welcome.md	10.00	0.0	0.0	5.0	1.36	active_learning	Low (non-learning page)
01-prerequisites.md	10.00	0.0	0.0	5.0	1.36	active_learning	Low (non-learning page)
02a-setup-codespace.md	6.09	5.0	0.0	5.0	3.18	checkpoint_quality	Medium
04-github-actions-intro.md	5.42	4.6	0.0	5.0	3.04	checkpoint_quality	High
05-agentic-workflows-intro.md	5.43	2.4	0.0	5.0	2.24	checkpoint_quality	High
05b-agentic-workflows-security.md	5.75	2.5	0.0	5.0	2.27	checkpoint_quality	Medium
05c-agentic-workflows-practice.md	6.22	6.2	0.0	5.0	3.62	checkpoint_quality	Medium
06-install-gh-aw.md	10.00	3.5	0.0	5.0	2.64	checkpoint_quality	Low
07-your-first-workflow.md	6.25	6.3	0.0	5.0	3.65	checkpoint_quality	High
07d-confirm-model-access.md	6.25	6.3	0.0	5.0	3.65	checkpoint_quality	Critical
08-run-your-workflow.md	5.67	3.0	0.0	5.0	2.45	checkpoint_quality	Critical
08b-interpret-your-run.md	5.99	3.7	0.0	5.0	2.71	checkpoint_quality	Medium
09-agentic-editing.md	5.99	4.7	0.0	5.0	3.07	checkpoint_quality	Medium
14-next-steps.md	5.83	3.1	0.0	5.0	2.49	checkpoint_quality	Low
14b-pr-reviewer-workflow.md	5.45	4.3	0.0	5.0	2.93	checkpoint_quality	Medium
15-conditional-logic.md	5.53	3.8	0.0	5.0	2.75	checkpoint_quality	Medium
16-connect-data-source.md	5.71	3.8	0.0	5.0	2.75	checkpoint_quality	Medium
17-add-mcp-tools.md	5.61	3.1	0.0	5.0	2.49	checkpoint_quality	Medium
18-share-and-reuse.md	6.15	4.5	0.0	5.0	3.00	checkpoint_quality	Medium
19-research-driven-training-node.md	6.37	5.6	0.0	5.0	3.40	checkpoint_quality	Medium
20-persistent-memory.md	5.67	3.3	0.0	5.0	2.56	checkpoint_quality	Medium
21-inline-sub-agents.md	5.99	4.1	0.0	5.0	2.85	checkpoint_quality	Medium
22-error-handling-and-resilience.md	6.27	5.1	0.0	5.0	3.22	checkpoint_quality	Medium
23-ab-experiments.md	6.53	6.4	0.0	5.0	3.69	checkpoint_quality	Medium
24-self-hosted-runners.md	6.63	6.9	0.0	5.0	3.87	checkpoint_quality	Medium
25-audit-and-observability.md	6.23	4.9	0.0	5.0	3.15	checkpoint_quality	Medium
26-manage-costs-and-budgets.md	5.85	4.0	0.0	5.0	2.82	checkpoint_quality	Medium
27-evaluate-workflow-quality.md	6.59	6.7	0.0	5.0	3.80	checkpoint_quality	Medium
28-orchestrate-workflows.md	6.25	5.0	0.0	5.0	3.18	checkpoint_quality	Medium
29-skills-and-domain-knowledge.md	6.05	4.6	0.0	5.0	3.04	checkpoint_quality	Medium
Cohort mean	6.39	4.25	0.00	5.00	2.91	checkpoint_quality	—

Segment breakdowns

By technical level (all 0% — Step 8 blocks universally)

Level	n	Mean success rate
beginner	11	0.00%
github-basic	19	0.00%
actions-user	11	0.00%
advanced	5	0.00%

By personality

Personality	n	Mean success rate
confused	6	0.00%
curious	15	0.00%
impatient	6	0.00%
methodical	12	0.00%
skeptical	7	0.00%

By UI preference

UI preferred	n	Mean success rate
true (22)	22	0.00%
false (24)	24	0.00%

All segments show 0% completion because the Step 8 model-access-not-configured barrier is a hard simulator constraint independent of learner level, personality, or tool preference. Within that constraint, methodical personalities progress further through concept steps (04–06) before reaching Step 8, and actions-user learners have lower per-step dropout rates at Steps 4–6.

Notable student journeys (3)

Surprising partial success — Learner 004 (actions-user, methodical, backend-dev, vscode, work-project): This learner has the most consistent progression through Part 1 concept steps due to Actions familiarity and methodical reading. They still fail at Step 8 due to the billing state gap — illustrating the Step 8 blocker is a pure access barrier, not a learning deficit. With the billing gate fixed, this profile would likely complete.

Unexpected early dropout — Learner 002 (actions-user, confused, devops, cli, work-project): Actions background suggested lower concept-step dropout. However, the confused personality combined with a work-project urgency causes side-quest skipping — they bypass side-quest-07d-billing-paths.md to reach the "running" step faster, arriving at Step 8 with a valid .md but no matching billing configuration. Top failure: model-access-not-configured (713/1000 runs).

Content-gap case — Learner 001 (github-basic, confused, data-science, vscode, personal-learning): No Actions background; most commonly fails at Step 4 (concept-overload, 249/1000). The 25 new GitHub Actions concepts in 5 minutes exceed manageable scaffolding for this profile. Even if Step 4 were fixed, this learner faces the Step 8 billing state gap — a systemic gap in the beginner-to-completion pathway.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔬 Workshop Student Simulator · 101.5 AIC · ⌖ 7.82 AIC · ⊞ 10.4K · ◷

• expires on Aug 28, 2026, 4:15 AM UTC