Flatten the consumer-keyed tree into the published layout

[mmcky]

PLAN Phase 2. Moves the data out of lecture-python-intro/{static,dynamic}/ into lectures/ — the flat published root — and lifts scripts/ to the repo root, outside the published tree.

Why flat, and why now

The layout is flat because the URL is the interface: lectures/<filename> maps to data.quantecon.org/lectures/<filename>. A consumer-keyed folder implies a series owns a file, which is false — any lecture may consume any dataset — and would bake an owner into every URL.

Doing it now is the whole point. Zero lectures reference this repo, so every move is free. I verified that independently rather than relying on the audit — an org-wide code search for QuantEcon/data/raw, raw.githubusercontent.com/QuantEcon/data and the old static paths returns 0 hits; the only references to data-lectures anywhere are this repo's own docs. Once the first repoint merges (Phase 8), published filenames become an API and every move is a breaking change. AGENTS.md now says exactly that, where it previously promised the freedom.

Layout decisions (PLAN Phase 2's open question)

Path	Published
lectures/ — flat, all 9 datasets	yes
scripts/ — builders	no
manifest-schema.yml — the schema sketch	no

Manifests are the one exception: they live inside lectures/ as sidecars named <filename>.yml, so a dataset can't be moved or removed without its metadata and CI can assert the pairing with a glob. Naming uses the full filename (mpd2020.xlsx.yml, not mpd2020.yml) because a stem-keyed sidecar collides the moment one dataset ships in two formats — precisely the fig_3.xlsx/fig_3.ods case this repo already contained. All strawman until the pilot tests it.

Three files dropped rather than promoted

Rather than carry files with no consumer into the canonical namespace (each would then need a manifest, licence check and integrity verification under Phases 6–7):

• GDP_per_capita_world_bank.csv and Metadata_Country_API_NY.GDP.PCAP.CD_DS2_en_csv_v2_4770417.csv — an org-wide code search returns zero references to either, they are freely re-downloadable from the World Bank, and their licence was never established.
• fig_3.ods — verified to carry no provenance the published .xlsx lacks: both parse to a single Sheet1, identical 34×6 shape, DataFrame.equals → True. A pure format twin.

All three remain in git history if ever wanted. Happy to keep any of them if you'd rather — this is the Phase 6 keep-or-drop call made explicit.

Reviewing this — read the layout, not the schema

Two things worth knowing before you spend time here.

manifest-schema.yml is a strawman, and #12 already revises it by ~46 lines. It is labelled provisional in its own header ("every field here is provisional until proven by a real migration") and that is exactly what happened: the pilot in #12 met one real dataset and the schema needed integrity split into migration/upstream, a builder_status field, and known_nulls. So feedback on the schema's fields is better spent on #12, which carries the tested version. What is worth reviewing here is the layout decision — flat tree, sidecar manifests, full-filename naming — because that is what everything downstream inherits.

There is no CI in this repo (no .github/ — that is PLAN Phase 5). "MERGEABLE" here means "no conflicts", not "verified". The verification is the manual work described below.

Verification

• All 11 data files are tracked as renames (R in git status -M), so history follows them.
• business_cycle.py wrote to ../dynamic, which no longer exists — repointed at lectures/ and confirmed its path logic resolves to the real directory for all three of its outputs. A restructure that moved the data but left the builder writing into the void would have failed silently at the next refresh.

Also here

manifest-schema.yml sketches the per-dataset manifest. It is worked against a real example rather than a hypothetical one: business_cycle_metadata.md already records the World Bank series as CC BY-4.0 with source and periodicity, which proves those fields are actually obtainable and gives Phase 6 its model.

Part of #8
See QuantEcon/meta#336
Part of QuantEcon/workspace-lectures#14

[Copilot]

Pull request overview

Implements PLAN Phase 2 by flattening the legacy consumer-keyed layout under lecture-python-intro/ into a flat published tree at lectures/, while moving builder scripts to a non-published scripts/ directory at the repo root. It also documents the new layout and introduces a strawman per-dataset manifest schema to support upcoming metadata/integrity phases.

Changes:

• Flatten/move datasets into lectures/ and remove the legacy lecture-python-intro/{static,dynamic,scripts}/ READMEs and orphaned World Bank metadata CSV.
• Repoint the business_cycle.py builder to write outputs into lectures/ and document builders under scripts/.
• Update repo docs (README.md, PLAN.md, AGENTS.md) and add manifest-schema.yml to describe/standardize the new published layout and manifest approach.

Reviewed changes

Copilot reviewed 10 out of 23 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
scripts/README.md	New documentation for dataset builders living outside the published tree.
scripts/business_cycle.py	Updates output directory from legacy dynamic path to lectures/.
README.md	Documents the new flat layout and clarifies what is published vs not.
PLAN.md	Records Phase 2 completion details and updates roadmap language accordingly.
manifest-schema.yml	Adds a strawman manifest schema and sidecar placement convention.
lectures/nom_balances.npy	Moves dataset into the flat published lectures/ namespace.
lectures/fig_3.xlsx	Moves dataset into the flat published lectures/ namespace.
lectures/chapter_3.xlsx	Moves dataset into the flat published lectures/ namespace.
lectures/caron.npy	Moves dataset into the flat published lectures/ namespace.
lectures/business_cycle_metadata.md	Publishes World Bank metadata dump alongside the dataset for provenance/license capture.
lectures/business_cycle_info.md	Publishes series info dump alongside the dataset.
lectures/business_cycle_data.csv	Publishes the dynamic snapshot CSV under the new published root.
lecture-python-intro/static/README.md	Removes legacy consumer-keyed static README (layout is no longer consumer-keyed).
lecture-python-intro/static/Metadata_Country_API_NY.GDP.PCAP.CD_DS2_en_csv_v2_4770417.csv	Drops an orphan/no-consumer file rather than promoting it into the published namespace.
lecture-python-intro/scripts/README.md	Removes legacy consumer-keyed scripts README (scripts moved to repo root).
lecture-python-intro/dynamic/README.md	Removes legacy consumer-keyed dynamic README (dynamic output now lives in lectures/).
AGENTS.md	Updates repo map and documents the “published filenames are an API” constraint post-repoint.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.