Página web

[mgraffg]

Crear la página web del proyecto.

El proyecto está descrito en el siguiente artículo https://arxiv.org/pdf/2403.04693.

Hay que utilizar el artículo para hacer la página principal

La página debe de estar creada en quarto utilizando libretas de Python, para que puedan ser abiertas en colab.

La página debe de tener las isguiente secciones.

1. Introducción, basarse en el artículo (https://arxiv.org/pdf/2403.04693) para esta página
2. Guia rápida para usuarios de scikit-learn
3. Guia rápida para personas organizando competencias, aquí se pueden usar los datos de https://github.com/INGEOTEC/Delitos
4. Guía rápida para pruebas multiples

Considera también lo que se va implementar en el issue #35.

La página principal https://ingeotec.github.io/CompStats y la página que documenta la API de la librería que esta en readthedocs.

---

Subissues

Este issue se completa cuando se completen los siguientes subissues (vinculados como sub-issues nativos de GitHub, ver panel de sub-issues del issue):

• Crear la plantilla del sitio en Quarto (estructura tipo LegalIA) y la página de Introducción #43 — Crear la plantilla del sitio en Quarto y la sección de Introducción basada en el artículo
• Crear la página 'scikit-learn Users' (libreta) con la API actual (BiB en vez de score_func/error_func) #38 — Actualizar la sección 'scikit-learn Users' a la API actual (BiB en vez de score_func/error_func)
• Crear la página 'Guía rápida para organizadores de competencias' (libreta) usando el dataset de INGEOTEC/Delitos #39 — Agregar sección 'Guía rápida para organizadores de competencias' (dataset INGEOTEC/Delitos)
• Crear la página 'Guía rápida para pruebas múltiples' (libreta, issue #35) con corrección de comparaciones múltiples #40 — Agregar sección 'Guía rápida para pruebas múltiples' (issue Performance multiple comparison #35)
• Agregar enlaces 'Open in Colab' a las libretas del sitio #41 — Habilitar apertura en Google Colab de las libretas de la página web
• Enlazar la página principal (GitHub Pages) con la documentación de la API en ReadTheDocs #42 — Enlazar la página principal con la documentación de la API en ReadTheDocs

No cerrar este issue hasta que los 6 subissues anteriores estén completados.

[mgraffg]

Implementation plan (revised — one page per section following INGEOTEC/LegalIA's structure, plus a friendlier readthedocs API reference with a class diagram)

Verified before writing this plan: https://ingeotec.github.io/CompStats currently 404s (nothing published yet). Looked at INGEOTEC/LegalIA (website/ in that repo) as the reference for the header/sections structure you pointed at: it's a proper Quarto website project, not a single-file dashboard — website/_quarto.yml declares project: {type: website} and a website.navbar where every top-level header item (Home, Dataset, DOF Titles, Tools, References, plus an external Research link and a GitHub icon on the right) points at its own separate page file (index.ipynb, pages/dataset.ipynb, pages/titles.ipynb, pages/tools.ipynb, pages/references.qmd). Content pages are .ipynb notebooks rendered directly by Quarto; the one page with no code (references.qmd) is a plain .qmd backed by a references.bib file. This supersedes the single-quarto/CompStats.qmd-dashboard approach from my previous comment — the plan below replaces that with the same multi-page structure.

1. Project layout — replace the single dashboard file with a Quarto website project

• quarto/_quarto.yml (new), mirroring LegalIA's _quarto.yml:
  • project: {type: website, output-dir: _site}
  • execute: {freeze: auto} (kept from the current file)
  • bibliography: references.bib
  • website.navbar, one entry per requested section, each linking to its own page:
    • Home → index.ipynb (Introduction)
    • scikit-learn Users → pages/sklearn_users.ipynb
    • Competition Organizers → pages/organizers.ipynb
    • Multiple Comparisons → pages/multiple_tests.ipynb
    • Citing → pages/references.qmd
    • external entry API Docs → https://compstats.readthedocs.io/en/latest/ (same pattern as LegalIA's external Research navbar link to a GitHub issues view)
    • right-aligned GitHub icon → https://github.com/INGEOTEC/CompStats (same as LegalIA)
  • format.html: theme/toc settings similar to LegalIA (toc: true, code-fold: true where notebooks get long)
• Retire quarto/CompStats.qmd (the current single dashboard) and its build artifacts (quarto/CompStats.html, quarto/CompStats_files/) — content migrates into the pages below instead of living in one file with dashboard tabs.
• .github/workflows/publish.yml: change the publish step from quarto publish gh-pages CompStats.qmd (single-file publish) to quarto publish gh-pages run from inside quarto/ (project publish, since _quarto.yml now defines a website) — still targets the same gh-pages branch, i.e. still served at https://ingeotec.github.io/CompStats. No other workflow changes expected (same conda deps already cover scikit-learn/pandas/statsmodels).

2. Colab notebooks — no separate companion copies needed

Because each section is now its own real .ipynb under quarto/ (not a duplicate maintained separately under docs/, as my previous plan proposed), that same file is what's Colab-openable: https://colab.research.google.com/github/INGEOTEC/CompStats/blob/develop/quarto/pages/organizers.ipynb (and similarly for the other three notebook pages). Each notebook's first cell follows the existing docs/*.ipynb convention (try: import CompStats / except ImportError: !pip install CompStats), and each page gets an "Open in Colab" badge pointing at itself. This drops the earlier plan's docs/CompStats_organizers.ipynb / docs/CompStats_multiple_tests.ipynb items — they're no longer needed as separate files.

3. Page index.ipynb — Introduction

Same content plan as before: grounded in the arXiv paper (2403.04693 v2, Aug 2024, "Analysis of Systems' Performance in Natural Language Processing Competitions", Nava-Muñoz/Graff/Escalante), extending the current intro text with the two points not yet stated — illustrated on eight NLP competitions (classification and regression), and the paper's competition-difficulty metrics — plus a link to https://arxiv.org/abs/2403.04693 and a pointer to the Citing page.

4. Page pages/sklearn_users.ipynb — scikit-learn Users

Migrates the existing "scikit-learn Users" and "Multi-measure Support" sections from the current CompStats.qmd into one notebook page (multi-measure is a continuation of that same walkthrough, not one of the four requested top-level sections, so it stays folded into this page rather than getting its own navbar entry). No functional/content change otherwise — already satisfies issue item 2.

5. Page pages/organizers.ipynb — Competition Organizers (issue item 3)

Unchanged from the previous plan: uses INGEOTEC/Delitos (binary tweet classification, 1,351 negative / 449 positive, JSON-lines at corpus/delitos_train.json / corpus/delitos_test.json on main, loaded via pandas.read_json(url, lines=True)). Builds 3–4 stand-in "systems" (TfidfVectorizer + LinearSVC/MultinomialNB/RandomForestClassifier) trained on delitos_train.json, predicting on delitos_test.json, to reproduce the gold+submissions shape an organizer actually receives. Uses CompStats.metrics.macro_f1 given the class imbalance, then .plot() → .difference() → p_value() walkthrough, framed for an organizer deciding a winner.

6. Page pages/multiple_tests.ipynb — Multiple Comparisons (issue item 4, tied to issue #35)

Unchanged from the previous plan: reuses the multi-system comparison built in the Organizers page, explains why correction is needed, and shows it two ways since issue #35 is still open (issue #34 is now marked blocked-by #35 on GitHub, so this page — along with the rest of the site — won't be worked on until #35 closes):

1. Works today: diff.p_value() → statsmodels.stats.multitest.multipletests(...), zipped back into a dict (same snippet already in Performance multiple comparison #35's description and the current readthedocs quickstart).
2. Callout note: once Performance multiple comparison #35 ships Difference.p_value(correction=...) / Perf.plot(correction=...), this collapses to diff.p_value(correction='bonferroni') — noted now, swapped in as a fast-follow once Performance multiple comparison #35 merges.

7. Page pages/references.qmd — Citing

Plain markdown page (no code), following LegalIA's references.qmd pattern exactly: a quarto/references.bib file holding the existing BibTeX entry verbatim (@article{Nava:2023, title = {Comparison of Classifiers in Challenge Scheme}, ...}, copied byte-for-byte from docs/source/index.rst), referenced via nocite: | @Nava:2023 and a ::: {#refs} ::: block so Quarto renders the formatted citation plus the raw BibTeX. This preserves the exact citation you asked to keep, rendered through Quarto's native bibliography mechanism instead of a hardcoded card.

8. readthedocs (Sphinx, docs/source/) — friendlier API docs with a class diagram

New requirement: unlike the Quarto site above, this piece lives entirely in readthedocs, not the new website — its audience is someone wanting to extend CompStats, not use it. Today docs/source/*_api.rst is five bare automodule :members: pages (metrics_api, interface_api, performance_api, measurements_api, bootstrap_api) with no overview of how the pieces fit together before diving into per-function docstrings.

• New page docs/source/architecture.rst, inserted in index.rst's toctree before the existing API pages, so it's the on-ramp a contributor reads first. Content mirrors (and visualizes) the "Architecture" section already written in CLAUDE.md:
  • The one-paragraph data-flow narrative: raw predictions → bootstrap resampled statistic → derived comparisons/plots.
  • A class diagram showing how the modules relate:
    • StatisticSamples (bootstrap.py) — the foundational primitive: statistic, num_samples, n_jobs, calls: dict[name → ndarray], cached _samples indices, __sklearn_clone__.
    • Perf (interface.py) — has a StatisticSamples per system; score_func/error_func (XOR) drive BiB; difference(), dataframe(), plot(), __call__ to add predictions.
    • Difference (interface.py) — created from Perf.difference() via sklearn.base.clone (reuses bootstrap samples, doesn't redraw them); adds best, p_value().
    • CompStats.metrics wrappers (f1_score, accuracy_score, ...) — factories that close over an sklearn.metrics function and construct a Perf; the shared @metrics_docs decorator.
    • performance.py's module-level functions (performance, difference, all_differences) — shown as the older, parallel DataFrame-based API, explicitly not composed with Perf/Difference.
    • measurements.py (CI, SE, difference_p_value) — stateless helpers consumed by both APIs.
  • Callouts next to the diagram for the invariants a contributor must not break (already listed in CLAUDE.md's "Key invariants" section): paired bootstrap resampling across systems, BiB threaded consistently rather than re-derived, and __sklearn_clone__-based reuse of bootstrap samples instead of plain re-instantiation.
• Diagram implementation: sphinxcontrib-mermaid + a .. mermaid:: classDiagram block, rather than sphinx.ext.graphviz or the built-in sphinx.ext.inheritance_diagram. Reasoning: (a) our classes barely use inheritance — the interesting relationships are composition (Perf *-- StatisticSamples, Difference built from Perf's clone) — so inheritance_diagram (which walks the Python MRO) would render near-empty, unhelpful graphs; (b) Mermaid renders client-side in the browser with no system binary, whereas Graphviz needs the graphviz binary present on the readthedocs build image (an extra apt_packages entry in .readthedocs.yaml) for no real benefit here.
• Dependency/config changes:
  • environment.yml: add sphinxcontrib-mermaid under the existing pip: section (conda's docs build, per .readthedocs.yaml, uses this environment).
  • requirements.txt: add sphinxcontrib-mermaid too, for parity with local pip install -r requirements.txt setups.
  • docs/source/conf.py: add 'sphinxcontrib.mermaid' to extensions.
  • docs/source/index.rst: add architecture to the existing API toctree, ahead of metrics_api.

9. Supporting changes

• README.rst: no new Colab badges needed now (see §2) — leave as-is, or optionally point its existing two badges at the new notebook paths if the old docs/CompStats*.ipynb files are retired (open question, not required for this issue).

10. Verification

• quarto render from quarto/ to confirm the whole website (all five pages) builds without errors, including live-fetching the Delitos JSON and training the stand-in classifiers.
• Open each notebook page's Colab link to confirm it runs top-to-bottom in a clean runtime.
• cd docs && make html locally to confirm the new architecture.rst page builds and the Mermaid class diagram renders correctly, before relying on a full readthedocs build.
• Publish the Quarto site via the existing manual Quarto Publish workflow_dispatch once content is reviewed; confirm the navbar renders with all five sections plus the API Docs/GitHub links at https://ingeotec.github.io/CompStats, and confirm the new architecture page shows up at https://compstats.readthedocs.io/en/latest/.