Reviewers 1.1 / 1.4: Euler for Fig 1B, a legend for Fig 2A, a pathway panel, and the growth basis - #299
Open
freiburgermsu wants to merge 4 commits into
Open
Reviewers 1.1 / 1.4: Euler for Fig 1B, a legend for Fig 2A, a pathway panel, and the growth basis#299freiburgermsu wants to merge 4 commits into
freiburgermsu wants to merge 4 commits into
Conversation
… panel that does not exist
Four things, all touching the figures or the numbers they carry.
REVIEWER 1.4 -- "Fig 1B would be more informative as a Venn diagram."
Agreed, and for the reason implied: the stacked bar said 67% of MetaCyc's
reactions are unique but never said who the other 33% are shared WITH, which is
the question integrating Rhea raises. 1B is now a three-set Euler with every
region labelled. Circle areas are not proportional -- three-set areas cannot be
solved exactly -- so the counts carry the quantity and the circles carry only
membership, and the caption says so. The 9,406 reactions in none of the three
are labelled rather than dropped.
REVIEWER 1.4 -- "Fig 2A needs a color legend (and maybe a different color
scheme)." It had four segments and identified two; the caption was doing the
legend's job in prose. There is now a four-entry key. On the scheme: the two
non-Marvin segments were both grey, which merged a chemistry RESULT ("Marvin
ran and found no dissociable proton") with a curation GAP ("there is no
structure to run on"). The result now takes its own categorical slot and only
the genuine absence stays grey.
REVIEWER 1.1 -- "where are we still gaining new information." New panel 1D,
and two sentences in M09. The answer is a good one: 41 of the 12,261 reactions
added since 2020 fall in a central-carbon or energy-metabolism class. Growth is
in specialised metabolism and cell-envelope biosynthesis -- antibiotics,
polyketides, toxins, alkaloids, O-antigen, branched and acylated lipids.
Building that panel needed a join the released files cannot currently do:
Unique_ModelSEED_Reaction_Pathways.txt stops at rxn48568, just below the 2020
boundary, so every post-2020 reaction reads as unannotated. figure_common now
rebuilds the mapping from the MetaCyc source table, putting BOTH eras through
the same one-level parent join so the two bars are comparable -- reading the
pre-2020 side out of the alias file would compare a full ancestor closure
against a direct-parent lookup. The underlying annotation gap is a data defect
and is left to its own PR. Note MetaCyc_pathways.tsv mixes pathways and
ontology classes in one id column; treating every id as a pathway finds no
classes at all.
SELF-FOUND, not raised by either reviewer:
- M12 cited Figure 3 panel D. Figure 3 has A, B and C. Worse, B and C were
never cited by letter anywhere, so the only references into that figure
were A and a panel that does not exist. The grading sentence now points at
B and C, which are the panels that show what it describes.
- M09 claimed 55% growth in reactions against 34% in compounds. 55% is not
reproducible on any single population: it sets 2026's totals against a 2020
baseline with its 7,577 obsolete rows removed. On the all-records basis the
rest of the paper uses -- the abstract's ~56,000, Figure 2A's 56,002, M12's
25,855 -- it is 28%, which is what this now says. Live-vs-live would be 34%
and 34%, but adopting it means converting every other number in the paper
including the abstract, so it is raised in the PR rather than taken here.
- figure_common could not be imported at all in a clean checkout.
_silver_sigma_band() read two intermediates under results/thermo_grades/
that are .gitignore'd and absent, so EVERY figure was unregenerable, and
grade_thermo_sources.py cannot rebuild them under the pinned pandas. It now
derives the same band from files that ship: best_grade from
reaction_grades.tsv and each source's sigma from Biochemistry/reaction_*.json.
Reproduces the published band exactly (eQuilibrator 0.27-1.81, group
contribution 1.62-25.12, dGPredictor 2.09-38.84) once the documented
sentinels are excluded -- leaving them in put eQuilibrator's p95 at 23,901.
Also adds the two analysis scripts behind these numbers and the response plan
that maps every reviewer comment to its change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The class table, the enrichment ratios and the central-metabolism comparison in the R1.1 section were computed with an obsolete-row filter while sections A and B, Figure 1D and the PR body were on the all-records basis the manuscript uses. Same document, two populations. Regenerated on the manuscript's basis: carried-over counts rise (antibiotics 740 -> 824, Super-Pathways 3,650 -> 5,706), the join reaches 16,499 rather than 12,518 carried-over reactions, and the central/energy comparison is 770 of 43,751 rather than 328 of 36,142. The finding is unchanged and was re-derived two independent ways: 41 of the 12,261 reactions added since 2020 fall in a central-carbon or energy-metabolism class, on either population. The Figure 1D caption and M09 are correct as they stand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Follows the growth-basis question raised in this PR. The corresponding author
chose the live-only basis, so this converts the paper to it rather than
matching the reporting basis to the draft's existing numbers.
WHAT MOVES. The abstract's headline goes from ~56,000 reactions to ~48,000,
and from ~33,000 multi-source reactions to ~27,000. Growth becomes 34% in
compounds and 34% in reactions, measured the same way on both. Every count in
M09, M11, M12, M13 and Supplementary Table S1 moves with it, and all three
figures move with the text -- NONE of them filtered obsolete records before,
so Figure 1's source counts, Figure 2's coverage bars and Figure 3's direction
and grade panels were all silently on the other population.
HOW IT IS AUDITABLE. analysis/population_basis_table.py prints every quantity
the manuscript quotes on both populations side by side. Its all-records column
reproduces the submitted draft exactly, which is what makes the live column
trustworthy: 84.3% / 42.9% / 41.1%, 30,157 and 25,855, 95.0%, 4,248 and
13,289, 21,218 / 4,248 / 4,691, and the full grade table all come back
unchanged. Two definitions had to be recovered to get there and are documented
in the script, because both announce themselves loudly when wrong:
covers excludes each source's refusal sentinel -- group contribution
writes dg = 1e7 when it declines, eQuilibrator sigma >= 2500.
Counting them puts eQuilibrator's coverage at 25,175, not 21,789.
resolves states a direction OR an explicit reversibility ('>', '<', '=').
Restricting it to '>' and '<' turns 13,289 into 6,267.
figure_common gains a single LIVE_ONLY flag threaded through every loader.
Two entries were not derived at all -- energy_rxn and energy_total were
hardcoded constants, which is exactly why they would have stayed behind -- and
are now computed with the sentinel exclusion their own comment described.
Figure 2's x-axis was likewise pinned to 56,002 and now follows the total.
TWO THINGS THE CONVERSION EXPOSED, both now disclosed rather than absorbed:
- The anchor set is duplicated. 806 records carry a stereo-exact
measurement, but 441 are obsolete records that EACH link to a live
replacement -- duplicate entries for the same chemistry. The number of
distinct anchored reactions is 365. This is not a population preference;
it means the calibration's effective sample size is smaller than 806
suggests. S03 and M12 now say so. The calibration has NOT been refit:
that changes the published grades and belongs in its own PR with
co-author sign-off, and grade_thermo_sources.py currently dies under the
pinned pandas ("Encountered all NA values" from idxmin on an all-NA
slice), so anyone attempting it will need to fix that first.
- M13's atom-mapping numbers were stale independently of any population
question. The draft said 32,877 mapped and 7,819 salvaged; the shipped
data and Biochemistry/Structures/AtomMappings/README.md both say 32,378
and 7,320, the 499-reaction difference being the 2026-09-14 chain-salvage
withdrawal. On the live basis: 26,256 mapped, 19,810 clean, 6,446
salvaged.
NOT TOUCHED: M05's Uncertainty paragraph still quotes the old medians on this
branch. It is rewritten in the transport PR of this series, which removes those
numbers entirely (they were duplicated verbatim from M11 Results, where they
are now live: 0.62 / 10.83 / 17.56). Editing the same line here would turn a
clean merge into a conflict for no gain.
Verified: latexmk clean for main and supplementary, zero undefined references;
all three figures regenerated and inspected; a sweep for every superseded
all-records constant in sections/ and supplement/ returns only that M05 line.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… smaller fixes
Self-review of this PR against the shipped data after the live-basis
conversion. Six things, two of them substantive.
- Panel 1D's "already held" bars were still on the all-records population.
_pathway_classes iterates the alias file, which has no is_obsolete column,
and was the one loader the LIVE_ONLY threading missed -- antibiotics read
824 where the live count is 740, and the enrichment ratios moved with it
(branched fatty acids 98 vs 22 is 98 vs 8). Filtered against the live
reaction ids; the "added since 2020" bars are unchanged because every one
of the 12,261 additions is live.
- Figure 2C's histograms were drawn from figure_data_sigma.json, a static
cache with no generator that had been built on all records (n = 21,789 /
29,447 / 29,617). Its drawn median lines were therefore 0.63 / 10.41 /
17.01 while M11's text, converted to the live basis, says 0.62 / 10.83 /
17.56. The values are now derived from the records through the same
population filter and sentinel rule as the coverage bars, the cache is
deleted, and the medians match the text exactly.
- Neither the Figure 1 caption nor M09 said that panel D can only place
MetaCyc-sourced additions. Rhea supplied 8,409 of the 12,261 new reactions
and has no pathway ontology to put them in, so they are absent from the
panel -- a reviewer would have asked where they went. Both now say so.
- S03's sentence on the unpaired row had lost its denominator in the
conversion; it is 370 of the 4,690 reactions tagged unpaired.
- cover_letter.md still quoted ~56,000 reactions.
- The _growth docstring described the transcribed 2020 counts (19,138 /
17,760) as wrong against 19,172 / 17,793. They were the live-basis counts:
the differences are exactly the 34 obsolete 2020 compounds, and they are
what the function now returns. Reworded so the next reader is not told
the function's own output is an error.
The response plan's R1.1 table is on the live basis again (it had been moved
to all-records before the population decision). The Euler note in panel B is
repositioned; on the live basis it had landed on the Rhea set label.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update: the whole paper is now on the live-only population
Per the corresponding author's decision on the growth-basis question below, this PR no longer just fixes the 55% figure — it converts the entire manuscript to the live-only (non-obsolete) basis.
The abstract headline changes: ~56,000 reactions becomes ~48,000. That is the number people quote, so it is called out here rather than buried. Growth becomes 34% in compounds and 34% in reactions, measured identically. Every count in M09, M11, M12, M13 and Supplementary Table S1 moves with it, and so do all three figures — none of which filtered obsolete records before, so they were silently on the other population.
analysis/population_basis_table.pyprints every quantity on both populations. Its all-records column reproduces the submitted draft exactly (84.3% / 42.9% / 41.1%, 30,157 and 25,855, 95.0%, 4,248 and 13,289, the full grade table), which is what makes the live column trustworthy.Two things the conversion exposed, both now disclosed in the text:
grade_thermo_sources.pyalso currently dies under the pinned pandas). S03 and M12 now state that the effective sample size is smaller than the count implies.Answers reviewer 1's comments 1 (where is new information still arriving) and 4 (the figures), and fixes three things neither reviewer raised but both would have hit.
Reviewer 1, comment 4 — the figures
Fig 1B → Euler diagram
The reviewer is right, and for the reason implied. The stacked bar reported that 67% of MetaCyc's reactions are unique — but never said who the other 33% are shared with, which is exactly the question integrating Rhea raises. The regions are well separated:
Circle areas are not proportional — three-set areas cannot be solved exactly — so the counts carry the quantity and the circles carry only membership. The caption says this rather than leaving a reader to assume proportionality. The 9,406 reactions in none of the three are labelled rather than silently dropped.
matplotlib-vennis not in the environment, so the panel is drawn directly with matplotlib patches — no new dependency for one panel.Fig 2A → a legend, and a colour that means something
The panel had four segments and identified two; the caption was doing the legend's job in prose. There is now a four-entry key.
On the colour scheme, the reviewer's parenthetical was well aimed. Both non-Marvin segments were grey, which visually merged two different things:
These are not two shades of one thing. The result now takes its own categorical slot; only the genuine absence stays grey.
On "barplot abuse"
Fair. Figure 1B stops being a bar, and the new 1D earns its bars by carrying a comparison rather than three numbers.
Figure 2B is the remaining weak panel — it carries three numbers (39%, 53%, 53%) that the text already states, and folding it into 2C would be the right call. It is deliberately not done here: removing it renumbers 2C→2B, which edits the same line of
M05that PR #297 rewrites, and a guaranteed conflict is not worth a cosmetic gain. Flagged as a follow-up once #297 lands.Reviewer 1, comment 1 — where new information is still arriving
The reviewer's hypothesis is right and now quantified. Only 41 of the 12,261 reactions added since 2020 fall in a central-carbon or energy-metabolism class. New panel 1D shows the eight MetaCyc classes that gained most, against what was already held:
Named pathways make it concrete: mycolate biosynthesis (57), arachidonate metabolites (37), colibactin (34), apratoxin A, platensimycin, acylsucrose, H. pylori O-antigen. Mycobacterial cell wall, a bacterial genotoxin, eicosanoid signalling, two antibiotics.
A data gap this exposed
Building the panel required a join the released files cannot currently do.
Unique_ModelSEED_Reaction_Pathways.txtstops atrxn48568andUnique_ModelSEED_Reaction_ECs.txtatrxn48573, both just below the 2020 boundary (rxn48575) — so not one of the 12,261 post-2020 reactions carries a pathway or EC annotation, in the alias files or in the per-reaction records. The annotation step was never re-run over the MetaCyc and Rhea intake.figure_commonrebuilds the mapping from the shipped MetaCyc source table instead, putting both eras through the same one-level parent join so the two bars are comparable — reading the pre-2020 side out of the alias file would compare a full ancestor closure against a direct-parent lookup, which is meaningless. The underlying annotation gap is a data defect and gets its own PR; this panel does not depend on it.Two traps worth recording for whoever does that PR:
MetaCyc_pathways.tsvmixes individual pathways and ontology classes in oneidcolumn (Degradation,BiosynthesisandEnergy-Metabolismare all ids), so treating every id as a pathway finds zero classes; andSuper-Pathwaysis organisational rather than biological and would otherwise top the chart.Three things neither reviewer raised
1.
M12cites a Figure 3 panel that does not exist\ref{fig:direction}D— Figure 3 has A, B and C. Checking the whole manuscript turned up something worse: panels 3B and 3C were never cited by letter anywhere, so the only references into Figure 3 wereAand a panel that isn't there. The grading sentence now points at B and C, which are the two panels that actually show what it describes.2. "55% growth in reactions" is not reproducible on any single population
The 55% is the third row: 2026's totals against a 2020 baseline with its 7,577 obsolete rows removed. The compound figure is the first row and is correct.
This PR quotes 28%, keeping the whole paper on one population — the abstract's "~56,000 reactions", Figure 2A's 56,002, M12's 25,855 and Supplementary Table S1 are all all-records figures, and Figure 1's source counts (which apply no obsolete filter — verified in
_reaction_sources()) become consistent rather than wrong.This is a corresponding-author call, flagged rather than settled. Live-vs-live gives a tidier 34%/34%, but adopting it means converting every other number in the paper, including the abstract's headline. Say the word and I will do that instead.
(A correction on my own part: I first reported that the tree no longer reproduced the manuscript's grade counts — 27,355 graded against 33,099, 365 anchors against 806 — and attributed it to the September rebuilds. That was my own obsolete filter, not a regression. On the manuscript's basis the tree reproduces Supplementary Table S1 exactly, 806 anchors included, matching the
stereo_exactrows inopentecr_comparison.csv. Nothing needs re-measuring.)3. No figure in this paper could be regenerated
figure_common.pyraisedFileNotFoundErroron import in a clean checkout._silver_sigma_band()read two intermediates underresults/thermo_grades/that are.gitignore'd (Biochemistry/Thermodynamics/SourceGrading/.gitignoreline 12) and absent, andgrade_thermo_sources.pycannot rebuild them — it dies onValueError: Encountered all NA valuesunder the pinned pandas.It now derives the same band from files that ship:
best_gradefrom the committedreaction_grades.tsv, and each source's σ from the releasedBiochemistry/reaction_*.json, which is where the published uncertainty lives anyway. It reproduces the published band exactly — eQuilibrator 0.27–1.81, group contribution 1.62–25.12, dGPredictor 2.09–38.84 — once the sentinels the Results section already documents are excluded. Leaving them in put eQuilibrator's p95 at 23,901 kcal mol⁻¹, which is how the bug announced itself.Also included
analysis/pathway_distribution_of_growth.pyandanalysis/review_transport_and_llm.py— the two scripts behind every number in this PR series, registered inanalysis/README.md. Both read onlyBiochemistry/and this repository's git history, so unlike their siblings they need noEQUILIBRATOR_DIR.reviewer_response_plan.md— every reviewer comment mapped to its origin in the manuscript, the measurement, and the change.Verification
latexmk -pdfclean formain.texandsupplementary.tex; zero undefined references or citations.\lastpage{6}and the combined count need one pass after all four merge.🤖 Generated with Claude Code
Merge note (added after the live-basis conversion)
This branch and #297 both touch
M13_results_mapping.tex, which has a single-line body, so git cannot auto-merge them:\paragraph{Reconstruction scope.}after the atom-mapping sentenceThe resolution is to keep both, in this order:
No other pair of branches in this series conflicts — verified with
git merge-treeacross all four.Review pass (2026-09-23)
Six defects found and fixed, two substantive:
_pathway_classesiterates the alias file, which has no obsolete flag, and was the one loaderLIVE_ONLYdidn't reach — antibiotics read 824 where the live count is 740. The table above is corrected; "added" bars are unchanged (every addition is live).figure_data_sigma.json, a static all-records cache with no generator. Its drawn medians (0.63 / 10.41 / 17.01) contradicted M11's live text (0.62 / 10.83 / 17.56). Now derived from the records; cache deleted; medians match the text exactly.cover_letter.mdstill said 56,000._growthdocstring called the live-basis 2020 counts "wrong" (they differ from all-records by exactly the 34 obsolete compounds).Known and deliberately not fixed here, for the authors: S02's "~34,400 of ~56,000 reactions (61%) contain a magnesium-binding participant" is all-records and needs the eQuilibrator cache's 510-compound list to recompute. And M12's "eQuilibrator resolves 83.3% of the reactions it covers" uses a denominator that includes its refusal sentinels (20,890) while the "~17,900" two sentences later excludes them — a pre-existing inconsistency in the draft; with sentinels excluded eQuilibrator resolves 97.4%, which changes the narrative, so it is flagged rather than changed.