Jake update

README.md

@@ -7,10 +7,11 @@
 MolDscript is a Python workflow that converts Density Functional Theory (DFT) and related quantum chemistry outputs into descriptor tables ready for machine learning or benchmarking. It wraps `cclib`, `RDKit`, `DBSTEP`, and `pandas` to align Gaussian, ORCA, and xTB calculations and write consistent molecule-, bond-, and atom-level CSV files.
 
 ## Highlights
-- Parse optimization, single-point, NBO, NMR, charge, FMO, and Fukui calculations without manual file editing.
-- Match conformer ensembles, apply SMARTS-based substructure filters, and compute DBSTEP buried volumes on demand.
-- Generate ensembles (Boltzmann weighted, min/mnax within population windows, lowest-energy snapshots) in a single run.
-- Emit descriptor CSVs alongside module logs (`MOLDSCRIPT_*.dat`) for traceability.
+- Parse optimization, single-point, NBO, NMR, charge, FMO, and Fukui calculations without manual file editing.
+- Read CREST and ORCA GOAT multi-conformer XYZ files directly and summarize steric variation without requiring a higher-level calculation.
+- Match quantum-chemistry conformer ensembles, apply SMARTS-based substructure filters, and compute DBSTEP buried volumes on demand.
+- Generate ensembles (Boltzmann weighted, min/max within population windows, lowest-energy snapshots) in a single run.
+- Emit descriptor CSVs alongside a single run audit log (`MOLDSCRIPT.dat`) for traceability.
 
 ## Installation
 ```shell
@@ -31,28 +32,73 @@ python -m moldscript \
   --nbo calculations/nbo  
 
 ```
-Prefer storing options in a key:value text file? Use `--varfile inputs.txt`; command-line flags override values loaded from the file.
+Prefer storing options in a key:value text file? Use `--varfile inputs.txt`; command-line flags override values loaded from the file.
+
+For a geometry-only conformer ensemble:
+
+```shell
+python -m moldscript \
+  --ensemble conformer_search \
+  --ensemble_radii "[3.5]"
+```
+
+Directory input recognizes CREST `crest_conformers.xyz` /
+`*_crest_conformers.xyz` files and ORCA GOAT `*.finalensemble.xyz` files. Any
+equivalent multi-frame XYZ filename can be passed directly. Comment-line
+energies are interpreted as Hartree; the minimum is retained internally but is
+not added as an ensemble descriptor.
+
+Only compact steric summaries are added to `molecule_level.csv`: min, max, and
+range for mass-weighted radius of gyration and relative shape anisotropy.
+Buried volume is instead an atom-level descriptor: every atom is used as a
+center, and every requested radius adds the minimum, maximum,
+Boltzmann-weighted mean, and lowest-energy-conformer `%Vbur` to
+`atom_level.csv`. The Boltzmann mean uses `--temp` (298.15 K by default).
+
+An ensemble-only run with buried volume writes `molecule_level.csv` and
+`atom_level.csv`, but no bond table. When quantum outputs are also supplied,
+the ensemble descriptors are merged into their normal molecule and atom
+tables, and the normal bond table is retained. No raw conformer table or
+separate ensemble results folder is created. Buried volumes use DBSTEP's
+Bondi-radius table, molDscript's existing 1.17 scale factor, a 0.25 Angstrom
+grid, and non-hydrogen occupancy atoms by default.
 
 ## Core Inputs & Flags
-- `--opt PATH` (required) - baseline optimization files and conformer metadata.
+- `--opt PATH` - baseline optimization files and conformer metadata for quantum-output workflows; not required for single-point-only or ensemble-only analysis.
 - `--spc PATH` - single-point energies that replace optimization SCF energies.
 - `--nbo`, `--nmr`, `--charges`, `--fmo` PATH - add module-specific descriptors; pair with `--suffix_*` to specify filename tokens specific to calculation type (required for proper comformer matching). 
 - `--fukui_neutral`, `--fukui_reduced`, `--fukui_oxidized` PATH - supply all three charge states for vertical IE/EA and condensed Fukui functions. Again, pair with `--suffix_*` for proper conformer matching.
-- `--substructure SMARTS` - limit atom/bond descriptors to a SMARTS match; combine with `--volume` or `--vall` and optional `--radius` list for DBSTEP buried volumes.
-- `--boltz`, `--min_max`, `--lowe` - compute Boltzmann-weighted averages, min/max/range tables (using `--cut`), and lowest-energy snapshots. Adjust `--temp` (K) as needed.
-- `--output PREFIX` - prepend every generated filename; append a slash to target a directory. Use `--no_mol`, `--no_atom`, `--no_bond`, or `--no_bond_filter` to tailor CSV output.
+- `--substructure SMARTS` - limit atom/bond descriptors to a SMARTS match; combine with `--volume` or `--vall` and optional `--radius` list for DBSTEP buried volumes.
+- `--ensemble PATH` - analyze a multi-frame `.xyz` file, or recursively find standard CREST and GOAT ensemble files below a directory. This can be used alone or alongside quantum-output modules.
+- `--suffix_ensemble TAG` - remove a trailing filename tag when matching an ensemble to existing quantum-output molecule keys.
+- `--ensemble_radii "[3.5]"` - sphere radii (Angstrom) for atom-level buried-volume summaries; every atom is evaluated as the center. Set `[]` to omit this comparatively expensive descriptor.
+- `--ensemble_grid 0.25` - buried-volume voxel spacing in Angstrom.
+- `--ensemble_include_h` - allow hydrogen atoms to contribute to buried-volume occupancy. Hydrogen atoms still receive centered buried-volume descriptors when this flag is omitted.
+- `--ensemble_exclude "[1, 2]"` - additional 1-based atoms that cannot contribute to buried-volume occupancy. Excluded atoms still receive their own centered descriptors.
+
+Per-atom ensemble descriptors require the same atom identity and ordering in
+every XYZ frame, including among atoms of the same element. Changes in element
+order are rejected. A permutation of two same-element atoms is usually
+undetectable from XYZ element labels alone, but remains unsupported because it
+mixes atom identities across conformers.
+- `--boltz`, `--min_max`, `--lowe` - compute Boltzmann-weighted averages, min/max/range tables (using `--cut`), and lowest-energy snapshots for quantum-chemistry conformer rows. These reducers are ignored for ensemble-only XYZ input because its steric ranges are already summarized internally. Adjust `--temp` (K) as needed.
+- `--temp FLOAT` - temperature in Kelvin for standard conformer reducers and the atom-level ensemble `%Vbur` Boltzmann mean.
+- `--output PREFIX` - prepend every generated filename; append a slash to target a directory. Use `--no_mol`, `--no_atom`, `--no_bond`, or `--no_bond_filter` to tailor CSV output.
+- `--workers N` - parse independent quantum output files in parallel. Start with a modest value such as `--workers 4` for large batches, then increase if memory use is acceptable.
+- `--write_args arguments.txt` - save the effective options for the current run as a reusable `--varfile`.
 
 ## Output Artefacts
 - `molecule_level.csv`, `bond_level.csv`, `atom_level.csv` - aligned descriptors per calculation, bond pair, or atom.
-- `ensemble_*.csv`, `boltzmann_weights.csv` - created when `--boltz` is enabled.
-- `min_max_range_*.csv`, `lowest_energy_*.csv` - created when `--min_max` or `--lowe` are requested.
-- `MOLDSCRIPT_*.dat` - per-module logs capturing provenance and CPU-time summaries.
+- `ensemble_*.csv`, `boltzmann_weights.csv` - created when `--boltz` is enabled.
+- `min_max_range_*.csv`, `lowest_energy_*.csv` - created when `--min_max` or `--lowe` are requested.
+- `--ensemble` adds radius-of-gyration and shape min/max/range columns to `molecule_level.csv`, and per-atom buried-volume min/max/Boltzmann-mean/lowest-energy columns to `atom_level.csv` when radii are requested. It does not create additional ensemble-specific CSV files or directories.
+- `MOLDSCRIPT.dat` - a single run log capturing provenance, parsed files, module sections, and CPU-time summaries.
 
 ## Documentation
 The Read the Docs site (coming soon) will provide the full user guide: [https://moldscript.readthedocs.io](https://moldscript.readthedocs.io)
 
 ## Dependencies
-Key Python dependencies include `pandas`, `cclib` (latest GitHub version for the most up-to-date package compatability), `dbstep`, `rdkit`, `networkx`, `numpy`, and `periodictable`.
+Key Python dependencies include `pandas`, `cclib` (latest GitHub version for the most up-to-date package compatability), `dbstep`, `rdkit`, `networkx`, `numpy`, `periodictable`, `rich`, and `tqdm`.
 
 ## Supported Quantum Packages
 - Gaussian

docs/source/README.rst

@@ -18,17 +18,18 @@ Key Capabilities
 ----------------
 - Automate ingestion of optimization, single-point, NBO, NMR, Fukui, charge, and frontier-orbital calculations without hand editing.
 - Merge descriptors across conformers and calculation types into aligned CSV datasets.
-- Restrict analysis to user-defined SMARTS substructures and optionally compute DBSTEP buried volumes.
+- Restrict analysis to user-defined SMARTS substructures and optionally compute DBSTEP buried volumes.
+- Summarize steric variation in CREST or ORCA GOAT multi-frame XYZ ensembles without creating raw conformer tables.
 - Generate ensemble statistics such as Boltzmann-weighted averages, population windows, and lowest-energy snapshots.
-- Produce audit logs (``MOLDSCRIPT_*.dat``) alongside descriptor files for reproducibility.
+- Produce a single audit log (``MOLDSCRIPT.dat``) alongside descriptor files for reproducibility.
 
 Installation
 ------------
 1. Clone the repository: ``git clone https://github.com/patonlab/molDscript.git``.
 2. (Optional) create and activate a dedicated environment.
 3. Install the package from the repository root: ``pip install -e .`` (or ``pip install .`` for a standard install).
 
-Required Python dependencies are declared in ``setup.py`` and include ``pandas>=2.0.2``, ``cclib`` (latest from GitHub), ``dbstep``, ``rdkit``, ``networkx``, ``numpy``, and ``periodictable``. Install RDKit and Open Babel via conda-forge when pip wheels are not available:
+Required Python dependencies are declared in ``setup.py`` and include ``pandas>=2.0.2``, ``cclib`` (latest from GitHub), ``dbstep``, ``rdkit``, ``networkx``, ``numpy``, ``periodictable``, ``rich``, and ``tqdm``. Install RDKit and Open Babel via conda-forge when pip wheels are not available:
 
 .. code-block:: shell
 
@@ -95,16 +96,21 @@ CLI Options
 -----------
 The following options can be combined as needed. Paths can be absolute or relative.
 
-Core inputs
-^^^^^^^^^^^
-``--opt PATH``
-  Directory containing optimization log/out files. Required; provides the structural baseline, SCF energies, and conformer metadata.
+Core inputs
+^^^^^^^^^^^
+``--opt PATH``
+  Directory containing optimization log/out files. Provides the structural baseline, SCF energies, and conformer metadata when used.
 
 ``--spc PATH``
   Directory of single-point energy calculations. Overrides the SCF energies captured during the optimization step.
 
-``--suffix_opt TEXT`` / ``--suffix_spc TEXT``
-  Trailing text to strip from filenames before matching stems (for example, ``_opt`` or ``_spc``).
+``--suffix_opt TEXT`` / ``--suffix_spc TEXT``
+  Trailing text to strip from filenames before matching stems (for example, ``_opt`` or ``_spc``).
+
+``--ensemble PATH``
+  A multi-frame XYZ file, or a directory containing standard CREST
+  ``crest_conformers.xyz`` / ``*_crest_conformers.xyz`` files or ORCA GOAT
+  ``*.finalensemble.xyz`` files. Nonstandard XYZ names can be passed directly.
 
 Property modules
 ^^^^^^^^^^^^^^^^
@@ -137,11 +143,58 @@ Substructure and sterics
 ``--vall``
   Compute DBSTEP buried volumes for every atom in each molecule (ignores ``--substructure``).
 
-``--radius VALUE`` or ``--radius "[3.0, 3.5]"``
-  Probe radii (in Angstrom) passed to DBSTEP. Accepts a single float or a Python-like list string.
-
-Ensemble statistics
-^^^^^^^^^^^^^^^^^^^
+``--radius VALUE`` or ``--radius "[3.0, 3.5]"``
+  Probe radii (in Angstrom) passed to DBSTEP. Accepts a single float or a Python-like list string.
+
+XYZ ensemble sterics
+^^^^^^^^^^^^^^^^^^^^
+``--ensemble_radii "[3.5]"``
+  Sphere radii used for atom-level percent-buried-volume descriptors. Every
+  atom is evaluated as a center. For each atom and radius, molDscript reports
+  the minimum, maximum, Boltzmann-weighted mean, and value from the
+  lowest-energy conformer. Use ``[]`` to omit buried volume while retaining
+  molecule-level size and shape summaries.
+
+``--ensemble_grid 0.25``
+  Buried-volume voxel spacing in Angstrom. The default is 0.25 Angstrom.
+
+``--ensemble_include_h`` / ``--ensemble_exclude "[1, 2]"``
+  Control which atoms contribute to buried-volume occupancy.
+  ``--ensemble_include_h`` includes hydrogens as occupancy atoms; otherwise
+  they are omitted from occupancy. ``--ensemble_exclude`` removes additional
+  1-based atoms from occupancy. These controls do not remove atom rows or
+  centers: every atom, including a hydrogen or excluded atom, still receives
+  centered buried-volume descriptors.
+
+``--suffix_ensemble TEXT``
+  Strip a trailing XYZ filename tag when matching ensemble data to existing
+  quantum-output molecule keys.
+
+Ensemble XYZ input adds molecule-level min/max/range descriptors only for
+mass-weighted radius of gyration and relative shape anisotropy; buried volume
+is not a molecule-level descriptor. Requested buried volumes are written to
+``atom_level.csv`` as the min, max, Boltzmann mean, and
+lowest-energy-conformer value for every atom and radius. The Boltzmann mean
+uses the temperature supplied by ``--temp``.
+
+A standalone ensemble run with buried volume writes ``molecule_level.csv`` and
+``atom_level.csv`` but no bond table. With ``--ensemble_radii "[]"``, it
+writes only the molecule table. When combined with quantum outputs, ensemble
+values are merged into the existing molecule and atom tables and the normal
+bond table remains available. No raw conformer table, separate results folder,
+or other ensemble-specific CSV is created.
+
+These atom-level summaries require stable atom identity and ordering in every
+frame, including among same-element atoms. Different element ordering is
+rejected. Same-element permutations are usually impossible to detect from XYZ
+element labels alone, but they are unsupported because they mix atom
+identities across conformers.
+
+Ensemble statistics
+^^^^^^^^^^^^^^^^^^^
+These reducers operate on quantum-chemistry conformer rows and are ignored for
+ensemble-only XYZ input.
+
 ``--boltz``
   Produce Boltzmann-weighted averages at the temperature specified by ``--temp`` (default 298.15 K), along with the conformer weights.
 
@@ -151,8 +204,9 @@ Ensemble statistics
 ``--lowe``
   Keep only the lowest-energy conformer for each molecule.
 
-``--temp FLOAT``
-  Temperature (in Kelvin) used for Boltzmann and min/max population analyses.
+``--temp FLOAT``
+  Temperature (in Kelvin) used for Boltzmann and min/max population analyses,
+  as well as the atom-level ensemble buried-volume Boltzmann mean.
 
 ``--cut FLOAT``
   Cumulative Boltzmann weight cutoff for ``--min_max`` (expressed as the retained population fraction).
@@ -179,8 +233,12 @@ Running molDscript creates the following artefacts in the working directory (or
 - ``bond_level.csv`` - pairwise descriptors filtered by the default bond-order/length thresholds.
 - ``atom_level.csv`` - atomic descriptors including charges, Fukui indices, NMR shielding, and buried volumes when requested.
 - ``boltzmann_weights.csv`` plus ``ensemble_*.csv`` tables when ``--boltz`` is enabled.
-- ``min_max_range_*.csv`` tables when ``--min_max`` is enabled and ``lowest_energy_*.csv`` tables when ``--lowe`` is requested.
-- Module logs named ``MOLDSCRIPT_<MODULE>.dat`` that document parsing steps and CPU-time summaries.
+- ``min_max_range_*.csv`` tables when ``--min_max`` is enabled and ``lowest_energy_*.csv`` tables when ``--lowe`` is requested.
+- ``--ensemble`` writes radius-of-gyration and shape summaries directly into
+  ``molecule_level.csv`` and requested per-atom buried-volume summaries into
+  ``atom_level.csv``. It creates no additional ensemble-specific table or
+  directory.
+- ``MOLDSCRIPT.dat`` - a single run log that documents command provenance, parsed files, module sections, and CPU-time summaries.
 
 Each run also reports the cumulative CPU time associated with the parsed quantum chemistry jobs.