Skip to content

Commit cd054e3

Browse files
andiwandclaude
andcommitted
feat(csv)!: open a csv as a spreadsheet
A csv rendered as a line list, because it was a text file with no decoder. It is a table, and the library already knows how to render tables — the whole job is to expose one. `CsvDocument` is a one-sheet spreadsheet. The generic renderer walks it, and every binding gets table rendering without a line of format-specific code. Cells are not elements. The registry pattern the root `AGENTS.md` prescribes costs an entry per element, and a sheet has one per cell, so a large file would pay for millions before any were looked at. `ElementIdentifier` is 64 bits, so an id *is* the coordinate — kind, row, column packed in — and the adapter decodes rather than looks up. The consequence to respect is that a sheet's cells are not reachable by walking; they come from `sheet_cell(column, row)`, which is how the renderer asks for them anyway. Everything reaches the data through `cell` and `dimensions`, so the index and window work in `PLAN.md` can move in behind them later without the adapter noticing. The sheet is rectangular even where the file is not: a short row pads, a long one widens. That is the counterpart to detection having stopped rejecting ragged files. `FileCategory::text` becomes `document`, so `is_text_file()` is now false for a csv and `is_document_file()` true. Reading one as text stays available by opening it as `FileType::text_file`, which is also the escape hatch when detection was wrong about it being a csv at all. An encoding `internal/encoding` cannot decode has no document: cell text has to be UTF-8 by the time a binding sees it. The text path stays open to those files. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01QSgWdLTSLCWDeFvbVwZDVU
1 parent 1c21594 commit cd054e3

14 files changed

Lines changed: 1048 additions & 15 deletions

File tree

CMakeLists.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,7 @@ set(ODR_SOURCE_FILES
130130
"src/odr/internal/crypto/crypto_argon2.cpp"
131131
"src/odr/internal/crypto/crypto_util.cpp"
132132

133+
"src/odr/internal/csv/csv_document.cpp"
133134
"src/odr/internal/csv/csv_file.cpp"
134135
"src/odr/internal/csv/csv_util.cpp"
135136

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{
2-
"fileCategory": "text",
2+
"documentType": "spreadsheet",
3+
"fileCategory": "document",
34
"fileType": "csv",
45
"isEncrypted": false
56
}

src/odr/file.cpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -315,6 +315,10 @@ CsvFile CsvFile::from_file(const File &file, const CsvOptions &options,
315315
CsvFile::CsvFile(std::shared_ptr<internal::abstract::CsvFile> impl)
316316
: DecodedFile(impl), m_impl{std::move(impl)} {}
317317

318+
DocumentType CsvFile::document_type() const { return m_impl->document_type(); }
319+
320+
Document CsvFile::document() const { return Document(m_impl->document()); }
321+
318322
CsvOptions CsvFile::options() const { return m_impl->options(); }
319323

320324
CsvFile CsvFile::with_options(const CsvOptions &options) const {

src/odr/file.hpp

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -441,6 +441,12 @@ class CsvFile final : public DecodedFile {
441441

442442
explicit CsvFile(std::shared_ptr<internal::abstract::CsvFile>);
443443

444+
/// @brief Always @ref DocumentType::spreadsheet.
445+
[[nodiscard]] DocumentType document_type() const;
446+
/// @brief The csv as a one-sheet spreadsheet.
447+
/// @throws UnsupportedTextEncoding if the encoding cannot be decoded.
448+
[[nodiscard]] Document document() const;
449+
444450
/// @brief The options in use, every field resolved.
445451
[[nodiscard]] CsvOptions options() const;
446452

src/odr/internal/abstract/file.hpp

Lines changed: 16 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -67,16 +67,6 @@ class TextFile : public DecodedFile {
6767
[[nodiscard]] virtual TextEncoding encoding() const noexcept = 0;
6868
};
6969

70-
class CsvFile : public TextFile {
71-
public:
72-
/// The options in use, every field resolved.
73-
[[nodiscard]] virtual CsvOptions options() const = 0;
74-
75-
/// The same file read with @p options.
76-
[[nodiscard]] virtual std::shared_ptr<CsvFile>
77-
with_options(const CsvOptions &options) const = 0;
78-
};
79-
8070
class ImageFile : public DecodedFile {
8171
public:
8272
[[nodiscard]] FileCategory file_category() const noexcept final {
@@ -106,6 +96,22 @@ class DocumentFile : public DecodedFile {
10696
[[nodiscard]] virtual std::shared_ptr<Document> document() const = 0;
10797
};
10898

99+
/// A csv is a document, not a text file: it decodes to a one-sheet
100+
/// spreadsheet. Reading it as text is still available by opening it as
101+
/// @ref FileType::text_file.
102+
class CsvFile : public DocumentFile {
103+
public:
104+
/// The encoding the bytes were decoded with.
105+
[[nodiscard]] virtual TextEncoding encoding() const noexcept = 0;
106+
107+
/// The options in use, every field resolved.
108+
[[nodiscard]] virtual CsvOptions options() const = 0;
109+
110+
/// The same file read with @p options.
111+
[[nodiscard]] virtual std::shared_ptr<CsvFile>
112+
with_options(const CsvOptions &options) const = 0;
113+
};
114+
109115
class PdfFile : public DecodedFile {
110116
public:
111117
[[nodiscard]] FileType file_type() const noexcept final {

src/odr/internal/csv/AGENTS.md

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
# AGENTS.md — `internal/csv`
2+
3+
Read the root [`AGENTS.md`](../../../../AGENTS.md) first. This file covers what
4+
csv does differently, and why. The staged plan is in [`PLAN.md`](PLAN.md).
5+
6+
## Cells are not elements
7+
8+
The root `AGENTS.md` prescribes an `ElementRegistry`: a flat `std::vector` of
9+
elements, id = index + 1. **Csv does not use one**, deliberately.
10+
11+
A registry costs an entry per element. A sheet has one per *cell*, so a file
12+
with a million rows would cost millions of entries before a single one is
13+
looked at — and `spreadsheet_limit` means the renderer will ask for ten
14+
thousand rows of them at most.
15+
16+
`ElementIdentifier` is a `std::uint64_t`, which is room to spare:
17+
18+
```
19+
63..61 kind root | sheet | cell | text
20+
60..24 row 37 bits
21+
23..0 column 24 bits
22+
```
23+
24+
So an id *is* the coordinate, and the adapter decodes rather than looks up.
25+
`null_element_id` is zero, so no kind may be.
26+
27+
The consequence to respect: **a sheet's cells are not reachable by walking**.
28+
`element_first_child` of a sheet is `null_element_id`; cells come from
29+
`SheetAdapter::sheet_cell(column, row)`, which is how the renderer asks for
30+
them anyway (`html/document_element.cpp:163`).
31+
32+
## Everything goes through `cell` and `dimensions`
33+
34+
`CsvDocument` holds the whole file decoded, and the adapter never touches that
35+
storage — it calls `cell(column, row)` and `dimensions()`. That is the seam for
36+
the streaming work in `PLAN.md`'s last stage: an index and a window can move in
37+
behind those two without the adapter noticing.
38+
39+
## Detection rejects; the parser does not
40+
41+
Two jobs, two places, and mixing them is the mistake this module already made
42+
once.
43+
44+
- `probe` is detection. It scores a bounded sample and may say "not a csv".
45+
Its rules — at least two columns, no dangling quote in a complete file — are
46+
*heuristics for recognising an unknown file*, not statements about validity.
47+
- `RecordReader` is parsing. Given a separator it is total: ragged rows, one
48+
column, an empty file and a truncated quoted field all read as some csv.
49+
50+
So a one-column csv is perfectly legitimate and `CsvOptions{.separator = ','}`
51+
reads it. `NoCsvFile` is a detection failure only. An incoherent dialect — a
52+
separator equal to the quote, a line break as a separator — is
53+
`std::invalid_argument`, a caller mistake rather than bad input.
54+
55+
## A csv is a document
56+
57+
`FileCategory::document`, `DocumentType::spreadsheet` — so `is_text_file()` is
58+
false for a csv and `is_document_file()` is true. Reading one as text is still
59+
available by opening it as `FileType::text_file`, which is also the escape
60+
hatch when detection was wrong about it being a csv at all.
61+
62+
Text has to be UTF-8 by the time it reaches a cell: `Text::content()` returns
63+
`std::string` and every binding treats it as UTF-8. That is why an encoding
64+
`internal/encoding` cannot decode has no document at all, while the *text*
65+
rendering path stays open to it.

0 commit comments

Comments
 (0)