libleptris is a C99 library providing fast XML 1.0 parsing, complete XPath 1.0 evaluation, event-based SAX processing, and document canonicalization (C14N). It is designed for low-latency, memory-efficient workloads where predictable performance and safe memory handling matter.
-
XML 1.0 parsing with UTF-8 validation and optional encoding conversion (UTF-16, ISO-8859, Shift-JIS, EBCDIC, and others via iconv). Zero-copy text nodes — borrowed views into the input buffer, no per-node content copy (TODO 115).
-
XPath 1.0 engine implementing all 13 axes, 27 functions, and 15 operators. Bytecode VM for compile-once-eval-many dispatch (TODO 120).
-
Streaming SAX parser with an explicit state machine — events emit as chunks arrive, memory bounded by nesting depth, not document size (TODO 116). All SAX parsing routes through one state machine; the legacy recursive parser has been removed (~840 lines deleted).
-
XInclude 1.0 with ownership-transfer splice — included documents are moved (not deep-copied) into the parent tree. Cycle detection via ancestor-URI tracking (TODO 117).
-
DTD validation with content-model memoization — repeated element types with the same children signature skip the matcher on subsequent calls (TODO 119). Parameter entities, INCLUDE/IGNORE conditional sections, ENTITY/ENTITIES unparsed-entity attribute checking, and external subsets via
leptris_document_get_dtd+leptris_dtd_parse_external_subset(application-supplied I/O). -
Canonical XML (C14N) for digital signatures and cryptographic hashing.
-
Pool-based memory model — every allocation reachable from a document is released in a single
leptris_document_freecall. Zero leaks across the test suite. -
Compact-pointer architecture — tree edges stored as int32_t byte offsets with overflow-table fallback for macOS ASLR (TODO 121).
-
Recursion depth guard — deeply nested input is rejected with a parse error rather than crashing.
-
Per-document strict mode — strict and lenient parsing can coexist in the same thread.
-
Vtable-based dispatch — adding a new node type is purely additive; no switches to edit.
-
CLI tool —
leptris parse,leptris xpath,leptris format,leptris versionfor command-line XML processing. -
Ruby FFI binding —
Leptris::Document.parse, XPath, serialize, and event-driven SAX (one-shot + incremental streaming) via theffigem. No C extension compilation needed. See Ruby binding. -
Python binding —
leptrisviacffi(ABI mode, singlecdefmirroring the public headers):Document.parse, element navigation, typed XPath results, serialize. Shipped from its own repository, leptris/leptris-py. -
Zero required runtime dependencies — utf8proc and iconv are optional features, not prerequisites.
-
Stable C ABI with a documented FFI contract; bindings shipped for Ruby and Python (Rust planned). See FFI Design.
| Use libleptris when | Consider alternatives when |
|---|---|
You need a C library with a small footprint and no required runtime dependencies. |
You need full XML Schema 1.1 validation. |
You parse documents that may not fit in memory (SAX streaming). |
You need schema-aware, streaming or packaged XSLT 3.0. |
You need XSLT transformation — 1.0 complete, the 3.0 core ships (see XSLT transformation). |
You need XQuery 1.0 (XPath only here). |
You need deterministic memory usage and zero leaks in normal operation. |
You need a schema-validated or event-correlated read model. |
You want to embed XML processing in another language via FFI. |
You’re already on a platform with libxml2 + bindings you trust. |
==
#include <leptris.h>
#include <stdio.h>
#include <string.h>
int main(void) {
const char* xml = "<root><item>hello</item></root>";
LeptrisStatus status = LEPTRIS_OK;
LeptrisDocument doc = leptris_parse_string(xml, strlen(xml), &status);
if (!doc) {
fprintf(stderr, "parse failed: %d\n", status);
return 1;
}
LeptrisElement root = leptris_document_root(doc);
printf("root element: %s\n", leptris_element_name(root));
LeptrisXPathResult items = leptris_xpath_eval(doc, NULL, "//item");
printf("item count: %zu\n", leptris_xpath_result_count(items));
leptris_xpath_result_free(items);
leptris_document_free(doc); /* releases the entire pool */
return 0;
}Compile and run:
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build
./build/cli/leptris parse 'fixtures/basic.xml'
./build/cli/leptris xpath 'fixtures/basic.xml' 'count(//item)'cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build
sudo cmake --install build --prefix /usr/localAfter install, the library is discoverable via find_package(leptris):
cmake_minimum_required(VERSION 3.20)
project(myapp LANGUAGES C CXX)
find_package(leptris CONFIG REQUIRED)
target_link_libraries(myapp PRIVATE leptris::leptris)Or via pkg-config:
gcc myapp.c $(pkg-config --cflags --libs leptris)git clone https://github.com/microsoft/vcpkg
./vcpkg/vcpkg install leptrisSee vcpkg integration for the portfile template.
| Distro | Install command |
|---|---|
Alpine |
|
Debian/Ubuntu |
|
Homebrew |
|
MSYS2 (Windows) |
|
| Option | Default | Description |
|---|---|---|
|
|
Build the Google Test suite under |
|
|
Build the |
|
|
Build performance comparison targets (libxml2 / pugixml). |
|
|
Generate man pages from the AsciiDoc sources. |
|
|
UTF-8 validation via utf8proc. |
|
|
Encoding conversion via iconv (ISO-8859-1, Shift-JIS, etc.). |
|
|
Build with AddressSanitizer. |
|
|
Build the libFuzzer harness. |
|
|
Generate Doxygen API docs. |
cmake -B build-asan -S . -DLEPTRIS_ENABLE_ASAN=ON -DBUILD_TESTING=ON
cmake --build build-asan
ASAN_OPTIONS=detect_leaks=1 ctest --test-dir build-asanbrew install llvm # macOS
export CC=/opt/homebrew/opt/llvm/bin/clang
cmake -B build-fuzz -S . -DLEPTRIS_ENABLE_FUZZING=ON
cmake --build build-fuzz --target fuzz_parse
./build-fuzz/fuzz_parse -max_total_time=600 corpus/brew install doxygen
cmake -B build -S . -DLEPTRIS_BUILD_DOCS=ON
cmake --build build --target docs
open build/docs/api-generated/html/index.htmlThe library exposes a single import target:
target_link_libraries(your_app PRIVATE leptris::leptris)leptris_dep = dependency('leptris')
executable('your_app', 'main.c', dependencies: leptris_dep)leptris_parse_string is the entry point. It accepts a UTF-8 buffer
and a status output parameter:
LeptrisStatus status;
LeptrisDocument doc = leptris_parse_string(xml, strlen(xml), &status);
if (!doc) {
/* status is one of LEPTRIS_ERROR_PARSE, LEPTRIS_ERROR_MEMORY, ... */
}
/* Document is now a pool of nodes; no need to track them individually. */
/* Always release the document — the pool is destroyed too. */
leptris_document_free(doc);LeptrisXPathResult r = leptris_xpath_eval(doc, NULL, "//item[@price > 10]");
if (r) {
size_t n = leptris_xpath_result_count(r);
for (size_t i = 0; i < n; i++) {
LeptrisNodeRef node = leptris_xpath_result_node(r, i);
printf(" %s\n", leptris_node_name(node));
}
leptris_xpath_result_free(r);
}Supported: all 13 axes, all 27 functions, all 15 operators, full predicate syntax. See xpath-coverage for details.
One engine, four consumption models. Pick by how you want to pay:
| Model | API | Memory | Use when |
|---|---|---|---|
DOM (tree) |
|
Whole document |
You need random access, XPath, mutation, or serialization. |
The default; everything else is an optimization. |
SAX (push) |
|
Bounded by depth |
The engine calls YOU as it parses. Cheapest per event, but every |
callback crosses the FFI boundary (~1 µs each through bindings). |
StAX-style (pull) |
|
Bounded by the input slice |
YOU call the engine: |
demand. Same streaming guarantees as SAX with zero C→host |
callbacks — the binding-friendly form of streaming. |
Incremental (iterparse) |
|
Bounded by the largest subtree |
You want a TREE but not the whole document at once: each top-level |
SAX and pull share the same streaming core (set_streaming(1));
iterparse rides the pull API. File variants (_new_file) stream
from disk in bounded slices — no whole-document buffer.
Host-driven event loop over the streaming SAX core — no C→host callbacks (each costs ~a microsecond through FFI):
LeptrisPullParser p = leptris_pull_new(xml, len);
const LeptrisPullEvent* ev;
while ((ev = leptris_pull_next(p)) != NULL) {
if (ev->type == LEPTRIS_PULL_START_ELEMENT)
handle(leptris_pull_attr_count(p), ev->name);
if (ev->type == LEPTRIS_PULL_ERROR) break;
}
leptris_pull_free(p);Event strings live until the next leptris_pull_next call. Memory is
bounded by the internal input slice, not the document.
leptris_pull_new_file(path) streams the same events from disk —
huge documents parse with no whole-document buffer.
Bounded-memory tree iteration for huge documents
(TODO.bindings/02): each TOP-LEVEL child of the root is
materialized in its own pool and handed out when complete; the next
call releases it. Peak memory is bounded by the largest subtree, not
the document.
LeptrisIterparse it = leptris_iterparse_new(xml, len);
LeptrisElement e;
while ((e = leptris_iterparse_next(it)) != NULL)
process(e); /* valid until the next call */
leptris_iterparse_free(it);v1: element names are QNames as written (prefixes are not
re-resolved); use the DOM path when namespace URIs matter.
leptris_iterparse_new_file(path) iterates a document on disk with
memory bounded by the largest subtree.
/* Every element, in completion order (post-order: children before
* parents), each with its completed subtree attached. */
LeptrisIterparse it = leptris_iterparse_new_ex(
xml, len, LEPTRIS_ITERPARSE_FULL_DOCUMENT);
LeptrisElement e;
while ((e = leptris_iterparse_next(it)) != NULL) {
/* namespace context of THIS element, captured at parse time */
const char* uri = leptris_iterparse_ns_uri(it, "p"); /* or NULL
for the default namespace binding */
size_t nscopes = leptris_iterparse_ns_count(it); /* bulk FFI */
process(e, uri);
}
if (leptris_iterparse_error(it))
fprintf(stderr, "truncated: %s\n", leptris_iterparse_error(it));-
Full mode keeps the v1 memory discipline — a fresh pool per top-level child, released as you advance — and materializes the root in its own pool (it yields childless at document end).
-
Namespace bindings come from the streaming core’s prefix-mapping events; yielded elements carry resolved namespaces (
leptris_element_namespace), and the iterator snapshots the in-scope scope per yield. -
Truncated or malformed input stops iteration with a retrievable message instead of failing silently.
static void on_start(void* ud, const char* name, const char** attrs) {
fprintf(stderr, "<%s>\n", name);
}
LeptrisSAXHandler handler = {0};
handler.start_element = on_start;
leptris_sax_parse(xml, len, &handler, NULL);Callback SAX through an FFI binding pays one dispatch per event — the ffi gem’s generic callback machinery costs more per event than the parse itself (measured: leptris SAX slower than Nokogiri SAX through Ruby). The recorder buffers events C-side and hands them over in bulk:
LeptrisSaxRecorder rec = leptris_sax_recorder_new();
leptris_sax_recorder_feed(rec, chunk, chunk_len, is_final);
size_t n = 0, alen = 0;
const LeptrisSaxEventRecord* rs = leptris_sax_recorder_records(rec, &n);
const char* arena = leptris_sax_recorder_arena(rec, &alen);
/* iterate n fixed-size records; strings slice the arena by the
* record's offset+length fields (attributes: name\0value\0 pairs
* at attrs_off, attr_count pairs) */Each feed starts a fresh chunk (records and arena reset), so a
streaming host drains after every chunk and stays bounded. Event
semantics are identical to the callback API — the recorder is a
handler on the same streaming state machine. Callback count becomes
O(chunks), not O(events): 20 host calls for a 41,806-event document
(64 KB chunks).
LeptrisXslt xslt = leptris_xslt_parse(sheet, strlen(sheet));
char* out = leptris_xslt_apply_string(xslt, doc); /* owned */
leptris_free_string(out);
leptris_xslt_free(xslt);XSLT 1.0 is complete — every case of the libxslt regression suite
(205/205) matches byte-for-byte, including attribute sets,
xsl:number formats, keys, namespaces, decimal formats and DTD
attribute defaults. The XSLT 3.0 core ships and grows release by
release, each feature verified against Saxon-HE 12.7 before landing:
| Area | Supported |
|---|---|
Sequences |
|
Control |
|
Constructors |
|
Grouping |
|
Merging |
|
Strings |
|
Dynamic evaluation |
|
Secondary output |
|
Modes |
|
Accumulators (§18.2) |
|
The XPath engine inside carries the 3.1 composition core: let
bindings, the ! simple map with per-item focus, the ⇒ arrow
(every core and extension function serves it), and || string
concatenation, alongside current(), key(), document(),
format-number(), generate-id() and the EXSLT packs.
LeptrisSerializeOptions opts = { .indent = 2, .xml_declaration = 1 };
char* out = leptris_document_serialize(doc, &opts);
puts(out);
leptris_free_string(out);char* canonical = leptris_c14n_canonicalize(doc, LEPTRIS_C14N_1_0, 0);
fputs(canonical, stdout);
putchar('\n'); /* canonical output may not end with newline */
leptris_free_string(canonical);
leptris_free_string(canonical);TODO.bindings/03: compile once, evaluate many times — skips the
per-call expression hash and cache probe of leptris_xpath_eval.
LeptrisXPathCompiled c = leptris_xpath_compile("//book[@n > 3]");
LeptrisXPathResult r = leptris_xpath_compiled_eval(c, doc, NULL);
leptris_xpath_result_free(r);
leptris_xpath_compiled_free(c);Thread contract: the handle is immutable — any number of threads may
evaluate it concurrently (each against its own document); free it
only after the last evaluation returns. Context variants:
leptris_xpath_compiled_eval_ns (external namespace bindings) and
leptris_xpath_compiled_eval_vars ($var references) — compiled
counterparts of leptris_xpath_eval_ns / eval_with_vars_context.
leptris_node_children(parent, out, max) copies every child handle
in one call (elements, text, comments, CDATA, PIs) — out=NULL
returns the total across all kinds. leptris_document_serialize_into
/ leptris_element_serialize_into write into a caller buffer:
buf=NULL queries the needed size (including NUL); with capacity the
copy happens in the same call — the serialize + read + free FFI
pattern collapses to one call with zero library-side allocations.
TODO.bindings/05: scoped alternative to the thread-global
leptris_set_strict_mode / leptris_set_max_depth.
LeptrisParseOptions o = {LEPTRIS_PARSE_DEFAULT, /*strict*/ 1, /*depth*/ 4};
LeptrisDocument doc = leptris_parse_string_ex(xml, len, &o, &status);Options apply to that one call; the thread defaults are restored on return. Not reentrant (applied via thread-local state).
TODO.bindings/06: serialization output is ALWAYS UTF-8 — the body
is never transcoded (iconv is input-side only). The XML declaration
never lies: a document parsed as ISO-8859-1 serializes with
encoding="UTF-8" (the body was transcoded at parse time), and
requesting another output encoding is normalized to UTF-8.
serialize(parse(serialize(x))) is byte-stable.
Every byte the parser allocates that ends up referenced by a document
lives in the document’s pool. leptris_document_free destroys the
pool and releases everything in one call.
| Allocation | Where it lives |
|---|---|
Node structs (element, text, comment, CDATA, PI, doctype) |
Pool, allocated contiguously with content where possible. |
Node content strings |
Pool, contiguous with the struct (cache locality). |
Attribute names |
Pool hash table (interned; dedup across elements). |
Attribute values |
Pool, bypassing interning (attrs.xml regression fixed). |
DTD container + hash tables |
Pool, with DTD subsystem owned by the document. |
XPath intermediates |
Pool, freed at result destruction. |
For bindings: the C API has opaque handles. All freeing is
explicit. See the Memory: comment on each public function.
libleptris is thread-safe under the one-document-per-thread
contract (documented in TODO.concurrency/08):
-
Safe: any number of threads, each owning its own
LeptrisDocument— parse, evaluate, mutate, serialize, free independently. -
Safe: read-only sharing of one document across threads (parse once, evaluate concurrently from many threads).
-
Forbidden: mutating one document from two threads at once, or touching a document after another thread freed it.
Process-global state is either initialized at library load (SIMD dispatch, the XPath function registry) or mutex-guarded (the process-wide XPath AST/bytecode cache). Borrowed cache entries are reference-pinned for the duration of an evaluate, so eviction never frees an AST underneath a running thread.
Error reporting is channel-split:
-
leptris_last_error()— thread-local message of the most recent failure on the calling thread (e.g. a failedleptris_parse_string). -
leptris_document_last_error(doc)— per-document message from the last failed operation against that document (e.g. a failedleptris_xpath_eval); immune to concurrent activity on other documents.
Worker threads keep small per-thread caches (XPath free lists, the
root-map free list). C99 has no portable thread-exit hook, so a
thread that used libleptris and then exits retains those entries —
call leptris_thread_cleanup() from each worker just before it
exits to release them. Pooled, long-lived threads never need it.
The full pack is exercised by test/concurrency/test_concurrency.cpp
(parse/eval/serialize/free on four threads, thread-local error
isolation, per-document error slots), which is ThreadSanitizer-clean.
Opaque handles are pointer-sized — enforced at compile time:
_Static_assert(sizeof(LeptrisDocument) == sizeof(void*), "...");To pin enum values (bindings hard-code these):
ctest --test-dir build -R HeaderHygienelibleptris exposes a stable C ABI. Bindings:
-
Ruby — shipped. Uses
ffigem. See Ruby binding. -
Python — shipped as
leptrisviacffi(ABI mode), from leptris/leptris-py. -
Rust — shipped: crate
leptriswith safe wrappers, XPath, and SAX closures. See Rust binding.
require 'leptris'
doc = Leptris::Document.parse('<root><item id="1">hello</item></root>')
root = doc.root
puts root.name # => "root"
puts root.first_child_element['id'] # => "1"
puts doc.xpath('count(//item)') # => 1.0
doc.freeInstall: set LEPTRIS_LIB_PATH to your libleptris.dylib / .so, or
install libleptris system-wide. The ffi gem is the only dependency.
gem install ffi
LEPTRIS_LIB_PATH=/path/to/libleptris.0.dylib ruby -Ilib -rleptris -e '
doc = Leptris::Document.parse("<r/>")
puts doc.root.name
doc.free
'To parse the headers from a binding tool:
cc -DLEPTRIS_FOR_BINDGEN -E src/include/leptris.h # strips LEPTRIS_APIfrom pyleptris import Document
doc = Document.parse('<root><item id="1">hello</item></root>')
root = doc.root
print(root.name) # => "root"
print(root.first_child_element.attribute('id')) # => "1"
print(doc.xpath('count(//item)')) # => 1.0
doc.close()Install: pip install cffi and point LEPTRIS_LIB_PATH at your
libleptris.dylib / .so (or install libleptris system-wide).
cmake -B build -S . -DLEPTRIS_BUILD_SHARED=ON
cmake --build build --target leptris_shared
LEPTRIS_LIB_PATH=$PWD/build/src/libleptris.dylib \
python3 -m pytest bindings/python/tests/Event-driven parsing (TODO 118 Phase B) — a handlers Hash of procs; all keys optional. Strings arrive as UTF-8; entity references in text and attribute values arrive expanded (XML 1.0):
require 'leptris'
Leptris::SAX.parse('<books><book id="1">title & more</book></books>',
start_element: ->(name, attrs) { puts "#{name} #{attrs.inspect}" },
end_element: ->(name) { },
characters: ->(text) { },
error: ->(msg, line, col) { warn "#{line}:#{col} #{msg}" }
)
# prints: books {}
# book {"id"=>"1"}Incremental parsing for streams (streaming: true selects the
constant-memory state machine — the same machine the one-shot API
uses; the default buffers chunks and parses on the final feed):
parser = Leptris::SAX::Parser.new({ characters: ->(t) { print t } },
streaming: true)
IO.popen('curl -s https://example.com/feed.xml') do |io|
io.each(4096) { |chunk| parser.feed(chunk) }
end
parser.feed('', final: true)
parser.freeEvents: :start_document, :end_document, :start_element
(name, attrs-hash), :end_element (name), :characters (text),
:comment, :cdata, :processing_instruction (target, data),
:start_prefix_mapping / :end_prefix_mapping (namespace
declarations), :error (message, line, column). A characters
event may fire multiple times per text node — concatenate between
element events to coalesce (the SAX contract).
See docs/FFI.md for the full design document.
# Parse a document
leptris parse document.xml
# Round-trip via XPath count
leptris xpath --count document.xml 'count(//item)'
# Pretty-print
leptris format --indent 4 document.xml < ugly.xml > pretty.xml
# Validate / version
leptris versionExit codes: 0 on success, 1 on parse error or invalid usage.
Leptris is benchmarked against libxml2 and pugixml on every push via CI
(GitHub Actions, Linux + macOS). Numbers below are from Apple Silicon,
clang -O3 -flto=thin (LTO is default for Release builds since TODO 110).
| Benchmark | Leptris | libxml2 | Advantage |
|---|---|---|---|
SAX small (~1 KB) |
2.8 µs (377 MB/s) |
7.1 µs (124 MB/s) |
2.5× faster |
SAX medium (~5 KB) |
7.5 µs (624 MB/s) |
26.9 µs (175 MB/s) |
3.6× faster |
DOM parse (~5 KB) |
33 µs |
47 µs |
1.4× faster |
By default leptris keeps whitespace-only text nodes — the
faithful-DOM behavior of libxml2/Nokogiri, and the only mode that
round-trips pretty-printed XML byte-for-byte. pugixml discards these
nodes by default (its parse_ws_pcdata is opt-in). For an
apples-to-apples parse comparison on pretty-printed documents, use
leptris_parse_string_flags(…, LEPTRIS_PARSE_DROP_WS_TEXT, …)
— the equivalent of pugixml’s default and libxml2’s
XML_PARSE_NOBLANKS. Without the flag, leptris creates and wires
one text node per element that pugixml never materializes, which is
the entire difference on whitespace-heavy shapes (~1.1-1.4x in
default mode).
| Benchmark | Leptris | libxml2 | Advantage |
|---|---|---|---|
Attribute lookup by name |
1.6 µs |
3.0 µs |
1.9× faster |
Text content extraction |
1.4 µs |
3.5 µs |
2.6× faster |
Indexed child access (1000 × 50) |
2.3 µs (O(1) cached) |
2.5 µs |
9% faster |
Leptris’s XPath engine uses a bytecode VM with per-axis
specialization, predicate fast paths, absolute-path fusion, a
per-document element index with attribute buckets, fused
axis+predicate opcodes, and memcpy fast paths for index-backed
queries (TODO 120, TODO 125–137). Numbers below are from
benchmarks/xpath/bench_diagnostic on a ~5 KB catalog fixture,
CPU time.
| Benchmark | Leptris | libxml2 | Advantage |
|---|---|---|---|
|
0.57 µs |
0.89 µs |
1.6× faster |
|
0.71 µs |
0.94 µs |
1.3× faster |
|
0.63 µs |
2.52 µs |
4.0× faster |
|
0.72 µs |
0.96 µs |
1.3× faster |
|
0.74 µs |
0.99 µs |
1.3× faster |
|
0.55 µs |
~1 µs |
1.8× faster |
|
0.56 µs |
~1 µs |
1.8× faster |
|
1.13 µs |
~3 µs |
2.7× faster |
|
0.77 µs |
1.02 µs |
1.3× faster |
|
0.53 µs |
~1 µs |
1.9× faster |
Leptris BEATS libxml2 on all 10 XPath benchmarks.
Numbers from benchmarks/matrix (median of 20 interleaved Release
runs, fresh build dirs). Pugixml’s append_attribute does not
reject duplicates; the fair comparison for attribute setting is
find-then-set (or find-then-append), which is what a correct
program must do. Leptris rejects duplicates per the XML 1.0
attribute uniqueness rule.
| Benchmark | Leptris | pugixml | Advantage |
|---|---|---|---|
Append 10000 children (one parent) |
160 µs |
119 µs |
1.3× slower (measured; was 2.5× faster pre-attr-index — the doc-level index bookkeeping is the cost of O(1) duplicate-rejecting set_attribute at scale) |
Set 2000 attributes (duplicate-rejecting) |
347 µs |
~8–14 ms (find+set) |
~25× faster |
Serialize attr-heavy 2 MB |
490 µs |
799 µs |
1.6× faster |
Serialize text-heavy 2 MB |
566 µs |
521 µs |
1.1× slower (measured; the two are at parity within run noise) |
See also benchmarks/README.md for the decomposition benchmark
(benchmark_decomp: isolates text/element/attr/text-node costs)
and the SAX benchmark (benchmark_sax: leptris vs libxml2 SAX2 —
pugixml has no SAX interface; leptris leads every corpus shape).
LTO is enabled by default for Release and RelWithDebInfo builds.
Disable with -DLEPTRIS_ENABLE_LTO=OFF:
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release
cmake --build build # LTO is on automatically
# or explicitly:
cmake -B build -S . -DCMAKE_BUILD_TYPE=Release -DLEPTRIS_ENABLE_LTO=ONcmake -B build -S . \
-DCMAKE_BUILD_TYPE=Release \
-DLEPTRIS_BUILD_BENCHMARKS=ON
cmake --build build
# Run individual benchmarks:
./build/benchmarks/bench_dom_leptris
./build/benchmarks/bench_sax_leptris
./build/benchmarks/benchmark_write # vs pugixml + libxml2
./build/benchmarks/bench_xpath_pugixml # XPath vs pugixmlCI uploads a benchmark-results-<os> artifact per push with JSON
Markdown output from every benchmark binary.
A vcpkg port pattern follows the jemalloc convention (see
tamatebako/jemalloc/ports/jemalloc/). The library ships with:
-
A
vcpkg.json(manifest) for vcpkg consumption. -
A
portfile.cmaketemplate for vcpkg port submission. -
A
usagefile documenting the linkage pattern.
# portfile.cmake (excerpt — see repo for full version)
vcpkg_cmake_configure(
SOURCE_PATH "${SOURCE_PATH}"
OPTIONS
-DLEPTRIS_BUILD_CLI=OFF
-DLEPTRIS_ENABLE_UTF8PROC=ON
-DLEPTRIS_ENABLE_ICONV=ON
)
vcpkg_cmake_install()
vcpkg_cmake_config_fixup(CONFIG_PATH lib/cmake/leptris)-
CMake ≥ 3.20
-
C99 compiler (GCC, Clang, MSVC, MinGW)
-
Optional: utf8proc (Unicode), iconv (encoding conversion), Doxygen (API docs)
No runtime dependencies when built without utf8proc/iconv.
| Workflow | Triggers | What it does |
|---|---|---|
|
Every push/PR |
Build, run all 103 specs across 13 modules. |
|
Every push/PR |
Build with AddressSanitizer; verify zero leaks / errors. |
|
Nightly cron |
libFuzzer for 5 minutes; report any crashes. |
leptris/
src/ # library + CLI + tests source
include/ # public C API headers
leptris/ # internal C source
dom/ # DOM node types + pool
parse/ # parser
xpath/ # XPath evaluator
sax/ # SAX parser
encode/ # UTF-16 + iconv
serialize/ # output writer
memory/ # pool allocator
dtd/ # DTD subsystem
cli/ # command-line tool
test/ # 345 specs across 14 modules
benchmark/ # libxml2 / pugixml comparisons
bindings/ruby/ # Ruby FFI binding (TODO 118)
TODO.md # open-work tracker (TODO.fix/ planning docs live in git history)
.github/workflows/ # CI
docs/ # README, building guide, FFI design
====
== Roadmap
context, and link:docs/FFI.md[docs/FFI.md] for the FFI roadmap.
Shipped in v0.3.0:
* ✓ Zero-copy borrowed text nodes (TODO 115)
* ✓ Pool-routed Parser struct (TODO 114 Phase 3)
* ✓ Streaming SAX state machine (TODO 116 Phases A-C — recursive parser removed)
* ✓ XInclude ownership transfer + cycle detection (TODO 117 Phases A-C)
* ✓ DTD content-model memoization (TODO 119)
* ✓ XPath bytecode VM, complete + wired in (TODO 120 Phases A-E)
* ✓ Compact-pointer int32 overflow fix for macOS (TODO 121)
* ✓ Ruby FFI binding (TODO 118)
Shipped post-v0.3.0 (XPath perf track):
* ✓ SAX API exported from shared library (TODO 122 — unblocks Ruby SAX)
* ✓ XPath diagnostic benchmark suite (TODO 123)
* ✓ XPath bytecode VM inline dispatch + bytecode cache (TODO 120 Phase F)
* ✓ Lazy namespace init — 5-9× faster per-eval floor (TODO 125)
* ✓ Specialized child/attribute/self/parent axes (TODO 126)
* ✓ Specialized descendant / descendant-or-self axes (TODO 127)
* ✓ Simple predicate fast paths: `[@attr]`, `[@attr='lit']`, `[N]` (TODO 128)
* ✓ Specialized absolute paths `/foo` `//foo` with `//name` fusion (TODO 129)
* ✓ Inline VM opcodes for common functions — `count`, `sum`, `string`, etc. (TODO 130)
* ✓ Iterative descendant walk + result pre-alloc (TODO 131)
* ✓ Per-document element index for O(1) descendant queries (TODO 132)
* ✓ Attribute index infrastructure for predicate fast paths (TODO 133)
* ✓ Fused axis+predicate opcodes — `descendant::*[@id]` now BEATS libxml2 (TODO 134)
* ✓ Fast inline nodeset_add for VM hot paths (TODO 135)
* ✓ Descendant-or-self fused predicate opcodes (TODO 136)
* ✓ Memcpy fast path for index-backed descendant queries — Leptris BEATS libxml2 on ALL XPath benchmarks (TODO 137)
* ✓ Contiguous per-document arena + content-derived block sizing (TODO 183)
* ✓ Single-representation attribute strings — views only, one cache line per attr (TODO 184)
* ✓ Fused attribute/text scans — K=100 parse gap vs pugixml 7.8x → ~1.5x (TODO 184)
* ✓ O(K^2) attribute walks eliminated across finalize/DTD/c14n (TODO 185)
* ✓ Ruby SAX binding — full callback surface + streaming (TODO 118 Phase B)
Planned (see TODO.md for the full open list):
* Extend element index to relative-descendant queries (currently
only absolute paths and root-context descendant use it).
* XPointer `xmlns()` scheme (needs a public namespace-context API).
* Doxygen API reference polish.
Shipped since this list was last curated: Python bindings (cffi,
pyleptris) and Rust bindings (crate `leptris` in bindings/rust); VM
constant folding for `concat` / `contains` / `substring`.
Measured dead (recorded for honesty; see the TODO.remaining/08 and
TODO.fix perf-ledger planning docs in git history, v1.1.2-era main):
* 32 B split-stream attrs — the family's upper-bound probe (views-
only 32 B stride, no ctrl array) regressed mid-K and still lost at
K=100 (TODO 185 round 6).
* Parser two-pass SIMD (v3) — the floor probe (memcpy + scan + stub
walk) already cost 88.5% of the current full parse (TODO 193).
The single-pass parser is at a compiler-global optimum; leptris
parses 6-14x faster than libxml2 and sits ~1.5-1.8x behind
pugixml on attr-heavy shapes, with no remaining lever of any
known class.
== Contributing
Issues and pull requests at
https://github.com/leptris/leptris[github.com/leptris/leptris].
For C contribution, see link:docs/guide/building.md[docs/guide/building.md].
For the testing policy, see `test/README.md`.
== Acknowledgments
This project draws structural inspiration from several long-running
C projects in the wider ecosystem:
* *jemalloc* (Tebako fork) — memory model and CI patterns.
* *libxml2* — public API ergonomics.
* *pugixml* — pool-based XML DOM, performance targets.
== License
MIT. See link:LICENSE.md[LICENSE.md].