A catena is a commentary assembled out of excerpts drawn from many sources and chained in order, each one carrying the name of who said it and where. That is both what these programs produce and how they run.

Nothing here needs a database, a GPU, a queue or a config file. Each tool is a filter: bytes in, records out, diagnostics on stderr, and an exit code that means something.

Install

cargo install --git https://git.pumpin.online/funkpower/catena corpus-core

Each tool installs by name, so you can take one without taking the rest.

The contract

Every binary in this workspace obeys all of this, or it does not ship.

One job. A tool that needs a --mode flag to change what it is is two tools.
stdin to stdout. Input on stdin or a path argument, result on stdout, diagnostics on stderr only — never content and chatter on the same stream.
Streams, not documents. Anything list-shaped is NDJSON: one record per line, flushed as it is produced, so a slow stage never blocks the pipe.
Predicates answer with exit codes. 0 yes, 1 no, 2 could not tell. Failures use the sysexits range — 64 bad usage, 65 bad input, 69 a missing external program, 74 I/O — so "no" and "broke" are never the same number.
No network, no database, no config file, no state. Two exceptions, both declared in --help: a tool whose whole job is an HTTP call, and a tool that shells out to a named external program and fails clearly when it is absent.
No ambient knowledge. No hostnames, no local paths, no private vocabularies compiled in. General scheme data — Bible book names, ISBN check digits — may be; anything local is a --rules file.json.
Deterministic. Same bytes in, same bytes out, across runs and machines.
--json everywhere. The human-readable default is for a person at a terminal; --json is the contract, and it does not change shape without a major version.

The records

Three shapes travel between the tools, defined in corpus-core and versioned by a v field.

doc — what a file is.

{"v":1,"path":"scan.pdf","kind":"pdf","declared_ext":"pdf","bytes":9261184,
 "pages":412,"language":"eng","text_layer":"garbled","needs_ocr":true,"notes":[]}

chunk — a retrievable passage with enough provenance to cite it.

{"v":1,"idx":7,"text":"…","tokens":512,"page":37,"chapter":"III. The Second Estate",
 "role":"body","start_time":null,"end_time":null,"flags":[]}

page and chapter for documents, start_time and end_time for recordings. One shape either way, so a single index serves a book and a lecture and returns both with a location you can cite.

hit — one scored candidate, on its way into a fusion.

{"v":1,"id":"c81a","channel":"vec","rank":1,"score":0.83,"payload":{}}

The JSON Schema for all three:

cargo run -p corpus-core -- schema all

Build

cargo test
cargo clippy --all-targets -- -D warnings
cargo fmt --check

Licence

MPL-2.0. See LICENSE.