bindings: a C ABI and an npm package alongside the wheel

This commit is contained in:
funkpower
2026-09-19 18:40:08 -04:00
parent cf126dddc3
commit 48600bd90c
16 changed files with 1221 additions and 10 deletions
+82 -7
View File
@@ -19,6 +19,9 @@ What comes back is the same **record shapes** the commands write to stdout,
as dictionaries rather than lines of JSON. Those shapes are documented and
versioned; the structs behind them are free to move.
Three of them: Python, JavaScript through WebAssembly, and a C ABI. All three
call the same Rust and hand back the same records.
## Python
```sh
@@ -69,17 +72,89 @@ The tools that are not here are the ones a binding would gain nothing over a
subprocess for: they read files, shell out to another program, or talk to a
server, and none of that is cheaper in process.
## Building the wheel
## JavaScript
```sh
cd crates/bindings && maturin build --release
npm install catena
```
`cargo build --features python` does not work and is not supposed to: pyo3's
`extension-module` leaves the Python symbols to be resolved by the interpreter
at load time, so there is nothing to link against until an interpreter loads
it. `cargo clippy --features python` type-checks it without linking, which is
what the gate runs.
```js
import { chunkMarkdown, scrub, countTokens, fuseHits } from "catena";
const chunks = chunkMarkdown(markdown, 512, 64);
chunks[0].page; // 41
chunks[0].chapter; // 'III. The Second Estate'
fuseHits([...vectorHits, ...textHits], undefined, { vec: 2 }, 20);
```
Records come back as plain objects, not handles: nothing to free,
`JSON.stringify` works, and an array of chunks spreads into state without
ceremony. The declarations are hand-written, so a caller gets `Chunk[]` rather
than `any`.
`splitCollection` throws when the document is one work rather than a
collection, which is the usual answer.
Two builds, because the loaders differ — node reads the `.wasm` off disk at
import, the web build takes a URL a bundler rewrites:
```js
import { chunkMarkdown } from "catena"; // resolves per environment
import { chunkMarkdown } from "catena/node"; // or pick one
import { chunkMarkdown } from "catena/web";
```
## C
```c
#include <catena.h>
char *chunks = catena_chunk_markdown(markdown, 512, 64, 50, 768);
if (!chunks) fprintf(stderr, "%s\n", catena_last_error());
/* … a JSON array of chunk records … */
catena_string_free(chunks);
```
Text in as NUL-terminated UTF-8, records out as NUL-terminated JSON — the same
document the command line writes. Nothing here is an opaque handle with
accessors to keep in step: the records already have a documented shape, and
every language with an FFI has a JSON parser.
Every returned string is yours and must go back through `catena_string_free`.
A call that fails returns NULL and leaves a message in `catena_last_error`,
which is thread-local. `catena_abi_version` is bumped when anything here
changes meaning; additions leave it alone.
A budget of zero means that budget's default, so a caller that cares about one
of them can pass zero for the rest.
## Building
```sh
cd crates/bindings
maturin build --profile wheel # the Python wheel
npm/build.sh # the npm package
cargo build --features capi # the C library and its header
```
Three things worth knowing, each of which fails confusingly otherwise:
**`cargo build --features python` does not work and is not supposed to.**
pyo3's `extension-module` leaves the Python symbols to be resolved by the
interpreter at load time, so there is nothing to link against until an
interpreter loads it. `cargo clippy --features python` type-checks it without
linking, which is what the gate runs.
**The wheel needs `--profile wheel`, not `--release`.** The release profile
strips, and a stripped extension module is one macOS refuses to load at all —
"mis-aligned LINKEDIT string pool". The `wheel` profile is the release profile
without it. `[tool.maturin] profile` is ignored by maturin 1.15, so it has to
be the flag.
**The WebAssembly build is only compiled for wasm32.** On any other target the
module compiles to nothing, so `--all-features` on a host build does not reach
it; the gate checks it separately against `wasm32-unknown-unknown`.
## As a Rust library