From 9fd654ad5f59d8fa7edcc60fb2f7e2651b977543 Mon Sep 17 00:00:00 2001 From: funkpower Date: Sat, 19 Sep 2026 17:06:17 -0400 Subject: [PATCH] docs: the manual-page flag, and the records that belong to one tool --- crates/avplan/README.md | 1 + crates/bibclean/README.md | 2 ++ crates/booksniff/README.md | 1 + crates/chunkmd/README.md | 1 + crates/cuenorm/README.md | 1 + crates/cuepack/README.md | 1 + crates/cueseg/README.md | 1 + crates/emb/README.md | 1 + crates/mdscrub/README.md | 1 + crates/mdsplit/README.md | 1 + crates/pdfgutter/README.md | 17 +++++++++++++++++ crates/pdftriage/README.md | 1 + crates/rrf/README.md | 1 + docs/records.md | 13 +++++++++---- 14 files changed, 39 insertions(+), 4 deletions(-) diff --git a/crates/avplan/README.md b/crates/avplan/README.md index 85306bf..ebdc011 100644 --- a/crates/avplan/README.md +++ b/crates/avplan/README.md @@ -84,6 +84,7 @@ already plays produces no line and a non-zero status, so `&&` skips it. | `--output ` | Where the converted copy goes. Defaults to `.h264.mp4` beside the original | | `--encoder ` | Use this encoder rather than the best one installed | | `--vaapi-device ` | The render node to encode on. Defaults to `/dev/dri/renderD128` | +| `--man` | Write this page as a manual page, in roff | ## Exit codes diff --git a/crates/bibclean/README.md b/crates/bibclean/README.md index 62016e1..fdf94b6 100644 --- a/crates/bibclean/README.md +++ b/crates/bibclean/README.md @@ -71,6 +71,7 @@ bibclean books.ndjson | jq -r '.changes[] | "\(.why)\t\(.from)\t→\t\(.to)"' | Option | Effect | |---|---| | `--flagged` | Write only the records that need attention | +| `--man` | Write this page as a manual page, in roff | A missing ISBN alone does not count as needing attention: nearly every book lacks one. @@ -248,6 +249,7 @@ One heading per line — an `id`, a `name`, and the `count` of works under it: |---|---| | `--kind ` | What kind of heading the list holds | | `--rules ` | Vocabulary specific to this collection | +| `--man` | Write this page as a manual page, in roff | The rules file is JSON, and today holds one thing: diff --git a/crates/booksniff/README.md b/crates/booksniff/README.md index edbe14c..4991a99 100644 --- a/crates/booksniff/README.md +++ b/crates/booksniff/README.md @@ -66,6 +66,7 @@ booksniff library/*.pdf | jq -r 'select(.kind != "pdf") | "\(.kind)\t\(.path)"' | Option | Effect | |---|---| | `--plain` | Print a line a person can read instead of the record | +| `--man` | Write this page as a manual page, in roff | ## Exit codes diff --git a/crates/chunkmd/README.md b/crates/chunkmd/README.md index 87fb94e..2c043a4 100644 --- a/crates/chunkmd/README.md +++ b/crates/chunkmd/README.md @@ -60,6 +60,7 @@ chunkmd book.md | jq -c 'select(.role == "body")' | `--overlap ` | `64` | Carry this much of each chunk into the next | | `--min-tokens ` | `50` | Drop chunks smaller than this | | `--hard-max ` | `768` | Never emit a chunk larger than this | +| `--man` | Write this page as a manual page, in roff | The defaults suit a model with a 512-token comfortable window. A larger model wants roughly double everything: diff --git a/crates/cuenorm/README.md b/crates/cuenorm/README.md index eddd8ab..450408d 100644 --- a/crates/cuenorm/README.md +++ b/crates/cuenorm/README.md @@ -83,6 +83,7 @@ cuenorm episode.srt | cueseg --duration 7245 > chapters.ndjson | Option | Effect | |---|---| | `--keep-repeats` | Keep consecutive repeats instead of merging them | +| `--man` | Write this page as a manual page, in roff | ## Exit codes diff --git a/crates/cuepack/README.md b/crates/cuepack/README.md index 83afd74..571ad0a 100644 --- a/crates/cuepack/README.md +++ b/crates/cuepack/README.md @@ -57,6 +57,7 @@ cuepack episode.srt > passages.ndjson |---|---|---| | `--seconds ` | `75` | Flush a passage once it covers this much audio | | `--max-tokens ` | `512` | Never let a passage exceed this many tokens | +| `--man` | Write this page as a manual page, in roff | Seventy-five seconds is about a paragraph of speech: long enough to hold an argument, short enough that a result points at a place in the recording rather diff --git a/crates/cueseg/README.md b/crates/cueseg/README.md index fa1d4c5..f254613 100644 --- a/crates/cueseg/README.md +++ b/crates/cueseg/README.md @@ -65,6 +65,7 @@ cueseg --duration 7245 transcript.ndjson | jq -r '"\(.start | floor)\t\(.title)" | Option | Effect | |---|---| | `--duration ` | How long the recording is | +| `--man` | Write this page as a manual page, in roff | Without it, the duration is taken from the end of the last cue. That is earlier than the end of the file whenever a recording ends in music or diff --git a/crates/emb/README.md b/crates/emb/README.md index d943730..de6ccf3 100644 --- a/crates/emb/README.md +++ b/crates/emb/README.md @@ -72,6 +72,7 @@ paste -d'\t' <(chunkmd book.md | jq -c .) vectors.ndjson | `--max-input-tokens ` | `1800` | Cut each text to this many tokens before sending it; `0` sends them whole | | `--timeout ` | `600` | How long to wait for one request | | `--attempts ` | `3` | How many times to try a failing request | +| `--man` | Write this page as a manual page, in roff | The input cap is a defensive one, not a model limit. A server returns an error for an input past its context length, and that error fails the whole batch — diff --git a/crates/mdscrub/README.md b/crates/mdscrub/README.md index 861626b..e8baea2 100644 --- a/crates/mdscrub/README.md +++ b/crates/mdscrub/README.md @@ -83,6 +83,7 @@ pandoc -t commonmark chapter.xhtml | mdscrub | chunkmd > chunks.ndjson | Option | Effect | |---|---| | `--pages` | First turn page separators and form feeds into page markers | +| `--man` | Write this page as a manual page, in roff | `--pages` is for text straight out of a PDF or DjVu extractor, which marks its page boundaries with a form feed or a rule of dashes. It is off by default diff --git a/crates/mdsplit/README.md b/crates/mdsplit/README.md index 6412a70..535e0c6 100644 --- a/crates/mdsplit/README.md +++ b/crates/mdsplit/README.md @@ -85,6 +85,7 @@ for f in sections/*.md; do mdscrub "$f" | chunkmd > "chunks/$(basename "$f" .md) | Option | Effect | |---|---| | `--into ` | Write one file per section instead of records | +| `--man` | Write this page as a manual page, in roff | Files are named for their position and their title, so a directory listing is in reading order. diff --git a/crates/pdfgutter/README.md b/crates/pdfgutter/README.md index 9adc262..c58602e 100644 --- a/crates/pdfgutter/README.md +++ b/crates/pdfgutter/README.md @@ -92,6 +92,7 @@ done | `--quiet`, `-q` | Say nothing; the exit code is the answer | | `--min-ratio ` | How wide a sheet must be to hold two pages. Default `1.15` | | `--gutter-fraction ` | What share of wide sheets must show a gutter. Default `0.6` | +| `--man` | Write this page as a manual page, in roff | `--into` writes whatever the verdict was, so a loop over a directory does not have to branch. A document with no sheet wide enough to cut is not altered at @@ -156,6 +157,22 @@ the one that gets rendered. A wide sheet cropped down to a single page is therefore not a spread, which is what stops a second pass cutting a page in half. Pages with no crop box, which is most of them, are their media box. +## The record + +`--json` writes one of these, whatever the verdict: + +| Field | | +|---|---| +| `path` | The file it is about | +| `two_up` | Whether it is two pages to a sheet | +| `pages` | How many pages it has | +| `median_ratio` | The middle ratio of the pages looked at | +| `gutter_fraction` | What share of the wide ones showed a gutter | +| `sampled` | One entry per page looked at: its `number`, its `ratio`, and whether it had a `gutter` — `null` where the page was too narrow to be worth rendering | + +It is a shape of its own rather than one of [the records](../../docs/records.md) +the tools pass between them, because nothing downstream consumes it. + ## As a library ```rust diff --git a/crates/pdftriage/README.md b/crates/pdftriage/README.md index 13d6337..205718d 100644 --- a/crates/pdftriage/README.md +++ b/crates/pdftriage/README.md @@ -81,6 +81,7 @@ $ pdftriage --json book.pdf |---|---| | `--json` | Emit a [`doc` record](../../docs/records.md#doc--what-a-file-is) instead of a readable verdict | | `-q`, `--quiet` | Write no verdict; answer with the exit code alone. Diagnostics still go to stderr, because a silent failure is worse than a quiet one | +| `--man` | Write this page as a manual page, in roff | ## Exit codes diff --git a/crates/rrf/README.md b/crates/rrf/README.md index 35b7b8b..7662fdf 100644 --- a/crates/rrf/README.md +++ b/crates/rrf/README.md @@ -71,6 +71,7 @@ cat *.ndjson | rrf --weight fts=2 --weight meta=1 | `-n`, `--limit ` | all | How many results to keep | | `--rrf-k ` | `60` | The rank constant | | `--weight ` | `vec=1 fts=1 meta=1.5` | What a channel's rank is worth. Repeatable | +| `--man` | Write this page as a manual page, in roff | The rank constant flattens the difference between rank one and rank two. Sixty is what the method's own paper uses and what every other implementation uses, diff --git a/docs/records.md b/docs/records.md index eb19b16..5275536 100644 --- a/docs/records.md +++ b/docs/records.md @@ -102,10 +102,15 @@ Written by [`cueseg`](../crates/cueseg/README.md). ## Records that belong to one tool -Not everything is shared. [`bibclean`](../crates/bibclean/README.md) reads and -writes catalogue records, author suggestions, slug proposals and vocabulary -merge proposals; those shapes are described where that tool is, because -nothing else consumes them. +Not everything is shared. A tool whose answer nothing downstream consumes +has a shape of its own, described where that tool is: + +- [`bibclean`](../crates/bibclean/README.md) — catalogue records, author + suggestions, slug proposals and vocabulary merge proposals. +- [`avplan`](../crates/avplan/README.md) — what has to happen to a recording, + and the call that would do it. +- [`pdfgutter`](../crates/pdfgutter/README.md) — whether a scan is two pages + to a sheet, and what each sampled page measured. ## Getting the schemas