mirror of
https://git.pumpin.online/funkpower/catena.git
synced 2026-10-22 18:57:14 +00:00
docs: the manual-page flag, and the records that belong to one tool
This commit is contained in:
@@ -84,6 +84,7 @@ already plays produces no line and a non-zero status, so `&&` skips it.
|
||||
| `--output <PATH>` | Where the converted copy goes. Defaults to `<name>.h264.mp4` beside the original |
|
||||
| `--encoder <NAME>` | Use this encoder rather than the best one installed |
|
||||
| `--vaapi-device <PATH>` | The render node to encode on. Defaults to `/dev/dri/renderD128` |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
## Exit codes
|
||||
|
||||
|
||||
@@ -71,6 +71,7 @@ bibclean books.ndjson | jq -r '.changes[] | "\(.why)\t\(.from)\t→\t\(.to)"'
|
||||
| Option | Effect |
|
||||
|---|---|
|
||||
| `--flagged` | Write only the records that need attention |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
A missing ISBN alone does not count as needing attention: nearly every book
|
||||
lacks one.
|
||||
@@ -248,6 +249,7 @@ One heading per line — an `id`, a `name`, and the `count` of works under it:
|
||||
|---|---|
|
||||
| `--kind <author\|publisher\|tag>` | What kind of heading the list holds |
|
||||
| `--rules <FILE>` | Vocabulary specific to this collection |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
The rules file is JSON, and today holds one thing:
|
||||
|
||||
|
||||
@@ -66,6 +66,7 @@ booksniff library/*.pdf | jq -r 'select(.kind != "pdf") | "\(.kind)\t\(.path)"'
|
||||
| Option | Effect |
|
||||
|---|---|
|
||||
| `--plain` | Print a line a person can read instead of the record |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
## Exit codes
|
||||
|
||||
|
||||
@@ -60,6 +60,7 @@ chunkmd book.md | jq -c 'select(.role == "body")'
|
||||
| `--overlap <TOKENS>` | `64` | Carry this much of each chunk into the next |
|
||||
| `--min-tokens <TOKENS>` | `50` | Drop chunks smaller than this |
|
||||
| `--hard-max <TOKENS>` | `768` | Never emit a chunk larger than this |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
The defaults suit a model with a 512-token comfortable window. A larger model
|
||||
wants roughly double everything:
|
||||
|
||||
@@ -83,6 +83,7 @@ cuenorm episode.srt | cueseg --duration 7245 > chapters.ndjson
|
||||
| Option | Effect |
|
||||
|---|---|
|
||||
| `--keep-repeats` | Keep consecutive repeats instead of merging them |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
## Exit codes
|
||||
|
||||
|
||||
@@ -57,6 +57,7 @@ cuepack episode.srt > passages.ndjson
|
||||
|---|---|---|
|
||||
| `--seconds <SECONDS>` | `75` | Flush a passage once it covers this much audio |
|
||||
| `--max-tokens <TOKENS>` | `512` | Never let a passage exceed this many tokens |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
Seventy-five seconds is about a paragraph of speech: long enough to hold an
|
||||
argument, short enough that a result points at a place in the recording rather
|
||||
|
||||
@@ -65,6 +65,7 @@ cueseg --duration 7245 transcript.ndjson | jq -r '"\(.start | floor)\t\(.title)"
|
||||
| Option | Effect |
|
||||
|---|---|
|
||||
| `--duration <SECONDS>` | How long the recording is |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
Without it, the duration is taken from the end of the last cue. That is
|
||||
earlier than the end of the file whenever a recording ends in music or
|
||||
|
||||
@@ -72,6 +72,7 @@ paste -d'\t' <(chunkmd book.md | jq -c .) vectors.ndjson
|
||||
| `--max-input-tokens <TOKENS>` | `1800` | Cut each text to this many tokens before sending it; `0` sends them whole |
|
||||
| `--timeout <SECONDS>` | `600` | How long to wait for one request |
|
||||
| `--attempts <COUNT>` | `3` | How many times to try a failing request |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
The input cap is a defensive one, not a model limit. A server returns an error
|
||||
for an input past its context length, and that error fails the whole batch —
|
||||
|
||||
@@ -83,6 +83,7 @@ pandoc -t commonmark chapter.xhtml | mdscrub | chunkmd > chunks.ndjson
|
||||
| Option | Effect |
|
||||
|---|---|
|
||||
| `--pages` | First turn page separators and form feeds into page markers |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
`--pages` is for text straight out of a PDF or DjVu extractor, which marks its
|
||||
page boundaries with a form feed or a rule of dashes. It is off by default
|
||||
|
||||
@@ -85,6 +85,7 @@ for f in sections/*.md; do mdscrub "$f" | chunkmd > "chunks/$(basename "$f" .md)
|
||||
| Option | Effect |
|
||||
|---|---|
|
||||
| `--into <DIR>` | Write one file per section instead of records |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
Files are named for their position and their title, so a directory listing is
|
||||
in reading order.
|
||||
|
||||
@@ -92,6 +92,7 @@ done
|
||||
| `--quiet`, `-q` | Say nothing; the exit code is the answer |
|
||||
| `--min-ratio <RATIO>` | How wide a sheet must be to hold two pages. Default `1.15` |
|
||||
| `--gutter-fraction <FRACTION>` | What share of wide sheets must show a gutter. Default `0.6` |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
`--into` writes whatever the verdict was, so a loop over a directory does not
|
||||
have to branch. A document with no sheet wide enough to cut is not altered at
|
||||
@@ -156,6 +157,22 @@ the one that gets rendered. A wide sheet cropped down to a single page is
|
||||
therefore not a spread, which is what stops a second pass cutting a page in
|
||||
half. Pages with no crop box, which is most of them, are their media box.
|
||||
|
||||
## The record
|
||||
|
||||
`--json` writes one of these, whatever the verdict:
|
||||
|
||||
| Field | |
|
||||
|---|---|
|
||||
| `path` | The file it is about |
|
||||
| `two_up` | Whether it is two pages to a sheet |
|
||||
| `pages` | How many pages it has |
|
||||
| `median_ratio` | The middle ratio of the pages looked at |
|
||||
| `gutter_fraction` | What share of the wide ones showed a gutter |
|
||||
| `sampled` | One entry per page looked at: its `number`, its `ratio`, and whether it had a `gutter` — `null` where the page was too narrow to be worth rendering |
|
||||
|
||||
It is a shape of its own rather than one of [the records](../../docs/records.md)
|
||||
the tools pass between them, because nothing downstream consumes it.
|
||||
|
||||
## As a library
|
||||
|
||||
```rust
|
||||
|
||||
@@ -81,6 +81,7 @@ $ pdftriage --json book.pdf
|
||||
|---|---|
|
||||
| `--json` | Emit a [`doc` record](../../docs/records.md#doc--what-a-file-is) instead of a readable verdict |
|
||||
| `-q`, `--quiet` | Write no verdict; answer with the exit code alone. Diagnostics still go to stderr, because a silent failure is worse than a quiet one |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
## Exit codes
|
||||
|
||||
|
||||
@@ -71,6 +71,7 @@ cat *.ndjson | rrf --weight fts=2 --weight meta=1
|
||||
| `-n`, `--limit <COUNT>` | all | How many results to keep |
|
||||
| `--rrf-k <K>` | `60` | The rank constant |
|
||||
| `--weight <CHANNEL=WEIGHT>` | `vec=1 fts=1 meta=1.5` | What a channel's rank is worth. Repeatable |
|
||||
| `--man` | Write this page as a manual page, in roff |
|
||||
|
||||
The rank constant flattens the difference between rank one and rank two. Sixty
|
||||
is what the method's own paper uses and what every other implementation uses,
|
||||
|
||||
+9
-4
@@ -102,10 +102,15 @@ Written by [`cueseg`](../crates/cueseg/README.md).
|
||||
|
||||
## Records that belong to one tool
|
||||
|
||||
Not everything is shared. [`bibclean`](../crates/bibclean/README.md) reads and
|
||||
writes catalogue records, author suggestions, slug proposals and vocabulary
|
||||
merge proposals; those shapes are described where that tool is, because
|
||||
nothing else consumes them.
|
||||
Not everything is shared. A tool whose answer nothing downstream consumes
|
||||
has a shape of its own, described where that tool is:
|
||||
|
||||
- [`bibclean`](../crates/bibclean/README.md) — catalogue records, author
|
||||
suggestions, slug proposals and vocabulary merge proposals.
|
||||
- [`avplan`](../crates/avplan/README.md) — what has to happen to a recording,
|
||||
and the call that would do it.
|
||||
- [`pdfgutter`](../crates/pdfgutter/README.md) — whether a scan is two pages
|
||||
to a sheet, and what each sampled page measured.
|
||||
|
||||
## Getting the schemas
|
||||
|
||||
|
||||
Reference in New Issue
Block a user