mirror of
https://git.pumpin.online/funkpower/catena.git
synced 2026-10-23 03:07:13 +00:00
docs: the manual-page flag, and the records that belong to one tool
This commit is contained in:
@@ -84,6 +84,7 @@ already plays produces no line and a non-zero status, so `&&` skips it.
|
|||||||
| `--output <PATH>` | Where the converted copy goes. Defaults to `<name>.h264.mp4` beside the original |
|
| `--output <PATH>` | Where the converted copy goes. Defaults to `<name>.h264.mp4` beside the original |
|
||||||
| `--encoder <NAME>` | Use this encoder rather than the best one installed |
|
| `--encoder <NAME>` | Use this encoder rather than the best one installed |
|
||||||
| `--vaapi-device <PATH>` | The render node to encode on. Defaults to `/dev/dri/renderD128` |
|
| `--vaapi-device <PATH>` | The render node to encode on. Defaults to `/dev/dri/renderD128` |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
## Exit codes
|
## Exit codes
|
||||||
|
|
||||||
|
|||||||
@@ -71,6 +71,7 @@ bibclean books.ndjson | jq -r '.changes[] | "\(.why)\t\(.from)\t→\t\(.to)"'
|
|||||||
| Option | Effect |
|
| Option | Effect |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `--flagged` | Write only the records that need attention |
|
| `--flagged` | Write only the records that need attention |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
A missing ISBN alone does not count as needing attention: nearly every book
|
A missing ISBN alone does not count as needing attention: nearly every book
|
||||||
lacks one.
|
lacks one.
|
||||||
@@ -248,6 +249,7 @@ One heading per line — an `id`, a `name`, and the `count` of works under it:
|
|||||||
|---|---|
|
|---|---|
|
||||||
| `--kind <author\|publisher\|tag>` | What kind of heading the list holds |
|
| `--kind <author\|publisher\|tag>` | What kind of heading the list holds |
|
||||||
| `--rules <FILE>` | Vocabulary specific to this collection |
|
| `--rules <FILE>` | Vocabulary specific to this collection |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
The rules file is JSON, and today holds one thing:
|
The rules file is JSON, and today holds one thing:
|
||||||
|
|
||||||
|
|||||||
@@ -66,6 +66,7 @@ booksniff library/*.pdf | jq -r 'select(.kind != "pdf") | "\(.kind)\t\(.path)"'
|
|||||||
| Option | Effect |
|
| Option | Effect |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `--plain` | Print a line a person can read instead of the record |
|
| `--plain` | Print a line a person can read instead of the record |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
## Exit codes
|
## Exit codes
|
||||||
|
|
||||||
|
|||||||
@@ -60,6 +60,7 @@ chunkmd book.md | jq -c 'select(.role == "body")'
|
|||||||
| `--overlap <TOKENS>` | `64` | Carry this much of each chunk into the next |
|
| `--overlap <TOKENS>` | `64` | Carry this much of each chunk into the next |
|
||||||
| `--min-tokens <TOKENS>` | `50` | Drop chunks smaller than this |
|
| `--min-tokens <TOKENS>` | `50` | Drop chunks smaller than this |
|
||||||
| `--hard-max <TOKENS>` | `768` | Never emit a chunk larger than this |
|
| `--hard-max <TOKENS>` | `768` | Never emit a chunk larger than this |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
The defaults suit a model with a 512-token comfortable window. A larger model
|
The defaults suit a model with a 512-token comfortable window. A larger model
|
||||||
wants roughly double everything:
|
wants roughly double everything:
|
||||||
|
|||||||
@@ -83,6 +83,7 @@ cuenorm episode.srt | cueseg --duration 7245 > chapters.ndjson
|
|||||||
| Option | Effect |
|
| Option | Effect |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `--keep-repeats` | Keep consecutive repeats instead of merging them |
|
| `--keep-repeats` | Keep consecutive repeats instead of merging them |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
## Exit codes
|
## Exit codes
|
||||||
|
|
||||||
|
|||||||
@@ -57,6 +57,7 @@ cuepack episode.srt > passages.ndjson
|
|||||||
|---|---|---|
|
|---|---|---|
|
||||||
| `--seconds <SECONDS>` | `75` | Flush a passage once it covers this much audio |
|
| `--seconds <SECONDS>` | `75` | Flush a passage once it covers this much audio |
|
||||||
| `--max-tokens <TOKENS>` | `512` | Never let a passage exceed this many tokens |
|
| `--max-tokens <TOKENS>` | `512` | Never let a passage exceed this many tokens |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
Seventy-five seconds is about a paragraph of speech: long enough to hold an
|
Seventy-five seconds is about a paragraph of speech: long enough to hold an
|
||||||
argument, short enough that a result points at a place in the recording rather
|
argument, short enough that a result points at a place in the recording rather
|
||||||
|
|||||||
@@ -65,6 +65,7 @@ cueseg --duration 7245 transcript.ndjson | jq -r '"\(.start | floor)\t\(.title)"
|
|||||||
| Option | Effect |
|
| Option | Effect |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `--duration <SECONDS>` | How long the recording is |
|
| `--duration <SECONDS>` | How long the recording is |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
Without it, the duration is taken from the end of the last cue. That is
|
Without it, the duration is taken from the end of the last cue. That is
|
||||||
earlier than the end of the file whenever a recording ends in music or
|
earlier than the end of the file whenever a recording ends in music or
|
||||||
|
|||||||
@@ -72,6 +72,7 @@ paste -d'\t' <(chunkmd book.md | jq -c .) vectors.ndjson
|
|||||||
| `--max-input-tokens <TOKENS>` | `1800` | Cut each text to this many tokens before sending it; `0` sends them whole |
|
| `--max-input-tokens <TOKENS>` | `1800` | Cut each text to this many tokens before sending it; `0` sends them whole |
|
||||||
| `--timeout <SECONDS>` | `600` | How long to wait for one request |
|
| `--timeout <SECONDS>` | `600` | How long to wait for one request |
|
||||||
| `--attempts <COUNT>` | `3` | How many times to try a failing request |
|
| `--attempts <COUNT>` | `3` | How many times to try a failing request |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
The input cap is a defensive one, not a model limit. A server returns an error
|
The input cap is a defensive one, not a model limit. A server returns an error
|
||||||
for an input past its context length, and that error fails the whole batch —
|
for an input past its context length, and that error fails the whole batch —
|
||||||
|
|||||||
@@ -83,6 +83,7 @@ pandoc -t commonmark chapter.xhtml | mdscrub | chunkmd > chunks.ndjson
|
|||||||
| Option | Effect |
|
| Option | Effect |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `--pages` | First turn page separators and form feeds into page markers |
|
| `--pages` | First turn page separators and form feeds into page markers |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
`--pages` is for text straight out of a PDF or DjVu extractor, which marks its
|
`--pages` is for text straight out of a PDF or DjVu extractor, which marks its
|
||||||
page boundaries with a form feed or a rule of dashes. It is off by default
|
page boundaries with a form feed or a rule of dashes. It is off by default
|
||||||
|
|||||||
@@ -85,6 +85,7 @@ for f in sections/*.md; do mdscrub "$f" | chunkmd > "chunks/$(basename "$f" .md)
|
|||||||
| Option | Effect |
|
| Option | Effect |
|
||||||
|---|---|
|
|---|---|
|
||||||
| `--into <DIR>` | Write one file per section instead of records |
|
| `--into <DIR>` | Write one file per section instead of records |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
Files are named for their position and their title, so a directory listing is
|
Files are named for their position and their title, so a directory listing is
|
||||||
in reading order.
|
in reading order.
|
||||||
|
|||||||
@@ -92,6 +92,7 @@ done
|
|||||||
| `--quiet`, `-q` | Say nothing; the exit code is the answer |
|
| `--quiet`, `-q` | Say nothing; the exit code is the answer |
|
||||||
| `--min-ratio <RATIO>` | How wide a sheet must be to hold two pages. Default `1.15` |
|
| `--min-ratio <RATIO>` | How wide a sheet must be to hold two pages. Default `1.15` |
|
||||||
| `--gutter-fraction <FRACTION>` | What share of wide sheets must show a gutter. Default `0.6` |
|
| `--gutter-fraction <FRACTION>` | What share of wide sheets must show a gutter. Default `0.6` |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
`--into` writes whatever the verdict was, so a loop over a directory does not
|
`--into` writes whatever the verdict was, so a loop over a directory does not
|
||||||
have to branch. A document with no sheet wide enough to cut is not altered at
|
have to branch. A document with no sheet wide enough to cut is not altered at
|
||||||
@@ -156,6 +157,22 @@ the one that gets rendered. A wide sheet cropped down to a single page is
|
|||||||
therefore not a spread, which is what stops a second pass cutting a page in
|
therefore not a spread, which is what stops a second pass cutting a page in
|
||||||
half. Pages with no crop box, which is most of them, are their media box.
|
half. Pages with no crop box, which is most of them, are their media box.
|
||||||
|
|
||||||
|
## The record
|
||||||
|
|
||||||
|
`--json` writes one of these, whatever the verdict:
|
||||||
|
|
||||||
|
| Field | |
|
||||||
|
|---|---|
|
||||||
|
| `path` | The file it is about |
|
||||||
|
| `two_up` | Whether it is two pages to a sheet |
|
||||||
|
| `pages` | How many pages it has |
|
||||||
|
| `median_ratio` | The middle ratio of the pages looked at |
|
||||||
|
| `gutter_fraction` | What share of the wide ones showed a gutter |
|
||||||
|
| `sampled` | One entry per page looked at: its `number`, its `ratio`, and whether it had a `gutter` — `null` where the page was too narrow to be worth rendering |
|
||||||
|
|
||||||
|
It is a shape of its own rather than one of [the records](../../docs/records.md)
|
||||||
|
the tools pass between them, because nothing downstream consumes it.
|
||||||
|
|
||||||
## As a library
|
## As a library
|
||||||
|
|
||||||
```rust
|
```rust
|
||||||
|
|||||||
@@ -81,6 +81,7 @@ $ pdftriage --json book.pdf
|
|||||||
|---|---|
|
|---|---|
|
||||||
| `--json` | Emit a [`doc` record](../../docs/records.md#doc--what-a-file-is) instead of a readable verdict |
|
| `--json` | Emit a [`doc` record](../../docs/records.md#doc--what-a-file-is) instead of a readable verdict |
|
||||||
| `-q`, `--quiet` | Write no verdict; answer with the exit code alone. Diagnostics still go to stderr, because a silent failure is worse than a quiet one |
|
| `-q`, `--quiet` | Write no verdict; answer with the exit code alone. Diagnostics still go to stderr, because a silent failure is worse than a quiet one |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
## Exit codes
|
## Exit codes
|
||||||
|
|
||||||
|
|||||||
@@ -71,6 +71,7 @@ cat *.ndjson | rrf --weight fts=2 --weight meta=1
|
|||||||
| `-n`, `--limit <COUNT>` | all | How many results to keep |
|
| `-n`, `--limit <COUNT>` | all | How many results to keep |
|
||||||
| `--rrf-k <K>` | `60` | The rank constant |
|
| `--rrf-k <K>` | `60` | The rank constant |
|
||||||
| `--weight <CHANNEL=WEIGHT>` | `vec=1 fts=1 meta=1.5` | What a channel's rank is worth. Repeatable |
|
| `--weight <CHANNEL=WEIGHT>` | `vec=1 fts=1 meta=1.5` | What a channel's rank is worth. Repeatable |
|
||||||
|
| `--man` | Write this page as a manual page, in roff |
|
||||||
|
|
||||||
The rank constant flattens the difference between rank one and rank two. Sixty
|
The rank constant flattens the difference between rank one and rank two. Sixty
|
||||||
is what the method's own paper uses and what every other implementation uses,
|
is what the method's own paper uses and what every other implementation uses,
|
||||||
|
|||||||
+9
-4
@@ -102,10 +102,15 @@ Written by [`cueseg`](../crates/cueseg/README.md).
|
|||||||
|
|
||||||
## Records that belong to one tool
|
## Records that belong to one tool
|
||||||
|
|
||||||
Not everything is shared. [`bibclean`](../crates/bibclean/README.md) reads and
|
Not everything is shared. A tool whose answer nothing downstream consumes
|
||||||
writes catalogue records, author suggestions, slug proposals and vocabulary
|
has a shape of its own, described where that tool is:
|
||||||
merge proposals; those shapes are described where that tool is, because
|
|
||||||
nothing else consumes them.
|
- [`bibclean`](../crates/bibclean/README.md) — catalogue records, author
|
||||||
|
suggestions, slug proposals and vocabulary merge proposals.
|
||||||
|
- [`avplan`](../crates/avplan/README.md) — what has to happen to a recording,
|
||||||
|
and the call that would do it.
|
||||||
|
- [`pdfgutter`](../crates/pdfgutter/README.md) — whether a scan is two pages
|
||||||
|
to a sheet, and what each sampled page measured.
|
||||||
|
|
||||||
## Getting the schemas
|
## Getting the schemas
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user