cueseg: chapter by vectors somebody else made, where there are some

The hashed bag of words asks only whether two stretches repeat the same
words, and cannot tell that ships and vessels are one subject. An embedding
can. segment_with takes one vector per window and falls back to the words
when it is given none, or the wrong number of them — a caller that embedded
some other recording's windows is asking about that recording.

windows_of gives the text to embed, in the order the vectors come back in,
so nobody has to reimplement the windowing pass to use this.

Bound in all three languages, and on the command line as --windows and
--embeddings, which pipe through emb. The C ABI version goes to 2:
catena_segment_cues gained an argument.
This commit is contained in:
funkpower
2026-09-20 09:12:04 -04:00
parent dd09499e68
commit 0b849c39c4
14 changed files with 359 additions and 24 deletions
+29
View File
@@ -64,6 +64,8 @@ cueseg --duration 7245 transcript.ndjson | jq -r '"\(.start | floor)\t\(.title)"
| Option | Effect |
|---|---|
| `--windows` | Print the text to embed, one window per line, and stop |
| `--embeddings <FILE>` | One vector per window, as `emb` writes them |
| `--duration <SECONDS>` | How long the recording is |
| `--man` | Write this page as a manual page, in roff |
@@ -72,6 +74,33 @@ earlier than the end of the file whenever a recording ends in music or
silence, and a chapter that claims to run to the end of the transcript rather
than the end of the file is a mark that lands in the wrong place.
## Better boundaries, if you have embeddings
By default the windows are compared by the words they repeat. That is fast,
needs nothing, and cannot tell that a stretch about *ships* and a stretch
about *vessels* are about one thing. An embedding can.
`--windows` prints the text to embed, one window per line; `--embeddings`
takes one vector per line back, in the same order:
```sh
cueseg --windows episode.srt | emb --model nomic-embed-text > windows.ndjson
cueseg --embeddings windows.ndjson episode.srt
```
The count has to match, and a mismatch is refused rather than guessed at:
```
$ cueseg --embeddings two-vectors.ndjson episode.srt
cueseg: 2 vectors for 44 windows — run --windows to see what to embed
$ echo $?
65
```
Everything else is the same: the same windows, the same threshold, the same
minimum chapter length. Only the question "are these two stretches about the
same thing" is answered better.
## Exit codes
| Code | Meaning |