mirror of
https://git.pumpin.online/funkpower/catena.git
synced 2026-10-22 18:57:14 +00:00
funkpower
0b849c39c4
The hashed bag of words asks only whether two stretches repeat the same words, and cannot tell that ships and vessels are one subject. An embedding can. segment_with takes one vector per window and falls back to the words when it is given none, or the wrong number of them — a caller that embedded some other recording's windows is asking about that recording. windows_of gives the text to embed, in the order the vectors come back in, so nobody has to reimplement the windowing pass to use this. Bound in all three languages, and on the command line as --windows and --embeddings, which pipe through emb. The C ABI version goes to 2: catena_segment_cues gained an argument.
150 lines
6.1 KiB
Markdown
150 lines
6.1 KiB
Markdown
# cueseg
|
|
|
|
A timed transcript into topic chapters.
|
|
|
|
Part of [catena](../../README.md).
|
|
|
|
## What it does
|
|
|
|
Finds where a recording changes subject, and writes one [`segment`
|
|
record](../../docs/records.md#segment--one-topic-chapter) per chapter: a start, an end, a
|
|
title taken from the chapter's own first sentence, and an excerpt.
|
|
|
|
No model and no network. The text is hashed into a small lexical vector a
|
|
window at a time, and a topic boundary is where two consecutive windows stop
|
|
resembling each other.
|
|
|
|
## When you would use it
|
|
|
|
- **Chapter marks for a player.** A three-hour recording with no marks is
|
|
unnavigable; the boundaries this finds are exactly the marks a scrubber
|
|
needs.
|
|
- **Feeding a summariser one subject at a time.** A whole transcript is too
|
|
much context and a paragraph is not enough; a chapter is the unit that gives
|
|
a useful answer.
|
|
- **Navigation in a search result.** Knowing which chapter a hit falls in is
|
|
often more useful than the timestamp alone.
|
|
- **Titles before anybody has written any.** The extracted titles are meant to
|
|
be replaced by something better later, and to be usable until they are.
|
|
|
|
## Install
|
|
|
|
```sh
|
|
cargo install --git https://git.pumpin.online/funkpower/catena cueseg
|
|
```
|
|
|
|
## Example
|
|
|
|
A two-hour recording of two subjects:
|
|
|
|
```
|
|
$ cueseg --duration 7200 transcript.ndjson
|
|
{"v":1,"start":0.0,"end":1190.6666666666667,"title":"The Mill Stood At The Northern End Of","excerpt":"the mill stood at the northern end of the valley where the fall of the river was steepest and the ice came latest…"}
|
|
{"v":1,"start":1190.6666666666667,"end":2381.3333333333335,"title":"On One Household And The Benefit Was Spread","excerpt":"on one household and the benefit was spread across four hundred families…"}
|
|
{"v":1,"start":2381.3333333333335,"end":3572.0,"title":"Stood At The Northern End Of The Valley","excerpt":"stood at the northern end of the valley where the fall of the river was steepest…"}
|
|
{"v":1,"start":3572.0,"end":4479.0,"title":"Steepest And The Ice Came Latest Its Wheel","excerpt":"steepest and the ice came latest its wheel was undershot which was wasteful…"}
|
|
{"v":1,"start":4479.0,"end":5386.0,"title":"It From The Journals Of Either House Parliament","excerpt":"it from the journals of either house parliament dissolved itself again amid furious objections…"}
|
|
{"v":1,"start":5386.0,"end":6293.0,"title":"At All Which Left The Committee With No","excerpt":"at all which left the committee with no record of the vote…"}
|
|
{"v":1,"start":6293.0,"end":7200.0,"title":"House Parliament Dissolved Itself Again Amid Furious Objections","excerpt":"house parliament dissolved itself again amid furious objections from every bench…"}
|
|
```
|
|
|
|
Straight from a subtitle file:
|
|
|
|
```sh
|
|
cuenorm episode.vtt | cueseg --duration 7245 > chapters.ndjson
|
|
```
|
|
|
|
As chapter marks for a player:
|
|
|
|
```sh
|
|
cueseg --duration 7245 transcript.ndjson | jq -r '"\(.start | floor)\t\(.title)"'
|
|
```
|
|
|
|
## Options
|
|
|
|
| Option | Effect |
|
|
|---|---|
|
|
| `--windows` | Print the text to embed, one window per line, and stop |
|
|
| `--embeddings <FILE>` | One vector per window, as `emb` writes them |
|
|
| `--duration <SECONDS>` | How long the recording is |
|
|
| `--man` | Write this page as a manual page, in roff |
|
|
|
|
Without it, the duration is taken from the end of the last cue. That is
|
|
earlier than the end of the file whenever a recording ends in music or
|
|
silence, and a chapter that claims to run to the end of the transcript rather
|
|
than the end of the file is a mark that lands in the wrong place.
|
|
|
|
## Better boundaries, if you have embeddings
|
|
|
|
By default the windows are compared by the words they repeat. That is fast,
|
|
needs nothing, and cannot tell that a stretch about *ships* and a stretch
|
|
about *vessels* are about one thing. An embedding can.
|
|
|
|
`--windows` prints the text to embed, one window per line; `--embeddings`
|
|
takes one vector per line back, in the same order:
|
|
|
|
```sh
|
|
cueseg --windows episode.srt | emb --model nomic-embed-text > windows.ndjson
|
|
cueseg --embeddings windows.ndjson episode.srt
|
|
```
|
|
|
|
The count has to match, and a mismatch is refused rather than guessed at:
|
|
|
|
```
|
|
$ cueseg --embeddings two-vectors.ndjson episode.srt
|
|
cueseg: 2 vectors for 44 windows — run --windows to see what to embed
|
|
$ echo $?
|
|
65
|
|
```
|
|
|
|
Everything else is the same: the same windows, the same threshold, the same
|
|
minimum chapter length. Only the question "are these two stretches about the
|
|
same thing" is answered better.
|
|
|
|
## Exit codes
|
|
|
|
| Code | Meaning |
|
|
|---|---|
|
|
| `0` | Done |
|
|
| `64` | The command line was wrong, or the duration was not a length |
|
|
| `65` | The cue stream could not be read |
|
|
| `74` | The file could not be read |
|
|
|
|
## How it finds a boundary
|
|
|
|
1. **Windows.** Cues are gathered into stretches of about seventy-five
|
|
seconds — long enough to have a subject, short enough that a boundary
|
|
lands within a minute of the real one.
|
|
2. **Vectors.** Each window's content words are hashed into a small vector of
|
|
counts. Filler, discourse markers and the vague nouns that carry a
|
|
conversation are dropped: leaving them in makes every window resemble every
|
|
other.
|
|
3. **Valleys.** A boundary is where two consecutive windows resemble each
|
|
other less than their neighbours do, and less than the recording's own
|
|
average by half a standard deviation. Valleys are taken deepest first, each
|
|
at least a chapter's length from the ones already taken.
|
|
4. **A floor and a ceiling.** Between four and twenty chapters, targeting one
|
|
every fifteen minutes. A chapter mark every two minutes is not navigation,
|
|
it is noise; and a chapter that runs too long is no more navigable than no
|
|
chapter at all, so an over-long span is divided evenly whether or not the
|
|
text suggested a break there.
|
|
5. **A fallback.** A recording where nothing stands out is cut evenly, which
|
|
is better than one chapter of three hours.
|
|
|
|
## Input
|
|
|
|
Either the cue stream [`cuenorm`](../cuenorm/README.md) writes, or a subtitle file.
|
|
|
|
## As a library
|
|
|
|
```rust
|
|
for chapter in cueseg::segment(&cues, 7245.0) {
|
|
println!("{:.0}s {}", chapter.start, chapter.title);
|
|
}
|
|
```
|
|
|
|
## See also
|
|
|
|
- [`cuenorm`](../cuenorm/README.md) — what to run before this.
|
|
- [`cuepack`](../cuepack/README.md) — the other thing to do with a cue stream.
|