Commit Graph
27 Commits
Author SHA1 Message Date
funkpower 1387b1a62b chunkmd: a reading list with nothing marking it is citation material
Block, N. 1981. Psychologism and behaviorism. Philosophical Review 90:5-43.
    Dennett, D. C. 1987. The Intentional Stance. MIT Press.
    Geach, P. 1957. Mental Acts. Routledge and Kegan Paul.

That is how a reader or a companion sets the further-reading list at the end
of each chapter: indented under the last paragraph of the essay, with no
heading over it and no numbers beside it. Nothing announced it as a
bibliography, so the chunk stayed body text with no flag, the citation stage
never selected it, and a book made of such chapters came out with one or two
citations. *The Cambridge Companion to Ancient Greek Political Thought* had
three; Chalmers's *Philosophy of Mind* reader had one.

The hard part is that an index line opens identically — `Newton, Isaac, 5,
93, 141` — and a page of them would otherwise flood the table. What tells
them apart is that a reference names a work: five words of three letters or
more on the entry's own line, and not two page numbers written as links.

Over 15,925 unflagged body chunks sampled from the corpus this fires on 2.6%,
and seven of eight sampled were real reference lists.

Recorded in parity/KNOWN.md, and the harness allows this one difference by
name: a chunk the Rust flags and the Python does not, identical in every
other field. A flag the Python sets and the Rust does not is still a failure.
2026-09-20 16:58:07 -04:00
funkpower d47881ef3b idnorm and bibfetch: an identifier read, and a register asked 2026-09-19 19:19:54 -04:00
funkpower 4d14544409 bibclean: an abstract that arrived as an index 2026-09-19 19:09:27 -04:00
funkpower e6c51a5943 parity: the vocabulary rules gained a field 2026-09-19 19:07:43 -04:00
funkpower 484017fc18 vocabmerge: a subject scheme, supplied rather than compiled in 2026-09-19 19:06:57 -04:00
funkpower 05b363ea8a seriesof: the series a book belongs to, and which volume this is 2026-09-19 19:03:47 -04:00
funkpower 619672d403 pdfgutter: never rewrite a page tree it cannot account for 2026-09-19 16:51:12 -04:00
funkpower a03dcd400c avplan: paths an encoder can actually read, and a pipeline that runs 2026-09-19 15:31:17 -04:00
funkpower 3c697bee50 pdfgutter: two facing pages photographed as one, found and cut 2026-09-19 15:19:35 -04:00
funkpower 49f12c9718 avplan: whether a recording plays, and the one call that fixes it 2026-09-19 15:07:33 -04:00
funkpower ea0a3694b1 mdsplit: the works inside one file, and the books inside scripture 2026-09-19 14:58:54 -04:00
funkpower 9bf59f6ebb vocabmerge: the near-duplicate headings a catalogue accumulates 2026-09-19 14:37:22 -04:00
funkpower 739e6c82e9 lints 2026-09-19 14:30:22 -04:00
funkpower 3c16892ebf bibclean: a name from the book's own pages, where the record has none 2026-09-19 14:29:12 -04:00
funkpower 14e1fcf3ef bibclean: slugs, and whether the one a record has still names it 2026-09-19 14:23:36 -04:00
funkpower b0743d5bf7 a missing word, a hidden dependency, and a second pass that changed things 2026-09-19 13:00:29 -04:00
funkpower 6e26d5cf4b bibclean: record faults, weighed, with the repairs that need no lookup 2026-09-19 12:50:53 -04:00
funkpower 1071f0bed3 booksniff, and one place for the parts every tool shares 2026-09-19 12:40:20 -04:00
funkpower 36afe3350b cuepack: cues into passages that keep their timestamps 2026-09-19 12:32:41 -04:00
funkpower b6602ee1e9 rrf: rank fusion without the database 2026-09-19 12:29:54 -04:00
funkpower 17a03b0087 cueseg: topic chapters from a transcript, and a worked pipeline in the readme 2026-09-19 12:21:35 -04:00
funkpower c7d0767376 cuekit and cuenorm: three subtitle formats, one cue stream 2026-09-19 11:54:36 -04:00
funkpower 33c2e40a1a pdftriage: text layer verdicts, including the reflected-font recovery 2026-09-19 11:46:34 -04:00
funkpower bd0ae11d01 mdscrub: decode character references, keep block boundaries, narrow the fence 2026-09-19 11:36:31 -04:00
funkpower 1b3c002b3d mdscrub: residual html, control characters, headings, page markers 2026-09-19 11:30:05 -04:00
funkpower e0473fb8ae chunkmd: page and chapter aware chunking, roles, footnote flag 2026-09-19 11:22:56 -04:00
funkpower d8929371b4 parity harness, pinned to the python tokeniser over 10k lines 2026-09-19 11:06:55 -04:00