The bar a guessed title has to clear was waived when the entry also gave up
a container, on the reasoning that a container vouches for the rest of the
reading. Auditing the corpus rather than the reference chapters showed that
is wrong, and wrong on the commonest note there is:
As Lewis White Beck has rightly put it. See his Early German
Philosophy (Cambridge, Mass., 1969), p. 439.
The remark opens, the citation follows, and the second sentence becomes the
container — so the first was stored as the work and the book named in it was
lost. Neither a siglum nor a remark is the name of a work now, whatever else
came off the entry.
Measured where these notes actually live, which is flagged body chunks
rather than bibliography chapters: on the corpus the class ran at 1.43% of
rows naming a work. The cost is 46 rows in 34,359 on the reference holdout,
where the share naming an author rises from 88.1% to 88.2%.
The stop after `P.` is an initial rather than a sentence stop, so `Hoyt,
Edwin P. The Supersalesmen. New York: World Publishing, 1962.` never got cut
there. The name came back as `Hoyt, Edwin P. The Supersalesmen` and the next
segment along — the imprint — became the work. Both fields wrong from one
missing cut.
An inverted name whose forenames end in an initial, with two or more words
after it, now ends at that initial. Two words, because one is a forename:
`Price, M. Philips` and `Drexler, K. Eric` are whole names and must not be
cut.
And the segment the reader settles on for a title may not be an imprint —
which is what the function has said it did since it was written, in a comment
above a predicate that never checked. Asked narrowly enough that a title with
a colon in it survives.
On 199 reference chapters never used for tuning, imprints stored as the work
fall from 376 to 55, and the share of rows naming an author rises from 86.1%
to 88.1%.
Three refusals named on the page: a note's own remark, a siglum, and the
navigation strip a PDF reader leaves behind. Says where the line is drawn —
at the guess rather than at the title, so an entry that marked its own title
is taken at its word.
Three more shapes in the table — a reading list with nothing marking it, an
editor in the author's place, a reprint's two years — and three more refusals
named: a keyed short cite, a translator's signed note, and a page of
references whose lines happen to begin on numbers.
Also says what the cut does with a blank line now: it is often after the
running head rather than between the entries under it, so every block it
produces is looked at again.
The README now lists the forms it handles with an example of each, says that
where the italics sit is what decides whether they are the title or the
journal, and sets out what it refuses and why — scripture, Migne, a
contributor's biography, a note that only thanks people, a page footer.
Recipes gains "What a book leans on": one bibliography read, then the same
over a shelf, counted per citing book so a work cited forty times in one
bibliography still counts once. Every command was run — thirty converted
books gave 14,481 references, 10,536 naming a work, in about a second.
`records.md` now points at the five tools whose record shapes live in their
own pages rather than in `corpus-core`.
And pandoc escapes an underscore, so the rule of them that stands in for a
repeated author arrives as `\_\_\_\_\_` and was not being recognised.
Found by putting a real model at the end of the chain: a page of notes from a
theological book composed a prompt asking who wrote Matthew 4:17, and the
model rightly answered nothing for all twenty-three entries.
Those are scripture references, which is most of what the notes of such a book
are, and they are now not references at all — no record, and no question put
to anybody about them. A journal and its volume look similar on the page and
are still read as what they are.
Also: the stop that belonged to a marker no longer opens the entry after it.
Tried against three of the largest books in the corpus — 2,700 chunks, 44,000
citations the model had already extracted — and two things showed up that the
smaller sample had not.
A chunk that begins part-way through an entry leaves a closing quotation mark
with no opening one, and the title read off it starts mid-word and runs to the
end of the line. A title carrying a stray quote, an escape, or an editor's
credit is now no title, and `unsettled` says so.
The optional fields are written as null rather than left out. A key that comes
and goes is a key every consumer has to guard, and the documented Python use
raised KeyError on the first reference without a page.
Measured against the citations an LLM had already extracted from 40 books of
the same corpus — 19,340 of them over 3,424 chunks — and improved where the
comparison showed a real gap rather than a difference of task.
On pages of notes, where most citations live: year agreement with the model
44% -> 56%, title 18% -> 23%, author 16% -> 24%.
What changed:
- A note listing several works, semicolon-separated, becomes one record per
work. The note number is kept on all of them.
- A full stop is not a separator. It parts the author from the title in every
bibliography entry there is.
- A page of body text with notes at its foot no longer reads the body as the
first note.
- A run carrying a page reference or a semicolon is not reported as a title.
- Six sentences with no imprint, no page and no identifier is a remark about
the argument, not a reference.
- A translator standing where the journal goes is a credit, not a container,
and `(London, 1973)` is a place, not a publisher.
The model is not ground truth and the comparison is not a score: on several
sampled disagreements it named the journal where the article was wanted, or
the collected works instead of the essay.
Cuts a reference list into entries — at the note numbers where there are any,
at the layout where there are not — and reads each as far as its punctuation
allows. Fields that would not parse are named in `unsettled` rather than
guessed at, and `--prompt` asks a model about exactly those and no others.
Over 202 real reference sections from the corpus: 8,566 references, 95% with
a title, 82% with a year, 46% settling every field with no model involved.
A title is refused where the entry was never divided, so no record reports
its own text back as the work it cites.
`ibid.` is flagged and left alone — it points at a work, and the entry does
not say which. A rule of dashes is filled in, because the style defines it as
the author above.
chunkmd gains `note_starts` and `cites_something`: recognising a page of
notes and splitting one are the same knowledge, and two copies of that
pattern would drift.
funkpower