3 Commits
Author SHA1 Message Date
funkpower 6f39b75d94 citefind: the short note a history book uses for everything after the first
`Unger, Dutch Shipbuilding, 33-4` — a surname, a short title, and where in
it. No year, no publisher, nothing but commas. It is how a humanities book
cites a work it has already given in full, which is most of what its notes
are: one book in the corpus has 2,229 such chunks and this read almost none
of them.

The locators trailing a short note are not part of its title — `Sjoerds,
Algemene Beschryvinge, i, f I, 501-2` cites the Beschryvinge, not the
Beschryvinge-comma-i-comma-f-I. An entry ending on a year is an imprint and
goes on being read as one.

Also: a work after a semicolon may open on a quotation mark, and the marks a
style puts round a short title belong to the style.

626 more references over the 202 reference sections sampled from the corpus.
2026-09-20 14:22:43 -04:00
funkpower f3e70b9bf7 citefind: a note that cites three works is three references
Measured against the citations an LLM had already extracted from 40 books of
the same corpus — 19,340 of them over 3,424 chunks — and improved where the
comparison showed a real gap rather than a difference of task.

On pages of notes, where most citations live: year agreement with the model
44% -> 56%, title 18% -> 23%, author 16% -> 24%.

What changed:

- A note listing several works, semicolon-separated, becomes one record per
  work. The note number is kept on all of them.
- A full stop is not a separator. It parts the author from the title in every
  bibliography entry there is.
- A page of body text with notes at its foot no longer reads the body as the
  first note.
- A run carrying a page reference or a semicolon is not reported as a title.
- Six sentences with no imprint, no page and no identifier is a remark about
  the argument, not a reference.
- A translator standing where the journal goes is a credit, not a container,
  and `(London, 1973)` is a place, not a publisher.

The model is not ground truth and the comparison is not a score: on several
sampled disagreements it named the journal where the article was wanted, or
the collected works instead of the essay.
2026-09-20 11:10:42 -04:00
funkpower c071f84829 citefind: the references in a bibliography, and what could not be read off them
Cuts a reference list into entries — at the note numbers where there are any,
at the layout where there are not — and reads each as far as its punctuation
allows. Fields that would not parse are named in `unsettled` rather than
guessed at, and `--prompt` asks a model about exactly those and no others.

Over 202 real reference sections from the corpus: 8,566 references, 95% with
a title, 82% with a year, 46% settling every field with no model involved.
A title is refused where the entry was never divided, so no record reports
its own text back as the work it cites.

`ibid.` is flagged and left alone — it points at a work, and the entry does
not say which. A rule of dashes is filled in, because the style defines it as
the author above.

chunkmd gains `note_starts` and `cites_something`: recognising a page of
notes and splitting one are the same knowledge, and two copies of that
pattern would drift.
2026-09-20 10:38:48 -04:00