citefind: the references in a bibliography, and what could not be read off them

Cuts a reference list into entries — at the note numbers where there are any,
at the layout where there are not — and reads each as far as its punctuation
allows. Fields that would not parse are named in `unsettled` rather than
guessed at, and `--prompt` asks a model about exactly those and no others.

Over 202 real reference sections from the corpus: 8,566 references, 95% with
a title, 82% with a year, 46% settling every field with no model involved.
A title is refused where the entry was never divided, so no record reports
its own text back as the work it cites.

`ibid.` is flagged and left alone — it points at a work, and the entry does
not say which. A rule of dashes is filled in, because the style defines it as
the author above.

chunkmd gains `note_starts` and `cites_something`: recognising a page of
notes and splitting one are the same knowledge, and two copies of that
pattern would drift.
This commit is contained in:
funkpower
2026-09-20 10:38:48 -04:00
parent 30ff3361ee
commit c071f84829
22 changed files with 2612 additions and 6 deletions
+24 -6
View File
@@ -94,6 +94,28 @@ static CITE_SIGNAL: LazyLock<FancyRegex> = LazyLock::new(|| {
.expect("a valid pattern")
});
/// Where each numbered note begins, in reading order.
///
/// Public because splitting a page of notes into the notes on it is the same
/// knowledge as recognising one, and two copies of the pattern above would
/// drift apart.
pub fn note_starts(text: &str) -> Vec<usize> {
NOTE_MARKER
.find_iter(text)
.filter_map(Result::ok)
.map(|found| found.start())
.collect()
}
/// Whether an entry reads as a citation rather than as a remark.
///
/// Also public: a note that cites nothing is a note a bibliography has no
/// use for, and the caller deciding that should ask the same question this
/// module asks.
pub fn cites_something(entry: &str) -> bool {
CITE_SIGNAL.is_match(entry).unwrap_or(false)
}
/// Whether a chunk carries footnote or endnote citation material.
///
/// The structural signal first, when the source had one. Otherwise each note
@@ -104,11 +126,7 @@ pub fn looks_like_citations(text: &str) -> bool {
return true;
}
let starts: Vec<usize> = NOTE_MARKER
.find_iter(text)
.filter_map(Result::ok)
.map(|found| found.start())
.collect();
let starts = note_starts(text);
if starts.len() < MIN_CITED_NOTES {
return false;
}
@@ -117,7 +135,7 @@ pub fn looks_like_citations(text: &str) -> bool {
for (at, start) in starts.iter().enumerate() {
let end = starts.get(at + 1).copied().unwrap_or(text.len());
let entry: String = text[*start..end].chars().take(ENTRY_WINDOW).collect();
if CITE_SIGNAL.is_match(&entry).unwrap_or(false) {
if cites_something(&entry) {
cited += 1;
if cited >= MIN_CITED_NOTES {
return true;