funkpower
c071f84829
Cuts a reference list into entries — at the note numbers where there are any, at the layout where there are not — and reads each as far as its punctuation allows. Fields that would not parse are named in `unsettled` rather than guessed at, and `--prompt` asks a model about exactly those and no others. Over 202 real reference sections from the corpus: 8,566 references, 95% with a title, 82% with a year, 46% settling every field with no model involved. A title is refused where the entry was never divided, so no record reports its own text back as the work it cites. `ibid.` is flagged and left alone — it points at a work, and the entry does not say which. A rule of dashes is filled in, because the style defines it as the author above. chunkmd gains `note_starts` and `cites_something`: recognising a page of notes and splitting one are the same knowledge, and two copies of that pattern would drift.
citefind
The references in a bibliography or a page of notes.
Part of catena.
What it does
A reference list is the one place in a book where the text is already structured. Every entry names a work, and most name it in one of a handful of shapes that have not changed in a century. What is missing is not information but punctuation: the entries run together, and nothing in the file says where one stops.
So this does two jobs. It cuts the block into entries — at the note numbers where there are any, at the layout where there are not. Then it reads each entry as far as its shape allows, and names the fields it could not settle rather than guessing at them.
That last part is the point. A made-up year and a real one are indistinguishable three stages downstream; a missing one is not.
When you would use it
- Building a citation graph — which works a book leans on, and how often.
- Finding the identifiers in a reference list, to hand to
idnormandbibfetch. - Filling a catalogue from the back of a book rather than from a register.
- Costing a model run: ask one about the entries that did not parse, and nothing else.
Install
cargo install --git https://git.pumpin.online/funkpower/catena citefind
Example
$ citefind bibliography.md
{"v":1,"idx":0,"text":"Smith, John. *The Making of the Second Estate*. Oxford: Clarendon Press, 1998.","authors":["Smith, John"],"year":1998,"title":"The Making of the Second Estate","container":"Clarendon Press","ids":[],"kind":"book","continuation":false,"unsettled":[]}
{"v":1,"idx":1,"text":"Doe, Jane, and Peter Brown. \"Obligation and Its Limits.\" *Journal of Medieval Studies* 12, no. 3 (2001): 45-67.","authors":["Doe, Jane","Peter Brown"],"year":2001,"title":"Obligation and Its Limits","container":"Journal of Medieval Studies","pages":"45-67","ids":[],"kind":"article","continuation":false,"unsettled":[]}
What a book cites, by year:
citefind bibliography.md | jq -r 'select(.year) | .year' | sort | uniq -c
The identifiers, checked and resolved:
citefind refs.md | jq -r '.ids[] | select(.kind == "doi") | .id' | idnorm | bibfetch
The works it leans on hardest, from a page of endnotes:
citefind notes.md | jq -r 'select(.title) | .title' | sort | uniq -c | sort -rn | head
Asking a model about the rest
--prompt writes a question about the entries that did not parse, and only
those. Nothing is sent from here: the prompt goes to stdout and what happens
to it is yours to decide.
$ citefind --prompt refs.md
Below are entries from the bibliography or notes of a book. Each was read as
far as its punctuation allowed, and the fields that could not be read off it
are listed with it.
--- 0
text: Marron, Kevin Ritual Abuse, Seal Books, 1988
year: 1988
wanted: authors, title, container
...
citefind --prompt refs.md | ollama run qwen3:8b > reply.json
citefind --merge reply.json refs.md
A reply fills in only the fields that were asked about. A model that volunteers a better title for an entry whose title was read off the page is overruled — the page is evidence and the model is a guess — and a reply that is not one JSON object is refused rather than half-applied.
One prompt covers every unsettled entry rather than one prompt each. A reference list is long and mostly ordinary; the entries worth asking about are the mangled few, and they are cheaper to ask about together.
Options
| Option | Effect |
|---|---|
--prompt |
Write the question about what did not parse, instead of the records |
--merge FILE |
Fill in what did not parse from a model's reply |
--quiet, -q |
Say nothing; the exit code is the answer |
--man |
Write this page as a manual page, in roff |
Exit codes
| Code | Meaning |
|---|---|
0 |
It found references |
1 |
It found none |
64 |
The command line was wrong |
65 |
The input was not readable as text, or a reply was not JSON |
74 |
The file could not be read |
The record
| Field | What it holds |
|---|---|
text |
The entry as it appeared, whitespace collapsed |
marker |
The note number it carried, where it carried one |
authors |
The names, in the order and the form the entry gave them |
year |
The year of publication |
title |
The work cited — for an article the article, not the journal |
container |
The journal it appeared in, or the publisher of a book |
pages |
The pages, as written |
ids |
Any DOI, ISBN, ISSN, arXiv id or URL, normalised |
kind |
book, article, chapter, web or other |
continuation |
Whether it only points back at an earlier entry |
unsettled |
The fields it could not read off the entry |
text is kept whatever else was read, because every other field is an
interpretation of it and somebody will want to check.
How it cuts
A page of notes is cut at its note numbers, in any of the shapes
conversion leaves them: 12., [12], fn39., a superscript, or a
back-reference link. The pattern is the one
chunkmd uses to recognise a page of notes, so the
two cannot drift apart, with one form added — a link pointing at another file
rather than at a fragment. chunkmd is deciding whether a chunk is a page of
notes and can afford to miss one marker in thirty; here a missed marker runs
two references together.
A note that cites nothing is dropped. A page of notes mixes bibliographic notes with discursive ones, and a remark about the argument is not a reference.
A bibliography numbers nothing, so the signal is layout: a blank line between entries, a hanging indent under each, or the surname-comma opening that nearly every author-first entry begins with. Whichever the block carries is the one used. When it carries none of them the fallback is one entry per line, which is right for a list that was never wrapped and visibly wrong for one that was.
What it will not do
Resolve ibid. A continuation refers to whatever was cited before it,
which is a fact about the surrounding page and not about the entry. It is
flagged continuation and left alone, and it is left out of --prompt
entirely — there is nothing in it to ask about.
Report a title it did not find. An entry held together by commas alone, with no names to anchor it, yields no title rather than its own text echoed back.
A rule of dashes where the author should be is the exception, and a
deliberate one: ———. *The Second Book*. means the author of the entry above,
the style says so, and the entry it refers to is the one immediately before.
That is read and filled in.
As a library
for one in citefind::read(&text) {
println!("{} ({}) {:?}", one.authors.join("; "), one.year.unwrap_or(0), one.title);
}
citefind::split::entries cuts a block without reading the entries.
citefind::parse::one reads an entry already cut. citefind::prompt::compose
and merge are the two halves of the model-facing path, and
prompt::PROMPT_VERSION says which wording produced a given answer.
From Python, JavaScript or C, through
bindings:
for one in catena.find_citations(text):
print(one["year"], one["title"], one["unsettled"])