funkpower/catena
1
0
Fork 0
mirror of https://git.pumpin.online/funkpower/catena.git synced 2026-10-22 18:57:14 +00:00
Code Issues Packages Projects Releases Wiki Activity

citefind: every record has every key, and a title the cut landed inside is refused

Browse Source 
Tried against three of the largest books in the corpus — 2,700 chunks, 44,000
citations the model had already extracted — and two things showed up that the
smaller sample had not.

A chunk that begins part-way through an entry leaves a closing quotation mark
with no opening one, and the title read off it starts mid-word and runs to the
end of the line. A title carrying a stray quote, an escape, or an editor's
credit is now no title, and `unsettled` says so.

The optional fields are written as null rather than left out. A key that comes
and goes is a key every consumer has to guard, and the documented Python use
raised KeyError on the first reference without a page.
This commit is contained in:
funkpower
2026-09-20 11:31:24 -04:00
parent f3e70b9bf7
commit 2a2ecdc018
3 changed files with 60 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/citefind/README.md
+6 -2
View File
@@ -39,8 +39,8 @@ cargo install --git https://git.pumpin.online/funkpower/catena citefind
```
$ citefind bibliography.md
{"v":1,"idx":0,"text":"Smith, John. *The Making of the Second Estate*. Oxford: Clarendon Press, 1998.","authors":["Smith, John"],"year":1998,"title":"The Making of the Second Estate","container":"Clarendon Press","ids":[],"kind":"book","continuation":false,"unsettled":[]}
{"v":1,"idx":1,"text":"Doe, Jane, and Peter Brown. \"Obligation and Its Limits.\" *Journal of Medieval Studies* 12, no. 3 (2001): 45-67.","authors":["Doe, Jane","Peter Brown"],"year":2001,"title":"Obligation and Its Limits","container":"Journal of Medieval Studies","pages":"45-67","ids":[],"kind":"article","continuation":false,"unsettled":[]}
{"v":1,"idx":0,"text":"Smith, John. *The Making of the Second Estate*. Oxford: Clarendon Press, 1998.","marker":null,"authors":["Smith, John"],"year":1998,"title":"The Making of the Second Estate","container":"Clarendon Press","pages":null,"ids":[],"kind":"book","continuation":false,"unsettled":[]}
{"v":1,"idx":1,"text":"Doe, Jane, and Peter Brown. \"Obligation and Its Limits.\" *Journal of Medieval Studies* 12, no. 3 (2001): 45-67.","marker":null,"authors":["Doe, Jane","Peter Brown"],"year":2001,"title":"Obligation and Its Limits","container":"Journal of Medieval Studies","pages":"45-67","ids":[],"kind":"article","continuation":false,"unsettled":[]}
```
What a book cites, by year:
@@ -129,6 +129,10 @@ the mangled few, and they are cheaper to ask about together.
| `continuation` | Whether it only points back at an earlier entry |
| `unsettled` | The fields it could not read off the entry |
Every field is written on every record, `null` where there is nothing. A key
that comes and goes is a key every consumer has to guard, and the point of
`unsettled` is that an absence be legible.
`text` is kept whatever else was read, because every other field is an
interpretation of it and somebody will want to check.