recipes, examples: shelving a library by what its books printed

The suite could read a cataloguing block and could ask a register, and
nothing showed the two together — which is the only way either is worth
acting on. A title page alone gives up a running head when the scan
starts late; an ISBN alone returns a confidently wrong book when the
catalogue has been carrying the wrong one for years.

The example's title comparison folds to letters and digits rather than
comparing strings. A register and a title page differ over case, over a
curly apostrophe against a straight one, and over whether the subtitle
counts as part of the title. An example that called those disagreements
would teach the wrong lesson, and did until it was fixed.

It runs on this library and finds both of the cases worth finding: a file
called Document5 by -ducati996 that both sources agree is Russell's
Problems of Philosophy, and a Pelikan whose stored ISBN belongs to a book
about sculpture.
This commit is contained in:
funkpower
2026-09-21 10:21:52 -04:00
parent 18cc217c0d
commit f4d936e620
3 changed files with 217 additions and 0 deletions
+92
View File
@@ -6,6 +6,7 @@ this suite, because a real pipeline has a database at the end of it.
- [A shelf of files, sorted before anything else touches it](#a-shelf-of-files-sorted-before-anything-else-touches-it)
- [A catalogue out of what the files already say](#a-catalogue-out-of-what-the-files-already-say)
- [Shelving a library by what its books printed](#shelving-a-library-by-what-its-books-printed)
- [Asking a model, without this suite calling one](#asking-a-model-without-this-suite-calling-one)
- [A scanned book into searchable passages](#a-scanned-book-into-searchable-passages)
- [A collected edition into one record per work](#a-collected-edition-into-one-record-per-work)
@@ -98,6 +99,97 @@ with:
langid converted/*.md | jq -r 'select(.language != "en") | [.path, .name] | @tsv'
```
## Shelving a library by what its books printed
A catalogue's subjects come from wherever the file came from, which is to say
from nowhere in particular. A book's own cataloguing block is the one place its
subjects are stated rather than guessed — and stated in Library of Congress
form, which is precise and useless for browsing:
```
$ colophon converted/hobbes.md | jq -r '.subjects[]'
Hobbes, Thomas, 1588–1679. Leviathan
Political science – Early works to 1800
```
Nobody has a shelf called that. Give `colophon` the list you actually file by
and it files them for you:
```
$ cat shelf.txt
Political Science = political science, politics
Byzantine History = byzantium, byzantine, constantinople
Theology = theology, christianity
Ethics
$ colophon --shelf shelf.txt converted/*.md | jq -c '{title, shelved, unshelved}'
{"title":"Leviathan","shelved":["Political Science"],"unshelved":["Hobbes, Thomas, 1588–1679. Leviathan"]}
{"title":"Christianity and Classical Culture","shelved":["Theology"],"unshelved":[]}
```
`unshelved` is the point of the exercise. A subject quietly dropped reads
exactly like a book that printed none, and the difference decides whether you
go looking for a heading your shelf is missing.
**The title page is not evidence on its own.** A scan that begins mid-frontmatter
gives up a running head, a copyright line, or the author's name where the title
should be. What makes a printed record trustworthy is a second one agreeing,
and the ISBN in the same block is how you get one:
```sh
colophon --shelf shelf.txt converted/*.md > printed.ndjson
jq -r 'select(.isbns != null) | .isbns[0]' printed.ndjson \
| idnorm --quiet \
| bibfetch --polite you@example.org > register.ndjson
```
Now two sources that cannot have copied each other have spoken, and you can
say which books are settled and which need a person:
```sh
jq -s -r '
(.[1] | map({key: .id, value: .}) | from_entries) as $reg
| .[0][]
| . as $printed
| ($reg[$printed.isbns[0] // ""] // {}) as $found
| [ $printed.title,
$found.title,
(if $printed.title and $found.title then "corroborated" else "one source" end)
] | @tsv
' printed.ndjson register.ndjson
```
```
The First World War The First World War: A Very Short Introduction corroborated
Document5 The Problems of Philosophy corroborated
Born Fighting — one source
```
The second row is the case worth the whole recipe: a file named `Document5`
whose author reads `-ducati996`, which the book itself and OpenLibrary agree
is Russell's *The Problems of Philosophy*. The third is the honest failure —
one source said something, nothing confirmed it, and it goes to a human.
In process, where a pipeline actually lives:
```python
import catena
shelf = open("shelf.txt").read()
printed = catena.read_colophon(markdown, shelf=shelf)
found = None
if printed["isbns"]:
found = catena.fetch_identifier(printed["isbns"][0], polite="you@example.org")
settled = bool(printed.get("title") and found and found.get("title"))
```
`catena.shelve(shelf, subjects)` files headings a catalogue already holds, by
the same rules, so a record's existing subjects and its book's printed ones are
judged alike.
## Asking a model, without this suite calling one
A tool here may compose a prompt. It does not send one. That keeps every