The commit that reordered the chain did not carry the version bump. The
bump was chained after check.sh with &&, the commit after a semicolon, so
a failing check skipped the bump and let the commit through. The tag then
pointed at a tree still calling itself 0.3.0, which the release workflow
asserts against and would have refused.
check.sh was failing because a test names which register is asked first
about an ISBN, which is exactly what changed. It now names the index, for
the reason the reorder happened.
The chain asked OpenLibrary's edition record first and its search index
second. The edition record is the slow one: measured against this
library it takes five to twenty-eight seconds, and it names its authors
by key, so a second request was needed anyway. The index, asked for its
fields by name, returns the title, the authors, the publisher, the
language, a page count and the subjects together, in about a quarter of
a second.
Reordering them turns three requests per book into one, and a pass over
five thousand books from fifty-three hours into well under one. The
edition record is still there for a subtitle or a publisher the index
did not give.
The chain asked three registers and returned whichever answered first
with a title and an author, discarding the rest. The registers are
complementary rather than redundant: OpenLibrary's edition record
carries the publisher, the pagination and the language and names its
authors by key, while its search index carries the authors and not the
publisher. One book, two half-answers, and the whole of it only if they
are put together.
They are merged now, first answer winning each field, so the order of
the chain is the order of trust. The record says which registers spoke.
It also carries what was being thrown away: subjects, language and a
page count, from whichever register has them. The index will not return
those unless asked for by name, which is how a language it held went
unread. Subjects are capped at twelve — the index lists every heading
every edition of a work has ever been given, which on a well-known book
is hundreds.
An edition record keeps its subtitle in a separate field, so a title
came back without it.
book-pipeline reaches for colophon and bibfetch on every book it repairs.
Shelling out would mean a process per book per tool across five thousand
books, and parsing back the JSON it is about to parse anyway.
bibfetch's fetch loop lived in main.rs, so binding it meant either
lifting it or writing a second one. A second implementation of 'which
register answers about this kind of identifier, and in what order' would
drift from the first, and the copy nobody runs from a terminal would
drift silently. It is now bibfetch::ask, and the command line calls it
too — verified by running the CLI before and after and diffing.
fetch_identifier releases the GIL around the request. It is the only
function here that opens a socket, and holding the interpreter for the
length of an HTTP call would stop every other thread in it.
bibfetch is optional and only the python feature turns it on: its HTTP
client does not build for wasm32, and an unconditional dependency would
have taken the JavaScript package down with it.
An ISBN is not a DOI. The body that assigns a DOI is the only one that has
it, so one register is the whole answer. A book's identifier is assigned by
its publisher and recorded by whoever gets round to it, and the record that
comes back is as often as not missing the thing the caller wanted: asked
about 9780199205592, OpenLibrary's edition record names *The First World
War* and the press that printed it, and nobody at all as its author, because
an edition record names its authors by key.
So an ISBN now has a chain rather than a register. The edition record first,
then OpenLibrary's own index — which answers with `author_name` filled in
and costs nothing — and Google Books last, because it answers anonymous
callers only until a daily quota runs out, after which it replies 429 to
everyone behind the same address, as it is doing here today.
The first complete answer wins, an incomplete one is kept in case nothing
better comes, and a register being down is a failure only when every
register is. `source` on the record says which one answered.
colophon comes with it: the imprint reader was taking `Oxford is a
registered trade mark of Oxford University Press` for a publisher and
`Oxford N`, off `Oxford New York`, for a place. A legal notice mentioning a
press is not the press, and a city is the city and not the first letter of
the next one.
funkpower