0.3.1 — ask the index first, which answers in one request

The chain asked OpenLibrary's edition record first and its search index
second. The edition record is the slow one: measured against this
library it takes five to twenty-eight seconds, and it names its authors
by key, so a second request was needed anyway. The index, asked for its
fields by name, returns the title, the authors, the publisher, the
language, a page count and the subjects together, in about a quarter of
a second.

Reordering them turns three requests per book into one, and a pass over
five thousand books from fifty-three hours into well under one. The
edition record is still there for a subtitle or a publisher the index
did not give.
This commit is contained in:
funkpower
2026-09-22 09:36:46 -04:00
parent d551adc14c
commit e480f07c32
+9 -1
View File
@@ -55,9 +55,17 @@ impl Source {
match kind {
"doi" => &[Source::Crossref],
"arxiv" => &[Source::Arxiv],
// The index first. Asked for the fields by name it answers with
// the title, the authors, the publisher, the language, a page
// count and the subjects in one request, which is the whole of
// what a catalogue wants; the edition record is then needed only
// for a subtitle or a publisher the index did not give. Asking it
// second cost three requests per book where one usually does, and
// at three books in flight that is the difference between two
// days and two hours.
"isbn" => &[
Source::OpenLibrary,
Source::OpenLibrarySearch,
Source::OpenLibrary,
Source::GoogleBooks,
],
_ => &[],