Before trusting a catalogue you inherited. Metadata rots in recognisable ways, and the ways are worth naming: a title that is really a filename, an author field holding a publisher, a scanner's name where the work's name should be.
Finding the worst records first. Every fault carries a weight, so a thousand records sort by how much attention they deserve and a limited pass spends its time where it matters.
Recovering identifiers you already have. A file is often named exactly its ISBN, which is a free and certain identifier for a record that has none — certain because the check digit says so.
As a gate in a pipeline. It exits 1 when anything needed attention, so a nightly job can stop and ask.

Install

cargo install --git https://git.pumpin.online/funkpower/catena bibclean

Example

Five records, four of which have something wrong with them:

$ bibclean --flagged books.ndjson
{"v":1,"title":"The Human Condition","author":"Arendt, Hannah","isbn":"9780226586601","path":"/library/9780226586601.pdf","reasons":["allcaps_title","missing_isbn"],"severity":2,"changes":[{"field":"title","from":"THE HUMAN CONDITION","to":"The Human Condition","why":"allcaps_title"},{"field":"isbn","from":"","to":"9780226586601","why":"missing_isbn"}]}
{"v":1,"title":"Untitled","author":"Arguing About War","isbn":null,"path":null,"reasons":["placeholder_title","title_in_author","missing_isbn"],"severity":7,"changes":[]}
{"v":1,"title":"Microsoft Word - ch3.doc","author":"Oxford University Press","isbn":null,"path":null,"reasons":["filename_title","junk_title","publisher_as_author","missing_isbn"],"severity":9,"changes":[]}
{"v":1,"title":"The Fall: And the Rise","author":"Camus, Albert","isbn":null,"path":null,"shelf":"upstairs","reasons":["underscore_title","missing_isbn"],"severity":1,"changes":[{"field":"title","from":"The_Fall_ And the Rise_txt","to":"The Fall: And the Rise","why":"underscore_title"}]}

Three things to notice: the ISBN was recovered from the filename and the check digit confirmed it; the second record was not touched, because what it needs is a lookup and guessing would be worse; and "shelf": "upstairs" came through untouched, because fields this does not know about are carried.

The worst ten records, in order:

bibclean --flagged books.ndjson | jq -s 'sort_by(-.severity) | .[:10] | .[]' -c

Just the changes, for review:

bibclean books.ndjson | jq -r '.changes[] | "\(.why)\t\(.from)\t→\t\(.to)"'

Options

Option	Effect
`--flagged`	Write only the records that need attention

A missing ISBN alone does not count as needing attention: nearly every book lacks one.

Exit codes

Code	Meaning
`0`	Nothing needed attention
`1`	Something did — a fault worth fixing, or a slug worth replacing
`64`	The command line was wrong
`65`	A record could not be read
`74`	The file could not be read

Input

One JSON record per line. title, author, isbn, path, slug and id are read; everything else is carried through untouched, so a record can go through this and back into whatever it came from.

{"title":"THE HUMAN CONDITION","author":"Arendt, Hannah","path":"/library/9780226586601.pdf"}

Recovering a missing author

A book says who wrote it on its own copyright page. Hand in the opening pages as front_matter and a record whose author is missing gets a name proposed, with the evidence and how far it is worth trusting:

$ bibclean authors.ndjson
{"v":1,"title":"The Icon and the Axe","author":"Unknown",…,"reasons":["unknown_author","missing_isbn"],"severity":3,"changes":[],"author_proposal":{"author":"James H. Billington","method":"front_matter_copyright","confidence":0.9,"evidence":"…by James H. Billington","isbn":"0394708461"}}
{"v":1,"title":"Tragedy and Hope by Carroll Quigley","author":"",…,"reasons":["unknown_author"],"severity":3,"changes":[],"author_proposal":{"author":"Carroll Quigley","method":"title_by","confidence":0.7,"evidence":"by Carroll Quigley","isbn":null}}

Four signals, cheapest and most trustworthy first:

Method	Confidence	Read from
`front_matter_copyright`	0.9	"Copyright © 1966 by …"
`front_matter_loc`	0.8	The cataloguing block's main entry, written surname first
`title_by`	0.7	"… by Author Name" at the end of the title
`title_camel`	0.5	A run-together name at the front of the title

An ISBN found on the same page rides along with the name, because the two are facts about the same book.

A candidate that reads as a publisher rather than a person is refused — an imprint holds the copyright on a great many books and is the author of none of them — as is one with a digit in it, one that opens with an article, and one too long to be a name. Shouting is fixed, bare initials get their full stops, and co-authors are kept: John Schuster and John Sutton becomes John Schuster & John Sutton.

Nothing is applied. A record that already has an author is not second-guessed; the proposal is only ever made where there is a gap.

The front matter is read and never written back — it is evidence handed in, not part of the record.

Slugs

A slug is a public identifier — it reaches citations and notes, and outlives whatever produced it. When a record carries one, along with the id that makes it unique, its slug is judged too:

$ bibclean slugs.ndjson
{"v":1,"title":"Weaponizing Anthropology","author":"Laura Nader",…,"slug":"william-zinsser-on-writing-well-3451","id":3451,"reasons":[],"severity":0,"changes":[],"slug_proposal":{"current":"william-zinsser-on-writing-well-3451","proposed":"laura-nader-weaponizing-anthropology-3451","tier":"wrong-book","reasons":["only 0% of slug tokens appear in the current title/author — slug and metadata disagree"]}}
{"v":1,"title":"Mind and Brain","author":"Zinsser, William",…,"slug":"unknown-mind-and-brain-1234","id":1234,"reasons":[],"severity":0,"changes":[],"slug_proposal":{"current":"unknown-mind-and-brain-1234","proposed":"zinsser-william-mind-and-brain-1234","tier":"placeholder-author","reasons":["slug opens with placeholder author 'unknown'"]}}

Nothing is renamed. A proposal comes back beside the slug the record has, and the tier says how much the evidence is worth:

Tier	Evidence	Safe to apply in bulk?
`placeholder-author`	The slug opens with a placeholder, so it was minted without an author	yes
`filename-artefact`	The slug carries a mirror suffix, a wrapper format or a path	yes
`wrong-book`	Too few of the slug's words appear in the record at all	no
`noop-metadata-junk`	Regenerating gives the same slug: the record is the problem	no
`defer-new-slug-junk`	The new slug would be no better than the old one	no
`review-loses-volume`	The current slug names a volume the new one would lose	no
`review-low-recall`	The new slug recovers too little of the title	no

The first two rest on evidence inside the slug itself and cannot be wrong about it. The rest are disagreements between the slug and the record, and a disagreement does not say which side is at fault — which is exactly the case where renaming automatically makes things worse.

Only the safe tiers:

bibclean books.ndjson | jq -c 'select(.slug_proposal.tier | IN("placeholder-author","filename-artefact"))'

The faults it names

Reason	Weight	What it means
`placeholder_title`	4	The whole title is "Untitled", "Document", "N/A"
`placeholder_author`	4	The same, in the author field
`path_title`	4	A file path sitting in the title
`numeric_author`	4	An author field opening with a bare year or series number
`unknown_author`	3	"Unknown", "Anonymous", empty
`filename_title`	3	A bare filename, extension and all
`junk_title`	3	The name of the tool that made the file, not the work
`publisher_as_author`	3	An imprint in the author field
`title_in_author`	3	The title is a placeholder and the real title is in the author field
`allcaps_title`	2	Shouting
`lowercase_title`	2	A filename stub: "heidegger areopagite"
`missing_apostrophe`	2	"Lossky s Argument"
`possible_swap`	2	Title and author look to be the wrong way round
`underscore_title`	1	Underscores standing in for spaces and colons
`missing_isbn`	0	Reported, but not a reason to do anything

Each test is deliberately conservative. A real book called "The Document" or "A Test of Faith" must not be swept up, so the placeholder tests only fire when the entire field is the placeholder; and "Oxford University Press" in an author field is wrong while "Oxford, Michael" is a person.

The repairs it makes

Fault	Repair
`underscore_title`	A trailing format name comes off, an underscore before a subtitle becomes a colon, the rest become spaces
`allcaps_title`	Title-cased, keeping small words small, hyphenated parts cased on both sides, and accented first letters capitalised rather than skipped
`missing_isbn`	Filled from the filename, if a checksum-valid ISBN is in it

Everything else is named and left alone, because the fix needs a lookup and a wrong guess in a catalogue is harder to find than a missing value.

As a library

let cleaned = bibclean::clean(&book);
if bibclean::reasons::worth_fixing(&cleaned.reasons) {
    for change in &cleaned.changes {
        println!("{}: {} → {} ({})", change.field, change.from, change.to, change.why);
    }
}

The pieces are usable on their own: bibclean::isbn::valid, bibclean::isbn::from_filename, bibclean::fix::title_case, bibclean::matching::same_work, bibclean::slug::slug_for.

same_work is worth knowing about separately. It compares vocabulary rather than characters, because character similarity is not usable for titles: "The Cambridge Companion to Aquinas" and "The Cambridge Companion to Hume" are eighty-six per cent the same string, and believing that attaches one book's description to another.

README.md

bibclean

What it does

When you would use it