Two things between a converted EPUB and its chapter titles.
The heading level taken as the chapter map was the shallowest present. Pandoc
writes the book's title as the only `#` and every real chapter as a `##`, so
the shallowest level had one heading in it and the whole book came back as a
single chapter. The dividing level is now the shallowest one used more than
once — a level used once is a title.
And an empty bracketed span survived the heading cleaner, because the pattern
wanted at least one character between the brackets. Pandoc writes an anchor
as exactly that: `## []{#link2H_INTR} Introduction`.
Gibbon's first volume, converted from the Project Gutenberg EPUB: 2 distinct
chapter titles before, 61 after, and none of them carrying an element id.
The chapter a passage carries was resolved at the offset of the page it came
from, not the offset of the passage. A document with no page markers is a
single page beginning at zero, so every chunk in such a book answered with
whichever heading came first in the file — and took the role classification
with it, since that reads the chapter title.
Measured over a corpus of 5,386 books: all 1,232 unpaginated books with
twenty or more chunks carried exactly one chapter title from cover to cover,
as did 73% of the paginated ones. A bibliography chapter was therefore
invisible to anything selecting on the chapter title, which is the first of
the three ways the citation stage finds one.
The packer now hands back each passage with the offset it began at, and the
page's own offset is the true start of its text rather than the position of
the marker before it.
An entry opening was a surname at the start of a line, and a converted
bibliography is often a bulleted list — so the bullet sat exactly where the
surname was looked for and not one entry was seen:
• Skorupski, J. 1995. 'Agent-Neutrality, Consequentialism,
Utilitarianism', Utilitas, 7: 49-54.
The whole page went unflagged, no chunk of it was ever selected, and books
whose only bibliography arrived this way came back with no citations at all.
A bullet may now stand where the surname would.
Measured against the corpus: of 300 sampled body chunks that open lines on a
bullet and an inverted name, 0 were flagged before and 223 after. Across
3,000 randomly sampled unflagged body chunks, the change fires on none — it
buys the bulleted list and nothing else.
Block, N. 1981. Psychologism and behaviorism. Philosophical Review 90:5-43.
Dennett, D. C. 1987. The Intentional Stance. MIT Press.
Geach, P. 1957. Mental Acts. Routledge and Kegan Paul.
That is how a reader or a companion sets the further-reading list at the end
of each chapter: indented under the last paragraph of the essay, with no
heading over it and no numbers beside it. Nothing announced it as a
bibliography, so the chunk stayed body text with no flag, the citation stage
never selected it, and a book made of such chapters came out with one or two
citations. *The Cambridge Companion to Ancient Greek Political Thought* had
three; Chalmers's *Philosophy of Mind* reader had one.
The hard part is that an index line opens identically — `Newton, Isaac, 5,
93, 141` — and a page of them would otherwise flood the table. What tells
them apart is that a reference names a work: five words of three letters or
more on the entry's own line, and not two page numbers written as links.
Over 15,925 unflagged body chunks sampled from the corpus this fires on 2.6%,
and seven of eight sampled were real reference lists.
Recorded in parity/KNOWN.md, and the harness allows this one difference by
name: a chunk the Rust flags and the Python does not, identical in every
other field. A flag the Python sets and the Rust does not is still a failure.
\[←788 \]
\[[←193](index_split_011.xhtml#citation-193) \]
An arrow pointing back to where the note was called, sometimes a link and
sometimes not, inside escaped brackets. Neither pattern knew it, so a page of
notes set this way read as body prose: the chunk was never flagged, the
citation stage never selected it, and the book came out with nothing.
Ten books of the corpus use it and eight of them had zero citations —
including both volumes of *The Decline of the West*, whose 830 endnotes are
exactly what this is for. 142 of the 199 affected chunks are those two books.
Recorded in parity/KNOWN.md: the Python this was ported from did not know the
marker either, and no fixture carries one, so the suite agrees either way.
Cuts a reference list into entries — at the note numbers where there are any,
at the layout where there are not — and reads each as far as its punctuation
allows. Fields that would not parse are named in `unsettled` rather than
guessed at, and `--prompt` asks a model about exactly those and no others.
Over 202 real reference sections from the corpus: 8,566 references, 95% with
a title, 82% with a year, 46% settling every field with no model involved.
A title is refused where the entry was never divided, so no record reports
its own text back as the work it cites.
`ibid.` is flagged and left alone — it points at a work, and the entry does
not say which. A rule of dashes is filled in, because the style defines it as
the author above.
chunkmd gains `note_starts` and `cites_something`: recognising a page of
notes and splitting one are the same knowledge, and two copies of that
pattern would drift.
funkpower