mirror of
https://git.pumpin.online/funkpower/catena.git
synced 2026-10-22 18:57:14 +00:00
idnorm: a prefix is ASCII and the text it is cut from is not
`10.1093/<zwsp>oso/<zwsp>` is exactly sixteen bytes, the length of `https://doi.org/`, so testing that prefix against it sliced into the middle of the third zero-width space and brought the process down. Found by running citefind over the corpus: a real book prints its DOI that way, because the typesetter marked where the line could break. Both prefix tests — DOI and arXiv — now ask for the head rather than cutting one, so a byte length that is not a character boundary is an answer of no rather than a panic. And an invisible character is not part of an identifier. Zero-width spaces and joiners, the soft hyphen, a byte-order mark left at the front of a fragment: text out of a PDF or an EPUB is full of them, none of them can be seen, and a DOI carrying one does not resolve. They are dropped before anything is read.
This commit is contained in:
@@ -180,8 +180,6 @@ fn isbn(input: &str) -> Option<Identifier> {
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn an_identifier_copied_out_of_a_pdf_is_full_of_nothing() {
|
||||
// A typesetter's zero-width spaces, where the line was allowed to
|
||||
|
||||
Reference in New Issue
Block a user