From a1f7dea8df1212cae6f64d8457da6eead6d2367e Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Wed, 4 Nov 2026 15:27:38 -0800 Subject: [PATCH 1/7] Report a bare I or V ending the given part after a comma (#877) rules/C9 (a one-character suffix word inside a comma part) reads the bare letter behind a name word as the generation, and the comma path never reported the guess that the no-comma 'John Doe V' reports. FEC writes the letter as a middle initial about as often as letter frequency predicts, so a reader hesitates too. assign's given-part walk now emits suffix-or-name there, with the no-comma wording and the middle initial as the reading not taken. The reading is unchanged. AGENTS.md's "a kind is worth adding only if a reader would hesitate" bullet used 'Smith, John V' as a fork nobody hesitates over; it uses 'Smith, John V.' now. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 2 +- ...aracter-suffix-word-inside-a-comma-part.md | 7 ++++-- ...aracter-suffix-word-inside-a-comma-part.md | 7 ++++++ docs/release_log.rst | 1 + nameparser/_pipeline/_assign.py | 24 ++++++++++++++++++- tests/v2/cases.py | 9 +++++-- tests/v2/pipeline/test_assemble.py | 2 +- tests/v2/test_cases.py | 16 ++++++++----- 8 files changed, 55 insertions(+), 13 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 9874dcc6b..f6789accb 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -242,7 +242,7 @@ The 2.0 rewrite landed as underscore-private modules beside the v1 code, which 2 - **Validation is eager and fail-loud**: every `raise` states the offending value, the expected form, and the fix. Exception taxonomy: wrong type — including wrong element type inside a collection, bare `str` where an iterable of strings is expected, or a `Mapping` where a plain iterable is expected — raises `TypeError`; well-typed but unacceptable values raise `ValueError`; failed enum lookups stay `ValueError` for any input (stdlib `EnumType` precedent). **When the message hands the reader code to paste, that code has to survive a type checker, and then RUN** — nameparser ships `py.typed`, and a placeholder can type-check and still raise: the 2.4 release sweep's first fix to the `capitalization_exceptions` hint offered `('key', 'Value')`, which passes mypy and fails #459's mask check, so `test_add_capitalization_exceptions_raises_pointing_at_replace` now executes the offered code. #337's segmenterless warning offered `Policy(segment_scripts=())`, an `arg-type` error, because these fields are annotated with what they STORE rather than everything the constructor accepts. Prefer the `frozenset()` / `()` spellings in messages and docstrings, and pin the offered spelling in a test — the warning tests matched on `ja_segmenter` and never checked the actionable half of the message. **A warning emitted in `Parser.__post_init__` needs `parser_for` to re-emit it from its own frame** (the `catch_warnings(record=True)` block at its return): `__post_init__`'s `stacklevel` is sized for direct `Parser(...)` construction, and through `parser_for`'s extra frame the default one-line rendering attributes the warning to the library's own `return Parser(...)` — the exact call the message tells the user to change becomes invisible. No single stacklevel serves both entry points; a new construction warning gets the re-emission for free, but a new CONSTRUCTION SITE for `Parser` inside this package needs its own re-emission or its callers get library-attributed warnings (#337 review). - **Guard, hint, and emit for the WHOLE family, and parametrize the test over it**: a check added to one member of a set belongs on all of it, and the test must sweep the family, not one example. This session shipped `_reject_str_and_mapping` on `Policy` but not `PolicyPatch`, the bytes decode hint on three of five config entry points, and a regex-sync roster missing four of its copies — each a separate follow-up bug that a `{class} × {field} × {bad-value}` parametrization would have caught and a per-example test hid. When you find you're guarding member N, grep for the other members first. - **Ambiguities are emitted at the DECISION site**: an `Ambiguity` records a fork the parse had to call, not a token that sits in an ambiguous vocabulary, so emit where the branch is taken, in every stage that takes one, and name the reading not taken. The reading TAKEN (`Token.reading`, 3.0) is the one read later, at assemble, where roles are final, from the role and the tags the deciding stages left, its suffix and name sources by the predicates that name the reading not taken. The contract is mechanisms.md#AMBIGUITY-AT-THE-DECISION-SITE; the working rules (branches that change nothing, the field a report names, the stages that decide `PARTICLE_OR_GIVEN`, `alternative`, the reading taken, and how to test an emitter) are in `nameparser/_pipeline/AGENTS.md`, which loads when a session reads a pipeline stage. -- **A kind is worth adding only if a reader would hesitate too**: the test is not "does the code take a branch" but whether a person reading that input would genuinely be unsure. "Smith, John V" reads as a middle initial to anyone -- the comma settles it -- so reporting it would be noise that teaches callers to ignore the field, which costs more than the missing report. Reachability of the second branch is necessary, not sufficient. Prefer leaving a fork silent and documenting the omission over emitting on input nobody finds ambiguous. +- **A kind is worth adding only if a reader would hesitate too**: the test is not "does the code take a branch" but whether a person reading that input would genuinely be unsure. "Smith, John V." reads as a middle initial to anyone -- the period settles it -- so reporting it would be noise that teaches callers to ignore the field, which costs more than the missing report. Reachability of the second branch is necessary, not sufficient. The example was "Smith, John V" until #877 measured FEC writing that bare letter as a middle initial about as often as letter frequency predicts, where the parser reads the generation: a fork that feels settled is a claim to measure against the data before it decides a report. Prefer leaving a fork silent and documenting the omission over emitting on input nobody finds ambiguous. - **Parser owns config-dependent conveniences**: `Parser.matches`/`Parser.capitalized`/`Parser.revise` exist because the `ParsedName` equivalents fall back to DEFAULT config for str/omitted arguments (documented loudly in both docstrings). `revise` harvests tokens from a full sub-parse of each replacement value (tags kept minus `FOLDED_TAG`, roles forced, the R1 entry pass `suffix_entries` re-run over the forced state so a suffix value's entries follow its own commas, ambiguities discarded); the merge tail is shared with `replace()` via `ParsedName._with_field_tokens`. `Parser.capitalized` delegates through `name.capitalized(self.lexicon)` specifically so `_parser` never imports `_render` — keep it that way. - **A reading is of one word, save a birth marker and a title addressing the given name** (the build's check 8, which `_lexicon._make_edit` runs on every `add` and RAISES on, since a reading is now a caller's statement the compile can refuse as the build does: 3.0's 4i-b). The two lists, `surnames` and `given_names`, and `capitalization_exceptions` keys still WARN on a multi-word entry (`_normset`/`_normpairs` via `_warn_at_caller`, UserWarning, never a raise — an inert entry produces nothing). The default vocabulary and every locale pack must stay warning-free (`test_default_lexicon_builds_warning_free`, `test_pack_vocabulary_entries_are_single_words`). - **Invariants guard harm, not no-ops**: add a constructor check when violating it produces a *wrong parse*, not when it produces *nothing*. A false positive costs a working configuration; a true positive on an inert condition costs the user nothing, so that trade is never worth taking. Since 3.0's 4i-b a `Lexicon` compiles from readings, which hold the 2.x marker-over-base relations by construction; the checks a caller's edit reaches are the build's checks 3 (a kind and its properties, `may_be_initial` on a single cased letter alone), 4 (one suffix kind per word), 8 (a phrase), 9 (a bound given name that is a particle has a `given` reading) and 11's agnomen clause (an agnomen has a name reading), in `_lexicon._make_edit` and `_check_pool`, which `_lexicon._built` runs again under a choice of languages, refusing a caller's reading that clashes with one the choice turns on and naming the edit, each raising with the call that fixes it, because each is a wrong parse. A given-addressing title run the parse can never build is not checked, because an unreachable entry is simply never consulted (see Gotchas). Before adding one, construct the config it forbids and check what actually breaks. **Between raise and silence sits the construction-time `UserWarning`**, for a gap that is statically decidable, harmless to SOME deliberate caller, and indistinguishable-from-working for everyone else: the segmenterless activation (#337 — `parser_for(locales.JA)` without a segmenter behaved exactly like a working parser minus the feature) warns rather than raises because the inert JA registration is itself a pinned property, and a warning is filterable by the caller who wants exactly that. The message must carry every applicable remedy and no inapplicable one (the `ja_segmenter` hint fires only when a Japanese script is among the dead ones). Test fuzzers that legitimately construct such configs suppress the warning by MESSAGE, never by category — a blanket `UserWarning` ignore would mask the next construction diagnostic (`_quiet_parser` in `tests/v2/test_properties.py` is the pattern). diff --git a/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md b/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md index cfbc3347c..7d72da9bd 100644 --- a/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md +++ b/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md @@ -7,7 +7,8 @@ - The given part asks no initial in front of a numeral, the no-comma reading of one, so a numeral behind an initial stays the suffix (`Chang, Andy C I` → middle 'C', suffix 'I') — 2026-08-25 #430. - A lone letter ending the given part ahead of a further comma is the middle initial whatever stands before it, a credential run in front stopping short of it (`Smith, John PhD I, Jr.` → middle 'I', suffix 'PhD, Jr.'), at the cost of splitting a certification level from its credential (`Smith, John PSM I, Jr.`) — 2026-10-25 #819. - The given part's walk asks the numeral's predicate, which reads a one-letter numeral there by the vocabulary, so an unlisted letter stays the middle initial (`SMITH, JOHN X`), and a numeral of two or more letters as the generation (`Smith, John VI` → suffix 'VI') — 2026-10-25 #819. -- The given part's walk is strict-blind and the fork reports nothing: `Smith, John V` reads suffix 'V' under `lenient_comma_suffixes=False` as under the default, with no `suffix-or-name`, where `John Smith V` reports — 2026-08-25 #430. +- The given part's walk is strict-blind: `Smith, John V` reads suffix 'V' under `lenient_comma_suffixes=False` as under the default — 2026-08-25 #430; 2026-11-04 #877. +- The bare letter read as the generation behind a name word reports `suffix-or-name`, naming the middle initial, as `John Smith V` reports with no comma, and the reading stays: `Smith, John V` → suffix 'V', reported; a letter behind another suffix word, a period-marked one, one ahead of a further comma and one in lower case in a name of mixed case report nothing — 2026-11-04 (Derek), #877. Elsewhere: the strict switch, which vetoes the numeral in the comma's decision and in a part that is nothing but suffix words — decisions/C14's 2026-10-19 (Derek), #761 entry; a credential starting a run to the end of its part, which reads `Smith, John PSM I I.` as suffix 'PSM I I.' where #430 had accepted middle 'I I.' — decisions/S8's 2026-10-04 (Derek), #602 entry; a bare letter behind a name word with a suffix comma after it stays a middle initial (`Smith, John V, Jr.` → middle 'V', suffix 'Jr.') — decisions/v1-xfail-triage's 2026-09-01 (Derek's triage) entry; an ideographic or fullwidth full stop is no abbreviation mark there (`Smith, John V。` → suffix 'V。') — decisions/cjk-full-stops' 2026-09-10 #322/#323 bundle. @@ -22,8 +23,10 @@ Open: none in this file. BLAST RADIUS, over the 1069 corpus names against 56fb505c (PR #443): seven carry a one-character numeral behind a word inside a comma segment, every one bare, and none moves; the only movers are the three names this change added as examples, so the case table is the whole coverage. `Berg, abdul V` cannot reach the walk at all, the bound-given join making `abdul V` one piece, which is why the LENIENT mirror #401/#421 declined does not collide here. Two things the /simplify round settled by measurement rather than argument. The walk asks `is_suffix_piece(pieces[m - 1])` — a VOCABULARY test — where the reading carries the verdict it just recorded, and unifying them looks obviously right until you run it: the walk assigns `PSM` to GIVEN in `Smith, PSM I.`, so a reading-style flag says "behind a name" and sends the numeral to middle, losing the strict parity restored two commits earlier. The two sites ask different questions on purpose, and the vocabulary answer is the load-bearing one. The cost is a reading the walk cannot reach: `Smith, John PSM I I.` moves from middle 'I' + suffix 'PSM, I.' to middle 'I I.' + suffix 'PSM', two consecutive one-character numerals behind a credential — accepted, both readings being defensible for a shape nobody writes, and recorded here rather than chased at the price of two correct pins. The walk is also strict-blind, pre-existing and unchanged: it calls is_suffix_lenient unconditionally, so `Smith, John V` reads suffix 'V' under `lenient_comma_suffixes=False` as under the default. The reading consults the knob and the walk does not, which is now conspicuous where it was merely unexamined. Not this change's to fix. - Not done, and recorded rather than left implicit: neither half emits an ambiguity, though the no-comma `John Smith V` reports `suffix-or-name` for the same fork. The comma path has never reported it (decisions/C8's #428 review entry records PARTICLE_OR_GIVEN suppressed there for a related reason), and adding an emitter needs a trigger, a stage-ownership entry and a case row — more than this fix, and #432 raises it as a secondary observation rather than a requirement. + Not done, and recorded rather than left implicit (done by #877, 2026-11-04 below): neither half emits an ambiguity, though the no-comma `John Smith V` reports `suffix-or-name` for the same fork. The comma path has never reported it (decisions/C8's #428 review entry records PARTICLE_OR_GIVEN suppressed there for a related reason), and adding an emitter needs a trigger, a stage-ownership entry and a case row — more than this fix, and #432 raises it as a secondary observation rather than a requirement. - 2026-10-18 #728 — SPLIT FROM decisions/C1 with the rule. The entries above were written under decisions/C1 while this rule was part of rules/C1, and moved here unchanged but for the pointers the move broke, the measurement paragraphs, each cut to one line naming its population, counts, baseline and PR, and the superseded passages, each cut to its decision and what replaced it; where one says rules/C1 or C1, it means the rule as it then stood, which held this one. - 2026-10-25 #819 — A LONE LETTER ENDING THE GIVEN PART AHEAD OF A FURTHER COMMA IS THE MIDDLE INITIAL, AND THE WALK ASKS THE NUMERAL'S PREDICATE. Derek agreed reading (A) of #819 on 2026-10-23 with the numeral predicate (decisions/vocabulary-model's 2026-10-23 #752 entry, item 8): the further comma has named the suffix, so a letter ending the given part in front of it has no generation left to be, and #602's credential run (decisions/S8) stops short of it. `Smith, John PhD I, Jr.` and `Smith, John PhD I., Jr.` read middle 'I' and 'I.' again, as before #602, and the run's report on them goes with the reading; `Smith, John PhD I` (no further comma) keeps suffix 'PhD I'. The clause reads a letter written as an initial (rules/T5: a bare capital A to Z, or a letter with its period), numeral or not (`Smith, John PSM X, Jr.` → middle 'X'), since what it asks is the slot, not the word; a bare accented or non-Latin capital is no initial there and stays in the run (`Smith, John PhD É, Jr.` → suffix 'PhD É, Jr.'). WHAT IT COSTS: `Smith, John PSM I, Jr.`, a certification level written ahead of a generation, splits from its credential (rules/C9's Accepted bullet). The spike that proposed this measured no corpus moving; on this change the committed snapshot holds no such name either, and the local corpora were read with it (decisions/vocabulary-model's 2026-10-25 2f entry). THE WALK'S NUMERAL QUESTIONS are the predicate's (decisions/vocabulary-model, mechanisms.md#ONE-PREDICATE-PER-QUESTION): the last-piece test asks it of a bare one-letter numeral, and in this part, where neither the count nor an initial in front is asked, it answers by the vocabulary, so `Smith, John V` keeps suffix 'V' and `SMITH, JOHN X` middle 'X' (FEC writes a middle initial `X` there, the constraint the predicate's design recorded); a one-letter numeral behind a suffix word in a part of nothing but suffix words is asked the same way, its answer unchanged. A numeral of two or more letters, which this rule never governed, is read in this part by S10 since the same change (`Smith, John VI` → suffix 'VI', reported). +- 2026-11-04 (Derek), #877 — THE BARE LETTER BEHIND A NAME WORD IS REPORTED, AND STILL READ AS THE GENERATION. #430 recorded the missing report as not done; #835's salutation read (rules/H8) put more names into the shape (`DOE, JANE I MRS` reads title 'MRS', suffix 'I' where it read middle 'I MRS', 24 of FEC's 59 such names losing the `conjunction-or-initial` report they had carried). #877 counted FEC's itemized contributions (1,890,655 names with exactly one comma, 2026-10-24): 214,856 end the given part in a lone capital, `V` (1,956) 19th of 26 letters and `I` (1,305) 21st, about where `O` (1,324) falls, nearly all behind a full given name; `I.` and `V.` about as common as `I` and `V`; and `IV` 765 in the slot against `III` 6,254. So in that file the letter is mostly a middle initial, where #144's data wrote it as the generation, and nothing in the writing tells the two apart: the reader-would-hesitate test the kind needs (AGENTS.md's 2.0 conventions) is met, and the report is an emission of the existing `suffix-or-name`, not a new kind. DECIDED: report only, the reading kept (1.4.0's and #144's); flipping the default to the middle initial is not decided here and would need its own measurement of #144's shape. The emitter stands in assign's given-part walk, where the branch is taken, with the no-comma report's wording and an alternative naming the middle initial (`as_a_middle`), and only where the letter is read as a suffix behind a word the walk read as a name: behind another suffix word the run settles it (`Smith, PSM I`), a period marks an initial (`Smith, John V.`), a further comma has already named the suffix (`Smith, John V, Jr.`, middle 'V'), a letter in lower case in a name of mixed case is settled as the generation (S10), and a particle in front is no name word (`, Freiherr von V`, family 'von', whose stand-in P6 would attach). The strict switch is not involved: the walk does not consult it (#430), and this report does not change that. AGENTS.md's "a kind is worth adding only if a reader would hesitate too" bullet used `Smith, John V` as its example of a fork nobody hesitates over, contradicting both the code and this data; it now uses `Smith, John V.`, the period settling it. + MEASURED 2026-11-04 with `snapshot.py --local fec-individuals --base `: see the PR for the counts; no field moves, only reports. diff --git a/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md b/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md index 1f7af5c28..dfda9e1f3 100644 --- a/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md +++ b/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md @@ -12,6 +12,13 @@ "Smith, John PhD I, Jr." → suffix="PhD, Jr." "Smith, John PhD É, Jr." → suffix="PhD É, Jr." · boundary ``` +- **The bare letter read as the generation behind a name word is reported**, naming the middle initial it would otherwise be, as S10 reports the same letter with no comma: one letter is the shape a middle initial is written in, so a reader cannot tell the two apart there. Nothing is reported where the writing or the slot settles it: a letter behind another suffix word, a period-marked letter, one ahead of a further comma, and one in lower case in a name of mixed case (S10). + ``` + "Smith, John V" → ambiguities=("suffix-or-name",) + "SMITH, JOHN I" → ambiguities=("suffix-or-name",) + "Smith, PSM I" → ambiguities=() + "Smith, John v" → ambiguities=() + ``` - **Only the given part is touched**: a part after the comma that is nothing but suffix words is the credential run, and a letter in it continues that run up to the further comma. - **Longer suffix words are not in question either way**, a numeral of two or more letters being read as S10 reads one in this part. diff --git a/docs/release_log.rst b/docs/release_log.rst index f5e531ccf..05e7830cb 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -41,6 +41,7 @@ Release Log - **Read a compound given name written with a particle as a middle name.** ``parse("María del Carmen García")`` gives middle ``del Carmen``, family ``García``, where 1.4.0 through 2.8.0 gave family ``del Carmen García``. It reads 23 marked words behind ``de``, ``del``, ``de los`` and the like, Spanish and Portuguese alike (``del Pilar``, ``de los Ángeles``, ``do Socorro``); a surname written that way moves too (``Pedro del Valle Llufrio`` gives middle ``del Valle``), and ``Maria do Carmo Silva`` and ``Domingo Santa María González`` keep their family. :issue:`704` - **Fix a numeral VI to IX before a comma or ending a connective name costing the family name.** ``parse("Dumas VI, Jr.")`` gives family ``Dumas``, suffix ``VI, Jr.``, as ``Dumas III, Jr.`` does, where 2.0.0 through 2.8.0 gave given ``Dumas`` and no family; ``parse("Ortega y Gasset VI")`` gives family ``Gasset``, where 1.4.0 through 2.8.0 gave given ``Ortega y Gasset``. :issue:`893` + - **Change a bare I or V ending the given part after a comma to report suffix-or-name.** ``parse("Doe, John V")`` still gives suffix ``V``, and now reports that the letter may be a middle initial, as ``parse("John Doe V")`` already did; 1.4.0 through 2.8.0 reported nothing. :issue:`877` - **Fix two_family_names leaving a hyphenated surname out of the family.** ``Parser(policy=Policy(two_family_names=True)).parse("Andrés Sánchez-Cid Torres")`` gives family ``Sánchez-Cid Torres``, where 2.6.0 through 2.8.0 gave middle ``Sánchez-Cid``, family ``Torres``; a hyphenated given name is taken too unless every part of it is a listed given name, as under ``locales.ES`` (``Ana María-José López`` keeps middle ``María-José``), and hyphenated initials stay. :issue:`959` - **Fix particles joined by a connective reading the connective as the family's base.** ``parse("Fritz Freiherr und von Bar")`` gives family_base ``Freiherr und von Bar`` and initials ``F. F. v. B.``, where 2.0.0 through 2.8.0 gave family_base ``und``; a connective ending the family is still its base (``Juan de y`` gives ``y``), and one joining particles before a name word stays in it (``Alois von und zu Liechtenstein`` gives ``und Liechtenstein``). :issue:`939` - **Read père after a French name as the generation, as fils is.** ``parse("Alexandre Dumas père")`` gives family ``Dumas``, suffix ``père``, where 1.4.0 through 2.8.0 gave middle ``Dumas``, family ``père``. It is a French surname too, so Title case keeps it the family name (``Olivier Père``), as with ``fils``; ``M.`` before a name stays an initial. :issue:`684`, :issue:`658` diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 5b3d088f0..62425ec1e 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -104,7 +104,8 @@ split_credential_report, anchor_in_reach, credential_at_the_given_slot, given_slot_anchors, _NOT_A_RUN_START, has_name_content, is_lone_never_given_particle, - Peel, TailRead, is_suffix_piece, is_title_piece, leading_titles, + Peel, TailRead, is_prefix_piece, is_suffix_piece, is_title_piece, + leading_titles, particle_tail, listed_lean, peel_walk, settled_by_case, starts_a_credential_run, tail_reading, trailing_titles, @@ -1680,6 +1681,27 @@ def as_a_middle(m: int, token: WorkToken) -> PendingReading: tokens[pieces[m][0]].text.upper())): _set_roles(tokens, pieces[m], Role.MIDDLE) continue + # rules/C9: "behind a name word it is the generation only + # when written bare" -- and a bare letter there is the + # shape a middle initial is written in, so the reading + # is reported, as S10 reports it with no comma + # (decisions/C9, #877); `len` first, C-level + if (len(pieces[m]) == 1 + and len((tok := tokens[pieces[m][0]]).text) == 1 + and (m >= sticky_from and not k6 <= m < end6 + or m < sticky_from and suffix_here) + and _ROMAN.match(tok.text) is not None + and tokens[pieces[m - 1][0]].role in NAME_ROLES + and not is_prefix_piece(pieces[m - 1], ptags[m - 1], + tokens) + and not numeral_settled_by_case(tok.text, + state.one_case)): + ambiguities.append(PendingAmbiguity( + AmbiguityKind.SUFFIX_OR_NAME, + f"{tok.text!r} is a roman numeral, so it reads as a " + f"generational suffix; any other single letter " + f"there would be a middle initial", + (pieces[m][0],), alternative=as_a_middle(m, tok))) if m >= sticky_from and not k6 <= m < end6: # inside the run: a title word reads as a title, # every other word as a suffix, and a word the walk diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 4971da3a0..d00454658 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -7328,6 +7328,7 @@ def _check_cjk_shape_purity(self) -> None: Case("a_numeral_the_bound_join_would_take_stays_maiden", "Berg, abdul nee Smith V", {"given": "abdul", "middle": "nee Smith", "family": "Berg", "suffix": "V"}, + ambiguities=("suffix-or-name",), classification="fix(#601)", notes="#601: a marker in the given part after a family comma is an " "ordinary word (rules/M2). Before #601: given 'abdul', " @@ -8546,6 +8547,7 @@ def _check_cjk_shape_purity(self) -> None: tolerated=True), Case("family_comma_lenient_trailing", "Smith, John V", {"given": "John", "family": "Smith", "suffix": "V"}, + ambiguities=("suffix-or-name",), notes="v1 #144: the trailing piece of a two-part comma name " "takes the lenient suffix test"), Case("family_comma_first_piece_is_given", "Steven Hardman, RN - CRNA", @@ -8778,7 +8780,7 @@ def _check_cjk_shape_purity(self) -> None: Case("a_numeral_behind_a_class_member_keeps_the_listing_form", "John Smith, Ma I", {"given": "Ma", "family": "John Smith", "suffix": "I"}, - ambiguities=("suffix-or-name",), + ambiguities=("suffix-or-name", "suffix-or-name"), notes="contrast for the row below: C4's run test admits no " "single-letter numeral, which could be an initial, so " "'Ma' is the given name, as b547c247 read it"), @@ -9910,6 +9912,7 @@ def _check_cjk_shape_purity(self) -> None: "Smith, Freiherr von Berg V", {"title": "Freiherr", "given": "von Berg", "family": "Smith", "suffix": "V"}, + ambiguities=("suffix-or-name",), classification="fix(#424)", notes="S4: the given part's trailing slot after a family comma " "is read without the count, so the one-letter numeral " @@ -12618,6 +12621,7 @@ def _check_cjk_shape_purity(self) -> None: Case("family_comma_bare_numeral_after_a_name_is_the_suffix", "Smith, John V", {"given": "John", "family": "Smith", "suffix": "V"}, + ambiguities=("suffix-or-name",), notes="THE BOUNDARY, and v1 parity (#144): with no period " "there is no abbreviation, so the numeral is the " "generation it looks like. This row is what makes " @@ -13073,7 +13077,8 @@ def _check_cjk_shape_purity(self) -> None: Case("title_word_trailing_after_a_family_comma_behind_a_numeral", "Smith, John V Prof.", {"title": "Prof.", "given": "John", "family": "Smith", - "suffix": "V"}, classification="fix(#316)", + "suffix": "V"}, ambiguities=("suffix-or-name",), + classification="fix(#316)", notes="the mirror of the row above, and what makes the " "lenient test read the pieces as if the title were " "absent: the walk took the last piece, so 'V' is where " diff --git a/tests/v2/pipeline/test_assemble.py b/tests/v2/pipeline/test_assemble.py index 05d28dbf5..ec75ce4f3 100644 --- a/tests/v2/pipeline/test_assemble.py +++ b/tests/v2/pipeline/test_assemble.py @@ -159,7 +159,7 @@ def test_a_connective_or_initial_report_is_withdrawn_where_the_role_is_a_suffix( withdrawn = { "JOHN QUINCY SMITH I": ["suffix-or-name"], "john smith i": ["suffix-or-name"], - "SMITH, JOHN I": [], + "SMITH, JOHN I": ["suffix-or-name"], # rules/C9, #877 "HENRY SMITH I": ["suffix-or-name"], } for text, kinds in withdrawn.items(): diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 20a962b18..41ca8cb16 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -353,10 +353,12 @@ def test_the_field_sweep_sees_the_claims_it_checks() -> None: # suffix-or-name reports (five new rows one each, the 'Pedro Junior # Filho' row two, 'Pedro Júnior Neto' a second, 'José Silva Júnior' # one), each under three orders, and its review row 'Smith, - # Junior.', reporting suffix-or-name under three orders. - assert claims == 1582, ( - f"the field sweep checks {claims} claims, recorded as 1582 on " - f"2026-11-03") + # Junior.', reporting suffix-or-name under three orders; 1582 -> + # 1600 on 2026-11-04, #877's report on a bare letter ending the + # given part after a comma, six rows each under three orders. + assert claims == 1600, ( + f"the field sweep checks {claims} claims, recorded as 1600 on " + f"2026-11-04") #: The kinds that report a fork between two readings of a word, so carry @@ -719,8 +721,10 @@ def test_the_alternative_sweep_sees_the_fields_it_checks() -> None: # 'Nasreen Sultana, MBBS' row reporting no fork under the three # orders; 459 -> 462 the same day, #684's two père rows under the # three orders; 462 -> 477 the same day, #1039's junior rows under - # the three orders - assert (checked, skipped) == (477, 330), (checked, skipped) + # the three orders; 477 -> 495 on 2026-11-04, #877's report on a bare + # letter ending the given part after a comma, six rows under the + # three orders + assert (checked, skipped) == (495, 330), (checked, skipped) def test_every_case_name_is_unique() -> None: From cf9d45cbe7cc4d7554baf427354568c205d3332d Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Wed, 4 Nov 2026 15:30:49 -0800 Subject: [PATCH 2/7] decisions/C9: record what #877's report reaches in the local corpora Co-Authored-By: Claude Opus 5.5 --- .../C9-one-character-suffix-word-inside-a-comma-part.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md b/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md index 7d72da9bd..8571d7991 100644 --- a/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md +++ b/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md @@ -29,4 +29,4 @@ Open: none in this file. WHAT IT COSTS: `Smith, John PSM I, Jr.`, a certification level written ahead of a generation, splits from its credential (rules/C9's Accepted bullet). The spike that proposed this measured no corpus moving; on this change the committed snapshot holds no such name either, and the local corpora were read with it (decisions/vocabulary-model's 2026-10-25 2f entry). THE WALK'S NUMERAL QUESTIONS are the predicate's (decisions/vocabulary-model, mechanisms.md#ONE-PREDICATE-PER-QUESTION): the last-piece test asks it of a bare one-letter numeral, and in this part, where neither the count nor an initial in front is asked, it answers by the vocabulary, so `Smith, John V` keeps suffix 'V' and `SMITH, JOHN X` middle 'X' (FEC writes a middle initial `X` there, the constraint the predicate's design recorded); a one-letter numeral behind a suffix word in a part of nothing but suffix words is asked the same way, its answer unchanged. A numeral of two or more letters, which this rule never governed, is read in this part by S10 since the same change (`Smith, John VI` → suffix 'VI', reported). - 2026-11-04 (Derek), #877 — THE BARE LETTER BEHIND A NAME WORD IS REPORTED, AND STILL READ AS THE GENERATION. #430 recorded the missing report as not done; #835's salutation read (rules/H8) put more names into the shape (`DOE, JANE I MRS` reads title 'MRS', suffix 'I' where it read middle 'I MRS', 24 of FEC's 59 such names losing the `conjunction-or-initial` report they had carried). #877 counted FEC's itemized contributions (1,890,655 names with exactly one comma, 2026-10-24): 214,856 end the given part in a lone capital, `V` (1,956) 19th of 26 letters and `I` (1,305) 21st, about where `O` (1,324) falls, nearly all behind a full given name; `I.` and `V.` about as common as `I` and `V`; and `IV` 765 in the slot against `III` 6,254. So in that file the letter is mostly a middle initial, where #144's data wrote it as the generation, and nothing in the writing tells the two apart: the reader-would-hesitate test the kind needs (AGENTS.md's 2.0 conventions) is met, and the report is an emission of the existing `suffix-or-name`, not a new kind. DECIDED: report only, the reading kept (1.4.0's and #144's); flipping the default to the middle initial is not decided here and would need its own measurement of #144's shape. The emitter stands in assign's given-part walk, where the branch is taken, with the no-comma report's wording and an alternative naming the middle initial (`as_a_middle`), and only where the letter is read as a suffix behind a word the walk read as a name: behind another suffix word the run settles it (`Smith, PSM I`), a period marks an initial (`Smith, John V.`), a further comma has already named the suffix (`Smith, John V, Jr.`, middle 'V'), a letter in lower case in a name of mixed case is settled as the generation (S10), and a particle in front is no name word (`, Freiherr von V`, family 'von', whose stand-in P6 would attach). The strict switch is not involved: the walk does not consult it (#430), and this report does not change that. AGENTS.md's "a kind is worth adding only if a reader would hesitate too" bullet used `Smith, John V` as its example of a fork nobody hesitates over, contradicting both the code and this data; it now uses `Smith, John V.`, the period settling it. - MEASURED 2026-11-04 with `snapshot.py --local fec-individuals --base `: see the PR for the counts; no field moves, only reports. + MEASURED 2026-11-04 with `snapshot.py --local --base ` (head package digest 3bb08fd4c91df2bc, base 2036d954b9b61eb9): no field moves anywhere, only reports. fec-individuals: 3,174 names gain `suffix-or-name` on the letter (`ABELSON, TOM I`, `ABHYANKER, RAJ V`), near #877's estimate of the 3,180 bare `I`/`V` names read as the suffix. ctgov-officials: 4 (`Fazal H Shah, FCPS I`), where the unlisted `FCPS` is read as the given name and the letter is reported behind it, a level of a credential the vocabulary does not know. psc-individuals: none. In the committed corpora 4 entries gain it (`Smith, John V`, `Chang, Andy C I`). From 885c0f3d5fe177e8e24c2c049b0d92421f7ef01e Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Wed, 4 Nov 2026 15:46:32 -0800 Subject: [PATCH 3/7] Name the family for a letter P1's fold takes in; record the particle exclusion (#877 review) The reading not taken named the middle for 'Smith, de Mesnil V', where the fold of a never-given particle opening the given part takes a letter into the family; the shared numeral report had the same gap. The exclusion of a letter behind a particle is kept, its reason corrected and stated in rules/C9: the chain decides the field, and the walk cannot name it. The stage header lists the new site. Co-Authored-By: Claude Opus 5.5 --- ...one-character-suffix-word-inside-a-comma-part.md | 4 ++-- ...one-character-suffix-word-inside-a-comma-part.md | 3 ++- nameparser/_pipeline/_assign.py | 13 ++++++++++++- tests/v2/test_cases.py | 6 +++++- tools/differential/corpus_rules.jsonl | 1 + tools/differential/snapshot.jsonl | 1 + 6 files changed, 23 insertions(+), 5 deletions(-) diff --git a/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md b/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md index 8571d7991..cecc9d062 100644 --- a/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md +++ b/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md @@ -8,7 +8,7 @@ - A lone letter ending the given part ahead of a further comma is the middle initial whatever stands before it, a credential run in front stopping short of it (`Smith, John PhD I, Jr.` → middle 'I', suffix 'PhD, Jr.'), at the cost of splitting a certification level from its credential (`Smith, John PSM I, Jr.`) — 2026-10-25 #819. - The given part's walk asks the numeral's predicate, which reads a one-letter numeral there by the vocabulary, so an unlisted letter stays the middle initial (`SMITH, JOHN X`), and a numeral of two or more letters as the generation (`Smith, John VI` → suffix 'VI') — 2026-10-25 #819. - The given part's walk is strict-blind: `Smith, John V` reads suffix 'V' under `lenient_comma_suffixes=False` as under the default — 2026-08-25 #430; 2026-11-04 #877. -- The bare letter read as the generation behind a name word reports `suffix-or-name`, naming the middle initial, as `John Smith V` reports with no comma, and the reading stays: `Smith, John V` → suffix 'V', reported; a letter behind another suffix word, a period-marked one, one ahead of a further comma and one in lower case in a name of mixed case report nothing — 2026-11-04 (Derek), #877. +- The bare letter read as the generation behind a name word reports `suffix-or-name`, naming the middle initial, as `John Smith V` reports with no comma, and the reading stays: `Smith, John V` → suffix 'V', reported; a letter behind another suffix word, a period-marked one, one ahead of a further comma, one in lower case in a name of mixed case and one behind a particle (`Smith, Van V`) report nothing — 2026-11-04 (Derek), #877. Elsewhere: the strict switch, which vetoes the numeral in the comma's decision and in a part that is nothing but suffix words — decisions/C14's 2026-10-19 (Derek), #761 entry; a credential starting a run to the end of its part, which reads `Smith, John PSM I I.` as suffix 'PSM I I.' where #430 had accepted middle 'I I.' — decisions/S8's 2026-10-04 (Derek), #602 entry; a bare letter behind a name word with a suffix comma after it stays a middle initial (`Smith, John V, Jr.` → middle 'V', suffix 'Jr.') — decisions/v1-xfail-triage's 2026-09-01 (Derek's triage) entry; an ideographic or fullwidth full stop is no abbreviation mark there (`Smith, John V。` → suffix 'V。') — decisions/cjk-full-stops' 2026-09-10 #322/#323 bundle. @@ -28,5 +28,5 @@ Open: none in this file. - 2026-10-25 #819 — A LONE LETTER ENDING THE GIVEN PART AHEAD OF A FURTHER COMMA IS THE MIDDLE INITIAL, AND THE WALK ASKS THE NUMERAL'S PREDICATE. Derek agreed reading (A) of #819 on 2026-10-23 with the numeral predicate (decisions/vocabulary-model's 2026-10-23 #752 entry, item 8): the further comma has named the suffix, so a letter ending the given part in front of it has no generation left to be, and #602's credential run (decisions/S8) stops short of it. `Smith, John PhD I, Jr.` and `Smith, John PhD I., Jr.` read middle 'I' and 'I.' again, as before #602, and the run's report on them goes with the reading; `Smith, John PhD I` (no further comma) keeps suffix 'PhD I'. The clause reads a letter written as an initial (rules/T5: a bare capital A to Z, or a letter with its period), numeral or not (`Smith, John PSM X, Jr.` → middle 'X'), since what it asks is the slot, not the word; a bare accented or non-Latin capital is no initial there and stays in the run (`Smith, John PhD É, Jr.` → suffix 'PhD É, Jr.'). WHAT IT COSTS: `Smith, John PSM I, Jr.`, a certification level written ahead of a generation, splits from its credential (rules/C9's Accepted bullet). The spike that proposed this measured no corpus moving; on this change the committed snapshot holds no such name either, and the local corpora were read with it (decisions/vocabulary-model's 2026-10-25 2f entry). THE WALK'S NUMERAL QUESTIONS are the predicate's (decisions/vocabulary-model, mechanisms.md#ONE-PREDICATE-PER-QUESTION): the last-piece test asks it of a bare one-letter numeral, and in this part, where neither the count nor an initial in front is asked, it answers by the vocabulary, so `Smith, John V` keeps suffix 'V' and `SMITH, JOHN X` middle 'X' (FEC writes a middle initial `X` there, the constraint the predicate's design recorded); a one-letter numeral behind a suffix word in a part of nothing but suffix words is asked the same way, its answer unchanged. A numeral of two or more letters, which this rule never governed, is read in this part by S10 since the same change (`Smith, John VI` → suffix 'VI', reported). -- 2026-11-04 (Derek), #877 — THE BARE LETTER BEHIND A NAME WORD IS REPORTED, AND STILL READ AS THE GENERATION. #430 recorded the missing report as not done; #835's salutation read (rules/H8) put more names into the shape (`DOE, JANE I MRS` reads title 'MRS', suffix 'I' where it read middle 'I MRS', 24 of FEC's 59 such names losing the `conjunction-or-initial` report they had carried). #877 counted FEC's itemized contributions (1,890,655 names with exactly one comma, 2026-10-24): 214,856 end the given part in a lone capital, `V` (1,956) 19th of 26 letters and `I` (1,305) 21st, about where `O` (1,324) falls, nearly all behind a full given name; `I.` and `V.` about as common as `I` and `V`; and `IV` 765 in the slot against `III` 6,254. So in that file the letter is mostly a middle initial, where #144's data wrote it as the generation, and nothing in the writing tells the two apart: the reader-would-hesitate test the kind needs (AGENTS.md's 2.0 conventions) is met, and the report is an emission of the existing `suffix-or-name`, not a new kind. DECIDED: report only, the reading kept (1.4.0's and #144's); flipping the default to the middle initial is not decided here and would need its own measurement of #144's shape. The emitter stands in assign's given-part walk, where the branch is taken, with the no-comma report's wording and an alternative naming the middle initial (`as_a_middle`), and only where the letter is read as a suffix behind a word the walk read as a name: behind another suffix word the run settles it (`Smith, PSM I`), a period marks an initial (`Smith, John V.`), a further comma has already named the suffix (`Smith, John V, Jr.`, middle 'V'), a letter in lower case in a name of mixed case is settled as the generation (S10), and a particle in front is no name word (`, Freiherr von V`, family 'von', whose stand-in P6 would attach). The strict switch is not involved: the walk does not consult it (#430), and this report does not change that. AGENTS.md's "a kind is worth adding only if a reader would hesitate too" bullet used `Smith, John V` as its example of a fork nobody hesitates over, contradicting both the code and this data; it now uses `Smith, John V.`, the period settling it. +- 2026-11-04 (Derek), #877 — THE BARE LETTER BEHIND A NAME WORD IS REPORTED, AND STILL READ AS THE GENERATION. #430 recorded the missing report as not done; #835's salutation read (rules/H8) put more names into the shape (`DOE, JANE I MRS` reads title 'MRS', suffix 'I' where it read middle 'I MRS', 24 of FEC's 59 such names losing the `conjunction-or-initial` report they had carried). #877 counted FEC's itemized contributions (1,890,655 names with exactly one comma, 2026-10-24): 214,856 end the given part in a lone capital, `V` (1,956) 19th of 26 letters and `I` (1,305) 21st, about where `O` (1,324) falls, nearly all behind a full given name; `I.` and `V.` about as common as `I` and `V`; and `IV` 765 in the slot against `III` 6,254. So in that file the letter is mostly a middle initial, where #144's data wrote it as the generation, and nothing in the writing tells the two apart: the reader-would-hesitate test the kind needs (AGENTS.md's 2.0 conventions) is met, and the report is an emission of the existing `suffix-or-name`, not a new kind. DECIDED: report only, the reading kept (1.4.0's and #144's); flipping the default to the middle initial is not decided here and would need its own measurement of #144's shape. The emitter stands in assign's given-part walk, where the branch is taken, with the no-comma report's wording and an alternative naming the middle initial (`as_a_middle`), and only where the letter is read as a suffix behind a word the walk read as a name: behind another suffix word the run settles it (`Smith, PSM I`), a period marks an initial (`Smith, John V.`), a further comma has already named the suffix (`Smith, John V, Jr.`, middle 'V'), a letter in lower case in a name of mixed case is settled as the generation (S10), and a particle in front reports nothing: it may take the letter into its chain (`, Freiherr von Q` reads family 'von Q', `Smith, Freiherr von Q` given 'von Q') or stand alone as the given name (`Smith, Van Q` reads given 'Van', middle 'Q'), and the walk cannot name the initial's field without the chain's decision, so `Smith, Van V` goes unreported, an accepted gap for a rare shape (the code review found the first draft's rationale, which named P6, wrong, and the exclusion broader than its reason; the narrower exclusion it suggested named the wrong field in 120 of 1,695 grid reports). The review also found the alternative naming the middle where a never-given particle opens the given part and the fold takes the letter into the family (`Smith, de Mesnil V`, 870 grid reports, and the shared numeral report's `Smith, de Mesnil VI` before this change): the reading not taken now names the family wherever P1's fold opens the part and the letter stands past its first word. The strict switch is not involved: the walk does not consult it (#430), and this report does not change that. AGENTS.md's "a kind is worth adding only if a reader would hesitate too" bullet used `Smith, John V` as its example of a fork nobody hesitates over, contradicting both the code and this data; it now uses `Smith, John V.`, the period settling it. MEASURED 2026-11-04 with `snapshot.py --local --base ` (head package digest 3bb08fd4c91df2bc, base 2036d954b9b61eb9): no field moves anywhere, only reports. fec-individuals: 3,174 names gain `suffix-or-name` on the letter (`ABELSON, TOM I`, `ABHYANKER, RAJ V`), near #877's estimate of the 3,180 bare `I`/`V` names read as the suffix. ctgov-officials: 4 (`Fazal H Shah, FCPS I`), where the unlisted `FCPS` is read as the given name and the letter is reported behind it, a level of a credential the vocabulary does not know. psc-individuals: none. In the committed corpora 4 entries gain it (`Smith, John V`, `Chang, Andy C I`). diff --git a/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md b/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md index dfda9e1f3..1e06d5cdb 100644 --- a/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md +++ b/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md @@ -12,12 +12,13 @@ "Smith, John PhD I, Jr." → suffix="PhD, Jr." "Smith, John PhD É, Jr." → suffix="PhD É, Jr." · boundary ``` -- **The bare letter read as the generation behind a name word is reported**, naming the middle initial it would otherwise be, as S10 reports the same letter with no comma: one letter is the shape a middle initial is written in, so a reader cannot tell the two apart there. Nothing is reported where the writing or the slot settles it: a letter behind another suffix word, a period-marked letter, one ahead of a further comma, and one in lower case in a name of mixed case (S10). +- **The bare letter read as the generation behind a name word is reported**, naming the middle initial it would otherwise be, as S10 reports the same letter with no comma: one letter is the shape a middle initial is written in, so a reader cannot tell the two apart there. Nothing is reported where the writing or the slot settles it: a letter behind another suffix word, a period-marked letter, one ahead of a further comma, and one in lower case in a name of mixed case (S10). Nor behind a particle, which may take the letter into its chain or stand alone as the given name, so the report could not say which field the initial would land in. ``` "Smith, John V" → ambiguities=("suffix-or-name",) "SMITH, JOHN I" → ambiguities=("suffix-or-name",) "Smith, PSM I" → ambiguities=() "Smith, John v" → ambiguities=() + "Smith, Van V" → ambiguities=() ``` - **Only the given part is touched**: a part after the comma that is nothing but suffix words is the credential run, and a letter in it continues that run up to the further comma. - **Longer suffix words are not in question either way**, a numeral of two or more letters being read as S10 reads one in this part. diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 62425ec1e..28254c064 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -65,7 +65,8 @@ Emits SUFFIX_OR_NAME at these sites: the trailing roman numeral, taken or declined (since #718 for want of words to spare, since #752 for its Title case), a numeral ending the family part before a family comma, -one of two or more letters ending a given part's walk, each +one of two or more letters ending a given part's walk, -- since +#877 -- a bare one-letter numeral ending that walk behind a name word, each ambiguous acronym the trailing peel had to resolve, the bare-suffix carve-out where an input that is nothing but post-nominal vocabulary gets its first word made into the name (H4's suffix half, #491), @@ -1638,6 +1639,13 @@ def as_a_middle(m: int, token: WorkToken) -> PendingReading: # the name reading of a word the walk read as a suffix: # a middle name, or taken in by the name word in front front = pieces[m - 1] + # rules/P1's fold after a family comma takes the rest of + # the part into the family, so a word behind a part a + # never-given particle opens lands there too, however + # far behind it the word stands (#877 review) + if n < m - 1 and is_lone_never_given_particle(pieces[n], + tokens): + return name_reading(token, Role.FAMILY) return _name_alt( token, Role.MIDDLE, front if tokens[front[0]].role in NAME_ROLES else None, @@ -1692,6 +1700,9 @@ def as_a_middle(m: int, token: WorkToken) -> PendingReading: or m < sticky_from and suffix_here) and _ROMAN.match(tok.text) is not None and tokens[pieces[m - 1][0]].role in NAME_ROLES + # a particle in front may take the letter into + # its chain or stand alone, the field the name + # reading names depending on which (#877 review) and not is_prefix_piece(pieces[m - 1], ptags[m - 1], tokens) and not numeral_settled_by_case(tok.text, diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 41ca8cb16..3d7c88d3a 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -538,7 +538,11 @@ def test_an_alternative_names_the_field_the_other_reading_gives( "Sir abdul VI", "Dr. mc VI", "de VI", "Smith, Dr. de VI", # the PR review's: a given-name title licenses the # bound join only from the leading run, as group asks - "abdul VI Sir.", "Sir Dr. abdul VI", "Dr. Sir abdul VI") + "abdul VI Sir.", "Sir Dr. abdul VI", "Dr. Sir abdul VI", + # #877's review: P1's fold opening the given part + # takes a letter standing past its first word + "Smith, de Mesnil V", "Smith, de Mesnil VI", + "Smith, Mc V I") @pytest.mark.parametrize("order", _INVARIANT_ORDERS) diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 547c9a43e..78bb0b769 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -1120,6 +1120,7 @@ "Smith, Sir" "Smith, Sr." "Smith, V; MD" +"Smith, Van V" "Smith, XYZ" "Smith, de Mesnil Jean" "Smith. John" diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index e5feae5b0..90827cd7d 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -3597,6 +3597,7 @@ {"name": "Smith, Sir", "title": "Sir", "given": "Smith", "initials": "S."} {"name": "Smith, Sr.", "family": "Smith", "suffix": "Sr.", "initials": "S.", "generation": "Sr."} {"name": "Smith, V; MD", "given": "V;", "family": "Smith", "suffix": "MD", "initials": "V. S."} +{"name": "Smith, Van V", "given": "Van", "family": "Smith", "suffix": "V", "initials": "V. S.", "generation": "V"} {"name": "Smith, XYZ", "given": "XYZ", "family": "Smith", "initials": "X. S.", "capitalized_force": "Xyz Smith"} {"name": "Smith, de Mesnil Jean", "family": "Smith de Mesnil Jean", "initials": "S. M. J."} {"name": "Smith, meng", "given": "meng", "family": "Smith", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: credential as suffix (vocabulary)"], "initials": "m. S.", "capitalized_force": "Meng Smith"} From 416a823bb97f3da38f4be80a3dcd125f7d89202d Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Wed, 4 Nov 2026 15:53:44 -0800 Subject: [PATCH 4/7] Make the #877 review row reach the fold; record what the shared alternative moved Co-Authored-By: Claude Opus 5.5 --- .../C9-one-character-suffix-word-inside-a-comma-part.md | 2 +- nameparser/_pipeline/_assign.py | 2 +- tests/v2/test_cases.py | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md b/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md index cecc9d062..38361f39e 100644 --- a/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md +++ b/docs/design/decisions/C9-one-character-suffix-word-inside-a-comma-part.md @@ -28,5 +28,5 @@ Open: none in this file. - 2026-10-25 #819 — A LONE LETTER ENDING THE GIVEN PART AHEAD OF A FURTHER COMMA IS THE MIDDLE INITIAL, AND THE WALK ASKS THE NUMERAL'S PREDICATE. Derek agreed reading (A) of #819 on 2026-10-23 with the numeral predicate (decisions/vocabulary-model's 2026-10-23 #752 entry, item 8): the further comma has named the suffix, so a letter ending the given part in front of it has no generation left to be, and #602's credential run (decisions/S8) stops short of it. `Smith, John PhD I, Jr.` and `Smith, John PhD I., Jr.` read middle 'I' and 'I.' again, as before #602, and the run's report on them goes with the reading; `Smith, John PhD I` (no further comma) keeps suffix 'PhD I'. The clause reads a letter written as an initial (rules/T5: a bare capital A to Z, or a letter with its period), numeral or not (`Smith, John PSM X, Jr.` → middle 'X'), since what it asks is the slot, not the word; a bare accented or non-Latin capital is no initial there and stays in the run (`Smith, John PhD É, Jr.` → suffix 'PhD É, Jr.'). WHAT IT COSTS: `Smith, John PSM I, Jr.`, a certification level written ahead of a generation, splits from its credential (rules/C9's Accepted bullet). The spike that proposed this measured no corpus moving; on this change the committed snapshot holds no such name either, and the local corpora were read with it (decisions/vocabulary-model's 2026-10-25 2f entry). THE WALK'S NUMERAL QUESTIONS are the predicate's (decisions/vocabulary-model, mechanisms.md#ONE-PREDICATE-PER-QUESTION): the last-piece test asks it of a bare one-letter numeral, and in this part, where neither the count nor an initial in front is asked, it answers by the vocabulary, so `Smith, John V` keeps suffix 'V' and `SMITH, JOHN X` middle 'X' (FEC writes a middle initial `X` there, the constraint the predicate's design recorded); a one-letter numeral behind a suffix word in a part of nothing but suffix words is asked the same way, its answer unchanged. A numeral of two or more letters, which this rule never governed, is read in this part by S10 since the same change (`Smith, John VI` → suffix 'VI', reported). -- 2026-11-04 (Derek), #877 — THE BARE LETTER BEHIND A NAME WORD IS REPORTED, AND STILL READ AS THE GENERATION. #430 recorded the missing report as not done; #835's salutation read (rules/H8) put more names into the shape (`DOE, JANE I MRS` reads title 'MRS', suffix 'I' where it read middle 'I MRS', 24 of FEC's 59 such names losing the `conjunction-or-initial` report they had carried). #877 counted FEC's itemized contributions (1,890,655 names with exactly one comma, 2026-10-24): 214,856 end the given part in a lone capital, `V` (1,956) 19th of 26 letters and `I` (1,305) 21st, about where `O` (1,324) falls, nearly all behind a full given name; `I.` and `V.` about as common as `I` and `V`; and `IV` 765 in the slot against `III` 6,254. So in that file the letter is mostly a middle initial, where #144's data wrote it as the generation, and nothing in the writing tells the two apart: the reader-would-hesitate test the kind needs (AGENTS.md's 2.0 conventions) is met, and the report is an emission of the existing `suffix-or-name`, not a new kind. DECIDED: report only, the reading kept (1.4.0's and #144's); flipping the default to the middle initial is not decided here and would need its own measurement of #144's shape. The emitter stands in assign's given-part walk, where the branch is taken, with the no-comma report's wording and an alternative naming the middle initial (`as_a_middle`), and only where the letter is read as a suffix behind a word the walk read as a name: behind another suffix word the run settles it (`Smith, PSM I`), a period marks an initial (`Smith, John V.`), a further comma has already named the suffix (`Smith, John V, Jr.`, middle 'V'), a letter in lower case in a name of mixed case is settled as the generation (S10), and a particle in front reports nothing: it may take the letter into its chain (`, Freiherr von Q` reads family 'von Q', `Smith, Freiherr von Q` given 'von Q') or stand alone as the given name (`Smith, Van Q` reads given 'Van', middle 'Q'), and the walk cannot name the initial's field without the chain's decision, so `Smith, Van V` goes unreported, an accepted gap for a rare shape (the code review found the first draft's rationale, which named P6, wrong, and the exclusion broader than its reason; the narrower exclusion it suggested named the wrong field in 120 of 1,695 grid reports). The review also found the alternative naming the middle where a never-given particle opens the given part and the fold takes the letter into the family (`Smith, de Mesnil V`, 870 grid reports, and the shared numeral report's `Smith, de Mesnil VI` before this change): the reading not taken now names the family wherever P1's fold opens the part and the letter stands past its first word. The strict switch is not involved: the walk does not consult it (#430), and this report does not change that. AGENTS.md's "a kind is worth adding only if a reader would hesitate too" bullet used `Smith, John V` as its example of a fork nobody hesitates over, contradicting both the code and this data; it now uses `Smith, John V.`, the period settling it. +- 2026-11-04 (Derek), #877 — THE BARE LETTER BEHIND A NAME WORD IS REPORTED, AND STILL READ AS THE GENERATION. #430 recorded the missing report as not done; #835's salutation read (rules/H8) put more names into the shape (`DOE, JANE I MRS` reads title 'MRS', suffix 'I' where it read middle 'I MRS', 24 of FEC's 59 such names losing the `conjunction-or-initial` report they had carried). #877 counted FEC's itemized contributions (1,890,655 names with exactly one comma, 2026-10-24): 214,856 end the given part in a lone capital, `V` (1,956) 19th of 26 letters and `I` (1,305) 21st, about where `O` (1,324) falls, nearly all behind a full given name; `I.` and `V.` about as common as `I` and `V`; and `IV` 765 in the slot against `III` 6,254. So in that file the letter is mostly a middle initial, where #144's data wrote it as the generation, and nothing in the writing tells the two apart: the reader-would-hesitate test the kind needs (AGENTS.md's 2.0 conventions) is met, and the report is an emission of the existing `suffix-or-name`, not a new kind. DECIDED: report only, the reading kept (1.4.0's and #144's); flipping the default to the middle initial is not decided here and would need its own measurement of #144's shape. The emitter stands in assign's given-part walk, where the branch is taken, with the no-comma report's wording and an alternative naming the middle initial (`as_a_middle`), and only where the letter is read as a suffix behind a word the walk read as a name: behind another suffix word the run settles it (`Smith, PSM I`), a period marks an initial (`Smith, John V.`), a further comma has already named the suffix (`Smith, John V, Jr.`, middle 'V'), a letter in lower case in a name of mixed case is settled as the generation (S10), and a particle in front reports nothing: it may take the letter into its chain (`, Freiherr von Q` reads family 'von Q', `Smith, Freiherr von Q` given 'von Q') or stand alone as the given name (`Smith, Van Q` reads given 'Van', middle 'Q'), and the walk cannot name the initial's field without the chain's decision, so `Smith, Van V` goes unreported, an accepted gap for a rare shape (the code review found the first draft's rationale, which named P6, wrong, and the exclusion broader than its reason; the narrower exclusion it suggested named the wrong field in 120 of 1,695 reports over a dated grid, 2026-11-04: heads `Smith`, `van der Berg`, empty, `(Bob)` and `Smith (née Jones)`, fourteen given parts from `John` to `St`, tails `V`, `I`, `V Prof.`, `V Mrs` and `V I`, under the three orders and the strict switch both ways, each reported letter replaced by `Q` and the report's field compared with where `Q` lands). The review also found the alternative naming the middle where a never-given particle opens the given part and the fold takes the letter into the family (`Smith, de Mesnil V`, and the shared numeral report's `Smith, de Mesnil VI` before this change): the reading not taken now names the family wherever P1's fold opens the part and the word stands past its first word. `as_a_middle` is shared, so the change reaches every report of the given part's walk that names a middle: S10's numeral report, S8's absorbed-word report (`Smith, de John PhD Jones` → Jones as family) and #531's class-member report (`Smith, de John PhD Ma` → family), each of which named the middle there before. The fix commit's review ran the project's oracle (`tests/v2/test_cases.py`'s `_check_alternatives`) over a dated grid of 5,987 family-comma texts under the three orders, 2026-11-04: failures fell from 1,722 to 294 and none appeared, the 294 left being a split credential (`Pharm D`) and a chain behind a title-particle (`Smith, Freiherr de Mesnil Dr. VI`), both failing before. The strict switch is not involved: the walk does not consult it (#430), and this report does not change that. AGENTS.md's "a kind is worth adding only if a reader would hesitate too" bullet used `Smith, John V` as its example of a fork nobody hesitates over, contradicting both the code and this data; it now uses `Smith, John V.`, the period settling it. MEASURED 2026-11-04 with `snapshot.py --local --base ` (head package digest 3bb08fd4c91df2bc, base 2036d954b9b61eb9): no field moves anywhere, only reports. fec-individuals: 3,174 names gain `suffix-or-name` on the letter (`ABELSON, TOM I`, `ABHYANKER, RAJ V`), near #877's estimate of the 3,180 bare `I`/`V` names read as the suffix. ctgov-officials: 4 (`Fazal H Shah, FCPS I`), where the unlisted `FCPS` is read as the given name and the letter is reported behind it, a level of a credential the vocabulary does not know. psc-individuals: none. In the committed corpora 4 entries gain it (`Smith, John V`, `Chang, Andy C I`). diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 28254c064..36cbe4009 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -65,7 +65,7 @@ Emits SUFFIX_OR_NAME at these sites: the trailing roman numeral, taken or declined (since #718 for want of words to spare, since #752 for its Title case), a numeral ending the family part before a family comma, -one of two or more letters ending a given part's walk, -- since +one of two or more letters ending a given part's walk -- since #877 -- a bare one-letter numeral ending that walk behind a name word, each ambiguous acronym the trailing peel had to resolve, the bare-suffix carve-out where an input that is nothing but post-nominal vocabulary diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 3d7c88d3a..871d80b2b 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -542,7 +542,7 @@ def test_an_alternative_names_the_field_the_other_reading_gives( # #877's review: P1's fold opening the given part # takes a letter standing past its first word "Smith, de Mesnil V", "Smith, de Mesnil VI", - "Smith, Mc V I") + "Smith, Mc John V I") @pytest.mark.parametrize("order", _INVARIANT_ORDERS) From 7f7bacb196ab7fd66a0004c713ab4914cf5f9e2d Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Wed, 4 Nov 2026 16:17:24 -0800 Subject: [PATCH 5/7] Tighten #877's comments: cite the report's own C9 clause, decisions pointers (comment review) Co-Authored-By: Claude Opus 5.5 --- nameparser/_pipeline/_assign.py | 29 ++++++++++++++++------------- tests/v2/cases.py | 4 +++- tests/v2/pipeline/test_assemble.py | 4 +++- tests/v2/test_cases.py | 7 ++++--- 4 files changed, 26 insertions(+), 18 deletions(-) diff --git a/nameparser/_pipeline/_assign.py b/nameparser/_pipeline/_assign.py index 36cbe4009..0abf6d7b2 100644 --- a/nameparser/_pipeline/_assign.py +++ b/nameparser/_pipeline/_assign.py @@ -65,8 +65,9 @@ Emits SUFFIX_OR_NAME at these sites: the trailing roman numeral, taken or declined (since #718 for want of words to spare, since #752 for its Title case), a numeral ending the family part before a family comma, -one of two or more letters ending a given part's walk -- since -#877 -- a bare one-letter numeral ending that walk behind a name word, each +one of two or more letters ending a given part's walk, a bare I or V +read as the generation ending that walk behind a name word (since +#877), each ambiguous acronym the trailing peel had to resolve, the bare-suffix carve-out where an input that is nothing but post-nominal vocabulary gets its first word made into the name (H4's suffix half, #491), @@ -1637,12 +1638,14 @@ def reads_as_a_suffix(m: int, titled: frozenset[int]) -> bool: def as_a_middle(m: int, token: WorkToken) -> PendingReading: # the name reading of a word the walk read as a suffix: - # a middle name, or taken in by the name word in front + # a middle name, taken in by the name word in front, or + # the family where P1's fold opens the part front = pieces[m - 1] - # rules/P1's fold after a family comma takes the rest of - # the part into the family, so a word behind a part a - # never-given particle opens lands there too, however - # far behind it the word stands (#877 review) + # P1's fold after a family comma takes the rest of the + # part into the family whatever the order, so a word past + # the opening particle's own piece lands there; the word + # directly behind it is `_name_alt`'s, which takes it into + # that piece; decisions/C9 (#877) if n < m - 1 and is_lone_never_given_particle(pieces[n], tokens): return name_reading(token, Role.FAMILY) @@ -1689,11 +1692,10 @@ def as_a_middle(m: int, token: WorkToken) -> PendingReading: tokens[pieces[m][0]].text.upper())): _set_roles(tokens, pieces[m], Role.MIDDLE) continue - # rules/C9: "behind a name word it is the generation only - # when written bare" -- and a bare letter there is the - # shape a middle initial is written in, so the reading - # is reported, as S10 reports it with no comma - # (decisions/C9, #877); `len` first, C-level + # rules/C9: "The bare letter read as the generation behind + # a name word is reported" -- the shape a middle initial + # is written in, as S10 reports it with no comma; + # decisions/C9 (#877); `len` first, C-level if (len(pieces[m]) == 1 and len((tok := tokens[pieces[m][0]]).text) == 1 and (m >= sticky_from and not k6 <= m < end6 @@ -1702,7 +1704,8 @@ def as_a_middle(m: int, token: WorkToken) -> PendingReading: and tokens[pieces[m - 1][0]].role in NAME_ROLES # a particle in front may take the letter into # its chain or stand alone, the field the name - # reading names depending on which (#877 review) + # reading names depending on which; decisions/C9 + # (#877) and not is_prefix_piece(pieces[m - 1], ptags[m - 1], tokens) and not numeral_settled_by_case(tok.text, diff --git a/tests/v2/cases.py b/tests/v2/cases.py index d00454658..051a991d1 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -12625,7 +12625,9 @@ def _check_cjk_shape_purity(self) -> None: notes="THE BOUNDARY, and v1 parity (#144): with no period " "there is no abbreviation, so the numeral is the " "generation it looks like. This row is what makes " - "#432's fix a period test rather than a numeral test"), + "#432's fix a period test rather than a numeral test. " + "Reported since #877, the letter being the shape a " + "middle initial is written in (rules/C9)"), Case("family_comma_title_resets_the_credential_run", "Smith, RA Dr. I", {"title": "RA Dr.", "given": "I", "family": "Smith"}, notes="THE RESET. A title ends the run: what follows a bare " diff --git a/tests/v2/pipeline/test_assemble.py b/tests/v2/pipeline/test_assemble.py index ec75ce4f3..fce97e587 100644 --- a/tests/v2/pipeline/test_assemble.py +++ b/tests/v2/pipeline/test_assemble.py @@ -159,7 +159,9 @@ def test_a_connective_or_initial_report_is_withdrawn_where_the_role_is_a_suffix( withdrawn = { "JOHN QUINCY SMITH I": ["suffix-or-name"], "john smith i": ["suffix-or-name"], - "SMITH, JOHN I": ["suffix-or-name"], # rules/C9, #877 + # C9's report on the letter (#877); the conjunction-or-initial + # one is still withdrawn + "SMITH, JOHN I": ["suffix-or-name"], "HENRY SMITH I": ["suffix-or-name"], } for text, kinds in withdrawn.items(): diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index 871d80b2b..ca1288711 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -539,8 +539,9 @@ def test_an_alternative_names_the_field_the_other_reading_gives( # the PR review's: a given-name title licenses the # bound join only from the leading run, as group asks "abdul VI Sir.", "Sir Dr. abdul VI", "Dr. Sir abdul VI", - # #877's review: P1's fold opening the given part - # takes a letter standing past its first word + # #877's review: P1's fold opening the given part takes + # a numeral standing past the part's first word into + # the family "Smith, de Mesnil V", "Smith, de Mesnil VI", "Smith, Mc John V I") @@ -727,7 +728,7 @@ def test_the_alternative_sweep_sees_the_fields_it_checks() -> None: # three orders; 462 -> 477 the same day, #1039's junior rows under # the three orders; 477 -> 495 on 2026-11-04, #877's report on a bare # letter ending the given part after a comma, six rows under the - # three orders + # three orders, each stand-in landing in the alternative's field assert (checked, skipped) == (495, 330), (checked, skipped) From adb29360aed8c2347838fb56bc304fda5b33fb5f Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Wed, 4 Nov 2026 16:23:35 -0800 Subject: [PATCH 6/7] Pin #877's report behind a particle run and a connective (test review) A run of particles and a connective in front are name words whose field the report can name ('Smith, John de la Q' reads middle 'de la Q'), so the letter behind them is reported; rules/C9 says 'a lone particle' for the exclusion. Co-Authored-By: Claude Opus 5.5 --- ...aracter-suffix-word-inside-a-comma-part.md | 3 ++- tests/v2/cases.py | 21 +++++++++++++++++++ tests/v2/test_cases.py | 14 ++++++++----- tools/differential/snapshot.jsonl | 1 + 4 files changed, 33 insertions(+), 6 deletions(-) diff --git a/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md b/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md index 1e06d5cdb..6a745836a 100644 --- a/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md +++ b/docs/design/rules/C9-one-character-suffix-word-inside-a-comma-part.md @@ -12,13 +12,14 @@ "Smith, John PhD I, Jr." → suffix="PhD, Jr." "Smith, John PhD É, Jr." → suffix="PhD É, Jr." · boundary ``` -- **The bare letter read as the generation behind a name word is reported**, naming the middle initial it would otherwise be, as S10 reports the same letter with no comma: one letter is the shape a middle initial is written in, so a reader cannot tell the two apart there. Nothing is reported where the writing or the slot settles it: a letter behind another suffix word, a period-marked letter, one ahead of a further comma, and one in lower case in a name of mixed case (S10). Nor behind a particle, which may take the letter into its chain or stand alone as the given name, so the report could not say which field the initial would land in. +- **The bare letter read as the generation behind a name word is reported**, naming the middle initial it would otherwise be, as S10 reports the same letter with no comma: one letter is the shape a middle initial is written in, so a reader cannot tell the two apart there. Nothing is reported where the writing or the slot settles it: a letter behind another suffix word, a period-marked letter, one ahead of a further comma, and one in lower case in a name of mixed case (S10). Nor behind a lone particle, which may take the letter into its chain or stand alone as the given name, so the report could not say which field the initial would land in; behind a run of particles, which P6 attaches to the family, it is reported, a name word in its place joining the run as the middle name. ``` "Smith, John V" → ambiguities=("suffix-or-name",) "SMITH, JOHN I" → ambiguities=("suffix-or-name",) "Smith, PSM I" → ambiguities=() "Smith, John v" → ambiguities=() "Smith, Van V" → ambiguities=() + "Smith, John de la V" → ambiguities=("suffix-or-name", "particle-or-given") ``` - **Only the given part is touched**: a part after the comma that is nothing but suffix words is the credential run, and a letter in it continues that run up to the further comma. - **Longer suffix words are not in question either way**, a numeral of two or more letters being read as S10 reads one in this part. diff --git a/tests/v2/cases.py b/tests/v2/cases.py index 051a991d1..a36b44e3e 100644 --- a/tests/v2/cases.py +++ b/tests/v2/cases.py @@ -12628,6 +12628,27 @@ def _check_cjk_shape_purity(self) -> None: "#432's fix a period test rather than a numeral test. " "Reported since #877, the letter being the shape a " "middle initial is written in (rules/C9)"), + Case("family_comma_bare_numeral_behind_a_particle_run_reports", + "Smith, John de la V", + {"given": "John", "family": "de la Smith", "suffix": "V"}, + ambiguities=("suffix-or-name", "particle-or-given"), + classification="fix(#877)", + notes="rules/C9's report behind a particle RUN: P6 attaches " + "the run to the family, and with a name word in the " + "letter's place the run chains it as the middle name " + "('Smith, John de la Q' reads middle 'de la Q'), so the " + "report can name its field and is made; behind a lone " + "particle ('Smith, John de V') it is not"), + Case("family_comma_bare_numeral_behind_a_connective_reports", + "Smith, Maria y V", + {"given": "Maria", "middle": "y", "family": "Smith", + "suffix": "V"}, + ambiguities=("suffix-or-name",), + classification="fix(#877)", + notes="rules/C9's report behind a connective, which is a " + "name word to the walk: with a name word in the letter's " + "place the connective joins it as the middle name " + "('Smith, Maria y Q' reads middle 'y Q')"), Case("family_comma_title_resets_the_credential_run", "Smith, RA Dr. I", {"title": "RA Dr.", "given": "I", "family": "Smith"}, notes="THE RESET. A title ends the run: what follows a bare " diff --git a/tests/v2/test_cases.py b/tests/v2/test_cases.py index ca1288711..e08e09ffd 100644 --- a/tests/v2/test_cases.py +++ b/tests/v2/test_cases.py @@ -355,9 +355,11 @@ def test_the_field_sweep_sees_the_claims_it_checks() -> None: # one), each under three orders, and its review row 'Smith, # Junior.', reporting suffix-or-name under three orders; 1582 -> # 1600 on 2026-11-04, #877's report on a bare letter ending the - # given part after a comma, six rows each under three orders. - assert claims == 1600, ( - f"the field sweep checks {claims} claims, recorded as 1600 on " + # given part after a comma, six rows each under three orders; 1600 -> + # 1609 the same day, its test review's particle-run and connective + # rows, three reports each under three orders. + assert claims == 1609, ( + f"the field sweep checks {claims} claims, recorded as 1609 on " f"2026-11-04") @@ -728,8 +730,10 @@ def test_the_alternative_sweep_sees_the_fields_it_checks() -> None: # three orders; 462 -> 477 the same day, #1039's junior rows under # the three orders; 477 -> 495 on 2026-11-04, #877's report on a bare # letter ending the given part after a comma, six rows under the - # three orders, each stand-in landing in the alternative's field - assert (checked, skipped) == (495, 330), (checked, skipped) + # three orders, each stand-in landing in the alternative's field; + # 495 -> 504 the same day, its test review's two rows, three + # reports under the three orders + assert (checked, skipped) == (504, 330), (checked, skipped) def test_every_case_name_is_unique() -> None: diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 90827cd7d..964b41711 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -3540,6 +3540,7 @@ {"name": "Smith, John VI MD", "given": "John", "family": "Smith", "suffix": "VI MD", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: name as middle (position)"], "initials": "J. S.", "generation": "VI", "credentials": "MD"} {"name": "Smith, John Vi", "given": "John", "middle": "Vi", "family": "Smith", "initials": "J. V. S."} {"name": "Smith, John X", "given": "John", "middle": "X", "family": "Smith", "initials": "J. X. S."} +{"name": "Smith, John de la V", "given": "John", "family": "de la Smith", "suffix": "V", "ambiguities": ["PARTICLE_OR_GIVEN", "SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as middle (shape)", "PARTICLE_OR_GIVEN: name as middle (declared)"], "initials": "J. S.", "generation": "V"} {"name": "Smith, John e, III, Jr", "given": "John", "middle": "e", "family": "Smith", "suffix": "III, Jr", "initials": "J. e. S.", "generation": "III, Jr"} {"name": "Smith, John van der Berg Prof.", "title": "Prof.", "given": "John", "middle": "van der Berg", "family": "Smith", "initials": "J. B. S."} {"name": "Smith, John, Bob (MA)", "given": "John", "family": "Smith", "suffix": "Bob MA", "ambiguities": ["COMMA_STRUCTURE"], "initials": "J. S."} From bfae56f098e2ec2f1cddad12240ae1b2fd6867a9 Mon Sep 17 00:00:00 2001 From: Derek Gulbranson Date: Wed, 4 Nov 2026 19:52:39 -0800 Subject: [PATCH 7/7] Regenerate the shared records on master after #893 (#877 rebase) Co-Authored-By: Claude Opus 5.5 --- tests/v2/rules_budget.json | 2 +- tools/differential/corpus_rules.jsonl | 3 +++ tools/differential/snapshot.jsonl | 10 ++++++---- 3 files changed, 10 insertions(+), 5 deletions(-) diff --git a/tests/v2/rules_budget.json b/tests/v2/rules_budget.json index cf4918a95..310a54611 100644 --- a/tests/v2/rules_budget.json +++ b/tests/v2/rules_budget.json @@ -9,7 +9,7 @@ "C6": {"words": 207, "then_words": 162, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "C7": {"words": 119, "then_words": 68, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "C8": {"words": 176, "then_words": 110, "because_sentences": 1, "except": 0, "precedence": 1, "accepted": 1, "accepted_words": 36}, - "C9": {"words": 298, "then_words": 219, "because_sentences": 1, "except": 0, "precedence": 1, "accepted": 1, "accepted_words": 48}, + "C9": {"words": 446, "then_words": 367, "because_sentences": 1, "except": 0, "precedence": 1, "accepted": 1, "accepted_words": 48}, "C10": {"words": 434, "then_words": 272, "because_sentences": 1, "except": 1, "precedence": 1, "accepted": 7, "accepted_words": 480}, "C11": {"words": 391, "then_words": 226, "because_sentences": 1, "except": 2, "precedence": 0, "accepted": 0, "accepted_words": 0}, "C12": {"words": 316, "then_words": 107, "because_sentences": 2, "except": 2, "precedence": 1, "accepted": 2, "accepted_words": 59}, diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 78bb0b769..672c05e72 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -975,6 +975,7 @@ "SMITH, JOHN B. MAJ." "SMITH, JOHN DO" "SMITH, JOHN GEN." +"SMITH, JOHN I" "SMITH, MA" "SMITH, SAM E." "SMITH, VD DE LA MA" @@ -1077,6 +1078,8 @@ "Smith, John VI MD" "Smith, John Vi" "Smith, John X" +"Smith, John de la V" +"Smith, John v" "Smith, John van der Berg Prof." "Smith, John, Bob (MA)" "Smith, John, Jr." diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 964b41711..43b2a5d89 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -567,7 +567,7 @@ {"name": "Cetto, August von", "given": "August", "family": "von Cetto", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: name as middle (declared)"], "initials": "A. C."} {"name": "Chancellor Jane Smith", "given": "Chancellor", "middle": "Jane", "family": "Smith", "ambiguities": ["TITLE_OR_NAME"], "alternatives": ["TITLE_OR_NAME: title as title (vocabulary)"], "initials": "C. J. S."} {"name": "Chancellor Smith", "given": "Chancellor", "family": "Smith", "ambiguities": ["TITLE_OR_NAME"], "alternatives": ["TITLE_OR_NAME: title as title (vocabulary)"], "initials": "C. S."} -{"name": "Chang, Andy C I", "given": "Andy", "middle": "C", "family": "Chang", "suffix": "I", "initials": "A. C. C.", "generation": "I"} +{"name": "Chang, Andy C I", "given": "Andy", "middle": "C", "family": "Chang", "suffix": "I", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as middle (shape)"], "initials": "A. C. C.", "generation": "I"} {"name": "Charles Claude de La Billarderie d'Angiviller", "given": "Charles", "middle": "Claude", "family": "de La Billarderie d'Angiviller", "initials": "C. C. B. d.", "capitalized_force": "Charles Claude de la Billarderie D'Angiviller"} {"name": "Charles Wilkins", "given": "Charles", "family": "Wilkins", "initials": "C. W."} {"name": "Charles van der van der Berg", "given": "Charles", "family": "van der van der Berg", "initials": "C. B."} @@ -2555,7 +2555,7 @@ {"name": "Maider", "given": "Maider", "ambiguities": ["GIVEN_OR_FAMILY"], "alternatives": ["GIVEN_OR_FAMILY: name as family (position)"], "initials": "M."} {"name": "Maider", "order": "FAMILY_FIRST", "family": "Maider", "ambiguities": ["GIVEN_OR_FAMILY"], "alternatives": ["GIVEN_OR_FAMILY: name as given (position)"], "initials": "M."} {"name": "Maier, Amy I, Jr.", "given": "Amy", "middle": "I", "family": "Maier", "suffix": "Jr.", "initials": "A. I. M.", "generation": "Jr."} -{"name": "Maier, Amy Lauren I", "given": "Amy", "middle": "Lauren", "family": "Maier", "suffix": "I", "initials": "A. L. M.", "generation": "I"} +{"name": "Maier, Amy Lauren I", "given": "Amy", "middle": "Lauren", "family": "Maier", "suffix": "I", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as middle (shape)"], "initials": "A. L. M.", "generation": "I"} {"name": "Maj Gen John Smith", "title": "Maj Gen", "given": "John", "family": "Smith", "initials": "J. S."} {"name": "Maj Sjöwall", "given": "Maj", "family": "Sjöwall", "ambiguities": ["TITLE_OR_NAME"], "alternatives": ["TITLE_OR_NAME: title as title (vocabulary)"], "initials": "M. S."} {"name": "Maj Tom Mouat", "given": "Maj", "middle": "Tom", "family": "Mouat", "ambiguities": ["TITLE_OR_NAME"], "alternatives": ["TITLE_OR_NAME: title as title (vocabulary)"], "initials": "M. T. M."} @@ -3279,6 +3279,7 @@ {"name": "SMITH, JOHN B. MAJ.", "title": "MAJ.", "given": "JOHN", "middle": "B.", "family": "SMITH", "initials": "J. B. S.", "capitalized": "Maj. John B. Smith"} {"name": "SMITH, JOHN DO", "given": "JOHN", "family": "DO SMITH", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: name as middle (declared)"], "initials": "J. S.", "capitalized": "John do Smith"} {"name": "SMITH, JOHN GEN.", "title": "GEN.", "given": "JOHN", "family": "SMITH", "initials": "J. S.", "capitalized": "Gen. John Smith"} +{"name": "SMITH, JOHN I", "given": "JOHN", "family": "SMITH", "suffix": "I", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as middle (shape)"], "initials": "J. S.", "capitalized": "John Smith I", "generation": "I"} {"name": "SMITH, MA", "given": "MA", "family": "SMITH", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: credential as suffix (vocabulary)"], "initials": "M. S.", "capitalized": "Ma Smith"} {"name": "SMITH, SAM E.", "given": "SAM", "middle": "E.", "family": "SMITH", "initials": "S. E. S.", "capitalized": "Sam E. Smith"} {"name": "SMITH, VD DE LA MA", "family": "SMITH VD DE LA MA", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: credential as suffix (vocabulary)"], "initials": "S. M.", "capitalized": "Smith vd de la Ma"} @@ -3510,7 +3511,7 @@ {"name": "Smith, John ,", "given": "John", "family": "Smith", "initials": "J. S."} {"name": "Smith, John - Paul", "given": "John", "middle": "- Paul", "family": "Smith", "initials": "J. -. P. S."} {"name": "Smith, John E, III, Jr", "given": "John", "middle": "E", "family": "Smith", "suffix": "III, Jr", "initials": "J. E. S.", "generation": "III, Jr"} -{"name": "Smith, John I", "given": "John", "family": "Smith", "suffix": "I", "initials": "J. S.", "generation": "I"} +{"name": "Smith, John I", "given": "John", "family": "Smith", "suffix": "I", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as middle (shape)"], "initials": "J. S.", "generation": "I"} {"name": "Smith, John Jr.", "given": "John", "family": "Smith", "suffix": "Jr.", "initials": "J. S.", "generation": "Jr."} {"name": "Smith, John Jr. Jones", "given": "John", "family": "Smith", "suffix": "Jr. Jones", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: name as middle (position)"], "initials": "J. S.", "generation": "Jr.", "credentials": "Jones"} {"name": "Smith, John Junior", "given": "John", "family": "Smith", "suffix": "Junior", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: name as middle (declared)"], "initials": "J. S.", "generation": "Junior"} @@ -3530,7 +3531,7 @@ {"name": "Smith, John Pharm D Jones", "given": "John", "family": "Smith", "suffix": "Pharm D Jones", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: name as middle (position)", "SUFFIX_OR_NAME: name as middle (position)"], "initials": "J. S."} {"name": "Smith, John Prof.", "title": "Prof.", "given": "John", "family": "Smith", "initials": "J. S."} {"name": "Smith, John Sam E.", "given": "John", "family": "Smith", "suffix": "Sam E.", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: name as middle (position)"], "initials": "J. S.", "capitalized_force": "John Smith SAM E."} -{"name": "Smith, John V", "given": "John", "family": "Smith", "suffix": "V", "initials": "J. S.", "generation": "V"} +{"name": "Smith, John V", "given": "John", "family": "Smith", "suffix": "V", "ambiguities": ["SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as middle (shape)"], "initials": "J. S.", "generation": "V"} {"name": "Smith, John V MD", "given": "John", "middle": "V", "family": "Smith", "suffix": "MD", "initials": "J. V. S."} {"name": "Smith, John V, Jr.", "given": "John", "middle": "V", "family": "Smith", "suffix": "Jr.", "initials": "J. V. S.", "generation": "Jr."} {"name": "Smith, John V.", "given": "John", "middle": "V.", "family": "Smith", "initials": "J. V. S."} @@ -3542,6 +3543,7 @@ {"name": "Smith, John X", "given": "John", "middle": "X", "family": "Smith", "initials": "J. X. S."} {"name": "Smith, John de la V", "given": "John", "family": "de la Smith", "suffix": "V", "ambiguities": ["PARTICLE_OR_GIVEN", "SUFFIX_OR_NAME"], "alternatives": ["SUFFIX_OR_NAME: initial as middle (shape)", "PARTICLE_OR_GIVEN: name as middle (declared)"], "initials": "J. S.", "generation": "V"} {"name": "Smith, John e, III, Jr", "given": "John", "middle": "e", "family": "Smith", "suffix": "III, Jr", "initials": "J. e. S.", "generation": "III, Jr"} +{"name": "Smith, John v", "given": "John", "family": "Smith", "suffix": "v", "initials": "J. S.", "capitalized_force": "John Smith V", "generation": "v"} {"name": "Smith, John van der Berg Prof.", "title": "Prof.", "given": "John", "middle": "van der Berg", "family": "Smith", "initials": "J. B. S."} {"name": "Smith, John, Bob (MA)", "given": "John", "family": "Smith", "suffix": "Bob MA", "ambiguities": ["COMMA_STRUCTURE"], "initials": "J. S."} {"name": "Smith, John, Extra, Jr.", "given": "John", "family": "Smith", "suffix": "Extra, Jr.", "ambiguities": ["COMMA_STRUCTURE"], "initials": "J. S.", "generation": "Jr.", "credentials": "Extra"}