diff --git a/AGENTS.md b/AGENTS.md index e8dd46c3..9b961bd9 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -223,6 +223,7 @@ The compiled sets, read by `Lexicon` (mostly as the `_sets` field of the same na - particles — `PARTICLES` (family-name particles, e.g. "de", "van") and `NON_GIVEN_NAME_PARTICLES`, the curated subset that is *never* a standalone given name (the particles whose word has no `given` reading, each stating `may_be_given = false`; a `given` reading on the word is what makes a particle possibly given, the phase 1 flag `may_be_given = true` refused since 2026-10-25). What the subset licenses is rules/P1's fold in `post_rules` (the "rule 1b" of older issue comments — decisions/legacy-rule-numbers maps the old numbers), and the fold has ONE site: the piece that OPENS the name. A lone never-given particle there, with at least one more name word behind it, starts the family. Titles do not move the opening position (rules/P4), so `Sir de Mesnil` reads family `de Mesnil` under every `name_order`, as `de Mesnil` does. How far the fold reaches depends on the order the name was read under: by default it takes the rest of the name (`de Mesnil Jean` → family `de Mesnil Jean`); under either family-first order it takes the particle run plus ONE name unit and lays the rest out by the order (`de Mesnil Jean` → family `de Mesnil`, given `Jean`); after a family comma it takes the rest of that part whatever the order (`Smith, de Mesnil Jean` → family `Smith de Mesnil Jean`). The leading particle stays a lone piece for the fold to find — `group` does not chain it forward, and the fold walks the particles after it itself — so `de la Vega` reads family `de la Vega` under all three orders. The GIVEN position is not a site. It was a second one until #467, which removed it because under a family-first order that slot holds what the caller DECLARED to be the given name: `Mesnil de` under `FAMILY_FIRST` or `FAMILY_FIRST_GIVEN_LAST` reads given `de`, family `Mesnil`. Nor is a particle inside the name: `Juan de la Vega` under a family-first order reads family `Juan`, given `de la Vega`, with no ambiguity recorded — what the vocabulary forbids is the bare particle as a given name, not a name part that begins with one. A bare `de` has nothing to fold into and is placed by position like any one-word name: given `de` by default, family `de` under both family-first orders. A leading particle OUTSIDE the set is left to position — `van Gogh` is given `van`, family `Gogh` by default and family `van`, given `Gogh` under both family-first orders — and records a `PARTICLE_OR_GIVEN` ambiguity under all three; a never-given particle opening the name records none (`de Mesnil`), and neither does an ambiguous one inside it (`Vincent van Gogh`). The internal `Lexicon._sets.particles_ambiguous`, the particles holding a `given` reading, is its complement within `PARTICLES`, so the two mark OPPOSITE sets — see the flip warning in `docs/migrate.rst` before translating either - bound given names — `BOUND_GIVEN_NAMES` (`bound_given` readings, e.g. "abdul", "abu"; one that is also a particle needs a `given` reading, a build check); a group-stage rule joins the first non-title piece to its following piece before roles are assigned, reserving a family word unless a family comma has already fixed it or a given-name title stands ahead (#369, rules/P5) (ported from v1's `_join_bound_first_name`) - given names written short — `GIVEN_NAME_ABBREVIATIONS` (#789, a `given` reading carrying `same_as`, the full name: "chas", "wm", Filipino "ma"), the given names written short that rules/H2's shape yields to; nothing else reads it +- names the two-name switches read — `PARTICLE_GIVEN_NAMES` (#704, a `given` reading written `after_particle`: "carmen", "ángeles", the Spanish words a compound given name writes behind a particle, `María del Carmen`) and `SHARED_NAMES` (#704, a name reading written `shared`: Sikh "kaur", "singh"), private; only rules/O7 and O6 read them, through classify's tags, so neither moves a parse with both switches off - conjunctions — `CONJUNCTIONS` (e.g. "and", "of") used to chain multi-word titles and to join name parts (rules/P3), plus `CONJUNCTIONS_AMBIGUOUS` (#383/#479, rules/P8, `may_be_initial`, refused by the build on anything but a cased single letter), the subset of single cased letters that read as an INITIAL in a name written wholly in one case — a marker set - maiden markers — `MAIDEN_MARKERS` (`birth_marker` readings, e.g. "née", "geb.") routing the following name to `maiden`; with an `addresses = "given"` title, the one kind a phrase may be (`z domu`) - `config/surnames.py`, a public module — `KOREAN_SURNAMES`, the census list the 2.0 API splits unspaced hangul on (#271), still written in Python with an import-time `assert_normalized`: a segmentation list, outside the per-word model, like the locale packs' lists. diff --git a/docs/customize.rst b/docs/customize.rst index a8724a97..5a6932c6 100644 --- a/docs/customize.rst +++ b/docs/customize.rst @@ -124,7 +124,8 @@ A kind belongs to a category, which decides what the reading does: - none * - name - ``given``, ``family`` - - ``same_as="..."`` on ``given`` + - ``same_as="..."`` and ``after_particle=True`` on ``given``; + ``shared=True`` on either ``nameparser/vocab/README.md`` in the repository says what each kind and property means. A property given its unwritten value (``None``, @@ -1063,7 +1064,8 @@ hyphen or ``y`` joins them (``"Ana María García-López"``, ``"José Ortega y Gasset"``) or a particle chain took them both: a compound given name opening with a particle, such as ``"María del Carmen García López"``, reads family ``del Carmen García López`` with the switch on -or off. A name of two words is unchanged, and a comma naming the family +or off, until ``two_given_names`` below takes ``del Carmen`` into the +given name. A name of two words is unchanged, and a comma naming the family decides by itself, so ``"García López, Ana"`` needs no switch at all. Under a family-first ``name_order`` the *first* two name words are the family. @@ -1149,11 +1151,30 @@ two: >>> both.parse("Gabriel García Márquez").given 'Gabriel' -A given name a particle completes reads whole where a second particle -starts the family (``"María del Carmen de la Vega"`` gives given -``María del Carmen``); where the particle's words run to the end of -the name, as in ``"Maria do Carmo Silva"``, they stay the family with -the switch on or off, the same shape as ``"João da Silva Santos"``. +A given name a particle completes reads whole too. Where the +particle's words would otherwise run on into the family, the switch +takes them only if the vocabulary marks the word behind the particle +as a given name written there (``after_particle``). The Spanish ones +are marked, those that ORCID's researchers from Spain and +Spanish-speaking America write in the given box at least nine times +for each ten they also open a family box with it (``del +Carmen``, ``de los Ángeles``, ``del Mar``, ``del Pilar``: 23 words in +all on 2026-11-03). A phrase too often a surname to mark, as Portuguese ``do Carmo`` +is, stays the family, the same shape as ``"João da Silva Santos"``: + +.. doctest:: + + >>> carmen = two_given.parse("María del Carmen García") + >>> carmen.given, carmen.family + ('María del Carmen', 'García') + >>> two_given.parse("Maria do Carmo Silva").family + 'do Carmo Silva' + +To mark a word of your own, add it with ``Lexicon.default().add("carmo", +kind="given", after_particle=True)``. The two switches also leave alone +the Sikh names Kaur and Singh, given to every Sikh woman and man and +written after the personal name: ``"Harpreet Kaur Dhillon"`` keeps +middle ``Kaur`` under either switch, as it does by default. Turn it on only for data written this way: under it every name whose second given word is a middle name gains it in the given name diff --git a/docs/design/decisions/O6-two-family-names.md b/docs/design/decisions/O6-two-family-names.md index 16f26aab..160ee4ce 100644 --- a/docs/design/decisions/O6-two-family-names.md +++ b/docs/design/decisions/O6-two-family-names.md @@ -13,8 +13,9 @@ - The switch runs before `middle_as_family` and is not refused beside it: given-first, the two together read as `middle_as_family` alone — 2026-10-24 #754. - The switch reports nothing — 2026-10-24 #754. - The switch reports no alternative either, so the reading it did not take is not surfaced: the caller declared the convention — 2026-10-27 #752 (Derek). -- A listed given name (`Lexicon.given_names`) standing alone is not taken, under every order, and the name then reads as with the switch off; a listed word behind a particle is taken as before, and with the list empty the switch reads as before; no other rule reads the list, the default carries none, and the veto reports nothing — 2026-10-30 #959. +- A listed given name (`Lexicon.given_names`) standing alone is not taken, under every order, and the name then reads as with the switch off; a listed word behind a particle is taken as before (unless written `after_particle` and standing given-first beside a given name of one word, the 2026-11-03 #704 entry), and with the list empty the switch reads as before; no other rule reads the list, the default carries none, and the veto reports nothing — 2026-10-30 #959. - The veto stays a clause of this rule though it reads a second signal, the taken word looked up in a list, and the rule is a split candidate — 2026-10-30 #959, decided. +- Given-first, a particle run and the one word behind it whose `given` reading is written `after_particle` (rules/O7), standing beside a given name of one word, is not taken; under a family-first order it is, completing no given name there. Neither side of a join holding a name reading written `shared` (Sikh `kaur`, `singh`) is joined: each is no family name, and with the switch alone the name reads as with it off — 2026-11-03 #704, decided in chat. - The Spanish list, which `locales.ES` carries as its lexicon fragment since sub-phase 3g (a caller may pass a list of its own), is generated, not per-word vocabulary: INE's 2025 census tables, pooled over every name borne in Spain, cut at a measured rank (1,000 words, 90% given use) by `tools/es_given_names.py`; a change to it is a cutoff change — 2026-10-30 #959. Open: why the Spanish list does less in Latin America, and whether a list per region or another selection rule would serve it better than the one list `locales.ES` carries (the 2026-10-30 #959 entry's Latin America bullet; the 2026-10-30 #753 phase 3g entry measures the pack against the default there) — #959. A name written with ñ dropped (`Begona`, `Inigo`) matches no entry, the key keeping the tilde; whether to fold ñ for data that strips it — #959. @@ -75,7 +76,7 @@ Open: why the Spanish list does less in Latin America, and whether a list per re - **The cutoff: 1,000 words** (84.3% of the given-word uses in INE's table). Measured per country with O6 on, the list cut at 100 to all 16,559 candidates; agreement in each of the six double-surname countries is within 0.3 points of the 2,000-word list at 1,000 (Venezuela the 0.3), and a band past it buys at most a tenth of a net fix per added word in those six (1,000 → 2,000: 134 fixed, 35 broken; 2,000 → 5,000: 128 fixed, 36 broken, Chile breaking as many as it fixes), where the first hundred words bought sixteen. Argentina gains 1.6 points more by 2,000 and 2.4 by all, which 3g weighs if it turns the switch on there. - **Family-first, symmetric.** Under a family-first order the switch takes the unit after the family, and a listed one stays where it is: `Pérez José Luis` keeps family `Pérez`. No population the corpora hold writes Spanish names family-first with the fields stated, so it is measured on ORCID's boxes rendered family box first, which is a rendering, not anyone's writing. The cost moves with the order: a listed word that is a second surname, written after the first, becomes the given name under FAMILY_FIRST (`Flores Jorge Ana` gives given `Jorge`, middle `Ana`) and a middle name under FAMILY_FIRST_GIVEN_LAST, where given-first it is a first surname left as a middle name; on the rendering the list breaks 38 (Chile) to 205 (Peru) records a country, the family-first line below (found by the #961 review). - **A bare word only.** The veto reads a unit that is one word with no particle; a listed word behind a particle is a surname unit and is taken as before (`Ana de Miguel de la Vega` keeps family `de Miguel de la Vega`), which is what the tool's count assumes in leaving such surnames out. The first draft read the word inside a particle unit too, and the design-docs review found it reading `Ana de Miguel de la Vega` middle `de Miguel`. - - **O7.** With both switches on, the word O6 leaves is a middle O7 takes where it stands beside the given name, so `José Luis Pérez` reads given `José Luis`, family `Pérez` (rules/O7's precedence); `Ana María Luis Pérez` keeps middle `Luis`, O7 taking `María`. Whether O7 reads the list itself is decisions/O7's Open question. + - **O7.** With both switches on, the word O6 leaves is a middle O7 takes where it stands beside the given name, so `José Luis Pérez` reads given `José Luis`, family `Pérez` (rules/O7's precedence); `Ana María Luis Pérez` keeps middle `Luis`, O7 taking `María`. Whether O7 reads the list itself is decisions/O7's Open question (declined 2026-11-03 #704: O7 reads a word written `after_particle` instead). - **No report**, as for the switch (the 2026-10-24 entry): the caller declared the convention and the list. - **Latin America, measured in the #961 review.** Of the 27 words commonest among the Argentine records the default reads right and the switch with the list reads wrong, each record keyed by the last word of its given box (836 records), 722 are lost to the 90% bar measured on Spain's population and 114 to the 1,000 cut (a word at or over the bar ranked past 1,000), none to anything else: `Martín` (10% given in INE's tables), `Andrés` 77%, `Nicolás` 76%, `Sebastián` 81%, `Esteban` 36%, `Cecilia` 89.75% (84 Argentine records, just under the bar), `Matías` 76%, `Hernán` 63%, `Gastón` 40%, `Santiago` 57% and `Tomás` 61%, the shares recomputed here. Peru is net negative because the switch alone already agrees on 96.0% of its records, leaving little to fix, and its 189 breaks spread over the whole list, ranks 1 to 994 (`maria` 1st, `jose` 2nd, `chiara` 994th), so they come from the selection rule rather than the cutoff (the records the switch reads right and the switch with the list wrong, each keyed by the word the veto left, the switch's family word missing from the vetoed family, ranked in the list). Whether 3g needs a list per region or another selection rule is Open above; the list is not changed here. Both counts recomputed on the branch at e06b2612 with the review's recipes as stated, over the records of `orcid-researchers` with both boxes, read with `local_agreement.read`, words folded by `given_name_key`. - **Pickling.** The field carries `_ADDED_IN` "3.0.0" and an empty default, so a `Lexicon`, `Parser` or locale pack pickled by 2.x loads with no list. That is what such a pickle meant, no pack filling the field before 3.0, so this field is the case decisions/A1's 2026-10-30 caveat does not reach: the silent drift it warns of comes from a marked field the default vocabulary fills, and the default leaves this one empty. `tests/v2/test_lexicon.py` loads the committed 2.8.0 pickle with the field empty, and refuses it with the marker taken off. @@ -166,4 +167,5 @@ Open: why the Spanish list does less in Latin America, and whether a list per re Every broken record, in all three, has two or more words in the given box, read as a given name and a family name: no population-wide Czech or Slovak given-name list is published to veto the word, the register's frequencies having been withdrawn after 2016's; ČSÚ's top 100 newborn given names a year (CC BY 4.0) is a list, and the open lead. The circular ceiling such a list could reach, and the sources looked for, are in the vocabulary-model entry. - 2026-11-03 #1024 — UNDER A FAMILY-FIRST ORDER, A PARTICLE CHAIN AFTER THE FAMILY IS CUT WHERE P1'S FAMILY-FIRST FOLD CUTS ITS RUN. rules/O6's Except said a particle chain (P2) is never split, so under FAMILY_FIRST `Pérez de la Vega Juan` reached O6 as one unit, `de la Vega Juan`, which counts two name words, and O6 declined: family 'Pérez', given 'de la Vega Juan', under both family-first orders, the switch on or off. The caller has declared two things the chain contradicts, an order putting the given name after the family (decisions/P1's 2026-08-17 #395 entry reads a declared family-first order as evidence of where the surname ends) and a family of two names; and a chain running on through `Juan` makes the given name part of the surname P2 says a particle is written as part of, which is `Vega`. So where the first unit after the family begins with a particle and runs past the one name word its particles attach to, O6 cuts it as `_fold_reach` cuts P1's run, takes the particles and that word as the second family name, and lays out what is left by the order: `Pérez de la Vega Juan Carlos` gives given 'Juan', middle 'Carlos' under FAMILY_FIRST and middle 'Juan', given 'Carlos' under FAMILY_FIRST_GIVEN_LAST, as `García Márquez Juan Carlos` reads. The rule's own example `Juan de la Vega Rodríguez` (family 'Juan' under FAMILY_FIRST, where it illustrated the decline) is this shape, and now reads family 'Juan de la Vega', given 'Rodríguez'; the decline's example is `Pérez de la Vega`, where taking the chain would leave no given name. Unchanged: the switch off (whether the chain's reach should follow a family-first order by itself, which would move `Nguyen Van Thanh Long`, is #1024's open question 1, not decided here), the default order, a family comma, and a chain whose first unit is a conjunction join (`Pérez de la Vega y Santos Juan`, the unit `de la Vega y Santos` counting two words, so O6 still declines). With the switch on, a Vietnamese name with `Van` after the family now gives up a word to the family as any name of three or more words does under the switch (`Nguyen Van Thanh Long` → family 'Nguyen Van Thanh', given 'Long'), the switch declaring two family names. No ambiguity is reported, as for every reading of the switch. A lone particle a family-first order left as the family takes the cut chain after it as it takes any word (rules/O6's Accepted), so `Van der Merwe Jan` reads family 'Van der Merwe', surname2 'der Merwe', where it read family 'Van', given 'der Merwe Jan': the family field right, one surname read as two (the review; corpus examples `van den Berg Cornelia …`). Under the default order nothing is cut, held by test_cases' `two_family_names_cuts_no_chain_under_the_default_order`, whose recorded negative control without the order guard reads `du Ménil Thi Mai` family 'Thi Mai'; 29 corpus names move under that mutant. Measured 2026-11-03 by the issue's recipe on the cached `orcid-researchers` corpus (ES, MX, CO, PE, CL, VE, AR, at most 20,000 records a country with both boxes, 135,735 records): of the 1,577 whose family box holds a particle after its first word, the family box is read right under `Policy(name_order=FAMILY_FIRST, two_family_names=True)` in 2 on 39839b91 and in 1,273 on the branch, and of the 1,296 in the issue's exact shape (one word, a particle run, one word) in 1 and 1,272; by country, of the 1,577, ES 264/383, MX 164/221, CO 59/63, PE 152/199, CL 113/142, VE 406/433, AR 115/136. Recompute: the issue's recipe, run against each tree with `nameparser.__file__` asserted; the comparator is the base tree, so 2 and 1 are the same parser's reading before the change. - 2026-11-03 #959 (Derek) — A HYPHENATED WORD COUNTS AS ONE WHEN IT IS THE WORD TAKEN, UNLESS EVERY PART OF IT IS A LISTED GIVEN NAME OR AN INITIAL. SUPERSEDES, in part, the 2026-10-24 #754 entry's count of the word taken, in which a hyphen joined two words without making them one, so `Andrés Sánchez-Cid Torres` kept `Sánchez-Cid` a middle name (the #754 review's reading of `Ana María-José López`, middle 'María-José', which the first draft had read as family 'María-José López'). The issue found this shape behind 271 of the 466 Spanish records the given-name veto still read wrong. Measured 2026-11-03 on the cached `orcid-researchers` corpus, ES, MX, CO, PE, CL, VE and AR at most 20,000 records a country with both boxes (135,735 records), over the 10,794 distinct records holding a hyphen in either box, rendered given first and read right where both the family and the given names match the boxes (`local_agreement.fold`), under `parser_for(locales.ES)`, on 39839b91 against prototypes of three readings: a hyphen counting as one everywhere O6 counts (family and word taken) reads 10,038 right against 10,281, fixing 277 and breaking 520, most of them a family box of two surnames written with a hyphen (`Garcia-Gomez`), which then counts as one and takes a given name; a hyphen counting as one in the word taken alone reads 10,464, fixing 186 (ES 125, MX 24, CL 20, PE 6, CO 5, VE 5, AR 1) and breaking 3 (`Aristide Tanyi-Jong Akem`, `Hilda Ines-Romero Menacho`, `Benjamin Moraga-Arias .`); and the same with a veto on a hyphenated word every part of which is a listed given name, which moves none of these records. Decided: the word-taken count with the veto, the veto keeping the rule's own `Ana María-José López` where the Spanish list is loaded (middle 'María-José'), while without a list the switch takes it (family 'María-José López'), its usual cost (rules/O6's Accepted). Found building it and in its review, and folded into the veto, which reads each part: a part that is an initial counts as a listed given name does, so hyphenated initials (`Ana M.-J. López`) stay as an initial written alone stays, and `M.-José` stays under the list, which moves no ORCID record. A hyphenated bound given-name pair (`Ana Abdul-Rahman López`, middle 'Abdul-Rahman' in 2.8.0) is taken like any hyphenated word, family 'Abdul-Rahman López', where the pair written as two words (P5) stays: a veto on a hyphenated word whose first part is a bound given name was built and backed out the same day, since in this population it cost 2 records whose hyphenated word is a surname (`Paola Valentina Abdul-Hadi Pru`, `Giorgia Anette Abd-El-Kader Espinoza`) and kept none, the hyphen being how those surnames are written; rules/O6's Accepted records the cost. A hyphen in the family still counts its words, so `Javier Olmedo Pérez-Montaut` keeps `Olmedo` a middle name: reading `Pérez-Montaut` as one surname is the 520 above. The built change reads the same 10,464 and moves the same 186 and 3, and with the switch alone (`Policy(two_family_names=True)`, no list) 10,292 to 10,475, the same records. Recompute: for each record of `local_agreement.records(local_corpora.load("orcid-researchers"))` in those countries, both boxes non-empty, at most 20,000 a country, keep those with a hyphen (`_post_rules._HYPHENS`) in either box, distinct by given box, family box and country (two records are duplicates), parse `f"{given} {family}"` with each tree's parser, `nameparser.__file__` asserted, and compare. -- 2026-11-03 #959 — BOTH SWITCHES OVER A GIVEN NAME A PARTICLE COMPLETES GET AN EXAMPLE. The 2026-10-30 entry's Open named the shape no rule example showed: `María del Carmen de la Vega` reads family 'del Carmen de la Vega' with both switches on, O6 taking the chain before O7 can complete the given name with it, and given 'María del Carmen' under O7 alone. rules/O7's precedence bullet against two-family-name handling now says so and carries both rows; no behavior moves. +- 2026-11-03 #959 — (Superseded for a word written `after_particle` by the 2026-11-03 #704 entry: `María del Carmen de la Vega` reads given 'María del Carmen' with both switches on.) BOTH SWITCHES OVER A GIVEN NAME A PARTICLE COMPLETES GET AN EXAMPLE. The 2026-10-30 entry's Open named the shape no rule example showed: `María del Carmen de la Vega` reads family 'del Carmen de la Vega' with both switches on, O6 taking the chain before O7 can complete the given name with it, and given 'María del Carmen' under O7 alone. rules/O7's precedence bullet against two-family-name handling now says so and carries both rows; no behavior moves. +- 2026-11-03 #704, decided in chat — A GIVEN NAME A PARTICLE COMPLETES, AND A NAME A COMMUNITY GIVES EVERY MEMBER, ARE NO FAMILY NAME. SUPERSEDES, for a marked word, the 2026-11-03 #959 entry's reading of `María del Carmen de la Vega` with both switches on: this rule now leaves `del Carmen` where it stands, so the name reads middle `del Carmen` with this switch alone and given `María del Carmen` with both (decisions/O7's entry of this date holds the property, the word list and the measurements). The second clause is the Kaur and Singh decline that entry records: `Harpreet Kaur Dhillon` read family `Kaur Dhillon` under this switch and reads middle `Kaur`, family `Dhillon`, as by default; a shared word as the family declines too, so `Harpreet Simran Kaur` keeps family `Kaur`. Like the given-name list (the 2026-10-30 #959 entry), each reads the vocabulary, the signal this rule already reads beside its count, so they stay clauses of this rule; the shared-name clause reads it at a second site, the family's own words as well as the word taken, recorded on the rule's Signal line. The given-name clause reads only the shape rules/O7 takes, given-first beside a given name of one word: the review of #1057 found the first draft declining under a family-first order (`García del Valle María` lost family `García del Valle`) and behind a middle (`Juan Antonio del Valle de la Vega` under `locales.ES` lost family `del Valle de la Vega`), where no given name is completed. diff --git a/docs/design/decisions/O7-two-given-names.md b/docs/design/decisions/O7-two-given-names.md index daa84f06..cfab4c6f 100644 --- a/docs/design/decisions/O7-two-given-names.md +++ b/docs/design/decisions/O7-two-given-names.md @@ -5,7 +5,10 @@ - A compound given name is an opt-in `Policy` switch, `two_given_names`, off by default, which a pack's `[rules]` may turn on (`two_given_names = true`): given-first it is the same string shape as a name with a middle name, and `middle` keeps its meaning — 2026-10-30 #704. - The given name takes ONE unit, the middle unit beside it (after it, or before it under FAMILY_FIRST_GIVEN_LAST with no family comma, the part after one reading given-first), counted as rules/O6 counts a name word, so a particle chain the parse left in the middle moves whole; a given name already holding two words keeps what it has — 2026-10-30 #704. - The switch reads after `two_family_names` and before `middle_as_family`, and is refused beside neither — 2026-10-30 #704. -- A particle chain running to the end of the name stays the family under the switch (`Maria do Carmo Silva`); reading it as a given name is not this rule's — 2026-10-30 #704. +- A particle chain running to the end of the name stays the family under the switch (`Maria do Carmo Silva`) unless the word behind its particles carries a `given` reading written `after_particle`; then, the given name standing alone before the chain and a name word left for the family, the particles and that word join the given name (`María del Carmen García`). The property is on the word, not a list the rule holds; the shipped words are the Spanish ones ORCID holders write behind a particle in the given box at least nine times in ten, with ten records or more — 2026-11-03 #704, decided in chat (option B). +- The two vocabulary clauses stay clauses of this rule though they read a signal beside the caller's switch, the word looked up where it stands, as O6's given-name veto does (decisions/O6, 2026-10-30 #959): the rule is a split candidate, its When widened to the particle phrase that opens the family — 2026-11-03 #704, for review. +- A name a community gives every member, a name reading written `shared` (Sikh `kaur`, `singh`), is neither taken nor completed, on either side of the join, as a patronymic is not — 2026-11-03 #704, decided in chat. +- An English given name written beside a Chinese one (`Lim Mei Ling Jessica`) is not read: it is the same shape as a middle name, and the caller who writes it in delimiters with the declared order gets it as the nickname (`Lim Mei Ling (Jessica)` under `FAMILY_FIRST` with the switch reads given `Mei Ling`, family `Lim`, nickname `Jessica`) — 2026-11-03 #704, recorded as accepted. - The switch reports nothing — 2026-10-30 #704. - No shipped pack turns the switch on; `locales.ES` and `locales.PT` leave it opt-in, decided in chat — 2026-10-30 #753, phase 3g. - With O6's switch on, a listed given name O6 leaves where it is (decisions/O6's 2026-10-30 #959 entry) is a middle this rule takes where it stands beside the given name, so `José Luis Pérez` reads given `José Luis` with both on — 2026-10-30 #959. @@ -19,7 +22,7 @@ - **A patronymic is no given name, on either side of the join.** A patronymic particle's run is neither taken nor completed, read off the `vocab:patronymic` tag classify left (rules/R9's vocabulary arm): `Ahmad bin Ali bin Hassan` keeps middle `bin Ali`, which the first draft moved into the given name while the patronymic view still claimed it (found by the #954 review), and `Ahmad, bin Ali`, `bin Ali Hassan` and, under FAMILY_FIRST_GIVEN_LAST, `Hassan Yusof bin Ali` keep their given name, which the second draft, testing only the unit taken, joined to the word beside it (found by the third review); so do, under an ending rule, a word the rule reads as a patronymic, a Turkic marker and the marker's father (`Ivan Ivanovich Ivanov` under the East Slavic rule keeps middle `Ivanovich`); with no rule on, a patronymic written without a particle is a name word and joins, as O6 accepts. The #954 review also found the first draft's two edge tests on the unit's position unreachable, over the committed corpora's 6,428 texts under every order, patronymic rule and O6 and O3 setting: the positions lay the middles out on the side of the given name the order says, so the unit beside it is the first middle unit or the last, and the tests were removed. - **Orders and commas.** A family comma lays the part after it out given-first, so the switch reads it (`Pérez, Juan Carlos`), an empty head (C15) included; no comma, the switch stands down where the name's script set its order (W4), as O6 does. Under FAMILY_FIRST_GIVEN_LAST the unit beside the given name is the one before it, so the two stay in written order. - **Precedence.** O6 reads first: turning both on says the family holds two names, and O6 takes its word before this rule takes from what is left. Measured in ORCID's records from Spain, Mexico, Colombia, Chile, Peru and Ecuador whose two boxes hold three words with no particle or hyphen, the family box holds two of them in 73% (Ecuador) to 93% (Peru) of the records (Spain 6,586 of 7,433), and where they hold four, two and two in 96% to 99.9% (Spain 2,484 of 2,582; 2026-10-30, the fetch of 2026-10-27), so O6-first reads `Gabriel García Márquez` and `María José García López` as those writers split them. Argentina (2+1 in 7,184 of 8,457 three-word records), the United States and the United Kingdom go the other way, which is why neither switch is a default. `middle_as_family` (O3) reads after, as P5's join reads before it: the given name is decided first, and the fold takes the middles left. - - **The particle chain ending the name is out of scope.** The RFC's measured gap for `pt` is `Maria do Carmo Silva` (given `Maria`, family `do Carmo Silva`), where P2 chains every word after the particle into the family. Splitting that chain needs to know which particle phrases complete a given name, and the shape is `João da Silva Santos`'s, whose family it is. Measured in ORCID's records from Spain, Portugal, Brazil and 19 Latin American countries with a family box filled (2026-10-30): 5,506 given boxes hold a particle inside, against 5,625 family boxes that open with one; the given boxes' objects are mostly Marian titles (`Carmen` 1,283, `Jesús` 454 and `Jesus` 245, `los Ángeles` 564 in two spellings, `Pilar` 324, `Rosario` 260, `la Caridad` 230), several of which open family boxes too (`Jesus` 65, `Rosario` 35, `Valle` 47, `Conceição` 24, against 19 in given boxes). So the split is a question of vocabulary, which word completes a given name after which particle, and is left Open below rather than guessed at by shape. + - **The particle chain ending the name is out of scope.** (Superseded for a word written `after_particle` by the 2026-11-03 #704 entry.) The RFC's measured gap for `pt` is `Maria do Carmo Silva` (given `Maria`, family `do Carmo Silva`), where P2 chains every word after the particle into the family. Splitting that chain needs to know which particle phrases complete a given name, and the shape is `João da Silva Santos`'s, whose family it is. Measured in ORCID's records from Spain, Portugal, Brazil and 19 Latin American countries with a family box filled (2026-10-30): 5,506 given boxes hold a particle inside, against 5,625 family boxes that open with one; the given boxes' objects are mostly Marian titles (`Carmen` 1,283, `Jesús` 454 and `Jesus` 245, `los Ángeles` 564 in two spellings, `Pilar` 324, `Rosario` 260, `la Caridad` 230), several of which open family boxes too (`Jesus` 65, `Rosario` 35, `Valle` 47, `Conceição` 24, against 19 in given boxes). So the split is a question of vocabulary, which word completes a given name after which particle, and is left Open below rather than guessed at by shape. - **No report.** The caller declared the convention, as for O6. Measured 2026-10-30 on the branch, the default unchanged (the snapshot gains the rule's and case table's example texts and moves no reading; every agreement record matches): - **PSC** (`psc-individuals`, whose form has a middle-name box; every box agreeing as `local_agreement.py` scores it, records scored by nationality, a set-aside title box excluded): @@ -38,6 +41,14 @@ - **CLDR** (`cldr_fields.jsonl`'s given-first renderings with a stated given name, given agreeing): over all 880, default 636, O7 568, O6 636, O6+O7 746; over `es`'s 8, 6 → 8 under O6+O7; over `pt`'s 8, 7 under O6+O7 and 5 under O7 alone. - **TSE** (`snapshot.py --local tse-donors --base`, Brazilian donors' names as filed): 689,057 of 854,531 entries read differently under O7, 684,625 of them in given and middle alone and 310 in an ambiguity's alternative too, and with O6 on in both trees 270,169 (268,478 in given and middle alone), measured on the final commit; the corpus states no fields, so it counts reach, not agreement. Recompute: the ORCID counts read `raw.jsonl` of the 2026-10-27 fetch, records with a family box, the three- and four-word split over records whose boxes hold no hyphen and no word of `Lexicon.default().particles` (conjunctions not excluded), by the record's `country`; the particle-inside count over BR, PT, ES, MX, AR, CO, CL, PE, VE, EC, UY, BO, PY, CR, GT, CU, DO, HN, SV, NI, PA and PR, a given box whose word at neither end is in `particles`, its object the next word (the next two after a second particle), against family boxes of two or more words opening with one. The PSC table is `local_agreement.score(rows, FIELDED[PSC], Parser(policy=...), jobs=1)` in one process, `rows` the records whose `group_of` is the filers' nationality, each table cell a group's `agreed` over its `scored`: every box compared case-folded, the title too, a parse's suffix appended to the family before the surname box is compared, and a record whose title box holds an unlisted word that disagrees set aside and not scored. An earlier draft of this table compared the raw boxes without that read and gave a few tenths of a point less. The CLDR tables score each rendering's `given` against `parse()` under each `Policy` in one process; the local corpora's runs are `--tree` a checkout whose `two_given_names` (and for O6+O7 `two_family_names`) default is flipped, `--base` one at master `40675146` flipped the same way for O6. - - **Open:** which particle phrases complete a given name, and whether that is a reading on the word or a list the rule kind holds (the particle-chain bullet above), #704, and whether this rule reads O6's given-name list for it (`María del Carmen` against `João da Silva`), #959; deriving `DEVIATES` from the rules (decisions/vocabulary-model's 2026-10-29 4f entry, item 6), which this rule leaves as it found it, having no surface in the name to declare, #754. + - **Open:** which particle phrases complete a given name, and whether that is a reading on the word or a list the rule kind holds (the particle-chain bullet above), #704 (decided 2026-11-03, below), and whether this rule reads O6's given-name list for it (`María del Carmen` against `João da Silva`), #959 (declined 2026-11-03, below); deriving `DEVIATES` from the rules (decisions/vocabulary-model's 2026-10-29 4f entry, item 6), which this rule leaves as it found it, having no surface in the name to declare, #754. - 2026-10-30 #959 — THE WORD O6'S GIVEN-NAME LIST LEAVES IS THIS RULE'S. rules/O6 now leaves a listed given name where it stands (decisions/O6's entry of this date), so with both switches on the middle it leaves is this rule's to take where it is the unit beside the given name: `José Luis Pérez` reads given `José Luis`, family `Pérez`, where with the list empty O6 takes `Luis` first and the name keeps one given name. Where the given part holds two units before it, this rule takes the first and the listed word stays a middle (`Ana María Luis Pérez`: given `Ana María`, middle `Luis`), so a name of four words reads two and two only where O6 took its third (found by the #959 design-docs review). Nothing in this rule moved; its precedence clause states the reading with an example. Whether this rule reads the list itself is Open above. - 2026-10-30 #753, phase 3g, decided in chat — NEITHER `es` NOR `pt` TURNS THIS SWITCH ON. ORCID cannot score it: its given box holds the middle names too, so the switch moves no record's agreement (0 fixed, 0 broken in every Spanish- and Portuguese-speaking country measured). PSC, whose form has a middle-name box, scores it against the filer's own split, and filers put the second given name in that box: under a `es` choice the switch alone takes Spanish filers from 31.8% to 15.7% (1 fixed, 980 broken), and beside `locales.ES`'s `two_family_names` from 82.0% to 54.7%; Portuguese filers fall from 61.1% to 28.5%. CLDR's `es` renderings read it right (the 2026-10-30 #704 entry, 6 → 8 of 8). The two sources disagree about a convention, not about a misreading, and the pack follows the boxes callers fill: a caller whose data writes the compound given name whole turns the switch on beside the pack. Recompute: decisions/O6's phase 3g recipe with `PolicyPatch(two_given_names=True)` added. +- 2026-11-03 #704, decided in chat — A PARTICLE PHRASE COMPLETES THE GIVEN NAME WHERE THE VOCABULARY MARKS ITS WORD, AND KAUR AND SINGH ARE NO GIVEN NAME. Settles the 2026-10-30 entry's Open. Two name-reading properties, each read by this rule and rules/O6 and by no other, so a parse with both switches off moves nowhere (the snapshot gains the rule examples' new texts and moves no reading; a grid of 128 texts under every order and switch pairing, token readings and `Reading.languages` included, moved no default-policy parse against master `a1b3e186`). + - **The shape: a property on the word behind the particle.** The particle cannot carry it (`del` opens `del Carmen` and `del Río` alike), and the question is a vocabulary one (the 2026-10-30 entry's particle-chain bullet). Weighed: (A) a phrase reading (`text = "del carmen"`), most precise, telling Spanish `de la Luz` from Portuguese `da Luz`, but a third kind of phrase reading beside a birth marker and a given-addressing title; (C) a list the rule holds as a `Policy` field or `[rules]` key, vocabulary in `Policy` that no choice of languages or strict parser reaches; (D) O6's given-name list (`Lexicon.given_names`), no new data, but it would split `de la Cruz` (175 family boxes in ORCID against 38 given), `de León` (151 against 5) and `de la Rosa`, and is empty by default and has no Portuguese list. B was chosen: `after_particle = true` on a `given` reading, one word, compiled into the internal `PARTICLE_GIVEN_NAMES`. Its cost against A is a word whose phrase reads both ways by particle, which is left out (`luz`). + - **The reach.** Given-first, no family comma (a comma, and both family-first orders, already read `Silva, Maria do Carmo` given `Maria do Carmo` through the middle path), the given name one word with no middle between it and the chain, the chain opening with particles and then the marked word, and a name word left after them for the family, one with a letter that is no connective standing alone (a family of particles alone never reaches the path: assign leaves a trailing particle the family by itself, the phrase a middle, found by the second #1057 review), and no Turkic marker behind the phrase under that rule, as for the middle path (both found by the #1057 review: `María del Carmen y` and `Ali del Carmen oglu` had moved): `María del Carmen` alone keeps family `del Carmen`, as `Juan Carlos` keeps family `Carlos`. A given name of two words (`María-José del Carmen García`) keeps what it has, and a middle between leaves the chain to the family, this rule taking that middle instead (`Ana María del Carmen García` reads given `Ana María`). With rules/O6 on too, O6 leaves the phrase where it stands (decisions/O6, this date), so `María del Carmen de la Vega` reads given `María del Carmen` with both on, where the 2026-11-03 #959 entry in decisions/O6 recorded family `del Carmen de la Vega`; and the family the split leaves keeps its chained words as one unit, so `María del Carmen García López` gives no `surname2` (accepted in the rule). + - **The words** (es.toml; each records its counts): `carmen`, `ángeles` and `angeles`, `pilar`, `rosario`, `valle`, `caridad`, `socorro`, `mar`, `rocío` and `rocio`, `lourdes`, `mercedes`, `maría` and `maria`, `milagro`, `milagros`, `cisne`, `fátima`, `concepción`, `nieves`, `consuelo`, `jesús` — 23. Measured 2026-11-03 on ORCID's `raw.jsonl` of the 2026-10-27 fetch (20,000 records a country at most): for each word, the records whose given box ends with a particle run (`Lexicon.default()`'s particles, NFC lower case) and the word, against those whose family box opens with a particle run and the word and holds more words after them, the shape that competes; kept where the given side is at least 90% with ten given records or more, counted over Spain and Spanish-speaking America (ES MX AR CO CL PE VE EC UY BO PY CR GT CU DO HN SV NI PA PR AD) and, separately, Portugal, Brazil and Portuguese-speaking Africa (BR PT AO MZ CV GW ST TL). `carmen` 1,261 to 5, `valle` 235 to 26 (exactly 90.0%), `rosario` 260 to 27. Left out: no Portuguese-speaking word reaches the bar (`carmo` 21 to 6, `rosário` 21 to 8, `conceição` 25 to 25, `fátima` 32 to 4, which the Spanish counts carry, 27 to 1); unaccented `jesus`, 203 to 16 in Spanish-speaking countries but 67 to 41 in Portuguese-speaking ones; `luz` (12 to 6, and Portuguese `da Luz` 4 to 5), `dios`, `paz`, `cruz`; and two that pass by count but are no compound given name, `c.` (an initial) and `piero` (Italian `del Piero`, 10 to 0). The Catalan phrases (`dels Àngels`, `de les Neus`, `de la Mercè`) barely occur, so no Catalan word is marked. The PSC register agrees where it can, its filers putting `del Carmen` in the middle-name box 21 times and the surname box 2, `del Mar` 11 and 0, `del Pilar` 10 and 1. `dels` stays out of `ca.toml`: the property moves nothing with the switch off, so a `dels` particle would still move `Maria dels Àngels Puig` from middle `dels Àngels` to family `dels Àngels Puig` by default (the #704 comment of 2026-10-31). + - **Kaur and Singh.** Sikh women are given `Kaur` and men `Singh`, written after the personal name and before a family name where there is one, and often as the only surname. A name reading written `shared = true` (mul.toml, `family`, so that a strict choice of any language keeps it: in pa.toml, the review of #1057 found `parser_for(locales.ES, strict=True)` reading family `Kaur Dhillon`) compiles into the internal `SHARED_NAMES`, and this rule and rules/O6 decline a join that takes or completes one. Measured 2026-11-03: the PSC register's filers put `Kaur` in the middle-name box 2,354 times (2,264 as its only word), as the whole surname 674 (729 surname boxes in all) and in the forename box 25, and `Singh` 5,303 (5,015), 1,885 (2,092) and 48; ORCID holders whose family box holds another surname put `Kaur` in the given box 69 times (16 of them with `Singh` there) and open the family box with it 3. So the default's middle `Kaur` is what filers write, and both switches moved it, O7 into the given name and O6 into the family (`Kaur Dhillon`, found while measuring, not stated in the request). Weighed: (2) a view of its own, which nobody has asked for; (3) leaving both alone and accepting them under the middle-name bullet of rules/O7's Accepted. An agnomen-style suffix reading does not reach the word, which stands before the surname where the trailing peel never looks. The cost: `Singh` is an ordinary Rajput and Nepali surname too (301 of the ORCID records holding `Kaur` or `Singh` are Nepal's, 671 India's), so `Ram Bahadur Singh Thakuri` under O6 keeps middle `Singh` where it read family `Singh Thakuri`. + - **A bound pair whose bound word is a particle.** O6's count passes over a particle, so `Abu Bakr` (`abu` a particle and a bound word both) counted as one word, and the switch read `Abu Bakr Carlos García` given `Abu Bakr Carlos` from #954 on, against this rule's own Except (a bound given-name pair keeps what it has); the particle phrase would have repeated it (`Abu Bakr del Carmen García`). The given name's count reads a bound pair as two words now, `abdul Rahman` having always counted so (found by the #1057 test review, which mutated each guard). + - **Not decided here.** Whether the default, with both switches off, should read a marked phrase as a middle name, as PSC filers write it (`María del Carmen García` reads family `del Carmen García`): it moves the default, so it is left for its own decision. + - Recompute: the scripts' recipe above, over `~/.cache/nameparser-tools/local/orcid-researchers/raw.jsonl` and `psc-individuals/raw.jsonl`; the default-policy grid is 16 names in eight comma and order shapes under every `name_order` and switch pairing, compared field by field, token readings included, against a `git worktree add --detach` checkout of `a1b3e186`, each side's `nameparser.__file__` asserted. diff --git a/docs/design/mechanisms.md b/docs/design/mechanisms.md index 37206e5f..def49c3a 100644 --- a/docs/design/mechanisms.md +++ b/docs/design/mechanisms.md @@ -61,7 +61,7 @@ This catalog converts discovery into a one-time cost — it does not remove disc **Problem shape.** A rule counts "one name word", but the input holds words that another rule has already joined into one name — and the joining structure it would read has been merged away. -**Contract statement.** Three rules build multi-word units: a particle chain (P2), a conjunction join (P3), and a bound given-name pair (P5). A rule that counts name words counts those units, and takes each whole or not at all — with one stated exception, rules/C3's count of the words before a comma, which counts particle surnames alone (decisions/C3, #575). +**Contract statement.** Three rules build multi-word units: a particle chain (P2), a conjunction join (P3), and a bound given-name pair (P5). A rule that counts name words counts those units, and takes each whole or not at all — with stated exceptions: rules/C3's count of the words before a comma, which counts particle surnames alone (decisions/C3, #575), and three cuts of a particle chain at the run and the one word it attaches to, P1's family-first fold, rules/O6's family-first cut (#1024) and rules/O7's particle phrase, which takes that much of the family's chain only where the vocabulary marks the word (#704). **How it works.** group builds each join as a piece, but its own prefix chain then merges the joined piece into a longer one, so PIECES no longer carries the boundary — the units are rebuilt from the tags the vocabulary layer left (`particle`, `conjunction`, `vocab:bound-given`). The rebuild is RECURSIVE: what a conjunction or a bound word joins is the next UNIT, not the next word, and absorbing a single token instead strands a particle severed from the words it chains. Note the two joins arrive here for opposite reasons — the conjunction join was built and then swallowed, while the bound-given join was never built at all (P5 joins only at the first non-title piece), so restoring piece boundaries in group would fix the first and silently split the second. @@ -320,7 +320,7 @@ a fork for all ambiguous particles on "Dr. Van Jr.". **Problem shape.** A convention says a part of the name holds two units, a family of two surnames or a given name of two, but written in its order the name is the same string shape as a name whose extra word belongs to the part beside it, so no reading of the words can tell them apart. -**Contract statement.** A part that a convention spans over two units is a caller switch, a `Policy` field off by default that a pack's `[rules]` may turn on, read after the positional assignment: the part takes the ONE unit standing beside it in the part next to it, counted as UNIT-PARTITION counts one, and declines where the part already holds two words, where the unit is not one name word that part could hold, or, outside a family comma, where the order the name was read in is not the one the caller declared; after a family comma the family's switch declines, the comma having decided the family, and the given name's reads the part after it given-first. +**Contract statement.** A part that a convention spans over two units is a caller switch, a `Policy` field off by default that a pack's `[rules]` may turn on, read after the positional assignment: the part takes the ONE unit standing beside it in the part next to it, counted as UNIT-PARTITION counts one (the given name's switch may also take the particle phrase opening the family where the vocabulary marks its word, one of UNIT-PARTITION's stated cuts), and declines where the part already holds two words, where the unit is not one name word that part could hold, or, outside a family comma, where the order the name was read in is not the one the caller declared; after a family comma the family's switch declines, the comma having decided the family, and the given name's reads the part after it given-first. **How it works.** Each switch is a post rule that retags in place and never reorders: it reads the roles assign settled, takes the neighbouring part's units as `_units` rebuilds them from the tags, and moves the unit at the shared edge, which in text order already renders where it stands, so no view needs a tag to put it back (a nickname written inside the given part is the one exception, rendered after the joined name: `John "Jack" Fitzgerald Kennedy` under O7 renders `John Fitzgerald "Jack" Kennedy`, every word kept, accepted); only the family's switch under a family-first order, where the unit it takes is the given name's and leaves a remainder to lay out again, retags the rest by `_name_positions`. The two switches read in a fixed order, the family's first (rules/O6) and then the given name's (rules/O7), and compound-family handling (rules/O3) folds what both leave, so turning on more than one reads as each states its precedence and none needs a guard against another. A switch reports nothing, the caller having declared the convention, and a pack turns it on as data: one row in `tools/vocab/build.py`'s `RULE_KEYS` and in `_locale._RULE_VALUES`, its default refused there, so a pack can only turn it on (decisions/vocabulary-model, 2026-10-29 4f; decisions/O7, #704). diff --git a/docs/design/rules/O6-two-family-names.md b/docs/design/rules/O6-two-family-names.md index 0ae462d3..6040bc8d 100644 --- a/docs/design/rules/O6-two-family-names.md +++ b/docs/design/rules/O6-two-family-names.md @@ -70,7 +70,7 @@ ``` ", Ana García López" two_family_names → family="" ``` -- **What is taken must be one name word a family name could be**: a bound given-name pair (P5) written as two words stays where it is, and so do an initial written alone, which writes a given name, and a mark or number with no letter in it. A particle chain (P2) is split only where the family-first bullet cuts it and otherwise moves whole, and where taking it would leave no given name the rule declines. A listed given name stays where it is too, under either order, and so does a hyphenated word every part of which is one or an initial, and a list written without accents matches the word written with them: the caller's vocabulary names it a word only a given name is, so the name holds one family name. A listed word behind a particle is a surname unit, taken as before. +- **What is taken must be one name word a family name could be**: a bound given-name pair (P5) written as two words stays where it is, and so do an initial written alone, which writes a given name, and a mark or number with no letter in it. A particle chain (P2) is split only where the family-first bullet cuts it and otherwise moves whole, and where taking it would leave no given name the rule declines. A listed given name stays where it is too, under either order, and so does a hyphenated word every part of which is one or an initial, and a list written without accents matches the word written with them: the caller's vocabulary names it a word only a given name is, so the name holds one family name. A listed word behind a particle is a surname unit, taken as before, unless the vocabulary marks it a given name written there (below). ``` "José Luis Pérez" two_family_names+es_given_names → family="Pérez" "Jose Luis Perez" two_family_names+es_given_names → middle="Luis" @@ -86,6 +86,18 @@ "Pérez de la Vega" family-first+two_family_names → family="Pérez" "Pérez de la Vega y Santos Juan" family-first+two_family_names → given="de la Vega y Santos Juan" ``` +- **A given name a particle completes stays where it is**: given-first, a particle run and the word behind it that the vocabulary names a given name written that way (O7), standing beside a given name of one word, are the given name's second, not a surname. Under a family-first order the unit after the family completes no given name, and is taken as before. + ``` + "María del Carmen de la Vega" two_family_names → middle="del Carmen" + "Ana de Miguel de la Vega" two_family_names → family="de Miguel de la Vega" · boundary + "Juan Antonio del Valle de la Vega" two_family_names → family="del Valle de la Vega" · boundary + "García del Valle María" family-first+two_family_names → family="García del Valle" · boundary + ``` +- **A name a community gives every member is no family name of two**, on either side of the join: Sikh Kaur and Singh, written after the personal name and before a family name or as the only one, are neither taken nor joined. + ``` + "Harpreet Kaur Dhillon" two_family_names → family="Dhillon" + "Harpreet Simran Kaur" two_family_names → family="Kaur" + ``` - **An order a name's script set is not a declared one** (W4), and no script it keys writes two family names as separate words, so a name its script reads in an order other than the declared `name_order` is left alone. ``` "王 小 明" two_family_names → family="王" @@ -146,5 +158,5 @@ - The rule reports nothing: the caller declared the convention, as a declared order is followed without a report, and a report on most names of three words or more would teach the caller to ignore the field. --- -**Signal:** none in the name + position: a caller switch, the family's words and the taken word's shape read as written to see whether the family already holds two; and the word standing where the family would take one, looked up in the caller's given-name list, a vocabulary signal tagged by where the word stands as P1's is +**Signal:** none in the name + position: a caller switch, the family's words and the taken word's shape read as written to see whether the family already holds two; and the word standing where the family would take one, looked up in the caller's given-name list, a vocabulary signal tagged by where the word stands as P1's is, as are a given name written behind a particle and a name a community gives every member, which the family's own words are looked up for too **history:** decisions/O6 · **implemented:** nameparser/_pipeline/_post_rules.py diff --git a/docs/design/rules/O7-two-given-names.md b/docs/design/rules/O7-two-given-names.md index 5eb2aa3c..f5031a86 100644 --- a/docs/design/rules/O7-two-given-names.md +++ b/docs/design/rules/O7-two-given-names.md @@ -1,8 +1,8 @@ # O7 — two given names -**When:** two-given-name handling is active and the given name stands beside a middle name, as in "Juan Carlos Pérez". +**When:** two-given-name handling is active and the given name stands beside a middle name, as in "Juan Carlos Pérez", or alone before a family that a particle and a given name written behind it open, as in "María del Carmen García". -**Then:** the given name takes the middle unit beside it, so the given name is two name units and any middles past them stay middles. +**Then:** the given name takes the middle unit beside it, or that particle phrase, so the given name is two name units and any middles past them stay middles. - **Under the default given-first order, and after a family comma**, the middle written right after the given name joins it. ``` @@ -22,6 +22,14 @@ "María del Carmen de la Vega" two_given_names → given="María del Carmen" "Maria do Carmo da Silva" two_given_names → family="da Silva" ``` +- **A particle run and the given name written behind it complete the given name**, though the particle chain runs on into the family, where the given name stands alone before the chain and a name word, no lone connective, is left for the family: the word behind the particles is one the vocabulary names a given name written that way, as in María del Carmen or María de los Ángeles. + ``` + "María del Carmen García" two_given_names → given="María del Carmen" + "María del Carmen García" two_given_names → family="García" + "María de los Ángeles López Pérez" two_given_names → family="López Pérez" + "María del Carmen" two_given_names → family="del Carmen" · boundary + "Maria da Silva Santos" two_given_names → family="da Silva Santos" · boundary + ``` - **An initial is taken**, since it writes a given name short. ``` "J. Carlos Pérez" two_given_names → given="J. Carlos" @@ -47,6 +55,7 @@ ``` "María-José Ana López" two_given_names → given="María-José" "abdul Rahman Ali Khan" two_given_names → middle="Ali" + "Abu Bakr Carlos García" two_given_names → middle="Carlos" ``` - **What is taken must be one name word**: a part counting as two words stays where it is, and so does a mark or number with no letter in it; nor does a given name with no letter in it take anything, being no name to complete. ``` @@ -61,6 +70,11 @@ "Ivan Ivanovich Ivanov" east-slavic+two_given_names → given="Ivan" "Ivan Ivanovich Ivanov" east-slavic+two_given_names → middle="Ivanovich" ``` +- **A name a community gives every member is no given name**, on either side of the join: Sikh Kaur and Singh, written after the personal name and before a family name, stay in the middle. + ``` + "Harpreet Kaur Dhillon" two_given_names → middle="Kaur" + "Gurpreet Singh Sandhu" two_given_names → given="Gurpreet" + ``` - **An order a name's script set is not a declared one** (W4), so a name its script reads in an order other than the declared `name_order` is left alone. ``` "王 小 明" two_given_names → given="小" @@ -68,7 +82,7 @@ **Precedence:** -- **Against two-family-name handling** ("Gabriel García Márquez" with both switches on): that rule reads first, since the caller who turns both on says the family holds two names, and this rule takes from the middles it leaves. A name of three words then keeps one given name, and a name of four whose third word that rule takes reads two and two. A listed given name that rule leaves where it is (O6) is a middle, which this rule takes where it stands beside the given name, so a name of three words whose middle is listed reads two given names and one family name. A given name a particle completes is a particle chain that rule takes into the family first, so it is the family's with both on, where this rule alone reads it as the given name. +- **Against two-family-name handling** ("Gabriel García Márquez" with both switches on): that rule reads first, since the caller who turns both on says the family holds two names, and this rule takes from the middles it leaves. A name of three words then keeps one given name, and a name of four whose third word that rule takes reads two and two. A listed given name that rule leaves where it is (O6) is a middle, which this rule takes where it stands beside the given name, so a name of three words whose middle is listed reads two given names and one family name. A given name a particle and a word the vocabulary marks complete, standing beside a given name of one word, is one that rule leaves where it stands (O6), so this rule takes it with both on, and the family it leaves is read as it is written; any other particle chain left in the middle that rule takes into the family first. ``` "Gabriel García Márquez" two_family_names+two_given_names → given="Gabriel" "José Luis Pérez" two_family_names+two_given_names+es_given_names → given="José Luis" @@ -76,8 +90,10 @@ "Ana María Luis Pérez" two_family_names+two_given_names+es_given_names → middle="Luis" "María José García López" two_family_names+two_given_names → given="María José" "María José García López" two_family_names+two_given_names → family="García López" - "María del Carmen de la Vega" two_family_names+two_given_names → family="del Carmen de la Vega" - "María del Carmen de la Vega" two_given_names → given="María del Carmen" + "María del Carmen de la Vega" two_family_names+two_given_names → given="María del Carmen" + "María del Carmen García López" two_family_names+two_given_names → family="García López" + "Maria do Carmo da Silva" two_family_names+two_given_names → family="do Carmo da Silva" + "Juan Antonio del Carmen de la Vega" two_family_names+two_given_names → family="del Carmen de la Vega" ``` - **Against compound-family handling** ("Hassan Mohamad Ali" with both switches on): this rule reads first, as a bound given-name pair (P5) joins before the fold, and O3 then folds whatever middles are left. ``` @@ -96,10 +112,14 @@ ``` "Vincent van Gogh van Beethoven" two_given_names → given="Vincent van Gogh" ``` -- A particle chain running to the end of the name is the family (P2), so a given name a particle completes is read only where a second particle starts the family: in "Maria do Carmo Silva" the family keeps "do Carmo Silva", the same string shape as "João da Silva Santos", whose family it is. +- A particle chain running to the end of the name is the family (P2) unless the vocabulary names the word behind its particles, and it names only a word whose phrase ORCID holders write as a given name at least nine times in ten: in "Maria do Carmo Silva" the family keeps "do Carmo Silva", "do Carmo" being a surname too often to name, the same string shape as "João da Silva Santos", whose family it is. ``` "Maria do Carmo Silva" two_given_names → family="do Carmo Silva" ``` +- The family a particle phrase leaves behind keeps the words it was written with, so with both switches on it holds two words as one unit and gives no second-surname view. + ``` + "María del Carmen García López" two_family_names+two_given_names → surname2="" + ``` - Where the declared order and a name's script agree, nothing tells the two apart, so under a declared family-first order a spaced Han or Hangul name is read like any other, as O6 reads it. ``` "王 小 明" family-first+two_given_names → given="小 明" @@ -111,5 +131,5 @@ - The rule reports nothing: the caller declared the convention, as O6's switch reports nothing. --- -**Signal:** none in the name: a caller switch, the given name's words and the taken unit's shape read as written +**Signal:** none in the name + position: a caller switch, the given name's words and the taken unit's shape read as written; and the word behind a particle, looked up as a given name written there, and a name a community gives every member, vocabulary signals tagged by where the word stands, as O6's given-name list is **history:** decisions/O7 · **implemented:** nameparser/_pipeline/_post_rules.py diff --git a/docs/design/rules/P2-particles-join-forward.md b/docs/design/rules/P2-particles-join-forward.md index 8c7ba861..89629358 100644 --- a/docs/design/rules/P2-particles-join-forward.md +++ b/docs/design/rules/P2-particles-join-forward.md @@ -34,6 +34,10 @@ ``` "Pérez de la Vega Juan" family-first+two_family_names → given="Juan" ``` +- **Where two-given-name handling completes the given name with the particle run and a word the vocabulary marks, it takes that part of the final group** (O7), leaving the rest of the chain the family. + ``` + "María del Carmen García" two_given_names → family="García" + ``` **Because:** a particle is written as part of the surname it precedes, and a title stands outside the name entirely. diff --git a/docs/release_log.rst b/docs/release_log.rst index 06afcb95..1f2caaec 100644 --- a/docs/release_log.rst +++ b/docs/release_log.rst @@ -14,7 +14,7 @@ Release Log - **Add ParsedName.patronymic, the words naming the bearer's parent.** ``parse("Mahathir bin Mohamad")`` gives patronymic ``bin Mohamad`` beside family ``bin Mohamad``, which 1.4.0 through 2.8.0 gave alone and which is unchanged; under ``locales.RU``, ``Ivan Ivanovich Ivanov`` gives patronymic ``Ivanovich`` and keeps it as the middle name, and under ``locales.TR_AZ`` a marker such as ``oglu`` reads with the name before it. A view like ``family_base``, not a field of ``as_dict()``, read only from a patronymic particle (``bin``, ``binti``, ``binte``, ``ibn``, ``bint``, ``bar``, ``bat`` and their Arabic and Hebrew forms) or a rule a locale turns on, so ``Anwar Ibrahim`` has none, and a name written by ``replace()``, or unpickled from 2.x, has none either. ``Ambiguity.alternative`` of a leading ``bin`` read as a given name names a patronymic where 2.7.0 and 2.8.0 named a particle. :issue:`754` - **Add ParsedName.surname2, the second of two family names.** ``Parser(policy=Policy(two_family_names=True)).parse("Gabriel García Márquez")`` gives surname2 ``Márquez`` beside family ``García Márquez``, which 2.8.0 gave alone and which is unchanged. A view like ``family_base``, not a field of ``as_dict()``, filled only where ``two_family_names`` reads the family as two, so the default, a family comma (``Lloyd Webber, Andrew``) and a hyphenated family have none, and so does a name unpickled from 2.x or one whose family ``replace()`` rewrote. ``Reading.second_surname`` marks its words; it is not compared. :issue:`754` - **Add ParsedName.surname1, surname1_particles and surname1_base, the first of two family names.** ``Parser(policy=Policy(two_family_names=True)).parse("Juan Pérez de la Vega")`` gives surname1_base ``Pérez``, where ``family_base``, in 2.8.0 as now, gives ``Pérez Vega``, the base of both surnames. Views like ``surname2``, render keys and not fields of ``as_dict()``; a family read as one unit, the default's included, is its own first surname, and a middle name ``middle_as_family`` folds in belongs to neither. ``Reading.first_surname`` marks its words; it is not compared. :issue:`1022` - - **Add a switch for names with a compound given name.** ``Parser(policy=Policy(two_given_names=True)).parse("Juan Carlos Pérez")`` gives given ``Juan Carlos``, where 1.4.0 through 2.8.0 gave given ``Juan``, middle ``Carlos``. Off by default: under it a middle name beside the given name joins it (``John Quincy Adams`` gives given ``John Quincy``). With ``two_family_names`` on too, the family takes its word first, so ``María José García López`` reads given ``María José``, family ``García López``. A ``Policy`` pickled by 2.8 loads with it off. :issue:`704` + - **Add a switch for names with a compound given name.** ``Parser(policy=Policy(two_given_names=True)).parse("Juan Carlos Pérez")`` gives given ``Juan Carlos``, where 1.4.0 through 2.8.0 gave given ``Juan``, middle ``Carlos``. Off by default: under it a middle name beside the given name joins it (``John Quincy Adams`` gives given ``John Quincy``). With ``two_family_names`` on too, the family takes its word first, so ``María José García López`` reads given ``María José``, family ``García López``. A particle phrase the vocabulary marks completes the given name (``María del Carmen García`` gives given ``María del Carmen``), and the Sikh ``Kaur`` and ``Singh`` stay middle names under either two-name switch, as does a middle name behind a bound given-name pair (``Abu Bakr Carlos García`` keeps middle ``Carlos``); mark a word of your own with ``Lexicon.default().add("carmo", kind="given", after_particle=True)``. A ``Policy`` pickled by 2.8 loads with it off. :issue:`704` - **Add Lexicon.given_names, given names two_family_names leaves out of the family.** ``Parser(policy=Policy(two_family_names=True), lexicon=Lexicon.default() | Lexicon(given_names={"luis"})).parse("José Luis Pérez")`` gives middle ``Luis``, family ``Pérez``, where the switch alone, in 2.8.0 as now, gives family ``Luis Pérez``. Empty by default, so the switch alone reads as before; an entry without accents matches the word with them. :issue:`959` - **Add Token.reading, the reading the parse took for each word.** ``parse("John Smith PhD Jones").tokens[-1].reading`` is ``Reading('credential' as 'suffix', 'position')``, the reading 2.7's ``Ambiguity.alternative`` gave only for the other side of a fork, and ``ReadingKind`` gains ``patronymic``. The field is not compared, and a ``ParsedName`` pickled by 2.x loads with no readings. :issue:`754` - **Add Reading.languages, the language files that list a word the way it was read.** ``parse("Dr. Juan de la Vega III").tokens[2].reading.languages`` is ``('ca', 'es', 'fil', 'fr', 'it', 'nl', 'pt', 'si')`` for the particle ``de``, and ``Ambiguity.alternative`` names its files too; a reading by shape or position names none. Under a choice of languages it names the files the choice reads: ``parser_for(locales.PT)`` names ``('pt',)`` for that ``de``. It is not compared, and a ``Reading`` pickled by 2.7 or 2.8 loads with none. The first parse in a process that meets a listed word loads the reading index once: about 1 ms, or about 16 ms where Python writes no bytecode cache. :issue:`754`, :issue:`753` diff --git a/nameparser/AGENTS.md b/nameparser/AGENTS.md index 097b96a0..75ebba6d 100644 --- a/nameparser/AGENTS.md +++ b/nameparser/AGENTS.md @@ -10,7 +10,7 @@ No comma, a FULL name before the comma, a ONE-WORD name before the comma and NOT ### Counting the one-word path -Count the one-word path the way C3 counts, which is neither tokens nor pieces: a particle run and the one word it attaches to are one word (#575), a connective join and a bound given-name pair count as their words (#613, so `Ortega y Gasset, Dr.` is a whole name), and a suffix beside the surname counts as no name word at all. #603's review found the suffix half (`Vega y Lopez Jr., PhD Jones` lost its family where `Vega y Lopez, PhD Jones` kept it); #613 then reversed the connective half, and both now read given 'Vega', middle 'y', family 'Lopez' — so test a particle surname, a connective surname and a surname with a suffix, each before a comma. A suffix word OPENING the part is the other half, and counts as a name word where one follows it: from 2.4.0 the counts passed over it, so `JP Clancy, MD` lost its given name to initials that spell a credential and `Cem Akin, MD` to a given name a credential spells, until #1017, while `Md Rahman, PhD` had read title 'Md' or family 'Md Rahman' in every release. No comma grid had put a listed suffix word at the head, and real heads are full of them, so open a head with initials (`JP`, `A.M.`), a given name a credential spells (`Cem`, `Md`), a title/suffix dual (`Sr.`) and a split credential's member (`MB ChB`), behind a title too, and put a title behind the opening word (`Md Prof., PhD`): the first build counted it as the name word there in one count and not in the other, found by review. +Count the one-word path the way C3 counts, which is neither tokens nor pieces: a particle run and the one word it attaches to are one word (#575), a connective join and a bound given-name pair count as their words (#613, so `Ortega y Gasset, Dr.` is a whole name), and a suffix beside the surname counts as no name word at all. #603's review found the suffix half (`Vega y Lopez Jr., PhD Jones` lost its family where `Vega y Lopez, PhD Jones` kept it); #613 then reversed the connective half, and both now read given 'Vega', middle 'y', family 'Lopez' — so test a particle surname, a connective surname and a surname with a suffix, each before a comma. A suffix word OPENING the part is the other half, and counts as a name word where one follows it: from 2.4.0 the counts passed over it, so `JP Clancy, MD` lost its given name to initials that spell a credential and `Cem Akin, MD` to a given name a credential spells, until #1017, while `Md Rahman, PhD` had read title 'Md' or family 'Md Rahman' in every release. No comma grid had put a listed suffix word at the head, and real heads are full of them, so open a head with initials (`JP`, `A.M.`), a given name a credential spells (`Cem`, `Md`), a title/suffix dual (`Sr.`) and a split credential's member (`MB ChB`), behind a title too, and put a title behind the opening word (`Md Prof., PhD`): the first build counted it as the name word there in one count and not in the other, found by review. A bound given-name pair is the same trap from the other side: a count that passes over particles reads `Abu Bakr` as ONE word, `abu` being a particle and a bound word both, where `abdul Rahman` counts two, so a rule asking whether the given name already holds two words needs a bound pair of each spelling in its grid; #1057's grid held only `abdul` until its test review found O7 completing `Abu Bakr` (`_post_rules._one_given_word` counts it now). ### Orders diff --git a/nameparser/_lexicon.py b/nameparser/_lexicon.py index 44b13d16..be789380 100644 --- a/nameparser/_lexicon.py +++ b/nameparser/_lexicon.py @@ -532,7 +532,8 @@ def _normpairs( #: compiled constant of its own. _KIND_FIELDS = ("generation_words", "credential_words", "honorific_words", "patronymic_particles", "title_abbreviations", - "title_case_name_words") + "title_case_name_words", "shared_names", + "particle_given_names") @dataclass(frozen=True, slots=True, eq=False, repr=False) class _LexiconSets: @@ -593,6 +594,15 @@ class _LexiconSets: #: `kc`), whose writing leans where a name reading gates them: #: lower case is the suffix, Title case the name (rules/S6, #1039) title_case_name_words: frozenset[str] + #: the names written `shared=True` (Sikh `kaur`, `singh`), which a + #: community gives every member: rules/O6 and O7 neither take nor + #: complete one (#704) + shared_names: frozenset[str] + #: the given names written `after_particle=True` (`carmen`), the + #: second of a compound given name written behind a particle + #: (`María del Carmen`): rules/O7 takes one, and O6 leaves it beside + #: a one-word given name (#704) + particle_given_names: frozenset[str] #: Lexicon field -> the compiled constant it is read from, for every @@ -626,6 +636,8 @@ class _LexiconSets: "patronymic_particles": "PATRONYMIC_PARTICLES", "title_abbreviations": "TITLE_ABBREVIATIONS", "title_case_name_words": "TITLE_CASE_NAME_WORDS", + "shared_names": "SHARED_NAMES", + "particle_given_names": "PARTICLE_GIVEN_NAMES", } @@ -710,7 +722,7 @@ def _prop_value(kind: str, prop: str, value: object) -> object: and value == _unwritten(kind, prop)): return None if prop in ("abbreviation", "acronym", "after_given", "may_be_initial", - "title_case_is_name"): + "title_case_is_name", "shared", "after_particle"): if not isinstance(value, bool): raise TypeError(f"{prop} must be True or False, got {value!r}") return value @@ -808,6 +820,10 @@ def _edit_words(words: tuple[object, ...], op: str, def _edit_props(kind: str, props: Mapping[str, object]) -> _Props: ri = _index_module() allowed = ri.indexed(kind) - {"default", "cased"} + if kind == "family": + # a name category's property that only a given name takes: the + # raise below names it, so the generic message leaves it out + allowed -= {"after_particle"} out: dict[str, object] = {} for prop, value in props.items(): if prop == "default": @@ -821,6 +837,11 @@ def _edit_props(kind: str, props: Mapping[str, object]) -> _Props: "capitalization_exceptions, e.g. dataclasses.replace(" "lexicon, capitalization_exceptions=(*lexicon." "capitalization_exceptions, ('phd', 'PhD')))") + if prop == "after_particle" and kind == "family": + raise ValueError( + "after_particle names a given name written behind a " + "particle (rules/O7), so only a given reading takes it: " + "write add(word, kind='given', after_particle=True)") if prop not in allowed: raise ValueError( f"a {kind} reading takes " @@ -1564,7 +1585,8 @@ def _shown_value(kind: str | None, prop: str, value: object) -> str: its unwritten value where it has one (`may_be_initial=False`).""" if value is None and kind is not None and prop in ( "abbreviation", "acronym", "addresses", "after_given", - "may_be_initial", "title_case_is_name"): + "may_be_initial", "title_case_is_name", "shared", + "after_particle"): value = _unwritten(kind, prop) if isinstance(value, tuple) and len(value) > _SHOWN_WORDS: # bounded: a phrase taken out of a kind's reach lists every @@ -2226,8 +2248,9 @@ def add(self, *words: str, kind: str | None = None, `properties` that kind's behavioral properties as the files spell them (``addresses="given"``, ``after_given=True``, ``abbreviation=True``, ``attaches=[...]``, ``acronym=...``, ``attached="after"``, - ``may_be_initial=True``, ``same_as=...`` on a ``given`` - reading); a property given its unwritten value (``None``, + ``may_be_initial=True``, ``same_as=...`` and + ``after_particle=True`` on a ``given`` reading, ``shared=True`` + on a name reading); a property given its unwritten value (``None``, ``False``) is removed. Where the word holds a reading of `kind` already, the add updates its properties in every file holding it; otherwise it adds the caller's own reading, which names no diff --git a/nameparser/_pipeline/_assemble.py b/nameparser/_pipeline/_assemble.py index f20c6097..7a642c11 100644 --- a/nameparser/_pipeline/_assemble.py +++ b/nameparser/_pipeline/_assemble.py @@ -44,6 +44,7 @@ AGNOMEN_TAG, AMBIGUOUS_ACRONYM_TAG, AMBIGUOUS_WORD_TAG, ATTACHED_TAG, CREDENTIAL_WORD_TAG, FIELD_READING, GENERATION_WORD_TAG, HONORIFIC_WORD_TAG, NAME_ROLES, PATRONYMIC_TAG, SECOND_SURNAME_TAG, SHAPE_ACRONYM_TAG, + SHARED_NAME_TAG, PARTICLE_GIVEN_TAG, FIRST_SURNAME_TAG, SPLIT_CREDENTIAL_TAG, SPLIT_MEMBER_TAG, TITLE_AMBIGUOUS_TAG, TRAILING_TITLE_TAG, Beside, Deferred, ParseState, PendingAmbiguity, @@ -221,9 +222,11 @@ def _split_partner(tokens: tuple[WorkToken, ...], dropped: set[int], #: particle, and P6's attachment mark (rules/R9), whose kind #: `Token.reading` carries out; and the written-short mark rules/H5's #: chain reads (#999), for which the published token keeps -#: TITLE_AMBIGUOUS_TAG. +#: TITLE_AMBIGUOUS_TAG; and the shared-name and particle-given marks +#: rules/O6 and O7 read (#704). _KIND_TAGS = frozenset({GENERATION_WORD_TAG, CREDENTIAL_WORD_TAG, HONORIFIC_WORD_TAG, - PATRONYMIC_TAG, ATTACHED_TAG, TRAILING_TITLE_TAG}) + PATRONYMIC_TAG, ATTACHED_TAG, TRAILING_TITLE_TAG, + SHARED_NAME_TAG, PARTICLE_GIVEN_TAG}) #: O6's second-surname mark (rules/R10), which no published token #: carries either: `Reading.second_surname` names it. diff --git a/nameparser/_pipeline/_classify.py b/nameparser/_pipeline/_classify.py index 8a1a7384..83a50a64 100644 --- a/nameparser/_pipeline/_classify.py +++ b/nameparser/_pipeline/_classify.py @@ -16,7 +16,9 @@ of its suffix words and particles (Lexicon's generation_words, credential_words, honorific_words and patronymic_particles, internal sets, 3.0) and the -titles written short (title_abbreviations, #999); and, since +titles written short (title_abbreviations, #999) and the names a +community gives every member (shared_names) and the given names +written behind a particle (particle_given_names, #704); and, since 2.4, Policy.unlisted_dotted_suffixes and Policy.unlisted_caps_suffixes, which decide whether an UNLISTED dotted or all-caps token joins the ambiguous credential class by SHAPE (#516) -- the all-caps half here @@ -42,7 +44,9 @@ stages and dropped by assemble, Token.reading carrying the kind out, save the last, which marks no kind), "vocab:particle-ambiguous", "vocab:bound-given", -"vocab:given-abbreviation", "vocab:maiden-marker", +"vocab:given-abbreviation", "vocab:shared-name" and +"vocab:particle-given" (which rules/O6 and O7 read and assemble drops, +carrying no kind out), "vocab:maiden-marker", "vocab:maiden-marker-cont"; in a namespace of its own, "shape:acronym"; and in another, "split:credential" and "split:member", written on the first word of a credential split across @@ -79,7 +83,7 @@ class (rules/S12). from nameparser._pipeline._state import ( AGNOMEN_TAG, AMBIGUOUS_ACRONYM_TAG, AMBIGUOUS_WORD_TAG, CREDENTIAL_WORD_TAG, GENERATION_WORD_TAG, GIVEN_ABBREVIATION_TAG, HONORIFIC_WORD_TAG, - PATRONYMIC_TAG, SHAPE_ACRONYM_TAG, SPLIT_CREDENTIAL_TAG, SPLIT_MEMBER_TAG, TITLE_AMBIGUOUS_TAG, + PARTICLE_GIVEN_TAG, PATRONYMIC_TAG, SHAPE_ACRONYM_TAG, SHARED_NAME_TAG, SPLIT_CREDENTIAL_TAG, SPLIT_MEMBER_TAG, TITLE_AMBIGUOUS_TAG, TITLE_CASE_NAME_TAG, TRAILING_TITLE_TAG, ParseState, Deferred, PendingAmbiguity, PendingReading, @@ -266,6 +270,14 @@ def _tags_for(token: WorkToken, n: str, state: ParseState, # carries it and only that test acts on it if n in lex._sets.given_name_abbreviations: tags.add(GIVEN_ABBREVIATION_TAG) + # rules/O6 and O7 read this: a name a community gives every member + # is neither taken nor completed by the two-name switches (#704) + if n in lex._sets.shared_names: + tags.add(SHARED_NAME_TAG) + # rules/O7 and O6 read this: a given name written behind a particle + # completes the given name it follows (#704) + if n in lex._sets.particle_given_names: + tags.add(PARTICLE_GIVEN_TAG) # maiden markers are NOT tagged here: an entry may be a phrase whose # words are not markers on their own, and this function sees one # token with no neighbours. `_vocab.tag_marker_runs` does the whole diff --git a/nameparser/_pipeline/_post_rules.py b/nameparser/_pipeline/_post_rules.py index 40c6a788..e0775e83 100644 --- a/nameparser/_pipeline/_post_rules.py +++ b/nameparser/_pipeline/_post_rules.py @@ -51,6 +51,7 @@ ) from nameparser._pipeline._state import ( AMBIGUOUS_ACRONYM_TAG, ATTACHED_TAG, NAME_ROLES, PATRONYMIC_TAG, + PARTICLE_GIVEN_TAG, SHARED_NAME_TAG, FIRST_SURNAME_TAG, SECOND_SURNAME_TAG, ParseState, PendingAmbiguity, PendingReading, Structure, WorkToken, _NEVER_FLIPPED, comma_bucket, copy_with, @@ -335,6 +336,22 @@ def _take_second_family_name(state: ParseState, is_listed_given_name(tokens[take[0]].text, state.lexicon) or _hyphenated_given(tokens[take[0]].text, state.lexicon)): return + # rules/O6: "a name a community gives every member is no family + # name of two" -- Kaur and Singh, on either side of the join, tagged + # at classify from Lexicon.shared_names (decisions/O6, #704) + if any(SHARED_NAME_TAG in tokens[i].tags for i in take + family): + return + # rules/O6: "a given name a particle completes stays where it is" -- + # given-first, the unit taken being the one beside a given name of + # one word, a particle run and a word classify tags from + # Lexicon.particle_given_names, in the shape rules/O7 reads; under a + # family-first order the unit after the family completes no given + # name (decisions/O6, #704) + if (state.order[0] is Role.GIVEN and len(units) == 2 + and _one_given_word(tokens, units[0]) + and all(tokens[i].role is Role.GIVEN for i in units[0]) + and _completes_a_given_name(tokens, take)): + return if state.order[0] is Role.GIVEN: # given, middles..., family: the middle unit standing right # before the family joins it, and text order already renders @@ -362,6 +379,60 @@ def _take_second_family_name(state: ParseState, _retag(tokens, i, role) +def _one_given_word(tokens: list[WorkToken], given: list[int]) -> bool: + """Whether the given name is ONE word, as rules/O7 counts it: O6's + count, save that a bound given-name pair (P5) is two words though + its bound word is also a particle the count passes over ('Abu + Bakr').""" + return (_family_word_count(tokens, given) == 1 + and not (len(given) > 1 + and "vocab:bound-given" in tokens[given[0]].tags)) + + +def _completes_a_given_name(tokens: list[WorkToken], + unit: list[int]) -> bool: + """Whether `unit` is a particle run and ONE word written + `after_particle` ('del Carmen'), the second of a compound given + name as rules/O7 reads one.""" + n = 0 + while n < len(unit) and "particle" in tokens[unit[n]].tags: + n += 1 + return (0 < n == len(unit) - 1 + and PARTICLE_GIVEN_TAG in tokens[unit[-1]].tags) + + +def _take_given_phrase(state: ParseState, tokens: list[WorkToken], + given: list[int]) -> None: + # rules/O7: "a particle run and the given name written behind it + # complete the given name" -- given-first with no middle between, + # the family's opening particle run and the one word after it, read + # off the tag classify gives a word of Lexicon.particle_given_names, + # move to the given name where a name word is left for the family + # (decisions/O7, #704) + family = _idx(tokens, Role.FAMILY) + # the family written after the given name: a family-first order, + # whose given name also stands before its middles, is turned away + if not family or family[0] < given[-1]: + return + cut = _fold_reach(tokens, family) + if cut >= len(family) or not _completes_a_given_name(tokens, + family[:cut]): + return + # a name word left for the family: one with a letter that is no + # connective standing alone ('María del Carmen y') + if not any(any(ch.isalpha() for ch in tokens[i].text) + and "conjunction" not in tokens[i].tags + for i in family[cut:]): + return + # and, as for the middle path, no Turkic marker behind the phrase, + # whose father's name it would be + if (PatronymicRule.TURKIC in state.policy.patronymic_rules + and is_patronymic_marker(tokens[family[cut]].text)): + return + for i in family[:cut]: + _retag(tokens, i, Role.GIVEN) + + def _take_second_given_name(state: ParseState, tokens: list[WorkToken]) -> None: # rules/O7: "the given name takes the middle unit beside it" -- @@ -382,7 +453,20 @@ def _take_second_given_name(state: ParseState, # a given name already holding two words keeps what it has: a # hyphenated one, or a bound given-name pair (P5), counted as O6 # counts the family - if not given or not middle or _family_word_count(tokens, given) != 1: + if not given or not _one_given_word(tokens, given): + return + if not middle: + # rules/O7's particle phrase, opening the family: after a family + # comma the part reads it as a middle already (the path below), + # and a given name that is a patronymic, a shared name or no + # name at all is not completed, as on the middle path + if (given_first and state.structure is not Structure.FAMILY_COMMA + and not any(PATRONYMIC_TAG in tokens[i].tags + or SHARED_NAME_TAG in tokens[i].tags + for i in given) + and any(ch.isalpha() for i in given + for ch in tokens[i].text)): + _take_given_phrase(state, tokens, given) return # the unit beside the given name: the positions lay a given-first # part out given, middles..., and FAMILY_FIRST_GIVEN_LAST's @@ -403,6 +487,11 @@ def _take_second_given_name(state: ParseState, # a Turkic marker's father if any(PATRONYMIC_TAG in tokens[i].tags for i in take + given): return + # rules/O7: "a name a community gives every member is no given + # name" -- Kaur and Singh, on either side of the join, tagged at + # classify from Lexicon.shared_names (decisions/O7, #704) + if any(SHARED_NAME_TAG in tokens[i].tags for i in take + given): + return rules = state.policy.patronymic_rules if rules: slavic = PatronymicRule.EAST_SLAVIC in rules diff --git a/nameparser/_pipeline/_state.py b/nameparser/_pipeline/_state.py index 1b0c0331..e56a022c 100644 --- a/nameparser/_pipeline/_state.py +++ b/nameparser/_pipeline/_state.py @@ -163,6 +163,20 @@ class WorkToken: #: (`_pieces.is_leading_title`, `_vocab.is_title_shaped`). GIVEN_ABBREVIATION_TAG = "vocab:given-abbreviation" +#: On a word of `Lexicon.shared_names`, a name a community gives every +#: member (Sikh 'Kaur', 'Singh'): rules/O6 and O7 decline a join that +#: takes or completes one +#: (`_post_rules._take_second_family_name`, `_take_second_given_name`). +#: Dropped from the published token like the kind tags above (#704). +SHARED_NAME_TAG = "vocab:shared-name" + +#: On a word of `Lexicon.particle_given_names`, a given name written +#: behind a particle as the second of a compound given name ('del +#: Carmen'): rules/O7 takes the particle run and the word into the +#: given name, and O6 leaves them beside a one-word given name. Dropped from the published token +#: like the kind tags above (#704). +PARTICLE_GIVEN_TAG = "vocab:particle-given" + #: EITHER way a token joins the ambiguous credential class -- the #: vocabulary's claim and the writing's. The two emitters that report #: a fork the peel called and declined ask exactly this: `_group`'s diff --git a/nameparser/_reading_index.py b/nameparser/_reading_index.py index 7ca3bca5..bb29b616 100644 --- a/nameparser/_reading_index.py +++ b/nameparser/_reading_index.py @@ -81,9 +81,10 @@ #: Behavioral properties per category: each compiles into a field, and #: its allowed values are exactly what today's pipeline does. A name -#: reading's are `default` (phase 3a) and `cased`, the word's case mask -#: (the shipped ones on `family` readings, #1014): its other keys are -#: evidence. +#: reading's are `default` (phase 3a), `cased`, the word's case mask +#: (the shipped ones on `family` readings, #1014), and `shared` and +#: `after_particle`, which rules/O6 and O7 read (#704): its other keys +#: are evidence. BEHAVIORAL: dict[str, frozenset[str]] = { "title": frozenset({"abbreviation", "addresses", "after_given", "attaches", "cased", "default"}), @@ -93,7 +94,7 @@ "conjunction": frozenset({"may_be_initial", "cased", "default"}), "bound_given": frozenset({"cased", "default"}), "relation": frozenset({"cased", "default"}), - "name": frozenset({"cased", "default"}), + "name": frozenset({"after_particle", "cased", "default", "shared"}), } #: The suffix word kinds a name reading gates (rules/S11): the third @@ -152,8 +153,9 @@ def on_by_default(self) -> bool: return self.props.get("default", True) is not False def flag(self, prop: str) -> bool: - """A `may_be_*` flag, `after_given`, `abbreviation` or - `title_case_is_name`: unwritten, false.""" + """A `may_be_*` flag, `after_given`, `abbreviation`, + `title_case_is_name`, `shared` or `after_particle`: unwritten, + false.""" return bool(self.props.get(prop, False)) @property @@ -535,6 +537,8 @@ class Compiled(TypedDict): PATRONYMIC_PARTICLES: frozenset[str] TITLE_ABBREVIATIONS: frozenset[str] TITLE_CASE_NAME_WORDS: frozenset[str] + SHARED_NAMES: frozenset[str] + PARTICLE_GIVEN_NAMES: frozenset[str] def compile_readings(readings: Iterable[ReadingProperties]) -> Compiled: @@ -577,11 +581,17 @@ def compile_readings(readings: Iterable[ReadingProperties]) -> Compiled: honorific_words: set[str] = set() patronymics: set[str] = set() title_case_names: set[str] = set() + shared: set[str] = set() + after_particle: set[str] = set() cased: dict[str, str] = {} for r, t in texts: p = r.props if "cased" in p: cased[t] = p["cased"] + if r.category == "name" and r.flag("shared"): + shared.add(t) + if r.kind == "given" and r.flag("after_particle"): + after_particle.add(t) if r.category == "title": titles.add(t) if r.addresses == "given": @@ -690,6 +700,16 @@ def compile_readings(readings: Iterable[ReadingProperties]) -> Compiled: # (`fils`, `kc`): of a word gated by a name reading, the one # kind whose writing leans (rules/S6, S11, #1039) TITLE_CASE_NAME_WORDS=frozenset(title_case_names), + # the names a community gives every member (Sikh `kaur`, + # `singh`), no personal or family name of the bearer's own, + # which the two-name switches neither move nor complete + # (rules/O6, O7, #704); internal, like the kind sets above + SHARED_NAMES=frozenset(shared), + # the given names written behind a particle as the second of a + # compound given name (`del Carmen`, `de los Ángeles`), which + # rules/O7 takes into the given name and O6 leaves beside a + # one-word given name (#704) + PARTICLE_GIVEN_NAMES=frozenset(after_particle), ) # One form per post-nominal: the word test strips only edge periods # and the acronym test strips all of them, so a word in both is diff --git a/nameparser/config/_compiled.py b/nameparser/config/_compiled.py index 415b8693..35cee2f0 100644 --- a/nameparser/config/_compiled.py +++ b/nameparser/config/_compiled.py @@ -3083,3 +3083,34 @@ 'pc', 'père', }) + +SHARED_NAMES: frozenset[str] = frozenset({ + 'kaur', + 'singh', +}) + +PARTICLE_GIVEN_NAMES: frozenset[str] = frozenset({ + 'angeles', + 'caridad', + 'carmen', + 'cisne', + 'concepción', + 'consuelo', + 'fátima', + 'jesús', + 'lourdes', + 'mar', + 'maria', + 'maría', + 'mercedes', + 'milagro', + 'milagros', + 'nieves', + 'pilar', + 'rocio', + 'rocío', + 'rosario', + 'socorro', + 'valle', + 'ángeles', +}) diff --git a/nameparser/config/_index.py b/nameparser/config/_index.py index 23d84e3e..571f1517 100644 --- a/nameparser/config/_index.py +++ b/nameparser/config/_index.py @@ -108,6 +108,7 @@ ('analytics', (('en', 'modifier', ()),)), ('anarchist', (('en', 'occupation', ()),)), ('and', (('en', 'conjunction', ()),)), + ('angeles', (('es', 'given', (('after_particle', True),)),)), ('animator', (('en', 'occupation', ()),)), ('anp', (('en', 'credential', ()),)), ('anthropologist', (('en', 'occupation', ()),)), @@ -263,6 +264,8 @@ ('capt', (('en', 'military', ()),)), ('captain', (('en', 'military', ()),)), ('cardinal', (('en', 'religious', ()),)), + ('caridad', (('es', 'given', (('after_particle', True),)),)), + ('carmen', (('es', 'given', (('after_particle', True),)),)), ('caro', (('en', 'credential', ()), ('en', 'given', ()), ('en', 'family', ()))), ('cartographer', (('en', 'occupation', ()),)), ('cartoonist', (('en', 'occupation', ()),)), @@ -399,6 +402,7 @@ ('ciro', (('en', 'credential', ()), ('en', 'given', ()), ('en', 'family', ()))), ('cisa', (('en', 'credential', ()),)), ('cism', (('en', 'credential', ()),)), + ('cisne', (('es', 'given', (('after_particle', True),)),)), ('cissp', (('en', 'credential', ()),)), ('civil', (('en', 'modifier', ()),)), ('cla', (('en', 'credential', ()),)), @@ -444,8 +448,10 @@ ('comptroller', (('en', 'office', (('attaches', ('gen',)),)),)), ('computer', (('en', 'modifier', ()),)), ('comtesse', (('fr', 'nobility', ()),)), + ('concepción', (('es', 'given', (('after_particle', True),)),)), ('conductor', (('en', 'occupation', ()),)), ('consort', (('en', 'modifier', (('addresses', 'given'),)),)), + ('consuelo', (('es', 'given', (('after_particle', True),)),)), ('consultant', (('en', 'occupation', ()),)), ('controller', (('en', 'office', ()),)), ('corporal', (('en', 'military', ()),)), @@ -808,6 +814,7 @@ ('fsa', (('en', 'credential', ()),)), ('fsdp', (('en', 'credential', ()),)), ('fws', (('en', 'credential', ()),)), + ('fátima', (('es', 'given', (('after_particle', True),)),)), ('född', (('sv', 'birth_marker', ()),)), ('fødd', (('nn', 'birth_marker', ()),)), ('født', (('da', 'birth_marker', ()), ('nb', 'birth_marker', ()))), @@ -939,6 +946,7 @@ ('jaunākais', (('lv', 'generation', ()),)), ('jaunākā', (('lv', 'generation', ()),)), ('jd', (('en', 'credential', ()), ('en', 'given', ()))), + ('jesús', (('es', 'given', (('after_particle', True),)),)), ('jno', (('en', 'given', (('same_as', 'john'),)),)), ('jnr', (('mul', 'generation', ()),)), ('jo', (('fil', 'given', (('same_as', 'josé'),)),)), @@ -955,6 +963,7 @@ ('justice', (('en', 'office', ()), ('en', 'given', ()), ('en', 'family', ()))), ('júnior', (('pt', 'generation', ()), ('pt', 'given', ()), ('pt', 'family', ()))), ('kalbatoni', (('ka', 'salutation', (('after_given', True),)),)), + ('kaur', (('mul', 'family', (('shared', True),)),)), ('kbe', (('en', 'credential', ()),)), ('kc', (('en', 'credential', (('acronym', False), ('cased', 'KC'), ('title_case_is_name', True))), ('en', 'given', ()), ('en', 'family', ()))), ('kcb', (('en', 'credential', ()),)), @@ -1039,6 +1048,7 @@ ('lmt', (('en', 'credential', ()),)), ('lord', (('en', 'nobility', (('attaches', ('chancellor', 'justice')),)),)), ('los', (('es', 'article', ()), ('fil', 'article', ()))), + ('lourdes', (('es', 'given', (('after_particle', True),)),)), ('lp', (('en', 'credential', ()),)), ('lpa', (('en', 'credential', ()),)), ('lpc', (('en', 'credential', ()),)), @@ -1126,14 +1136,17 @@ ('major', (('en', 'military', (('attaches', ('gen', 'general')),)), ('en', 'given', ()), ('en', 'family', ()))), ('malam', (('ha', 'given', ()), ('ha', 'family', ()))), ('manager', (('en', 'occupation', ()),)), + ('mar', (('es', 'given', (('after_particle', True),)),)), ('marcher', (('en', 'modifier', ()),)), ('marchess', (('en', 'nobility', ()),)), ('marchioness', (('en', 'nobility', ()),)), + ('maria', (('es', 'given', (('after_particle', True),)),)), ('marketing', (('en', 'modifier', ()),)), ('marquess', (('en', 'nobility', ()),)), ('marquis', (('en', 'nobility', ()), ('en', 'given', ()), ('en', 'family', ()), ('fr', 'nobility', ()))), ('marquise', (('en', 'nobility', ()), ('en', 'given', ()), ('en', 'family', ()), ('fr', 'nobility', ()))), ('marshal', (('en', 'military', ()), ('en', 'given', ()), ('en', 'family', ()))), + ('maría', (('es', 'given', (('after_particle', True),)),)), ('master', (('en', 'salutation', (('addresses', 'given'),)), ('en', 'family', ()))), ('mathematician', (('en', 'occupation', ()),)), ('mathematics', (('en', 'modifier', ()),)), @@ -1177,6 +1190,7 @@ ('memoirist', (('en', 'occupation', ()),)), ('meneer', (('nl', 'salutation', (('after_given', True),)),)), ('meng', (('en', 'credential', ()), ('en', 'given', ()), ('en', 'family', ()))), + ('mercedes', (('es', 'given', (('after_particle', True),)),)), ('merchant', (('en', 'occupation', ()),)), ('met', (('en', 'religious', ()),)), ('metropolitan', (('en', 'religious', ()),)), @@ -1197,6 +1211,8 @@ ('micfor', (('en', 'credential', (('cased', 'MICFor'),)),)), ('micp', (('en', 'credential', ()),)), ('mieee', (('en', 'credential', ()),)), + ('milagro', (('es', 'given', (('after_particle', True),)),)), + ('milagros', (('es', 'given', (('after_particle', True),)),)), ('military', (('en', 'modifier', ()),)), ('minister', (('en', 'office', ()),)), ('mirm', (('en', 'credential', ()),)), @@ -1305,6 +1321,7 @@ ('netto', (('pt-BR', 'agnomen', ()), ('pt-BR', 'family', ()))), ('neuroscientist', (('en', 'occupation', ()),)), ('nic', (('ga', 'clan', (('cased', 'Nic'), ('default', False))), ('ga', 'given', ()), ('ga', 'family', ()), ('gd', 'clan', (('cased', 'Nic'), ('default', False))))), + ('nieves', (('es', 'given', (('after_particle', True),)),)), ('nmd', (('en', 'credential', ()),)), ('nnp', (('en', 'credential', ()),)), ('novelist', (('en', 'occupation', ()),)), @@ -1379,6 +1396,7 @@ ('physicist', (('en', 'occupation', ()),)), ('pi', (('en', 'family', ()),)), ('pianist', (('en', 'occupation', ()),)), + ('pilar', (('es', 'given', (('after_particle', True),)),)), ('pilot', (('en', 'occupation', ()),)), ('pioneer', (('en', 'occupation', ()),)), ('pir', (('fa', 'religious', ()),)), @@ -1512,7 +1530,10 @@ ('rndr', (('cs', 'academic', (('cased', 'RNDr'), ('default', False))), ('sk', 'academic', (('cased', 'RNDr'), ('default', False))))), ('rnfa', (('en', 'credential', ()),)), ('robt', (('en', 'given', (('same_as', 'robert'),)),)), + ('rocio', (('es', 'given', (('after_particle', True),)),)), ('rock', (('en', 'modifier', ()),)), + ('rocío', (('es', 'given', (('after_particle', True),)),)), + ('rosario', (('es', 'given', (('after_particle', True),)),)), ('rouva', (('fi', 'salutation', (('after_given', True),)),)), ('royal', (('en', 'modifier', (('addresses', 'given'),)), ('en', 'given', ()), ('en', 'family', ()))), ('roz', (('cs', 'birth_marker', (('default', False),)),)), @@ -1601,6 +1622,7 @@ ('siie', (('en', 'credential', ()),)), ('singer', (('en', 'occupation', ()),)), ('singer-songwriter', (('en', 'occupation', ()),)), + ('singh', (('mul', 'family', (('shared', True),)),)), ('sir', (('en', 'nobility', (('addresses', 'given'),)),)), ('sister', (('en', 'religious', (('addresses', 'given'),)),)), ('slečna', (('cs', 'salutation', (('after_given', True),)),)), @@ -1619,6 +1641,7 @@ ('soccer', (('en', 'modifier', ()),)), ('social', (('en', 'modifier', ()),)), ('sociologist', (('en', 'occupation', ()),)), + ('socorro', (('es', 'given', (('after_particle', True),)),)), ('software', (('en', 'modifier', ()),)), ('soldier', (('en', 'occupation', ()),)), ('solicitor', (('en', 'occupation', (('attaches', ('gen',)),)),)), @@ -1713,6 +1736,7 @@ ('uí', (('ga', 'clan', (('cased', 'Uí'), ('default', False))),)), ('v', (('mul', 'generation', ()),)), ('vadm', (('en', 'military', ()),)), + ('valle', (('es', 'given', (('after_particle', True),)),)), ('van', (('nl', 'preposition', ()), ('nl', 'given', ()))), ('vande', (('nl', 'preposition', ()), ('nl', 'given', ()))), ('vander', (('nl', 'preposition', ()), ('nl', 'given', ()))), @@ -1759,6 +1783,7 @@ ('z domu', (('pl', 'birth_marker', ()),)), ('zoologist', (('en', 'occupation', ()),)), ('zu', (('de', 'preposition', ()),)), + ('ángeles', (('es', 'given', (('after_particle', True),)),)), ('ó', (('ga', 'clan', (('cased', 'Ó'),)),)), ('ƙarami', (('ha', 'generation', ()),)), ('δρ', (('el', 'academic', ()),)), diff --git a/nameparser/vocab/README.md b/nameparser/vocab/README.md index 9355386c..9b785ca7 100644 --- a/nameparser/vocab/README.md +++ b/nameparser/vocab/README.md @@ -182,6 +182,8 @@ A property is **behavioral** when it compiles into a set and so changes how name | `acronym` | behavioral | `suffix` | true for `credential`, false for every other kind | true: `SUFFIX_ACRONYMS`, matched with every period removed (`M.D.` is `md`); false: `SUFFIX_WORDS`, and a credential's `CREDENTIAL_WORDS`, a credential written as a word | | `title_case_is_name` | behavioral | `suffix` | false | `TITLE_CASE_NAME_WORDS`, on a `generation` or a `credential` written as a word (`acronym = false`) only: says the word's Title case is its writing as a name, so where a name reading gates the word ([rules/S11](../../docs/design/rules/S11-suffix-word-that-is-also-a-name.md), a suffix word that is also a name), its writing leans: Title case is the name, lower case in a name of mixed case the suffix, capitals nothing ([rules/S6](../../docs/design/rules/S6-case-signal-at-the-suffix-slot.md), the case signal at the suffix slot), as in `Marie Fils` and `Alexandre Dumas fils`. Unwritten, a gated word's writing says nothing and the count decides, as for `Junior`, whose Title case is how the generation is written too; an agnomen's writing never speaks (#1039) | | `attached` | behavioral | `suffix` | absent | `GLUED_HONORIFICS`, on a word reading only: `{ side = "after" }` splits the word off the end of a name token, as in `田中さん` ([rules/W2](../../docs/design/rules/W2-glued-honorific-peel.md), the glued-honorific peel) | +| `shared` | behavioral | `name` | false | `SHARED_NAMES`: a name a community gives every member, written after the personal name and before a family name where there is one (Sikh `kaur` and `singh`), no personal or family name of the bearer's own: two-family-name handling ([rules/O6](../../docs/design/rules/O6-two-family-names.md)) and two-given-name handling ([rules/O7](../../docs/design/rules/O7-two-given-names.md)) neither take it nor complete one with it, so `Harpreet Kaur Dhillon` keeps middle `Kaur` under either. It moves nothing in a parse with both switches off (#704) | +| `after_particle` | behavioral | `name` | false | `PARTICLE_GIVEN_NAMES`, on a `given` reading only: a given name written behind a particle as the second of a compound given name (`carmen` in `María del Carmen`); under two-given-name handling ([rules/O7](../../docs/design/rules/O7-two-given-names.md)) the particle run and the word join the given name though the chain would run on into the family, as in `María del Carmen García`, and two-family-name handling ([rules/O6](../../docs/design/rules/O6-two-family-names.md)) does not take them beside a one-word given name; rules/O7 states the conditions. It moves nothing in a parse with both switches off. A word below the 90% bar decisions/O7 sets is left out (`luz`) (#704) | | `may_be_given` | stated | `particle` | none: required where the word has no `given` reading | `false` is required where the word has no `given` reading in any file, and that absence, not the property, is what puts the word in `NON_GIVEN_NAME_PARTICLES`: a particle opening the name starts the family name ([rules/P1](../../docs/design/rules/P1-lone-particle-fold.md), the lone particle fold). `true` is refused, naming the `given` reading to write instead (check 11) | | `may_be_initial` | behavioral | `conjunction` | false | `CONJUNCTIONS_AMBIGUOUS`, on a cased single letter only: read as an initial in a name written in one case, as in `JOHN E SMITH` ([rules/P8](../../docs/design/rules/P8-single-letter-connective-against-the-initial-it-spells.md), a single-letter connective against the initial it spells) | | `default` | behavioral | `title`, `suffix`, `particle`, `conjunction`, `bound_given`, `relation`, `name` | true | `false` keeps the reading out of every compiled set, `SUFFIX_WORDS` and `GENERATION_WORDS` included for a generation: recorded and checked as a reading, for a pack to turn on in phase 3, where its writing offers the default no cue (`i`, the German regnal ordinal, written `I.`, exactly as an initial is). An exclusion does not refuse it, and it cannot read two ways beside the word's live readings (checks 4 to 6); a name reading beside it marks nothing. On a `given` or `family` reading (3.0's phase 3a) it derives no marker in the default, so the word's other readings compile as they would without it, until a pack turns the name on. It does not count as the name reading an agnomen on by default needs (check 11), and a particle whose word's only `given` readings are off is refused, which side of check 11 it takes being left to the first pack that needs one; a bound given name that is also a particle is refused with it, before check 9 would ask for its `given` reading. `true` is refused: it is what every reading is unwritten | @@ -227,7 +229,7 @@ A `given` or `family` reading takes its own keys: | `gender` | `"f"`, `"m"` or `"n"` | | `same_as` | on a `given` reading only: the full given name the word abbreviates, normalized (`chas` is `charles`), which puts the word in `GIVEN_NAME_ABBREVIATIONS`: written with its period at the head of the part of the name that carries the given name, the word is that given name, where an unlisted abbreviation there is a title by its shape ([rules/H2](../../docs/design/rules/H2-leading-abbreviation-title.md), the leading-abbreviation title), so `Chas. Smith` reads given `Chas.`. Anywhere else the reading acts on nothing. A name is no word of the vocabulary, so this `same_as`, unlike a reading's, names no entry, and an entry holding only such a reading is not inert | -A name reading may stand without a count, as a particle kept possibly given under C-i's default with no source counting it does, but it then needs a `note` saying why it is declared (check 3). It may also be written `default = false`, the behavioral property in [Properties](#properties): recorded and checked, it derives no marker until a pack turns it on. And it may carry `cased`, the word's case mask, the other behavioral property a name reading takes: case repair title-cases a Mac surname, so one its bearers write with the inner capital carries the mask (`macgregor` is `MacGregor`, #1014), and its entry, holding only that name reading, compiles. +A name reading may stand without a count, as a particle kept possibly given under C-i's default with no source counting it does, but it then needs a `note` saying why it is declared (check 3). It may also be written `default = false`, the behavioral property in [Properties](#properties): recorded and checked, it derives no marker until a pack turns it on. And it may carry `cased`, the word's case mask, another behavioral property a name reading takes: case repair title-cases a Mac surname, so one its bearers write with the inner capital carries the mask (`macgregor` is `MacGregor`, #1014), and its entry, holding only that name reading, compiles. So do the two properties the two-name switches read: `after_particle` on a `given` reading (`carmen`, of `María del Carmen`) and `shared` on either (`kaur`, `singh`), each an entry holding only that name reading (#704). ## Exclusions diff --git a/nameparser/vocab/ar.toml b/nameparser/vocab/ar.toml index 56b34452..90cb7a8a 100644 --- a/nameparser/vocab/ar.toml +++ b/nameparser/vocab/ar.toml @@ -524,7 +524,7 @@ bound_given = { expands = "father of", same_as = "abu" } [[word]] text = "umm" -bound_given = { expands = "mother of" } +bound_given = { expands = "mother of", note = "no particle reading, unlike abu: every use counted opens or continues a given name (Pakistani Umm-e-Laila, Umm E Farwa; Umm Kulthum), never a surname: UK PSC filers write it 7 times, in the forename or middle-name box, and ORCID holders 70 times in the given box and in no family box but a university's name typed there; Wikidata 3 given, 0 family. A particle reading would move Syeda Umm E Farwa Zaidi to family Umm E Farwa Zaidi (2026-11-04, #704)" } [[word]] text = "أم" diff --git a/nameparser/vocab/es.toml b/nameparser/vocab/es.toml index 0e906854..99f283b7 100644 --- a/nameparser/vocab/es.toml +++ b/nameparser/vocab/es.toml @@ -177,6 +177,98 @@ text = "y" [word.conjunction.excluded] may_be_initial = { reason = "the commonest Hispanic double-surname link and this library's oldest fixture, 'Velasquez y Garcia': marking it would read the link of every one-case Hispanic record as a middle initial; a caller who wants the Dutch reading adds it", issue = [383, 479] } +[[word]] +text = "angeles" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de los Ángeles written without its accent): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 327 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "caridad" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de la Caridad): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 228 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "carmen" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Carmen): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 1,261 times and open a family box of more words with it behind a particle 5 (2026-11-03, #704)" } + +[[word]] +text = "cisne" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Cisne): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 29 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "concepción" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de la Concepción): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 17 times and open a family box of more words with it behind a particle 1 (2026-11-03, #704)" } + +[[word]] +text = "consuelo" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Consuelo): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 11 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "fátima" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de Fátima): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 27 times and open a family box of more words with it behind a particle 1; in Portugal, Brazil and Portuguese-speaking Africa 32 and 4, below the 90% the other words meet, so it is Spanish here alone (2026-11-03, #704)" } + +[[word]] +text = "jesús" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de Jesús): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 446 times and open a family box of more words with it behind a particle 11 (2026-11-03, #704)" } + [[word]] text = "lady" given = { default = false, note = "a given name in Spanish-speaking Latin America (Lady Heidenreich, Lady Guillén) as well as the British title (en.toml's lady): ORCID records whose given box holds it 137, 111 in Spanish-speaking countries (Ecuador 41, Colombia 30, Peru 12) and 9 in the Philippines, against psc-individuals' 160 title boxes and 19 forename boxes and ukec-individuals' 59 names opening with it, all titles (2026-11-03); off by default, and turned on by a Spanish choice, which makes the title one only inside a title phrase (rules/H9, #913)" } + +[[word]] +text = "lourdes" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de Lourdes): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 76 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "mar" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Mar): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 86 times and open a family box of more words with it behind a particle 3 (2026-11-03, #704)" } + +[[word]] +text = "maria" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de Maria, without its accent): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 18 times and open a family box of more words with it behind a particle 1 (2026-11-03, #704)" } + +[[word]] +text = "maría" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de María): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 35 times and open a family box of more words with it behind a particle 3 (2026-11-03, #704)" } + +[[word]] +text = "mercedes" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de las Mercedes): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 56 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "milagro" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Milagro): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 30 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "milagros" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de los Milagros): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 27 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "nieves" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de las Nieves): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 15 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "pilar" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Pilar): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 322 times and open a family box of more words with it behind a particle 1 (2026-11-03, #704)" } + +[[word]] +text = "rocio" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Rocio, without its accent): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 57 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "rocío" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Rocío): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 77 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "rosario" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Rosario): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 260 times and open a family box of more words with it behind a particle 27 (2026-11-03, #704)" } + +[[word]] +text = "socorro" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Socorro): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 154 times and open a family box of more words with it behind a particle 0 (2026-11-03, #704)" } + +[[word]] +text = "valle" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (del Valle): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 235 times and open a family box of more words with it behind a particle 26 (2026-11-03, #704)" } + +[[word]] +text = "ángeles" +given = { after_particle = true, note = "the second of a compound given name written behind a particle (de los Ángeles): ORCID holders in Spain and Spanish-speaking America end the given box with it behind a particle 230 times and open a family box of more words with it behind a particle 1 (2026-11-03, #704)" } diff --git a/nameparser/vocab/mul.toml b/nameparser/vocab/mul.toml index de9fe187..96dd1e9f 100644 --- a/nameparser/vocab/mul.toml +++ b/nameparser/vocab/mul.toml @@ -121,3 +121,11 @@ credential = { expands = "Doctor of Philosophy", cased = "PhD", note = "written [[word]] text = "&" conjunction = {} + +[[word]] +text = "kaur" +family = { shared = true, us_census_2010 = 48753, wikidata = 204, as_of = 2026-11-03, native = "ਕੌਰ", gender = "f", note = "the name every Sikh woman is given, written after her personal name and before a family name where she has one (Harpreet Kaur Dhillon): in mul.toml so every choice of languages keeps it; UK PSC filers put it in the middle-name box 2,354 times (2,264 as its only word), the whole surname box 674 and the forename box 25; ORCID holders whose family box holds another surname put it in the given box 69 times and open the family box with it 3 (2026-11-03, #704). Wikidata is a lower bound" } + +[[word]] +text = "singh" +family = { shared = true, us_census_2010 = 116749, wikidata = 2192, as_of = 2026-11-03, native = "ਸਿੰਘ", gender = "m", note = "the name every Sikh man is given, written as kaur is: UK PSC filers put it in the middle-name box 5,303 times (5,015 as its only word), the whole surname box 1,885 and the forename box 48 (2026-11-03, #704); also an ordinary Rajput and Nepali surname, which shared costs only under the two-name switches. Wikidata is a lower bound" } diff --git a/tests/v2/pipeline/test_post_rules.py b/tests/v2/pipeline/test_post_rules.py index 77e27ada..30ded80d 100644 --- a/tests/v2/pipeline/test_post_rules.py +++ b/tests/v2/pipeline/test_post_rules.py @@ -1,11 +1,12 @@ import dataclasses import json +import pickle import sys from pathlib import Path import pytest -from nameparser import Parser, parse +from nameparser import Locale, Parser, PolicyPatch, locales, parse, parser_for from nameparser._lexicon import Lexicon from nameparser._pipeline import run from nameparser._pipeline import _post_rules @@ -50,6 +51,9 @@ # #789), read so only period-marked at the head of the given part; # the given reading is also what makes its credential ambiguous .add("ma", kind="given", same_as="maría") + # `maría` is a given name written behind a particle too ('de + # María'), which rules/O7 reads (#704) + .add("maría", "maria", kind="given", after_particle=True) .add("y", kind="conjunction") .add("abdul", kind="bound_given") ) @@ -1438,3 +1442,141 @@ def test_the_list_reads_nothing_without_two_family_names( assert texts assert [t for t in texts if plain.parse(t).as_dict() != listed.parse(t).as_dict()] == [] + + +#: rules/O6 and O7 read two properties of a name reading (#704): a +#: given name written behind a particle (`after_particle`, `carmen` in +#: "María del Carmen") and a name a community gives every member +#: (`shared`, Sikh `kaur`). Each row's reading with the shipped readings +#: and with them removed, the recorded negative control (AGENTS.md, +#: "Guard tests"): rows whose two columns agree are the shapes the +#: properties must leave alone -- nothing left for the family, a family +#: comma (whose part O7 already reads whole), a word nobody listed, a +#: given name of two words, a middle between, and the default policy. +_POLICIES_704 = { + "O7": Policy(two_given_names=True), + "O6": Policy(two_family_names=True), + "both": Policy(two_family_names=True, two_given_names=True), + "family-first+O7": Policy(name_order=FAMILY_FIRST, two_given_names=True), + "default": Policy(), + "family-first+O6": Policy(name_order=FAMILY_FIRST, two_family_names=True), + "family-first-given-last+O7": Policy(name_order=FAMILY_FIRST_GIVEN_LAST, + two_given_names=True), + "family-first+both": Policy(name_order=FAMILY_FIRST, two_given_names=True, + two_family_names=True), + "turkic+O7": Policy(two_given_names=True, + patronymic_rules=frozenset({PatronymicRule.TURKIC})), +} +_BARE_704 = (Lexicon.default().remove("carmen", "mar", "ángeles", kind="given") + .remove("kaur", "singh", kind="family")) +_EFFECT_704 = [ + ("María del Carmen García", "O7", ("María del Carmen", "", "García"), ("María", "", "del Carmen García")), + ("María de los Ángeles López Pérez", "O7", ("María de los Ángeles", "", "López Pérez"), ("María", "", "de los Ángeles López Pérez")), + ("Dr. María del Mar García Jr.", "O7", ("María del Mar", "", "García"), ("María", "", "del Mar García")), + ("María del Carmen", "O7", ("María", "", "del Carmen"), ("María", "", "del Carmen")), + ("García, María del Carmen", "O7", ("María del Carmen", "", "García"), ("María del Carmen", "", "García")), + ("Maria da Silva Santos", "O7", ("Maria", "", "da Silva Santos"), ("Maria", "", "da Silva Santos")), + ("María-José del Carmen García", "O7", ("María-José", "", "del Carmen García"), ("María-José", "", "del Carmen García")), + ("Ana María del Carmen García", "O7", ("Ana María", "", "del Carmen García"), ("Ana María", "", "del Carmen García")), + ("García María del Carmen", "family-first+O7", ("María del Carmen", "", "García"), ("María del Carmen", "", "García")), + ("María del Carmen de la Vega", "O6", ("María", "del Carmen", "de la Vega"), ("María", "", "del Carmen de la Vega")), + ("María del Carmen de la Vega", "both", ("María del Carmen", "", "de la Vega"), ("María", "", "del Carmen de la Vega")), + ("María del Carmen García López", "both", ("María del Carmen", "", "García López"), ("María", "", "del Carmen García López")), + ("María del Carmen García", "default", ("María", "", "del Carmen García"), ("María", "", "del Carmen García")), + ("Harpreet Kaur Dhillon", "O7", ("Harpreet", "Kaur", "Dhillon"), ("Harpreet Kaur", "", "Dhillon")), + ("Harpreet Kaur Dhillon", "O6", ("Harpreet", "Kaur", "Dhillon"), ("Harpreet", "", "Kaur Dhillon")), + ("Harpreet Simran Kaur", "O6", ("Harpreet", "Simran", "Kaur"), ("Harpreet", "", "Simran Kaur")), + ("Dhillon, Harpreet Kaur", "O7", ("Harpreet", "Kaur", "Dhillon"), ("Harpreet Kaur", "", "Dhillon")), + ("Harpreet Kaur", "both", ("Harpreet", "", "Kaur"), ("Harpreet", "", "Kaur")), + ("Harpreet Kaur Dhillon", "default", ("Harpreet", "Kaur", "Dhillon"), ("Harpreet", "Kaur", "Dhillon")), + # the #1057 review's rows: O6 declines only the shape O7 takes, not a + # family-first unit after the family or a phrase behind a middle; O7 + # leaves no lone connective as the family, nor the father's name + # before a Turkic marker + ("García del Carmen María", "family-first+O6", ("María", "", "García del Carmen"), ("María", "", "García del Carmen")), + ("Juan Antonio del Carmen de la Vega", "O6", ("Juan", "Antonio", "del Carmen de la Vega"), ("Juan", "Antonio", "del Carmen de la Vega")), + ("María del Carmen y", "O7", ("María", "", "del Carmen y"), ("María", "", "del Carmen y")), + ("Ali del Carmen oglu", "turkic+O7", ("Ali", "", "del Carmen oglu"), ("Ali", "", "del Carmen oglu")), + # ... and a given name of two words, which O7 would not complete, + # leaves O6 the phrase (the second #1057 review) + ("María-José del Carmen de la Vega", "O6", ("María-José", "", "del Carmen de la Vega"), ("María-José", "", "del Carmen de la Vega")), + ("abdul Rahman del Carmen de la Vega", "O6", ("abdul Rahman", "", "del Carmen de la Vega"), ("abdul Rahman", "", "del Carmen de la Vega")), + # the #1057 test review's rows: a shared word as the given name (the + # family-first-given-last given name is the last word), a family + # written before the given name, the given name's own guards on the + # phrase path, and a bound pair whose bound word is also a particle, + # which O6's count passes over ('Abu Bakr', where 'abdul' is none) + ("Dhillon Harpreet Kaur", "family-first-given-last+O7", ("Kaur", "Harpreet", "Dhillon"), ("Harpreet Kaur", "", "Dhillon")), + ("Kaur Harpreet Dhillon", "O7", ("Kaur", "Harpreet", "Dhillon"), ("Kaur Harpreet", "", "Dhillon")), + ("de los Ángeles López María", "family-first+both", ("María", "", "de los Ángeles López"), ("María", "", "de los Ángeles López")), + ("3 del Carmen García", "O7", ("3", "", "del Carmen García"), ("3", "", "del Carmen García")), + ("María del Carmen 3", "O7", ("María", "", "del Carmen 3"), ("María", "", "del Carmen 3")), + ("bin del Carmen García", "O7", ("bin", "", "del Carmen García"), ("bin", "", "del Carmen García")), + ("Kaur del Carmen García", "O7", ("Kaur", "", "del Carmen García"), ("Kaur", "", "del Carmen García")), + ("Abu Bakr del Carmen García", "O7", ("Abu Bakr", "", "del Carmen García"), ("Abu Bakr", "", "del Carmen García")), + ("Abu Bakr del Carmen de la Vega", "O6", ("Abu Bakr", "", "del Carmen de la Vega"), ("Abu Bakr", "", "del Carmen de la Vega")), + ("Abu Bakr Carlos García", "O7", ("Abu Bakr", "Carlos", "García"), ("Abu Bakr", "Carlos", "García")), +] + + +@pytest.mark.parametrize("text,policy,shipped,bare", _EFFECT_704, + ids=[f"{r[1]}:{r[0]}" for r in _EFFECT_704]) +def test_the_two_name_switches_read_particle_given_and_shared_names( + text: str, policy: str, shipped: tuple[str, ...], + bare: tuple[str, ...]) -> None: + assert _roles(text, _POLICIES_704[policy], Lexicon.default()) == shipped + assert _roles(text, _POLICIES_704[policy], _BARE_704) == bare + + +def test_the_704_table_counts_its_rows() -> None: + # the table is the properties' negative control only while some rows + # move without them and some do not + moved = [r for r in _EFFECT_704 if r[2] != r[3]] + assert len(moved) == 12 and len(_EFFECT_704) - len(moved) == 23 + + +def test_a_callers_reading_reaches_the_switches() -> None: + # a caller's add() states the property as a file does (3.0's 4i-b) + o7 = Policy(two_given_names=True) + mine = (Lexicon.default().add("zorba", kind="given", after_particle=True) + .add("zeta", kind="family", shared=True)) + assert _roles("Ana del Zorba García", o7, mine) == ( + "Ana del Zorba", "", "García") + assert _roles("Ana del Zorba García", o7, Lexicon.default()) == ( + "Ana", "", "del Zorba García") + assert _roles("Ana Zeta García", o7, mine) == ("Ana", "Zeta", "García") + assert _roles("Ana Zeta García", o7, Lexicon.default()) == ( + "Ana Zeta", "", "García") + # shared on a given reading too, a name reading of either kind + given_shared = Lexicon.default().add("zeta", kind="given", shared=True) + assert _roles("Ana Zeta García", o7, given_shared) == ( + "Ana", "Zeta", "García") + # and both properties survive a pickle round trip + loaded = pickle.loads(pickle.dumps(mine)) + assert _roles("Ana del Zorba García", o7, loaded) == ( + "Ana del Zorba", "", "García") + assert _roles("Ana Zeta García", o7, loaded) == ("Ana", "Zeta", "García") + + +def test_a_strict_choice_keeps_the_704_readings() -> None: + # kaur and singh stand in mul.toml, which every choice reads, and the + # Spanish words in es.toml, so a strict Spanish parser, whose pack + # turns two_family_names on, keeps both (the #1057 review found the + # first, in pa.toml, read family 'Kaur Dhillon' there) + strict_es = parser_for(locales.ES, strict=True) + name = strict_es.parse("Harpreet Kaur Dhillon") + assert (name.middle, name.family) == ("Kaur", "Dhillon") + with_o7 = parser_for(locales.ES, Locale( + code="x", lexicon=Lexicon.empty(), + policy=PolicyPatch(two_given_names=True)), strict=True) + assert with_o7.parse("María del Carmen García").given == "María del Carmen" + + +def test_the_704_tags_reach_no_published_token() -> None: + # SHARED_NAME_TAG and PARTICLE_GIVEN_TAG serve the post rules only, + # dropped at assemble like the kind tags (_assemble._KIND_TAGS) + names = [parse("Harpreet Kaur Dhillon"), + Parser(policy=Policy(two_given_names=True)).parse( + "María del Carmen García")] + published = {tag for n in names for t in n.tokens for tag in t.tags} + assert not published & {"vocab:shared-name", "vocab:particle-given"} diff --git a/tests/v2/rules_budget.json b/tests/v2/rules_budget.json index 9971f4ea..931c1a87 100644 --- a/tests/v2/rules_budget.json +++ b/tests/v2/rules_budget.json @@ -39,10 +39,10 @@ "O3": {"words": 47, "then_words": 18, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "O4": {"words": 91, "then_words": 57, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "O5": {"words": 478, "then_words": 160, "because_sentences": 2, "except": 6, "precedence": 1, "accepted": 0, "accepted_words": 0}, - "O6": {"words": 752, "then_words": 242, "because_sentences": 2, "except": 5, "precedence": 4, "accepted": 8, "accepted_words": 321}, - "O7": {"words": 633, "then_words": 175, "because_sentences": 2, "except": 4, "precedence": 3, "accepted": 6, "accepted_words": 220}, + "O6": {"words": 870, "then_words": 242, "because_sentences": 2, "except": 7, "precedence": 4, "accepted": 8, "accepted_words": 321}, + "O7": {"words": 792, "then_words": 249, "because_sentences": 2, "except": 5, "precedence": 3, "accepted": 7, "accepted_words": 273}, "P1": {"words": 415, "then_words": 287, "because_sentences": 2, "except": 0, "precedence": 1, "accepted": 4, "accepted_words": 319}, - "P2": {"words": 344, "then_words": 308, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 3, "accepted_words": 167}, + "P2": {"words": 378, "then_words": 342, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 3, "accepted_words": 167}, "P3": {"words": 639, "then_words": 598, "because_sentences": 1, "except": 0, "precedence": 0, "accepted": 0, "accepted_words": 0}, "P4": {"words": 190, "then_words": 82, "because_sentences": 1, "except": 1, "precedence": 0, "accepted": 0, "accepted_words": 0}, "P5": {"words": 717, "then_words": 399, "because_sentences": 1, "except": 2, "precedence": 3, "accepted": 3, "accepted_words": 192}, diff --git a/tests/v2/test_lexicon.py b/tests/v2/test_lexicon.py index 28498488..f2d85af2 100644 --- a/tests/v2/test_lexicon.py +++ b/tests/v2/test_lexicon.py @@ -744,7 +744,8 @@ def test_a_derived_marker_follows_from_the_readings() -> None: (lambda: Lexicon.empty().add("x", kind="title", default=False), ValueError, "a caller's reading is on"), (lambda: Lexicon.empty().add("x", kind="title", cased="X"), ValueError, "capitalization_exceptions"), (lambda: Lexicon.empty().add("x", kind="title", acronym=True), ValueError, "a title reading takes abbreviation, addresses, after_given, attaches"), - (lambda: Lexicon.empty().add("x", kind="family", same_as="y"), ValueError, "a family reading takes no properties"), + (lambda: Lexicon.empty().add("x", kind="family", same_as="y"), ValueError, "a family reading takes shared; got 'same_as'"), + (lambda: Lexicon.empty().add("x", kind="family", after_particle=True), ValueError, "only a given reading takes it"), (lambda: Lexicon.empty().add("x", kind="title", after_given="yes"), TypeError, "after_given must be True or False"), (lambda: Lexicon.empty().add("x", kind="title", abbreviation="yes"), TypeError, "abbreviation must be True or False"), (lambda: Lexicon.empty().add("x", kind="title", addresses="first"), ValueError, "addresses must be 'given' or 'family'"), @@ -1894,7 +1895,8 @@ def test_migrate_s_table_covers_every_compiled_constant() -> None: from nameparser.config import _compiled public_in_2x = {n for n in vars(_compiled) if n.isupper()} - { "GENERATION_WORDS", "CREDENTIAL_WORDS", "HONORIFIC_WORDS", - "PATRONYMIC_PARTICLES", "TITLE_ABBREVIATIONS", "TITLE_CASE_NAME_WORDS"} + "PATRONYMIC_PARTICLES", "TITLE_ABBREVIATIONS", "TITLE_CASE_NAME_WORDS", + "SHARED_NAMES", "PARTICLE_GIVEN_NAMES"} assert public_in_2x <= named assert len(_migrate_constant_rows()) >= 20 # the rows with code # the one row naming two constants, which the row pattern skips diff --git a/tests/v2/test_reading_index.py b/tests/v2/test_reading_index.py index d7ae7156..64af1cbd 100644 --- a/tests/v2/test_reading_index.py +++ b/tests/v2/test_reading_index.py @@ -63,7 +63,8 @@ def test_the_index_compiled_with_no_choice_is_the_committed_module() -> None: _SET = {"abbreviation": True, "addresses": "given", "after_given": True, "attaches": ["office"], "cased": "ZzQ", "acronym": None, "attached": {"side": "after"}, - "may_be_initial": True, "title_case_is_name": True} + "may_be_initial": True, "title_case_is_name": True, + "shared": True, "after_particle": True} def _probe_readings() -> list[Any]: diff --git a/tests/v2/test_vocab.py b/tests/v2/test_vocab.py index 760df717..87fcfdb9 100644 --- a/tests/v2/test_vocab.py +++ b/tests/v2/test_vocab.py @@ -506,6 +506,12 @@ def _w(body: str) -> str: ("same-as-on-a-family-reading", {"en.toml": _w('[[word]]\ntext = "smyth"\nfamily = { note = "x", same_as = "smith" }')}, "only a `given` reading takes one"), ("same-as-on-a-given-reading-unnormalized", {"en.toml": _w('[[word]]\ntext = "chas"\ngiven = { note = "x", same_as = "Charles" }')}, "not in normalized form; write 'charles'"), ("same-as-on-a-given-reading-names-itself", {"en.toml": _w('[[word]]\ntext = "chas"\ngiven = { note = "x", same_as = "chas" }')}, "names the word itself"), + # 3, the two name-reading properties the two-name switches read + # (#704): each is true or unwritten, and after_particle names a + # given name, so only a given reading takes it + ("shared-not-true", {"pa.toml": _w('[[word]]\ntext = "kaur"\nfamily = { note = "x", shared = false }')}, "shared must be true, or left unwritten"), + ("after-particle-not-true", {"es.toml": _w('[[word]]\ntext = "carmen"\ngiven = { note = "x", after_particle = 1 }')}, "after_particle must be true, or left unwritten"), + ("after-particle-on-a-family-reading", {"es.toml": _w('[[word]]\ntext = "carmen"\nfamily = { note = "x", after_particle = true }')}, "only a `given` reading takes one"), ] @@ -1099,6 +1105,8 @@ def _sets_fed(kind: str, props: str = "{}", text: str = "zzq", "abbreviation": ("military", "{ abbreviation = true }", "zzq"), "default": ("generation", "{ default = false }", "zzq"), "title_case_is_name": ("generation", "{ title_case_is_name = true }", "zzq"), + "shared": ("family", '{ shared = true, note = "probe" }', "zzq"), + "after_particle": ("given", '{ after_particle = true, note = "probe" }', "zzq"), # a phrase needs the head it attaches to: a second title word "attaches": ("office", '{ attaches = ["occupation"] }', "zzq", '\n\n[[word]]\ntext = "zzr"\noccupation = {}'), diff --git a/tools/differential/corpus_rules.jsonl b/tools/differential/corpus_rules.jsonl index 441f850b..f2ee28bb 100644 --- a/tools/differential/corpus_rules.jsonl +++ b/tools/differential/corpus_rules.jsonl @@ -38,6 +38,7 @@ "Abdul Rahman bin Ahmad" "Abu Bakar" "Abu Bakar Salim" +"Abu Bakr Carlos García" "Ahmad al-ʿAbbas" "Ahmad bin Abdul Rahman" "Ahmad bin Ali bin Hassan" @@ -279,6 +280,7 @@ "García Márquez, MJ JK" "García Márquez, MJ PhD" "García Márquez, Ms G.J." +"García del Valle María" "Gen Goto" "Gen Pinochet" "General Major John Smith" @@ -293,9 +295,12 @@ "Greve Anna" "Greve Smith, Anna" "Greve, Anna" +"Gurpreet Singh Sandhu" "Guy Andre Junior Francois" "Gyani Zail Singh" "Hans „Erster“ und “Zweiter” Müller" +"Harpreet Kaur Dhillon" +"Harpreet Simran Kaur" "Hassan Mohamad Ali" "Hassan Mohamad Ali Omar" "Hassan, Mohamad Ahmad Ali" @@ -712,6 +717,8 @@ "Jr." "Juan & Garcia" "Juan 3 Pérez" +"Juan Antonio del Carmen de la Vega" +"Juan Antonio del Valle de la Vega" "Juan Carlos" "Juan Carlos Pérez" "Juan Carlos Pérez née López" @@ -783,6 +790,7 @@ "Maria Kowalska z domu Nowak" "Maria Luisa y de la Cruz" "Maria López née García Pérez" +"Maria da Silva Santos" "Maria do Carmo Silva" "Maria do Carmo da Silva" "Marie FILS" @@ -801,6 +809,10 @@ "Mary Smith née Jones Prof." "Mary-​Jane Smith" "María José García López" +"María de los Ángeles López Pérez" +"María del Carmen" +"María del Carmen García" +"María del Carmen García López" "María del Carmen de la Vega" "María-José Ana López" "Mc Donald" diff --git a/tools/differential/snapshot.jsonl b/tools/differential/snapshot.jsonl index 28867589..e53714e6 100644 --- a/tools/differential/snapshot.jsonl +++ b/tools/differential/snapshot.jsonl @@ -73,6 +73,7 @@ {"name": "Abu Bakar", "given": "Abu", "family": "Bakar", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: particle as family (vocabulary)"], "initials": "A. B."} {"name": "Abu Bakar Salim", "given": "Abu Bakar", "family": "Salim", "initials": "A. B. S."} {"name": "Abu Bakr Al Baghdadi, MD", "given": "Abu Bakr", "family": "Al Baghdadi", "suffix": "MD", "initials": "A. B. B.", "capitalized_force": "Abu Bakr al Baghdadi, MD"} +{"name": "Abu Bakr Carlos García", "given": "Abu Bakr", "middle": "Carlos", "family": "García", "initials": "A. B. C. G."} {"name": "Abu Karim", "given": "Abu", "family": "Karim", "ambiguities": ["PARTICLE_OR_GIVEN"], "alternatives": ["PARTICLE_OR_GIVEN: particle as family (vocabulary)"], "initials": "A. K."} {"name": "Abu-'l-Fidāʾ Ismāʿīl Ibn-ʿAlī", "given": "Abu-'l-Fidāʾ", "middle": "Ismāʿīl", "family": "Ibn-ʿAlī", "initials": "A. I. I.", "capitalized_force": "Abu-'L-Fidāʾ Ismāʿīl ibn-ʿAlī"} {"name": "Abu-'l-Qāsim Maslama Ibn-Aḥmad al-Maǧrīṭī", "given": "Abu-'l-Qāsim", "middle": "Maslama Ibn-Aḥmad", "family": "al-Maǧrīṭī", "initials": "A. M. I. a.", "capitalized_force": "Abu-'L-Qāsim Maslama ibn-Aḥmad al-Maǧrīṭī"} @@ -1164,6 +1165,7 @@ {"name": "García Márquez, MJ PhD", "given": "MJ", "family": "García Márquez", "suffix": "PhD", "initials": "M. G. M.", "capitalized_force": "Mj García Márquez PhD"} {"name": "García Márquez, Ms G.J.", "title": "Ms", "given": "G.J.", "family": "García Márquez", "initials": "G. G. M."} {"name": "García Márquez, PhD G.J.", "given": "García", "family": "Márquez", "suffix": "PhD G.J.", "initials": "G. M."} +{"name": "García del Valle María", "given": "García", "family": "del Valle María", "initials": "G. V. M."} {"name": "García y Bellido, Antonio", "given": "Antonio", "family": "García y Bellido", "initials": "A. G. B."} {"name": "Gardner, Augustus P.", "given": "Augustus", "middle": "P.", "family": "Gardner", "initials": "A. P. G."} {"name": "Gary Strawsburg, PMP, CSCP, CLSSBB", "given": "Gary", "family": "Strawsburg", "suffix": "PMP, CSCP, CLSSBB", "ambiguities": ["COMMA_STRUCTURE"], "initials": "G. S.", "capitalized_force": "Gary Strawsburg, PMP, CSCP, Clssbb"} @@ -1243,6 +1245,7 @@ {"name": "Gurbanow, Meretgeldi Durdyýewiç", "given": "Meretgeldi", "middle": "Durdyýewiç", "family": "Gurbanow", "initials": "M. D. G."} {"name": "Gurdow Atajan Saparowiç, magistr, ylymlaryň kandidaty", "order": "FAMILY_FIRST", "given": "magistr", "family": "Gurdow Atajan Saparowiç", "suffix": "ylymlaryň kandidaty", "ambiguities": ["COMMA_STRUCTURE"], "initials": "m. G. A. S.", "capitalized_force": "Magistr Gurdow Atajan Saparowiç Ylymlaryň Kandidaty"} {"name": "Gurdow, Atajan Saparowiç", "given": "Atajan", "middle": "Saparowiç", "family": "Gurdow", "initials": "A. S. G."} +{"name": "Gurpreet Singh Sandhu", "given": "Gurpreet", "middle": "Singh", "family": "Sandhu", "initials": "G. S. S."} {"name": "Guru Gobind Singh", "title": "Guru", "given": "Gobind", "family": "Singh", "initials": "G. S."} {"name": "Guru Nanak", "title": "Guru", "given": "Nanak", "initials": "N."} {"name": "Gustavo León González", "given": "Gustavo", "middle": "León", "family": "González", "initials": "G. L. G."} @@ -1295,6 +1298,7 @@ {"name": "Harietta Keopuolani Nahi'ena'ena", "given": "Harietta", "middle": "Keopuolani", "family": "Nahi'ena'ena", "initials": "H. K. N.", "capitalized_force": "Harietta Keopuolani Nahi'Ena'Ena"} {"name": "Harms, Claus", "given": "Claus", "family": "Harms", "initials": "C. H."} {"name": "Harpreet Kaur Dhillon", "given": "Harpreet", "middle": "Kaur", "family": "Dhillon", "initials": "H. K. D."} +{"name": "Harpreet Simran Kaur", "given": "Harpreet", "middle": "Simran", "family": "Kaur", "initials": "H. S. K."} {"name": "Harrieta Keōpūolani Nāhiʻenaʻena", "given": "Harrieta", "middle": "Keōpūolani", "family": "Nāhiʻenaʻena", "initials": "H. K. N."} {"name": "Hasani Maria", "order": "FAMILY_FIRST", "given": "Maria", "family": "Hasani", "initials": "M. H."} {"name": "Hasani, Maria", "given": "Maria", "family": "Hasani", "initials": "M. H."} @@ -2154,6 +2158,8 @@ {"name": "Juan Anglada y Ruiz", "given": "Juan", "family": "Anglada y Ruiz", "initials": "J. A. R."} {"name": "Juan Antonio Rascón Navarro Seña y Redondo", "given": "Juan", "middle": "Antonio Rascón Navarro", "family": "Seña y Redondo", "initials": "J. A. R. N. S. R."} {"name": "Juan Antonio de los Heros Fernández", "given": "Juan", "middle": "Antonio", "family": "de los Heros Fernández", "initials": "J. A. H. F."} +{"name": "Juan Antonio del Carmen de la Vega", "given": "Juan", "middle": "Antonio del Carmen", "family": "de la Vega", "initials": "J. A. C. V."} +{"name": "Juan Antonio del Valle de la Vega", "given": "Juan", "middle": "Antonio del Valle", "family": "de la Vega", "initials": "J. A. V. V."} {"name": "Juan C. Garaikoetxea Urriza", "given": "Juan", "middle": "C. Garaikoetxea", "family": "Urriza", "initials": "J. C. G. U."} {"name": "Juan Carlos", "given": "Juan", "family": "Carlos", "initials": "J. C."} {"name": "Juan Carlos Pérez", "given": "Juan", "middle": "Carlos", "family": "Pérez", "initials": "J. C. P."} @@ -2596,6 +2602,7 @@ {"name": "Maria Popescu", "given": "Maria", "family": "Popescu", "initials": "M. P."} {"name": "Maria Silva", "given": "Maria", "family": "Silva", "initials": "M. S."} {"name": "Maria Wijaya", "given": "Maria", "family": "Wijaya", "initials": "M. W."} +{"name": "Maria da Silva Santos", "given": "Maria", "family": "da Silva Santos", "initials": "M. S. S."} {"name": "Maria do Carmo Silva", "given": "Maria", "family": "do Carmo Silva", "initials": "M. C. S."} {"name": "Maria do Carmo da Silva", "given": "Maria", "middle": "do Carmo", "family": "da Silva", "initials": "M. C. S."} {"name": "Mariam Geedi Hamish Hussein", "given": "Mariam", "middle": "Geedi Hamish", "family": "Hussein", "initials": "M. G. H. H."} @@ -2657,6 +2664,10 @@ {"name": "María Paz Alonso Romero", "given": "María", "middle": "Paz Alonso", "family": "Romero", "initials": "M. P. A. R."} {"name": "María Rubio Tejero", "given": "María", "middle": "Rubio", "family": "Tejero", "initials": "M. R. T."} {"name": "María de la O Lejárraga y García", "given": "María", "family": "de la O Lejárraga y García", "initials": "M. O. L. G."} +{"name": "María de los Ángeles López Pérez", "given": "María", "family": "de los Ángeles López Pérez", "initials": "M. Á. L. P."} +{"name": "María del Carmen", "given": "María", "family": "del Carmen", "initials": "M. C."} +{"name": "María del Carmen García", "given": "María", "family": "del Carmen García", "initials": "M. C. G."} +{"name": "María del Carmen García López", "given": "María", "family": "del Carmen García López", "initials": "M. C. G. L."} {"name": "María del Carmen de la Vega", "given": "María", "middle": "del Carmen", "family": "de la Vega", "initials": "M. C. V."} {"name": "María del Pilar Hurtado Montaño", "given": "María", "family": "del Pilar Hurtado Montaño", "initials": "M. P. H. M."} {"name": "María-José Ana López", "given": "María-José", "middle": "Ana", "family": "López", "initials": "M. A. L."} diff --git a/tools/vocab/build.py b/tools/vocab/build.py index bc806ca2..d0dee40c 100644 --- a/tools/vocab/build.py +++ b/tools/vocab/build.py @@ -111,7 +111,8 @@ check: a flag cannot exist without its reading, and a derived marker is read off the reading it marks. The one soft finding, a name reading on a word with no vocabulary reading and no exclusion -(save an abbreviated given name, which compiles), is +(save an abbreviated given name, a case mask, and a `shared` or +`after_particle` flag, each of which compiles), is a VocabWarning, which the shipped-data test turns into an error. """ from __future__ import annotations @@ -236,8 +237,9 @@ def _alnum(text: str) -> str: UNORDERED = "und.toml" #: Behavioral properties per category, each compiling into a field: -#: the compiler's, in nameparser/_reading_index.py. A name reading's one -#: is `default` (3a); its other keys are NAME_KEYS. +#: the compiler's, in nameparser/_reading_index.py. A name reading's +#: are `default` (3a), `cased`, `shared` and `after_particle`; its other +#: keys are NAME_KEYS. BEHAVIORAL = _ri.BEHAVIORAL #: Stated properties per category: compiled to nothing, but held to the #: word's readings (check 11). A particle states `may_be_given = false` @@ -854,6 +856,15 @@ def _check_reading(r: Reading, pool: Pool) -> None: if k not in allowed_here: raise _reserved_or_unknown(where, k, allowed_here) _check_default(where, p) + for flag in ("shared", "after_particle"): + if flag in p and p[flag] is not True: + # unwritten is false, so only true says anything + raise VocabError(f"{where}: {flag} must be true, or left " + f"unwritten") + if "after_particle" in p and r.kind != "given": + raise VocabError(f"{where}: after_particle names a given name " + f"written behind a particle (rules/O7), so " + f"only a `given` reading takes one") for src in COUNT_SOURCES & p.keys(): if not isinstance(p[src], int) or isinstance(p[src], bool) or p[src] < 1: raise VocabError(f"{where}: {src} must be a positive count " @@ -1289,8 +1300,8 @@ def check_pool(pool: Pool) -> None: check_order(pool.entries) # soft: a name reading compiles to nothing save a `same_as` (an - # abbreviated given name) or a `cased` (a case mask), so one without - # either on a word with no vocabulary reading and no exclusion + # abbreviated given name), a `cased` (a case mask) or a `shared` or + # `after_particle` flag (#704), so one with none of them on a word with no vocabulary reading and no exclusion # anywhere -- an exclusion's evidence is what a name reading usually # is -- informs nothing excluded_texts = {x.text for x in pool.exclusions if x.prop is None} @@ -1298,7 +1309,8 @@ def check_pool(pool: Pool) -> None: # an abbreviated given name compiles (GIVEN_NAME_ABBREVIATIONS), # and a name's case mask (CAPITALIZATION_EXCEPTIONS, #1014) if (all(r.category == "name" and "same_as" not in r.props - and "cased" not in r.props for r in rs) + and "cased" not in r.props and "shared" not in r.props + and "after_particle" not in r.props for r in rs) and text not in excluded_texts): warnings.warn(f"{rs[0].where()}: a name reading on a word with no " f"vocabulary reading is inert", VocabWarning, stacklevel=2) @@ -1313,7 +1325,7 @@ def _behavior(r: Reading, prop: str) -> object: att = r.props.get("attached") return None if att is None else att.get("side") if prop in ("may_be_initial", "after_given", "abbreviation", - "title_case_is_name"): + "title_case_is_name", "shared", "after_particle"): return r.flag(prop) if prop == "attaches": att = r.props.get("attaches") @@ -1354,6 +1366,8 @@ def _behavior(r: Reading, prop: str) -> object: "PATRONYMIC_PARTICLES", "TITLE_ABBREVIATIONS", "TITLE_CASE_NAME_WORDS", + "SHARED_NAMES", + "PARTICLE_GIVEN_NAMES", )