Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,7 +223,7 @@ The compiled sets, read by `Lexicon` (mostly as the `_sets` field of the same na
- particles — `PARTICLES` (family-name particles, e.g. "de", "van") and `NON_GIVEN_NAME_PARTICLES`, the curated subset that is *never* a standalone given name (the particles whose word has no `given` reading, each stating `may_be_given = false`; a `given` reading on the word is what makes a particle possibly given, the phase 1 flag `may_be_given = true` refused since 2026-10-25). What the subset licenses is rules/P1's fold in `post_rules` (the "rule 1b" of older issue comments — decisions/legacy-rule-numbers maps the old numbers), and the fold has ONE site: the piece that OPENS the name. A lone never-given particle there, with at least one more name word behind it, starts the family. Titles do not move the opening position (rules/P4), so `Sir de Mesnil` reads family `de Mesnil` under every `name_order`, as `de Mesnil` does. How far the fold reaches depends on the order the name was read under: by default it takes the rest of the name (`de Mesnil Jean` → family `de Mesnil Jean`); under either family-first order it takes the particle run plus ONE name unit and lays the rest out by the order (`de Mesnil Jean` → family `de Mesnil`, given `Jean`); after a family comma it takes the rest of that part whatever the order (`Smith, de Mesnil Jean` → family `Smith de Mesnil Jean`). The leading particle stays a lone piece for the fold to find — `group` does not chain it forward, and the fold walks the particles after it itself — so `de la Vega` reads family `de la Vega` under all three orders. The GIVEN position is not a site. It was a second one until #467, which removed it because under a family-first order that slot holds what the caller DECLARED to be the given name: `Mesnil de` under `FAMILY_FIRST` or `FAMILY_FIRST_GIVEN_LAST` reads given `de`, family `Mesnil`. Nor is a particle inside the name: `Juan de la Vega` under a family-first order reads family `Juan`, given `de la Vega`, with no ambiguity recorded — what the vocabulary forbids is the bare particle as a given name, not a name part that begins with one. A bare `de` has nothing to fold into and is placed by position like any one-word name: given `de` by default, family `de` under both family-first orders. A leading particle OUTSIDE the set is left to position — `van Gogh` is given `van`, family `Gogh` by default and family `van`, given `Gogh` under both family-first orders — and records a `PARTICLE_OR_GIVEN` ambiguity under all three; a never-given particle opening the name records none (`de Mesnil`), and neither does an ambiguous one inside it (`Vincent van Gogh`). The internal `Lexicon._sets.particles_ambiguous`, the particles holding a `given` reading, is its complement within `PARTICLES`, so the two mark OPPOSITE sets — see the flip warning in `docs/migrate.rst` before translating either
- bound given names — `BOUND_GIVEN_NAMES` (`bound_given` readings, e.g. "abdul", "abu"; one that is also a particle needs a `given` reading, a build check); a group-stage rule joins the first non-title piece to its following piece before roles are assigned, reserving a family word unless a family comma has already fixed it or a given-name title stands ahead (#369, rules/P5) (ported from v1's `_join_bound_first_name`)
- given names written short — `GIVEN_NAME_ABBREVIATIONS` (#789, a `given` reading carrying `same_as`, the full name: "chas", "wm", Filipino "ma"), the given names written short that rules/H2's shape yields to; nothing else reads it
- names the two-name switches read — `PARTICLE_GIVEN_NAMES` (#704, a `given` reading written `after_particle`: "carmen", "ángeles", the Spanish words a compound given name writes behind a particle, `María del Carmen`) and `SHARED_NAMES` (#704, a name reading written `shared`: Sikh "kaur", "singh"), private; only rules/O7 and O6 read them, through classify's tags, so neither moves a parse with both switches off
- names a given name's second half reads — `PARTICLE_GIVEN_NAMES` (#704, a `given` reading written `after_particle`: "carmen", "ángeles", the Spanish words a compound given name writes behind a particle, which rules/P10 reads as a middle name in `María del Carmen García`, by default), `SAINT_PARTICLES` (the `saint` kind, `san`, `santa`, `st`, `ste`, whose run P10 refuses, `Santa María` being a surname) and `SHARED_NAMES` (#704, a name reading written `shared`: Sikh "kaur", "singh", which rules/O6 and O7 neither take nor complete and P10 does not complete, `Kaur del Carmen García`), private; each is read through classify's tags or the lexicon in `_post_rules`
- conjunctions — `CONJUNCTIONS` (e.g. "and", "of") used to chain multi-word titles and to join name parts (rules/P3), plus `CONJUNCTIONS_AMBIGUOUS` (#383/#479, rules/P8, `may_be_initial`, refused by the build on anything but a cased single letter), the subset of single cased letters that read as an INITIAL in a name written wholly in one case — a marker set
- maiden markers — `MAIDEN_MARKERS` (`birth_marker` readings, e.g. "née", "geb.") routing the following name to `maiden`; with an `addresses = "given"` title, the one kind a phrase may be (`z domu`)
- `config/surnames.py`, a public module — `KOREAN_SURNAMES`, the census list the 2.0 API splits unspaced hangul on (#271), still written in Python with an import-time `assert_normalized`: a segmentation list, outside the per-word model, like the locale packs' lists.
Expand Down
31 changes: 17 additions & 14 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1061,11 +1061,8 @@ one family has no ``surname2``, and its ``surname1`` is the family:

A family that already holds two names keeps what it has, whether a
hyphen or ``y`` joins them (``"Ana María García-López"``, ``"José
Ortega y Gasset"``) or a particle chain took them both: a compound
given name opening with a particle, such as ``"María del Carmen García
López"``, reads family ``del Carmen García López`` with the switch on
or off, until ``two_given_names`` below takes ``del Carmen`` into the
given name. A name of two words is unchanged, and a comma naming the family
Ortega y Gasset"``) or a particle chain took them both (``"Ana María de la Vega
Rodríguez"`` reads family ``de la Vega Rodríguez``). A name of two words is unchanged, and a comma naming the family
decides by itself, so ``"García López, Ana"`` needs no switch at all. Under a family-first ``name_order`` the *first* two
name words are the family.

Expand Down Expand Up @@ -1152,26 +1149,32 @@ two:
'Gabriel'

A given name a particle completes reads whole too. Where the
particle's words would otherwise run on into the family, the switch
takes them only if the vocabulary marks the word behind the particle
as a given name written there (``after_particle``). The Spanish ones
are marked, those that ORCID's researchers from Spain and
Spanish-speaking America write in the given box at least nine times
for each ten they also open a family box with it (``del
particle's words would otherwise run on into the family, the parser
reads them as a middle name, with or without the switch, if the
vocabulary marks the word behind the particle as a given name written
there (``after_particle``), and the switch then takes them into the
given name. The Spanish ones are marked, those that ORCID's researchers
from Spain and Spanish-speaking America write in the given box at least
nine times for each ten they also open a family box with it (``del
Carmen``, ``de los Ángeles``, ``del Mar``, ``del Pilar``: 23 words in
all on 2026-11-03). A phrase too often a surname to mark, as Portuguese ``do Carmo``
is, stays the family, the same shape as ``"João da Silva Santos"``:
all on 2026-11-03). A phrase too often a surname to mark, as Portuguese
``do Carmo`` is, stays the family, the same shape as ``"João da Silva
Santos"``, and so does one opening with a saint (``"Domingo Santa María
González"``):

.. doctest::

>>> parse("María del Carmen García").middle
'del Carmen'
>>> carmen = two_given.parse("María del Carmen García")
>>> carmen.given, carmen.family
('María del Carmen', 'García')
>>> two_given.parse("Maria do Carmo Silva").family
'do Carmo Silva'

To mark a word of your own, add it with ``Lexicon.default().add("carmo",
kind="given", after_particle=True)``. The two switches also leave alone
kind="given", after_particle=True)``, which moves the default reading
too. The two switches also leave alone
the Sikh names Kaur and Singh, given to every Sikh woman and man and
written after the personal name: ``"Harpreet Kaur Dhillon"`` keeps
middle ``Kaur`` under either switch, as it does by default.
Expand Down
4 changes: 2 additions & 2 deletions docs/design/decisions/O6-two-family-names.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
- The switch reports no alternative either, so the reading it did not take is not surfaced: the caller declared the convention — 2026-10-27 #752 (Derek).
- A listed given name (`Lexicon.given_names`) standing alone is not taken, under every order, and the name then reads as with the switch off; a listed word behind a particle is taken as before (unless written `after_particle` and standing given-first beside a given name of one word, the 2026-11-03 #704 entry), and with the list empty the switch reads as before; no other rule reads the list, the default carries none, and the veto reports nothing — 2026-10-30 #959.
- The veto stays a clause of this rule though it reads a second signal, the taken word looked up in a list, and the rule is a split candidate — 2026-10-30 #959, decided.
- Given-first, a particle run and the one word behind it whose `given` reading is written `after_particle` (rules/O7), standing beside a given name of one word, is not taken; under a family-first order it is, completing no given name there. Neither side of a join holding a name reading written `shared` (Sikh `kaur`, `singh`) is joined: each is no family name, and with the switch alone the name reads as with it off — 2026-11-03 #704, decided in chat.
- Given-first, a particle run and the one word behind it whose `given` reading is written `after_particle` (rules/P10), standing beside a given name of one word, is not taken; under a family-first order it is, completing no given name there. Neither side of a join holding a name reading written `shared` (Sikh `kaur`, `singh`) is joined: each is no family name, and with the switch alone the name reads as with it off — 2026-11-03 #704, decided in chat.
- The Spanish list, which `locales.ES` carries as its lexicon fragment since sub-phase 3g (a caller may pass a list of its own), is generated, not per-word vocabulary: INE's 2025 census tables, pooled over every name borne in Spain, cut at a measured rank (1,000 words, 90% given use) by `tools/es_given_names.py`; a change to it is a cutoff change — 2026-10-30 #959.

Open: why the Spanish list does less in Latin America, and whether a list per region or another selection rule would serve it better than the one list `locales.ES` carries (the 2026-10-30 #959 entry's Latin America bullet; the 2026-10-30 #753 phase 3g entry measures the pack against the default there) — #959. A name written with ñ dropped (`Begona`, `Inigo`) matches no entry, the key keeping the tilde; whether to fold ñ for data that strips it — #959.
Expand Down Expand Up @@ -76,7 +76,7 @@ Open: why the Spanish list does less in Latin America, and whether a list per re
- **The cutoff: 1,000 words** (84.3% of the given-word uses in INE's table). Measured per country with O6 on, the list cut at 100 to all 16,559 candidates; agreement in each of the six double-surname countries is within 0.3 points of the 2,000-word list at 1,000 (Venezuela the 0.3), and a band past it buys at most a tenth of a net fix per added word in those six (1,000 → 2,000: 134 fixed, 35 broken; 2,000 → 5,000: 128 fixed, 36 broken, Chile breaking as many as it fixes), where the first hundred words bought sixteen. Argentina gains 1.6 points more by 2,000 and 2.4 by all, which 3g weighs if it turns the switch on there.
- **Family-first, symmetric.** Under a family-first order the switch takes the unit after the family, and a listed one stays where it is: `Pérez José Luis` keeps family `Pérez`. No population the corpora hold writes Spanish names family-first with the fields stated, so it is measured on ORCID's boxes rendered family box first, which is a rendering, not anyone's writing. The cost moves with the order: a listed word that is a second surname, written after the first, becomes the given name under FAMILY_FIRST (`Flores Jorge Ana` gives given `Jorge`, middle `Ana`) and a middle name under FAMILY_FIRST_GIVEN_LAST, where given-first it is a first surname left as a middle name; on the rendering the list breaks 38 (Chile) to 205 (Peru) records a country, the family-first line below (found by the #961 review).
- **A bare word only.** The veto reads a unit that is one word with no particle; a listed word behind a particle is a surname unit and is taken as before (`Ana de Miguel de la Vega` keeps family `de Miguel de la Vega`), which is what the tool's count assumes in leaving such surnames out. The first draft read the word inside a particle unit too, and the design-docs review found it reading `Ana de Miguel de la Vega` middle `de Miguel`.
- **O7.** With both switches on, the word O6 leaves is a middle O7 takes where it stands beside the given name, so `José Luis Pérez` reads given `José Luis`, family `Pérez` (rules/O7's precedence); `Ana María Luis Pérez` keeps middle `Luis`, O7 taking `María`. Whether O7 reads the list itself is decisions/O7's Open question (declined 2026-11-03 #704: O7 reads a word written `after_particle` instead).
- **O7.** With both switches on, the word O6 leaves is a middle O7 takes where it stands beside the given name, so `José Luis Pérez` reads given `José Luis`, family `Pérez` (rules/O7's precedence); `Ana María Luis Pérez` keeps middle `Luis`, O7 taking `María`. Whether O7 reads the list itself is decisions/O7's Open question (declined 2026-11-03 #704: a word written `after_particle` is read instead, by O7 until decisions/P10's 2026-11-04 entry and by P10 since, O7 taking it as any middle unit).
- **No report**, as for the switch (the 2026-10-24 entry): the caller declared the convention and the list.
- **Latin America, measured in the #961 review.** Of the 27 words commonest among the Argentine records the default reads right and the switch with the list reads wrong, each record keyed by the last word of its given box (836 records), 722 are lost to the 90% bar measured on Spain's population and 114 to the 1,000 cut (a word at or over the bar ranked past 1,000), none to anything else: `Martín` (10% given in INE's tables), `Andrés` 77%, `Nicolás` 76%, `Sebastián` 81%, `Esteban` 36%, `Cecilia` 89.75% (84 Argentine records, just under the bar), `Matías` 76%, `Hernán` 63%, `Gastón` 40%, `Santiago` 57% and `Tomás` 61%, the shares recomputed here. Peru is net negative because the switch alone already agrees on 96.0% of its records, leaving little to fix, and its 189 breaks spread over the whole list, ranks 1 to 994 (`maria` 1st, `jose` 2nd, `chiara` 994th), so they come from the selection rule rather than the cutoff (the records the switch reads right and the switch with the list wrong, each keyed by the word the veto left, the switch's family word missing from the vetoed family, ranked in the list). Whether 3g needs a list per region or another selection rule is Open above; the list is not changed here. Both counts recomputed on the branch at e06b2612 with the review's recipes as stated, over the records of `orcid-researchers` with both boxes, read with `local_agreement.read`, words folded by `given_name_key`.
- **Pickling.** The field carries `_ADDED_IN` "3.0.0" and an empty default, so a `Lexicon`, `Parser` or locale pack pickled by 2.x loads with no list. That is what such a pickle meant, no pack filling the field before 3.0, so this field is the case decisions/A1's 2026-10-30 caveat does not reach: the silent drift it warns of comes from a marked field the default vocabulary fills, and the default leaves this one empty. `tests/v2/test_lexicon.py` loads the committed 2.8.0 pickle with the field empty, and refuses it with the marker taken off.
Expand Down
Loading
Loading