Repository navigation
fix(verify-layout): find an orphaned entry header by where the text starts - #508
Conversation
…tarts find_orphans() compared each line's left edge with the document margin: an un-indented last line followed by an indented first line was an entry header orphaned from its bullets. A list marker moves a line's left edge without moving its text, so the rule misread two templates: * one that merges each bullet's marker into its first line (MadsLorentzen#481): a bullet wrapping across the break was reported as an orphaned header, and a real orphaned header followed by such bullets was not reported; * the stock CV, when its outer marker extracts as a line of its own: a [Job Title] header left at the foot of a page, bullets overleaf, was not reported (its text at x70.4 counts as indented), and a one-line list item whose outer marker is set 1.3pt below it - so the marker sorts last on the page - was reported as a split bullet although the whole item is on that page. Following discussion MadsLorentzen#481, the rule now reads text x: * parse_pdf records where a line's text starts after leading LIST_MARKERS words (Line.text_left, defaulted). The set is the agreed Unicode bullets without the dashes (`• ‣ ▪ ● ·`): a bullet's second line may open with an em dash, and the stock Honors item's second line opens with an en dash. Other glyphs are text, so a line that merges the stock outer marker (`○` on some setups, an emoji on others) keeps the marker's x, as the left-edge rule read it; * find_orphans compares the text x of a page's last printed line with that of the next page's first printed line, and reports an orphaned header when the latter is more than INDENT_PT further in. Bbox lines whose tops are within SAME_PRINTED_LINE_PT (3pt) are one printed line: pdftotext's rounded-yMin buckets split a line where a bold word sits 0.2pt off, or where a marker sits 1.3pt below its text. A line whose text starts more than SET_IN_LIMIT_PT (60pt) past the margin is set right or centred (a running header) and is skipped as the opener; * a page ending on a marker with no text beside it keeps the lone-marker report, with its condition and message unchanged. Stock cv/main_example.tex, \vspace* of 0-600pt in 8pt steps before four entries, 304 lualatex builds (and the same 304 with xelatex): this adds 15 reports, each a [Job Title] header at the foot of a page with its bullets overleaf; the other 289 builds give byte-identical output. The reported. A two-line outer item whose marker sorts last and that wraps across the break is no longer reported as a split bullet: it is read like the MadsLorentzen#481 wrapped bullet, which is expected clean. Tests: TestOrphansReadTextX (13; 7 fixes and the wrapped two-line item fail on master's logic, 5 pin behaviour master already had) and TestTextStart (4). On Python 3.10-3.14 `python -W error::SyntaxWarning -m unittest discover -s tests -t .` passes 520 tests (master 503); lint_skills.py, check_framework_version.py and security_guards.py OK.
ccfabde to
a76c7a8
Compare
|
Merged, thank you. This is the shape, and it held up under measurement. Reproduced on Windows (Poppler 24.04): both #481 fixtures behave exactly as stated. The false positive is now clean; the false negative now reports the orphan beside the thin-page note. 520 OK under My own sweep: The behaviour change you flagged (a two-line outer item wrapping across the break, no longer reported) did not come up at my positions. It is pinned by your test and named in the CHANGELOG, which is the right way to carry it. #481 is resolved by this. |
Implements the shape from discussion #481: the orphan rule reads where the text starts, not where the line starts.
Failing cases, on the real path
/applyStep 5b runspython tools/verify_layout.py <pdf>on whatever template is active. The rule compared line left edges against the document margin, and a list marker moves a line's left edge without moving its text.cv_orphan_false_positive.pdf(#481; a bullet wraps across the break)p1 ends on the un-indented line '• Laid a new cobbled path...' ... an entry header is orphaned from its bullets, exit 1cv_orphan_false_negative.pdf(#481; header on p1, first bullet on p2)p2 is the last page and 64% emptyp1 ends on 'Kitchen Gardener 1975-1980' while the text on p2 starts further in ('• Worked in the kitchen garden...'): an entry header is orphaned from its bulletscv/main_example.texas shippedOn the stock CV I swept a
\vspace*of 0-600pt in 8pt steps before four entries (Previous Role, the third role, both Education entries). That is 304 builds with lualatex and the same 304 with xelatex. On my TeX Live setup the outer\itemmarker extracts as a line of its own, 4.6pt above the title, and as🟤rather than○. So yes, the glyph depends on the font setup. There, master does not report a[Job Title]header at the foot of a page, because its text at x70.4 counts as indented. This PR reports 15 such builds, each a header at the foot of a page with its bullets overleaf. The other 289 builds give byte-identical output, and the xelatex sweep gives the same split.On the second stock shape, "the outer marker on its own line, which master currently misreports as a split bullet", here is how I read it; correct me if you meant something else. The misreport I can reproduce on my build comes from the outer marker being set 1.3pt below its item's text, so the marker sorts last on the page. The stock CV unchanged shows it with
\vspace*{440-470pt}before Core Competencies. Page 1 ends on a whole one-line item, page 2 opens with the next item, and master reportsp1 ends on a lone list marker whose text continues on p2. This PR reports nothing there. If you meant the entry-header shape, that one istest_stock_header_whose_outer_marker_is_its_own_line.Change
parse_pdfrecords where each line's text starts after leadingLIST_MARKERSwords (Line.text_left, defaulted, so aLinebuilt fromleftalone behaves as before). The set is the agreed Unicode one without the dashes:• ‣ ▪ ● ·.—stripped, the verify_layout's orphan rule assumes the stock template's bbox geometry (and can go silent on a real orphan) #481 false positive came back for a continuation— that rain ran off....-- [Event/Organization]).○on your build,🟤on mine) keeps the marker's x, 59.0, which is what the left-edge rule read. That is why a merged stock[Degree]line and its description stay reported, as they are today. This does depend on the marker extracting outside the set: a setup that mapped it to•would lose that case, as the separate-line case below already does.find_orphanscompares the text x of the page's last printed line with the text x of the next page's first printed line. It reports an orphaned header when the latter is more thanINDENT_PTfurther in: 59.5 -> 71.6 in the verify_layout's orphan rule assumes the stock template's bbox geometry (and can go silent on a real orphan) #481 repro, 70.4 -> 81.3 on stock, and about 11-25pt in every orphan I found.SAME_PRINTED_LINE_PT = 3.0count as one printed line. pdftotext's rounded-yMin buckets split a printed line where a bold word sits 0.2pt off (721.44 and 721.64 fall on either side of a .5 boundary), or where a marker sits 1.3pt below its text. Consecutive lines are about 12pt apart.SET_IN_LIMIT_PT = 60past the margin is set right or centred, for example a fancyhdr running header at x459.8. It is skipped as the next page's opener. Without that, a running header counted as "further in" on every break.find_orphanssay that the rule reads text x, and why.CHANGELOGentry under[Unreleased]/### Fixed.Not solved
[Degree]line and its description are level on text x (both 70.4), so neither master nor this PR reports them. As you said, telling that from a wrapped line needs font information. I tried a column-gap test (the header's dates column) and dropped it, because an\hfillyear on a list item, or a\textttword split into its own bbox line, set it off.is_headinghas no median to compare against. The verdict is right and master says nothing there, but the wording is off. I have only seen this with a\vspace*that leaves one line on a page.Tests
TestOrphansReadTextXhas 13 synthetic cases with the verify_layout's orphan rule assumes the stock template's bbox geometry (and can go silent on a real orphan) #481 and stock geometry.○merged in, the stock[Degree]/ description with○merged in, a printed line split by a bold word, a wrapped stock bullet, and a running header over a clean break.TestTextStart(4) checks that ASCII, the dashes and○are text.Rebased on v1.7.2.
python -W error::SyntaxWarning -m unittest discover -s tests -t .passes 520 tests (master 503) on Python 3.10, 3.11, 3.12, 3.13 and 3.14.lint_skills.py,check_framework_version.pyandsecurity_guards.pyare OK.