Skip to content

fix: normalize Nemotron formula delimiters before export - #4907

Open
PierreMesure wants to merge 1 commit into
docling-project:mainfrom
PierreMesure:fix/nemotron-formula-delimiters
Open

PierreMesure wants to merge 1 commit into
docling-project:mainfrom
PierreMesure:fix/nemotron-formula-delimiters

Conversation

@PierreMesure

Copy link
Copy Markdown

Nemotron formula responses such as \(x^2\) exported as $$\(x^2\)$$. Store the LaTeX body without its matching outer delimiters so the serializer supplies exactly one pair. Keep the original response in orig and preserve escapes, line breaks, and coordinates.

The normalizer is in shared VLM utilities. LightOnOCR PR #4789 now uses the same helper; the helper is identical in both branches.

Resolves #4903.

Checklist:

  • Documentation assessed: output correction; no API change.
  • Examples are included in regression tests.
  • Tests added for four delimiter styles, raw LaTeX, and multiline equations.

Review evidence:

  • Before the fix: 5 regression failures, with raw undelimited LaTeX already passing.
  • .venv/bin/python -m pytest -q tests/test_nemotron_formula_exports.py tests/test_nemotron_parse_v2.py: 14 passed on Python 3.12.
  • UV_NO_SYNC=1 make validate: all applicable hooks passed.
  • The existing semantic-class test now checks normalized formula text. No reference fixtures were regenerated.
  • Full Linux CI and model inference were not run locally.

Use a shared LaTeX delimiter normalizer and keep the raw response in orig. Verify all delimiter styles and multiline equations. Fixes docling-project#4903.

Signed-off-by: Pierre Mesure <pierre@mesu.re>
@github-actions

Copy link
Copy Markdown
Contributor

✅ DCO Check Passed

Thanks @PierreMesure, all your commits are properly signed off. 🎉

@mergify

mergify Bot commented Oct 19, 2026

Copy link
Copy Markdown
Contributor

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

  • title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

@github-actions github-actions Bot added ai:ci-needs-care AI triage: check CI-safety findings before you approve CI bug Something isn't working vlm-pipeline labels Oct 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

AI triage (advisory)

Checked commit 3bc0ecc63b. The analysis read the diff only. It did not run code from this pull request.

The PR adds strip_math_delimiters to the shared VLM utilities. parse_nemotron_parse_v2 now uses it to remove the outer math delimiters from Nemotron formula text, so the Markdown export has one $$ pair. The original response stays in orig.

Topic labels added: bug, vlm-pipeline.

CI safety: needs maintainer care before you approve CI.

  • tach.toml: changes CI test selection
2 related pull request(s) in the same code
PR State Signal Assessment
#4789 open referenced as #4789; changes the same code related: The PR adds the same strip_math_delimiters helper in vlm_utils.py. The two PRs share only this helper. #4789 adds LightOnOCR presets and parsing, and this PR fixes Nemotron formulas.
#4908 open changes the same code related: Both PRs edit parse_nemotron_parse_v2 and make the same tach.toml change. #4908 adds inline math handling for non-formula text and keeps formula text raw. This PR strips delimiters from formula items. The two PRs will probably conflict in the same else branch.

The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI first review (advisory): no blocker found

This review covers the full diff at 3bc0ecc63b. It read the code and did not run it.

The PR adds strip_math_delimiters in vlm_utils.py and calls it for FORMULA items in nemotron_parse_utils.py. The fix is correct, the delimiter priority order ($$ before $) is safe, the orig field keeps the raw text, and the regression tests cover all four delimiter styles plus multiline. The tach.toml dependency addition is accurate and introduces no cycle. No blockers found.

A maintainer makes the merge decision. This review never approves the pull request.

@github-actions github-actions Bot added the ai:review-lgtm AI first review found no blocker or major issue (advisory) label Oct 19, 2026
@codecov

codecov Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai:ci-needs-care AI triage: check CI-safety findings before you approve CI ai:review-lgtm AI first review found no blocker or major issue (advisory) bug Something isn't working vlm-pipeline

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Nemotron formula exports contain nested LaTeX delimiters

1 participant