[sarankumar1325]

_sniff_dialect() sniffs the CSV dialect from the first line only. When that line holds more than one candidate delimiter, for example a semicolon-separated file with a column named Weight (kg, net) (common for Excel exports in European locales), csv.Sniffer has no row-to-row consistency to go on and picks the first delimiter in its preference order (,), so every cell is split wrongly and silently.

This change keeps the first-line sniff (and its quoted-field fallback) as is, and only when the first line contains two or more of the candidate delimiters, it also sniffs the larger sample. If the sample-based sniff succeeds and its delimiter splits the header into at least as many fields as the first-line delimiter, that dialect is used. Otherwise the first-line dialect is kept, which leaves ragged comma files with a : or ; inside a header field (e.g. Date,Time (hh:mm),Value,Note with a short row) as on main. Files whose header has a single candidate delimiter never read the sample, so behaviour there is unchanged.

Issue resolved by this Pull Request:
Resolves #4895

Checklist:

• Documentation has been updated, if necessary. (docstring of _sniff_dialect)
• Examples have been added, if necessary. (n/a)
• Tests have been added, if necessary.

Review evidence:

• Before: Product;Weight (kg, net);Price\nApple;0,2;1,50\nPear;0,35;2\n converts to ['Product;Weight (kg', 'net);Price', ''], ['Apple;0', '2;1', '50'], ...
• After: ['Product', 'Weight (kg, net)', 'Price'], ['Apple', '0,2', '1,50'], ['Pear', '0,35', '2'].
• Added test_header_with_second_delimiter_uses_following_rows with the reproducer from the issue (fails on main, passes with the change) and the mirrored case (comma file with ; in the header), which already worked and guards against regressions. Also added the ragged comma cases from review: Date,Time (hh:mm),Value,Note and id,ratio (a;b),x,y with short rows, which the first version of this PR split on : / ;. Both fail with the earlier condition and pass now. A third ragged case covers the path where the sample can't be sniffed.
• uv run pytest tests/test_backend_csv.py: 61 passed (includes the CSV reference-data e2e tests; no reference data changed).
• ruff format / ruff check clean on the changed files. ty check docling/backend/csv_backend.py reports one pre-existing warning on an unchanged line (172).
• I did not run the full test suite, only the CSV backend tests.

This fix was prepared with AI assistance; I reviewed the change and ran the checks above.

[github-actions]

✅ DCO Check Passed

Thanks @sarankumar1325, all your commits are properly signed off. 🎉

[mergify]

Merge Protections

🟢 Merge protection satisfied — ready to merge.

Show 1 satisfied protection

🟢 Enforce conventional commit

Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/

• title ~= ^(fix|feat|docs|style|refactor|perf|test|build|ci|chore|revert)(?:\(.+\))?(!)?:

[github-actions]

AI triage (advisory)

Checked commit b038452b29. The analysis read the diff only. It did not run code from this pull request.

The PR changes _sniff_dialect() in the CSV backend. When the header line has two or more candidate delimiters, the function now also sniffs the larger sample. It uses the sample dialect only if that delimiter splits the header into at least as many fields. The PR adds parametrized tests for this case.

CI safety: no concern found. A maintainer must still approve CI.

_{The ai:* labels are advisory. They do not approve CI and they do not add tests:full. A new push runs this check again.}

[github-actions]

AI first review (advisory): no blocker found

This review covers the full diff at f2fd877a5e. It read the code and did not run it.

The fix correctly disambiguates CSV dialects when the header contains more than one candidate delimiter by re-sniffing the full sample and preferring the result when it finds a delimiter present in the header. Logic, test coverage, and scope are all sound. No blocker found; the human reviewer should confirm that csv.Sniffer reliably picks the true delimiter from a multi-row sample for the two parametrized test cases (it does for these inputs because the data rows are consistent).

_{A maintainer makes the merge decision. This review never approves the pull request.}

[codecov]

Codecov Report

❌ Patch coverage is 62.50000% with 3 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
docling/backend/csv_backend.py	62.50%	2 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

[dolfim-ibm]

Thanks for this fix, @sarankumar1325! Semicolon files with a comma in a column name are very common in European Excel exports, so this matters a lot.

One blocker: ragged comma files whose header contains another candidate delimiter are now split on that delimiter. When rows miss trailing fields, the comma count varies, so the sniffer prefers a delimiter that occurs once per row, like the : in times:

Date,Time (hh:mm),Value,Note
2024-01-01,10:30,5,a
2024-01-02,11:45,6

On main this gives ['Date', 'Time (hh:mm)', 'Value', 'Note']. With this PR it gives ['Date,Time (hh', 'mm),Value,Note']. The same happens with ; in a header such as id,ratio (a;b),x,y.

See the inline suggestion for a fix. Please also add the case above to the parametrized test.

[dolfim-ibm]

Only switch when the sample delimiter splits the header into at least as many fields. This keeps the fix for the issue case and for Name;Size (w, h);Price, and the ragged comma files get the main output again.

Suggested change

	if sample_dialect.delimiter in head:
	if head.count(sample_dialect.delimiter) >= head.count(dialect.delimiter):

[sarankumar1325]

Thanks for catching that! I applied your suggestion, so the delimiter now only switches when the sample's delimiter splits the header into at least as many fields. I also added the ragged-comma cases with : and ; in the header to the parametrized test, plus one more case to cover the remaining lines. Ready for another look.