Skip to content

[client-v2] Review hasNext() logic in com.clickhouse.client.api.data_formats.internal.AbstractBinaryFormatReader #3365

Open

Description

@chernser

Description

Methods like com.clickhouse.client.api.data_formats.internal.AbstractBinaryFormatReader#hasNext should be idempotent.
Beside current readers may do too complex logic of checking end of stream

  • make all reader idempotent method - idempotent
  • add tests reading enough data from DB to verify work of the reader
  • make sure no extra work is done for next operation.

Update:
Two failing tests:

[ERROR] Failures: 
[ERROR]   NativeFormatReaderDataSetTest>AbstractAccessorDataSetTest.testReadDataSet:118->AbstractBinaryReaderDataSetTest.readAndVerify:81 More rows than in the data set: 12 expected [true] but found [false]
[ERROR]   NativeFormatReaderDataSetTest>AbstractAccessorDataSetTest.testReadDataSet:118->AbstractBinaryReaderDataSetTest.readAndVerify:81 More rows than in the data set: 17 expected [true] but found [false]

Activity

  1. added this to the 0.12.0 milestone on Nov 17, 2026
  2. added theissue type on Nov 17, 2026
  3. chernser commented on Nov 17, 2026

    @chernser
    ContributorAuthor

    @polyglotAI-bot

    take this issue and first of all fix the issues with NativeFormat. Add enough docs to code to explain how it work.

  4. polyglotAI-bot commented on Nov 17, 2026

    @polyglotAI-bot
    Collaborator

    PR #3367 fixes the Native reader part of this issue. The two NativeFormatReaderDataSetTest failures had this cause: the reader decodes Array(Array(Int8)) and Map(String, Array(Nullable(Float64))) with the RowBinary layout, so it loses its position in the stream. It then read the leftover bytes as a block with 0 columns and N rows, and returned those rows. The reader now checks each block against the header of the first block. It sets the schema once and uses one method for hasNext() / readRecord(...). The Javadoc explains how the reader works. The PR does not change the other readers.

  5. added
    planned_workWork that is planned for nearest release. Good to take
    on Nov 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:client-readhandling data read in clientclient-api-v2planned_workWork that is planned for nearest release. Good to take

    Type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions