Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make GenericRecord and ClickHouseBinaryFormatReader easy to iterate #2004

Merged
merged 2 commits into from
Dec 10, 2024

Conversation

abcfy2
Copy link
Contributor

@abcfy2 abcfy2 commented Dec 6, 2024

Summary

Add some useful methods to GenericRecord and ClickHouseBinaryFormatReader to make it easy to iterate.

Both GenericRecord and ClickHouseBinaryFormatReader could also get the column size, column names, and the full row values.

Use case:

Records records = client.queryRecords("select * from mytable limit 100").get();
records.forEach(record -> {
    // easy to get column size, then user can use record.getObject(<index>) to get a specified column value
    System.out.println(record.getSize());

    // easy to get column names, then user can use record.getObject(<column_name>) to get a specified column value
    System.out.println(record.getNames());

    // easy to get all row values in kv format
    System.out.println(record.getAll());
});

@abcfy2 abcfy2 force-pushed the main branch 2 times, most recently from e2242b6 to 74aef04 Compare December 6, 2024 10:05

@Override
public int getColumnCount() {
return currentRecord.size();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be returned from schema because it may be a virtual columns (for example, when we do so for tuples)


@Override
public Set<String> getColumnNames() {
return currentRecord.keySet();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should come from schema because it may have additional columns.


@Override
public Map<String, Object> current() {
return currentRecord;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cannot be exposed because it is internal records and should not be accessed in such way. I plan to change internal later.

@chernser
Copy link
Contributor

chernser commented Dec 6, 2024

@abcfy2 Thank you for contribution!
However we have com.clickhouse.client.api.data_formats.ClickHouseBinaryFormatReader#getSchema for purpose of getting metadata - it returns a lot more.

@abcfy2
Copy link
Contributor Author

abcfy2 commented Dec 7, 2024

@chernser Thanks, please check again. 😃 I've never thought to use TableSchema because I find all the records were put into a Map<String, Object> map. I've modified the PR. Please check again. thanks.

@abcfy2
Copy link
Contributor Author

abcfy2 commented Dec 7, 2024

So the new use case should be:

records.forEach(record -> {
    TableSchema schema = record.getSchema();
    // easy to get column size, then user can use record.getObject(<index>) to get a specified column value
    System.out.println(schema.getColumns().size());

    // easy to get column names, then user can use record.getObject(<column_name>) to get a specified column value
    System.out.println(schema.getColumns().stream().map(ClickHouseColumn::getColumnName).toList());

    // easy to get all row values in kv format
    System.out.println(record.getAll());
});

@abcfy2 abcfy2 force-pushed the main branch 2 times, most recently from 376ca8c to bd363cf Compare December 7, 2024 13:18
*
* @return a Map of column names and values.
*/
default Map<String, Object> getAll() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets rename it to getValues or asMap.
I would prefer having no default implementation in interfaces because default was designed to not break things when new stream API was introduced in java. I think, that only this case is valid usage of default.

@chernser
Copy link
Contributor

chernser commented Dec 9, 2024

@abcfy2 than you for changing the PR!
I've left 2 comments. I mainly worried about name of the method.

Thanks!

@abcfy2
Copy link
Contributor Author

abcfy2 commented Dec 10, 2024

No worries @chernser . I've modified PR. Please check again.

Thanks.

@chernser
Copy link
Contributor

@abcfy2 looks good to me!
Thank you!

@abcfy2
Copy link
Contributor Author

abcfy2 commented Dec 10, 2024

🤨 Some tests failed. But seems it's non of my mistake.

@chernser chernser merged commit 16228b2 into ClickHouse:main Dec 10, 2024
56 of 60 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants