ClientV2 ignoring schema inference hints and infer integer settings

[alxhill]

Describe your feedback

We run the following query through the clickhouse java client:

CREATE TABLE {tmpTableName:Identifier}  ENGINE=Log() AS SELECT * FROM s3('...file.csv', 'CSVWithNames')

With the following QuerySettings:

date_time_input_format=best_effort,
input_format_try_infer_integers=0,
schema_inference_hints='timestamp Nullable(Float64)'
input_format_try_infer_exponent_floats=1,
precise_float_parsing=1,
input_format_try_infer_dates=1,
input_format_try_infer_datetimes=1,
schema_inference_use_cache_for_s3=0,
schema_inference_make_columns_nullable=1

In ClientV1, the created table would have Nullable(Float64) for the timestamp column. ClientV2 seems to ignore the inference hints (and the "input_format_try_infer_integers=0" setting), as the column is Nullable(Int64) instead:

TableSchema{tableName='tmp_csv_0be155a3_920d_4496_87a7_ee2facee2efc', databaseName='nominal', columns=[timestamp Nullable(Int64), a Nullable(Float64), b Nullable(String)], metadata={a={type=Nullable(Float64)}, b={type=Nullable(String)}, timestamp={type=Nullable(Int64)}}, colIndex={a=1, b=2, timestamp=0}, hasDefaults=false}

[chernser]

@alxhill thank you for reporting!
will look into it shortly.

[alxhill]

Okay, looks like the culprit is that ClientV1 appends all QuerySettings to the URI, while ClientV2 only appends server settings to the URI.

ClientV1

clickhouse-java/clickhouse-http-client/src/main/java/com/clickhouse/client/http/ClickHouseHttpConnection.java

Lines 164 to 170 in d9cbaba

	for (Entry<String, Serializable> entry : settings.entrySet()) {
	// Skip internal settings
	if (entry.getKey().equalsIgnoreCase("_set_roles_stmt")) {
	continue;
	}
	appendQueryParameter(builder, entry.getKey(), String.valueOf(entry.getValue()));
	}

ClientV2

clickhouse-java/client-v2/src/main/java/com/clickhouse/client/api/internal/HttpAPIClientHelper.java

Lines 511 to 515 in d9cbaba

	for (Map.Entry<String, Object> entry : requestConfig.entrySet()) {
	if (entry.getKey().startsWith(ClientSettings.SERVER_SETTING_PREFIX)) {
	req.addParameter(entry.getKey().substring(ClientSettings.SERVER_SETTING_PREFIX.length()), entry.getValue().toString());
	}
	}

Changing from settings.setOption to settings.serverSetting fixed my tests. Seems like a break worth resolving as they're silently being ignored.

[chernser]

@alxhill
sorry - my bad - I would need to document this thing.

Is the issue resolved and only documentation should be changed?

Thanks!

[alxhill]

We are unblocked, but this does seem like a break in the client & not something I would expect to see upgrading between minor versions

[chernser]

@alxhill sorry about that - my bad. This is consequences of using two different clients under the hood (old and new) .
We always keep in mind that changes may be breaking and so we need to handle them properly - why many fixes are getting feature flags.