MDEV-39969 Incorrect result of JSON_TABLE with BIT

[DerZc]

JSON_TABLE('[0,1]', '$[*]' COLUMNS (b BIT(1) PATH '$')) returns 1 for both JSON numbers. The scalar helper passes their decimal text to BIT's binary-string store; the byte for '0' is 48, which is clipped to 1.

Bug report: https://jira.mariadb.org/browse/MDEV-39969

Changes

• Introduce Type_handler::value_store_type() for values with both numeric and string forms. It defaults to the existing result type; BIT explicitly selects integer storage. JSON boolean conversion uses this policy.
• Introduce virtual Field::store_numeric() for numeric values serialized as text. The default delegates to the existing field conversion. BIT overrides it using the charset-aware rounded unsigned integer parser and the existing BIT range checks, including the full unsigned 64-bit range.
• Route JSON numbers through this interface so JSON_TABLE does not identify individual destination types. JSON strings retain their existing conversion path.
• Preserve datatype conversion warnings as warnings. They do not invoke NULL, DEFAULT, or ERROR ON ERROR; JSON structural errors continue to use those responses. The scalar helper has no BIT-specific error output parameter.

Regression coverage

The regression is in the existing json.json_table test. It covers numeric 0/1, multiple bit widths, the unsigned 64-bit boundary, fractions and exponents, negative values and overflow, boolean/string/null inputs, and numeric/string/date destination controls. It also checks all ON ERROR responses, strict SQL mode, and genuine array/object errors.

Validation

Built the revised code and a separate clean 11.4 baseline at d10e5d726799b1cd57cc866f40aad68c720803da with GCC 13 in Release mode.

• The expanded regression fails on the clean baseline at the numeric conversion cases. It also catches the previous PR's incorrect ON ERROR handling.
• All 11 selected MTR tests passed: json.json_table, json.json_table_mysql, json.json_table_notembedded, main.func_json, main.type_bit, main.type_bit_innodb, main.type_int, main.type_uint, main.type_newdecimal, main.type_float, and main.select.
• json.json_table and json.json_table_mysql also passed with --ps-protocol.
• The new regression block, extracted into an isolated test, passed with --view-protocol.
• Explicit SQL checks verified the new regression's result values, warning codes/messages, and expected errors. No test-state cleanup failures occurred in the passing MTR runs. git diff --check passed.

The complete existing JSON_TABLE tests have two pre-existing --view-protocol failures, reproduced on both the clean baseline and the revision: a latin1_bin/utf8mb4_bin collation expectation in json.json_table, and a DEFAULT NULL ON EMPTY serialization syntax error in json.json_table_mysql. The full regression suite and Windows builds were not run locally.

[CLAassistant]

All committers have signed the CLA.

[gkodinov]

Thank you for your contribution! This is a preliminary review.

This fix needs more work IMHO. I've tried outlining some of it below.

But, this is a stuff for the final review anyway. For the preliminary review, please squash the two commits together.

[gkodinov]

The expectation here is that JSON boolean values will store as integer 1/0 into "numeric" data types and as a string literal ("true"/"false") into "string" data types. However bit type is a special case: it can store integers but is not an integer.

This fix proposed is a workaround at best: what happens when the next data type that has different base result type is to receive a value of some other kind?

E.g. store a date? or some such. or if the bit data type is implemented as a pluggable type? Or even consider other operations that call Field->store() with a hybrid data type.

I believe a proper fix should involve one of:

1. Come up with a "value store type" for data types and use that instead of the result type. Have bit report an int "value store type".
2. Consider converting BIT to int result/cmp type altogether, since the longest boolean literal is 64 bits anyway

[DerZc]

Implemented your first suggestion in 17eaf8d6, and squashed the PR into one commit.

Type_handler::value_store_type() now expresses the preferred representation for values with numeric and string forms; BIT explicitly returns INT_RESULT, and JSON boolean conversion uses that policy. Numeric JSON text goes through a separate virtual Field::store_numeric() interface. Its default preserves existing field conversion, while BIT overrides it with charset-aware unsigned integer parsing and the existing range checks. JSON_TABLE no longer checks for a particular type handler, and data type implementations can override these interfaces.

On this 11.4 base, BIT already inherits integer result/comparison types, so its previous boolean special case was redundant. The actual regression is numeric JSON input. Regression coverage now includes BIT widths, uint64 boundaries, fractions/exponents, and boolean/string/null inputs, with other destination types as controls.

[gkodinov]

This is unrelated to the topic of the bug. Make it abundantly clear in the commit message what is it that you're fixing here please.

[DerZc]

Rewrote the commit message and PR description to make the numeric conversion bug explicit. The MDEV-39969 report uses JSON numbers: [0,1] stored into BIT(1) produces [1,1]. The old path passes the decimal text to BIT's binary-string store; the byte for '0' is 48 and gets clipped to 1.

The numeric branch addresses that reported failure. Boolean-to-integer conversion already worked on this 11.4 base, and the revised message no longer presents it as the bug being fixed. The new regression fails on a separately built clean baseline and passes with the revision.

[gkodinov]

Thank you! LGTM. Please stand by for the final review.

BTW, are you testing some sort of a tool? I'd be very interested to find out more about it. Reach out to me on https://mariadb.zulipchat.com please in case you can share details.