Skip to content

fix(tpchavoc): run the V7 variants on Presto and fix 16_v6 results - #3003

Merged
joeharris76 merged 2 commits into
BenchBox-dev:developfrom
karthikeyann:fix/tpchavoc-presto-variants
Oct 18, 2026
Merged

joeharris76 merged 2 commits into
BenchBox-dev:developfrom
karthikeyann:fix/tpchavoc-presto-variants

Conversation

@karthikeyann

Copy link
Copy Markdown
Contributor

Pull Request

Description

Fixes #3001. Presto rejected nine TPC-Havoc variants, and 16_v6 returned wrong results on every engine.

  • SELECT aliases in HAVING (3_v7, 4_v7, 5_v7, 7_v7, 9_v7, 10_v7, 11_v7): Presto does not
    resolve SELECT aliases in HAVING. These variants now use the existing alias inlining
    (_inline_select_aliases, previously PostgreSQL only) when the dialect is presto.
  • FILTER on a window aggregate (2_v7): Presto supports FILTER on plain aggregates but not on window
    aggregates. For presto, MIN(x) FILTER (WHERE c) OVER (...) becomes MIN(IF(c, x, NULL)) OVER (...).
    Plain aggregates keep FILTER.
  • DuckDB LIST functions (1_v7): for presto, the CTE collects with ARRAY_AGG and the outer query
    aggregates the zipped arrays with CROSS JOIN UNNEST(quantities, prices, discounts, taxes). Presto zips multiple
    arrays positionally, like LIST_ZIP. A REDUCE/ZIP_WITH translation like the Databricks one does not work on
    Presto C++, which limits REDUCE to 100,000 array elements; each Q1 group has millions at SF10.
  • 16_v6 semantics: the variant replaced NOT IN with a cross join and <>, which keeps rows whenever more
    than one supplier has complaints (4 at SF1). It now uses NOT IN over the same derived table.

The Presto rewrites are registered as query_adapter.presto.tpchavoc.* compatibility rules, and
docs/compat/capability-matrix.md is regenerated.

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Refactor / chore

Testing

  • make pr-preflight: lint and the mapped tests pass.
  • uv run -- python -m pytest tests/unit/tpchavoc tests/unit/sql_compat: the new tests pass. Two tests in
    tests/unit/tpchavoc fail identically on unmodified develop in this environment (DataFusion is not installed,
    and the BigQuery FILTER expectation differs with sqlglot 30.23.0).
  • New tests:
    • tests/unit/tpchavoc/test_presto_compat.py: no SELECT alias left in HAVING/WHERE, no FILTER on window
      aggregates, 1_v7 uses ARRAY_AGG and UNNEST, and every rewritten variant has a registered rule.
    • test_query16_derived_table_variant_excludes_every_complaint_supplier: base Q16 and 16_v6 on a three-supplier
      DuckDB fixture with two complaint suppliers. It fails before the fix (supplier_cnt 3 instead of 1).
  • make compat-docs-check, make comment-policy-check, and the pre-commit hooks pass.
  • End to end: all 242 TPC-Havoc queries on Presto master (native C++ workers) at SF10, each checked against its
    base query and against DuckDB on the same Parquet data. Before: 232/242 pass. After: 242/242.
    1_v7 returns the same row values as base Q1.

Public Contract Check

  • I updated docs/reference/public-contracts.md, or this PR does not change
    public, beta-public, deprecated, generated, experimental, or repo-only
    contract surfaces.
  • I checked result schema fields, MCP tool parameters, platform support
    status, wrapper facades, adapter subclassing hooks, and generated docs for
    contract-map impact.
  • Any platform/benchmark count claim in docs is generated/checked from
    registry metadata, or explicitly marked editorial/non-authoritative.

Documentation

  • I updated the relevant user-facing docs.
  • I added regression notes for behavior changes.
  • I confirmed API contract wording remains accurate.
  • This PR does not change results-explorer/, website/, landing/,
    docs/blog/, _blog/ or _project/design/site-inventory/, or its
    description says the change must be replayed into
    BenchBox-dev/benchbox-site to reach benchbox.dev (see
    CONTRIBUTING.md).

Code Quality

  • I ran formatting and lint/type checks.
  • I reviewed impacted modules for backwards compatibility and migration impact.
  • I added/updated tests for behavior changes and error paths.
  • I confirmed public contracts and release checklists are still valid.

Artifact Hygiene

  • I did not commit raw screenshots, browser reports, generated logs,
    benchmark outputs, or temporary binary artifacts.
  • Any committed binary/raw evidence file has durable repo value and is
    justified here with its consumer and size:

Notes

  • 16_v6 changes results on every engine. Stored results for that variant were wrong and will differ after this
    change.
  • The Presto rewrites only apply to the presto dialect. Trino likely has the same alias and window FILTER
    limitations, but that was not tested here.

Presto rejected nine TPC-Havoc variants and 16_v6 returned wrong results on
every engine:

- 3_v7, 4_v7, 5_v7, 7_v7, 9_v7, 10_v7, 11_v7 reference SELECT aliases in
  HAVING; inline them for Presto as already done for PostgreSQL.
- 2_v7 uses FILTER on a window aggregate; move the condition into the
  aggregate argument for Presto.
- 1_v7 uses DuckDB LIST functions; collect with ARRAY_AGG and aggregate the
  zipped arrays with UNNEST. REDUCE is limited to 100,000 elements in
  Presto C++, which Q1 groups exceed.
- 16_v6 rewrote NOT IN as a cross join with <>, which keeps rows when more
  than one supplier has complaints; use NOT IN over the derived table.

Fixes BenchBox-dev#3001
@joeharris76

Copy link
Copy Markdown
Collaborator

Thanks for the PR submission! I will review it and confirm.

@joeharris76

Copy link
Copy Markdown
Collaborator

Thanks for putting this together and validating all 242 TPC-Havoc queries at SF10 on Presto C++!

The root-cause diagnosis on 16_v6 (replacing the <> cross join against excluded with NOT IN while preserving the v6 derived-table structure) is spot on, and the Presto SELECT-alias inlining (3_v7–11_v7), window FILTER rewrite (2_v7), and sql_compat rule registrations are clean and well-structured.

Before we merge, there are a few small adjustments we will apply on the branch so we can take this out of draft and land it:

  1. Deterministic element ordering in the Presto 1_v7 ARRAY_AGG rewrite (benchbox/core/tpchavoc/dialect_compat.py):
    1_v7 computes sum_disc_price = SUM(price * (1 - discount)) and sum_charge = SUM(price * (1 - discount) * (1 + tax)) across positional elements from CROSS JOIN UNNEST(quantities, prices, discounts, taxes). In SQL and Presto, ARRAY_AGG(x) without ORDER BY does not guarantee cross-column element alignment across partial-aggregation splits. Ordering each of the four ARRAY_AGG calls by l_orderkey, l_linenumber (matching the existing _order_by_lineitem_key pattern in benchbox/core/tpchavoc/cloud_compat.py) makes the positional UNNEST alignment guaranteed by the query semantics, and we will assert that ordering on all four ARRAY_AGG calls in tests/unit/tpchavoc/test_presto_compat.py.
  2. Defensive guard in _window_filter_node (benchbox/core/tpchavoc/dialect_compat.py):
    Raise NotImplementedError if a window FILTER aggregate uses COUNT(*) (exp.Star) or DISTINCT (exp.Distinct), matching _clickhouse_filter_node.
  3. Targeted query translation and exact rule-set equality in unit tests (tests/unit/tpchavoc/test_variant_registry.py, tests/unit/tpchavoc/test_presto_compat.py):
    Use benchmark.get_query(...) instead of benchmark.get_queries(...) so the fast test suite only transpiles the variants under test rather than all 220 variants on each call, and tighten assert expected <= registered to assert registered == expected in test_presto_compat.py.

I'll push a follow-up commit with these updates to your PR branch, mark the PR ready for review, and run CI.

@joeharris76
joeharris76 marked this pull request as ready for review October 18, 2026 00:06
@joeharris76
joeharris76 enabled auto-merge (squash) October 18, 2026 00:13
@joeharris76
joeharris76 merged commit 07dda7a into BenchBox-dev:develop Oct 18, 2026
60 checks passed

This branch was previously deployed

1 inactive deployment
oracle — 6212fead Deployed Oct 18, 2026 by joeharris76 via post #1047
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] TPC-Havoc: 10 variants fail or return wrong results on Presto (V7 HAVING aliases, FILTER on window, LIST functions, 16_v6)

2 participants