fix(tpcds-obt): load the OBT table on Snowflake and BigQuery

[joeharris76]

Summary

On Snowflake and BigQuery, tpcds_obt could run its 89 queries against an empty OBT table and finish cleanly. A Snowflake SF1 bundle loaded the 25 normalized TPC-DS tables, never loaded tpcds_sales_returns_obt, and returned 0 rows for query 2 in every iteration. BigQuery and Databricks return 52.

Root cause, in order:

1. Platforms with a cloud default output location take _resolve_custom_output_root. It picked the local cache from get_data_source_benchmark() ("tpcds") and ignored GENERATES_OWN_OUTPUT. Fix transaction_primitives on Snowflake and BigQuery #2420 fixed the same redirect only on the construction path, which Databricks takes.
2. When the TPC-DS manifest in that folder was valid, the runner reused it, because the reuse gate accepts the data-source manifest. It filled benchmark.tables with the 25 source tables and never called OBT's generate_data(), the only code that registers the OBT table. Whether a run was correct depended on whether that manifest happened to be stale.
3. The OBT schema DDL creates the OBT table empty, and nothing checked before measuring that the query table had rows. Post-load validation is off by default and has no OBT expectations. The bundle validator's empty-result check does not fire, because aggregates over an empty table still return a row (48 of 267 measurements did).

Changes:

• benchbox/cli/orchestrator.py: both output-root paths choose the local datagen folder through one helper, _local_datagen_name, which honours GENERATES_OWN_OUTPUT.
• benchbox/platforms/base/adapter.py: a benchmark can list the tables its queries read in REQUIRED_LOADED_TABLES. After load and before measuring, the run fails if any of them is missing or empty. tpcds_obt lists its OBT table.
• benchbox/validation/bundle.py: a tpcds_obt bundle whose tables section lacks the OBT table, or shows it with 0 rows, is an error.

Verification

• New tests cover the cloud-staging cache choice for a benchmark that generates its own output and for an ordinary data-sharing benchmark, the load guard, and the validator check.
• Against real bundles: the wrong Snowflake bundle (tpcds_obt_sf1_snowflake_sql_20261001_112551_a3b5cd41) now fails with "queries table 'tpcds_sales_returns_obt', which is not in 'tables'". The correct BigQuery and Databricks bundles still pass.
• 4,850 tests pass across the OBT, CLI, runner, scripts and base adapter suites, plus the publication tests (34).

Review follow-ups (second commit): the public TPCDSOBT wrapper mirrors both class attributes; manifest reuse no longer accepts the TPC-DS manifest for a benchmark that generates its own output; the load-only path runs the same check; and a tpcds_obt bundle with measured queries but no tables block is an error. The three committed OBT bundles still pass.

Soundness review:

• External reviewer: codex
• Review output: fix(tpcds-obt): load the OBT table on Snowflake and BigQuery #2505 (comment)
• All Critical/High findings resolved.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-01T17:04:39.869101Z	46f7213	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[joeharris76]

External review (codex, model gpt-6.1-sol, low effort, read-only static review of git diff origin/develop...HEAD at 46f7213). Reviewer output, verbatim:

• High — adapter.py:126: The public TPCDSOBT wrapper bypasses the guard. REQUIRED_LOADED_TABLES exists only on its _impl; the wrapper neither declares nor forwards it. Calling TPCDSOBT(...).run_with_platform(adapter) with populated source-table mappings therefore makes this check return immediately. If DDL creates an empty OBT table, existing validation checks row counts only for the loaded source tables and checks OBT existence, allowing measurement against the empty OBT. The helper tests use SimpleNamespace, so they miss this supported execution path.

• High — bundle.py:1640: Omitting tables bypasses the required-table validator. A measured tpcds_obt bundle containing query results but no tables block receives no required-table error. Absence of table statistics does not establish that measurement never happened. The new test explicitly accepts this bypass, contrary to the requested rejection of bundles lacking the OBT table.

• Medium — runner.py:118: Manifest reuse still accepts source data for benchmarks that generate their own output. A tpcds_obt run with an explicit local output directory pointing at a valid TPC-DS cache accepts its manifest, populates the 25 source-table mappings, and skips OBT generation. The normal fresh-load guard eventually rejects the run, but only after staging/loading the wrong dataset. The load-only path calls load_data() directly without this guard and can report completion. Exclude source aliases when GENERATES_OWN_OUTPUT is true.

No Critical findings. Review was static; tests were not run.

---

Author dispositions (all accepted, fixed in aebfff5):

• High, public TPCDSOBT wrapper bypasses the guard: accepted. The wrapper now mirrors GENERATES_OWN_OUTPUT and REQUIRED_LOADED_TABLES from TPCDSOBTBenchmark. A test runs the guard against a wrapper instance.
• High, omitting tables bypasses the validator: accepted. A tpcds_obt bundle with measured queries and no tables block is now an error. The three committed OBT bundles in results-data/bundles (ClickHouse local, DataFusion, DuckDB) all still pass.
• Medium, manifest reuse accepts source data and the load-only path skips the guard: accepted. _resolve_manifest_allowed_names returns only the benchmark's own name when GENERATES_OWN_OUTPUT is set, and the load-only path calls require_loaded_tables after load_data. Both are covered by tests.

All Critical/High findings resolved.