feat: Add high precision TIMESTAMP values for queries by danieljbruce · Pull Request #7147 · googleapis/google-cloud-node

danieljbruce · 2026-02-05T16:19:43Z

Description

This PR makes it so that all requests to the /queries endpoint support returning high precision results for the TIMESTAMP type. This ensures that users will always get the highest precision possible for TIMESTAMP typed values requested in queries provided they request getting a higher precision timestamp by specifying a type of TIMESTAMP(12):

const query = {
    query: 'SELECT ? as ts',
    params: [bigquery.timestamp('2023-01-01T12:00:00.123456789123Z')],
    types: ['TIMESTAMP(12)']
};
const [rows] = await bigquery.query(query, options);

googleapis/nodejs-bigquery#1596 adds similar support for the /data REST endpoint which may be useful for reference. And googleapis/java-bigquery#4010 contains a similar PR to this one, but for the Bigquery Java codebase.

Impact

Allows users to opt in to fetching high precision timestamps for calls to the /queries endpoint.

Testing

System tests were added to make sure no matter what formatOptions they provide, the values sent back by the server will be delivered to the user without losing any precision when requested.

Additional Information

Notice that the PR adds timestampPrecision: 12 only when the user explicitly uses the TIMESTAMP(12) types. This is because if we automatically opt users into timestampPrecision: 12 then their code will suddenly fail for certain queries that were working before. Therefore, we are going to make timestamp precision an opt-in property.

precision timestamps

…is/google-cloud-node into big-query-query-changes

alvarowolfx · 2026-02-05T19:58:43Z

-          });
+          try {
+            /*
+            Without this try/catch block, calls to getRows will hang indefinitely if


nit: getRows = jobsQuery endpoint ?

Yes! This has been fixed now. This was a typo.

alvarowolfx · 2026-02-05T20:00:47Z

+import {describe, it, before} from 'mocha';
+import {BigQuery} from '../src';
+
+describe('High Precision Query System Tests', () => {


why do we need a separated file ? The previous PR we already deviated a bit from the pattern of having system tests in the https://github.com/googleapis/nodejs-bigquery/blob/main/system-test/bigquery.ts and added https://github.com/googleapis/nodejs-bigquery/blob/main/system-test/timestamp_output_format.ts, can we bundle all timestamp related tests into the same file ?

And gonna bring this again that we don't need the amount of combinations and system tests to check if the support for picosecond resolution is working. More tests doesn't necessarily means more coverage. With this PR and the previous one we are adding 28 query executions, where previously we had a total of around 50. Not saying that the coverage was perfect before, but adding too many system/integration test for a time format change seems bit too much, I'd lean more on unit tests. System tests makes CI slower, and I'm not seeing enough justification for the amount of combination and tests being added here.

We already got bitten by too much client side validation or tests trying to assert internal details of the service in the past that can cause problems. In this case, is not a SDK code, but is test that can fail in the future if things changes in the service.

See internal b/460198628

why do we need a separated file ?

I realize having a separate file deviates from the pattern of putting every test in system-test/bigquery.ts, but I really believe that having a separate file is a better pattern. One of the biggest drawbacks with putting every test in one file is that you lose encapsulation. For instance, the before/after hooks in the bigquery.ts test file don't have anything to do with the new tests we are adding. So if we add the new tests to this file then future maintainers have to think about how everything else in this file including the before/after hooks influence these tests and we don't want that.

we don't need the amount of combinations and system tests to check if the support for picosecond resolution is working

Ok. I needed the tests myself to understand how the client was behaving, but maybe now we can comment some of them out to improve the CI performance. I would prefer to comment the tests out rather than delete them though because the commented tests could be useful later when we want to investigate the product's behaviour. Which ones do you want to comment out?

I realize having a separate file deviates from the pattern of putting every test in system-test/bigquery.ts, but I really believe that having a separate file is a better pattern. One of the biggest drawbacks with putting every test in one file is that you lose encapsulation. For instance, the before/after hooks in the bigquery.ts test file don't have anything to do with the new tests we are adding. So if we add the new tests to this file then future maintainers have to think about how everything else in this file including the before/after hooks influence these tests and we don't want that.

I still don't understand why we can't bundle all timestamp related tests into the same file. I can understand the reasoning about the system-test/bigquery.ts currently being bloated ( and we should plan to refactor it), but those timestamp formats can definitely live together. Am I missing something ?

Ok. I needed the tests myself to understand how the client was behaving, but maybe now we can comment some of them out to improve the CI performance. I would prefer to comment the tests out rather than delete them though because the commented tests could be useful later when we want to investigate the product's behaviour. Which ones do you want to comment out?

Was a nice catch to add the nested query test, but I don't think is a good approach to keep tests commented out here. The tests added at test/bigquery.ts are already excising all the combinations. For the integration tests we can exercise a few scenarios like:

Not passing parameters -> should default to picosecond

Passing useInt64 -> should override default and return nanoseconds

Passing timestampOutputFormat: INT64 -> should override default and return nanoseconds

Passing two incompatible params (useInt64: true and timestampOutputFormat: Float64) -> error

but those timestamp formats can definitely live together. Am I missing something ?

I put the tests in the file like you wanted. I think this is just a discussion about pros and cons. Not bloating bigquery.ts has pros, putting the tests with the other similar test has pros. We can likely move on at this point.

For the integration tests we can exercise a few scenarios like

I added three of the scenarios, but I'm not sure what you mean by Passing timestampOutputFormat: INT64.

alvarowolfx · 2026-02-05T20:01:17Z

+
+describe('High Precision Query System Tests', () => {
+  let bigquery: BigQuery;
+  const expectedTsValueMicroseconds = '2023-01-01T12:00:00.123456000Z';


I think this was suppose to be Nanosecond. We missed that the previous PR

Yes. Good catch. I changed this in both places.

alvarowolfx · 2026-02-05T20:01:26Z

+describe('High Precision Query System Tests', () => {
+  let bigquery: BigQuery;
+  const expectedTsValueMicroseconds = '2023-01-01T12:00:00.123456000Z';
+  const expectedTsValueNanoseconds = '2023-01-01T12:00:00.123456789123Z';


and this picosecond. Same applies to the previous https://github.com/googleapis/nodejs-bigquery/pull/1596/changes#diff-e82cb9428a68678cc305e12e59f2e21761622b9fcfcf45d4caab5e80657e2b9a test file

Yes. Good catch. I changed this in both places.

alvarowolfx · 2026-02-05T20:02:21Z

+      expectedTsValue: expectedTsValueMicroseconds,
+    },
+    {
+      name: 'TOF: FLOAT64, UI64: true (error)',


the service side is going to reject it and is not a reasonable combination that customers would use. Not sure if we need to assert this. If the service side changes behavior (and can happen) we are gonna have a flaky test. I'd add just one error combination here.

I commented this test out. Let me know if you want more tests commented out.

danieljbruce

Just need to follow up on one more PR comment.

danieljbruce · 2026-02-06T15:07:39Z

-          });
+          try {
+            /*
+            Without this try/catch block, calls to getRows will hang indefinitely if


Yes! This has been fixed now. This was a typo.

danieljbruce · 2026-02-06T15:20:22Z

+import {describe, it, before} from 'mocha';
+import {BigQuery} from '../src';
+
+describe('High Precision Query System Tests', () => {


why do we need a separated file ?

I realize having a separate file deviates from the pattern of putting every test in system-test/bigquery.ts, but I really believe that having a separate file is a better pattern. One of the biggest drawbacks with putting every test in one file is that you lose encapsulation. For instance, the before/after hooks in the bigquery.ts test file don't have anything to do with the new tests we are adding. So if we add the new tests to this file then future maintainers have to think about how everything else in this file including the before/after hooks influence these tests and we don't want that.

we don't need the amount of combinations and system tests to check if the support for picosecond resolution is working

Ok. I needed the tests myself to understand how the client was behaving, but maybe now we can comment some of them out to improve the CI performance. I would prefer to comment the tests out rather than delete them though because the commented tests could be useful later when we want to investigate the product's behaviour. Which ones do you want to comment out?

danieljbruce · 2026-02-06T15:25:01Z

+
+describe('High Precision Query System Tests', () => {
+  let bigquery: BigQuery;
+  const expectedTsValueMicroseconds = '2023-01-01T12:00:00.123456000Z';


Yes. Good catch. I changed this in both places.

danieljbruce · 2026-02-06T15:26:27Z

+describe('High Precision Query System Tests', () => {
+  let bigquery: BigQuery;
+  const expectedTsValueMicroseconds = '2023-01-01T12:00:00.123456000Z';
+  const expectedTsValueNanoseconds = '2023-01-01T12:00:00.123456789123Z';


Yes. Good catch. I changed this in both places.

danieljbruce · 2026-02-06T15:29:57Z

+      expectedTsValue: expectedTsValueMicroseconds,
+    },
+    {
+      name: 'TOF: FLOAT64, UI64: true (error)',


I commented this test out. Let me know if you want more tests commented out.

Adds a new test suite to bigquery.ts that verifies the correct construction of QueryRequest formatOptions for various combinations of timestampOutputFormat and useInt64Timestamp. The test cases are modeled after the high-precision-query system tests. Co-authored-by: danieljbruce <8935272+danieljbruce@users.noreply.github.com>

alvarowolfx · 2026-02-09T16:52:21Z

+    return fromStringValue_(value);
  }

  static fromSchemaValue_(value: string, elementType: string): BigQueryRange {


this is an internal only method (because it ends with _), so we don't need to worry about keeping the public interface.

It sounds like you would like to move the code back to where it was and avoid creating a new method so I have reverted the change.

alvarowolfx · 2026-02-09T16:56:41Z

      assert.deepStrictEqual(req, expectedReq);
    });

+    describe('format options', () => {


nit: "timestamp format options"

This has been updated now.

alvarowolfx · 2026-02-09T17:05:41Z

+import {describe, it, before} from 'mocha';
+import {BigQuery} from '../src';
+
+describe('High Precision Query System Tests', () => {


I realize having a separate file deviates from the pattern of putting every test in system-test/bigquery.ts, but I really believe that having a separate file is a better pattern. One of the biggest drawbacks with putting every test in one file is that you lose encapsulation. For instance, the before/after hooks in the bigquery.ts test file don't have anything to do with the new tests we are adding. So if we add the new tests to this file then future maintainers have to think about how everything else in this file including the before/after hooks influence these tests and we don't want that.

I still don't understand why we can't bundle all timestamp related tests into the same file. I can understand the reasoning about the system-test/bigquery.ts currently being bloated ( and we should plan to refactor it), but those timestamp formats can definitely live together. Am I missing something ?

Ok. I needed the tests myself to understand how the client was behaving, but maybe now we can comment some of them out to improve the CI performance. I would prefer to comment the tests out rather than delete them though because the commented tests could be useful later when we want to investigate the product's behaviour. Which ones do you want to comment out?

Was a nice catch to add the nested query test, but I don't think is a good approach to keep tests commented out here. The tests added at test/bigquery.ts are already excising all the combinations. For the integration tests we can exercise a few scenarios like:

Not passing parameters -> should default to picosecond

Passing useInt64 -> should override default and return nanoseconds

Passing timestampOutputFormat: INT64 -> should override default and return nanoseconds

Passing two incompatible params (useInt64: true and timestampOutputFormat: Float64) -> error

alvarowolfx · 2026-02-09T17:17:08Z

+          useInt64Timestamp: false,
+        },
+        {
+          name: 'TOF: TIMESTAMP_OUTPUT_FORMAT_UNSPECIFIED, UI64: omitted (default INT64)',


both omitted it should default to ISO8601_STRING right ?

Yes. This combination results in ISO8601_STRING.

alvarowolfx · 2026-02-09T17:17:14Z

+          useInt64Timestamp: false,
+        },
+        {
+          name: 'TOF: omitted, UI64: omitted (default INT64)',


both omitted it should default to ISO8601_STRING right ?

Yes. This test case has actually been replaced by your most recent commit.

alvarowolfx · 2026-02-09T17:20:49Z

+          useInt64Timestamp: undefined,
+        },
+        {
+          name: 'TOF: ISO8601_STRING, UI64: omitted (error)',


why this should be an error ?

It shouldn't be, but these tests were failing anyway. We need to get the CI set up for the monorepo to catch stuff like this automatically.

In any case, these tests have now been updated. Keep in mind that the exact same logic we used for the /data endpoint is now being applied to the /queries endpoint so if you see any problems with these tests then the same correction should be applied to the /data test as well.

fns

This reverts commit f042cc4.

This reverts commit 8c4c2d5.

danieljbruce

Making changes. Work in progress.

danieljbruce · 2026-02-09T20:19:47Z

+    return fromStringValue_(value);
  }

  static fromSchemaValue_(value: string, elementType: string): BigQueryRange {


It sounds like you would like to move the code back to where it was and avoid creating a new method so I have reverted the change.

danieljbruce · 2026-02-09T22:05:58Z

+import {describe, it, before} from 'mocha';
+import {BigQuery} from '../src';
+
+describe('High Precision Query System Tests', () => {


but those timestamp formats can definitely live together. Am I missing something ?

I put the tests in the file like you wanted. I think this is just a discussion about pros and cons. Not bloating bigquery.ts has pros, putting the tests with the other similar test has pros. We can likely move on at this point.

danieljbruce · 2026-02-09T22:32:19Z

+import {describe, it, before} from 'mocha';
+import {BigQuery} from '../src';
+
+describe('High Precision Query System Tests', () => {


For the integration tests we can exercise a few scenarios like

I added three of the scenarios, but I'm not sure what you mean by Passing timestampOutputFormat: INT64.

Co-authored-by: Alvaro Viebrantz <aviebrantz@google.com>

danieljbruce

I addressed most of the comments, but it seems there are more regressions in the buildQueryRequest_ tests that I need to address.

danieljbruce · 2026-02-10T14:46:17Z

      assert.deepStrictEqual(req, expectedReq);
    });

+    describe('format options', () => {


This has been updated now.

danieljbruce · 2026-02-10T15:11:32Z

+          useInt64Timestamp: false,
+        },
+        {
+          name: 'TOF: omitted, UI64: omitted (default INT64)',


Yes. This test case has actually been replaced by your most recent commit.

danieljbruce · 2026-02-10T15:12:35Z

+          useInt64Timestamp: false,
+        },
+        {
+          name: 'TOF: TIMESTAMP_OUTPUT_FORMAT_UNSPECIFIED, UI64: omitted (default INT64)',


Yes. This combination results in ISO8601_STRING.

danieljbruce

Addressed comments. Ready for another review.

danieljbruce · 2026-02-10T20:03:09Z

+          useInt64Timestamp: undefined,
+        },
+        {
+          name: 'TOF: ISO8601_STRING, UI64: omitted (error)',


It shouldn't be, but these tests were failing anyway. We need to get the CI set up for the monorepo to catch stuff like this automatically.

In any case, these tests have now been updated. Keep in mind that the exact same logic we used for the /data endpoint is now being applied to the /queries endpoint so if you see any problems with these tests then the same correction should be applied to the /data test as well.

alvarowolfx · 2026-02-11T13:08:56Z

+        opts: QueryOptions;
+        expected?: any;
+        bail?: boolean;
+      }[] = [


nit: what are we asserting ?

Not passing anything and expecting a default makes sense to me

Passing X and expecting X makes sense too. But then passing Y and expecting Y, passing Z and expecting Z starts to get redundant

I created https://b.corp.google.com/issues/483681188 to address this in a separate PR.

https://github.com/googleapis/google-cloud-node/pull/7303/changes should address this issue

alvarowolfx

Still have nits on the unit tests, but function wise, LGTM

This reverts commit bea42b2. # Conflicts: # handwritten/bigquery/src/bigquery.ts # handwritten/bigquery/src/job.ts # handwritten/bigquery/test/bigquery.ts

* Revert "feat: Add high precision TIMESTAMP values for queries (#7147)" This reverts commit bea42b2. # Conflicts: # handwritten/bigquery/src/bigquery.ts # handwritten/bigquery/src/job.ts # handwritten/bigquery/test/bigquery.ts * Revert "feat: support high precision timestamp strings on getRows calls (#1596)" This reverts commit b3217a3. # Conflicts: # handwritten/bigquery/src/table.ts * skip a test * Remove the High Precision Query System tests * Source code changes for timestamp precision flag * Add tests back * Add test file back in * Modify tests to only run for high precision or not * listParams add * remove listParams * skip tests if picosecond support is not turned on * remove only * Don’t create separate code paths when not needed * extend the default options * Skip the before hook if picosecond support not on * Add documentation back in for methods * Remove the if block since release is done * Eliminate redundant feature flag check. * Add qs back in * Add commands to test for picoseconds in system t * Revert "Remove the if block since release is done" This reverts commit 757dfdc. * Eliminate environment variable for parsing check * Always use try block * Change type to GetRowsOptions * Remove exception * Some typescript simplifications * Eliminate default options branching * Change defaults for current requests * Don’t need extra import * Don’t gate with picosecond flag * Just add an extra test file * test(bigquery): parameterize system tests for picosecond support Wraps the BigQuery system tests in a parameterized function to run them both with and without the BIGQUERY_PICOSECOND_SUPPORT environment variable. Clients and resources are re-instantiated within a before hook to ensure the library picks up the environment changes for each run. The existing indentation is preserved to maintain a clean PR diff. Co-authored-by: danieljbruce <8935272+danieljbruce@users.noreply.github.com> * test(bigquery): add high precision timestamp wrapper system test Creates a new system test file that programmatically re-runs the entire BigQuery system test suite with BIGQUERY_PICOSECOND_SUPPORT=true. This is achieved by clearing the Node.js require cache for the package and requiring the existing bigquery.ts test file within a new describe block. This approach leaves the original test file untouched, ensuring a clean PR diff and avoiding linting issues. Co-authored-by: danieljbruce <8935272+danieljbruce@users.noreply.github.com> * linting change and copyright year * Correct package.json errors * This file is not working well with test runner * Run the timestamp output format tests as well * Add a wrapper --------- Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: danieljbruce <8935272+danieljbruce@users.noreply.github.com>

move over all code changes supporting high

aad102a

precision timestamps

danieljbruce changed the title ~~move over all code changes supporting high~~ feat: Add high precision TIMESTAMP values for queries Feb 5, 2026

danieljbruce added 2 commits February 5, 2026 11:22

feat: Add high precision TIMESTAMP values for queries

d4b2b0b

Merge branch 'big-query-query-changes' of https://github.com/googleap…

c369bdf

…is/google-cloud-node into big-query-query-changes

danieljbruce marked this pull request as ready for review February 5, 2026 19:56

danieljbruce requested a review from alvarowolfx February 5, 2026 19:56

alvarowolfx requested changes Feb 5, 2026

View reviewed changes

danieljbruce added 5 commits February 6, 2026 10:04

correct typo in comment

0c799a2

picoseconds / not nanoseconds

b033b9a

rename nanoseconds to picoseconds

5871908

nanoseconds not microseconds

d39d0a9

Add comment for non-meaningful use case

9b61a74

danieljbruce commented Feb 6, 2026

View reviewed changes

danieljbruce requested a review from alvarowolfx February 9, 2026 16:45

alvarowolfx requested changes Feb 9, 2026

View reviewed changes

danieljbruce added 6 commits February 9, 2026 15:10

Move the code back into the BigQueryRange static

00795de

fns

Move the “High Precision Query System Tests” block

f042cc4

Revert "Move the “High Precision Query System Tests” block"

8c4c2d5

This reverts commit f042cc4.

Reapply "Move the “High Precision Query System Tests” block"

48176dd

This reverts commit 8c4c2d5.

Move the tests to the end position

756d77b

delete unwanted test cases

4ad1233

danieljbruce commented Feb 9, 2026

View reviewed changes

danieljbruce and others added 3 commits February 10, 2026 09:41

Change the name used in the describe block

e38a4cb

Apply suggestion from @alvarowolfx

4886b51

Co-authored-by: Alvaro Viebrantz <aviebrantz@google.com>

Linter and clean up last commit

583ad83

danieljbruce commented Feb 10, 2026

View reviewed changes

Fix unit tests in test/bigquery to expect right va

3775cb4

danieljbruce commented Feb 10, 2026

View reviewed changes

danieljbruce requested a review from alvarowolfx February 10, 2026 20:06

alvarowolfx requested changes Feb 10, 2026

View reviewed changes

Comment thread handwritten/bigquery/src/bigquery.ts Outdated

Comment thread handwritten/bigquery/src/bigquery.ts Outdated

Comment thread handwritten/bigquery/system-test/bigquery.ts Outdated

Comment thread handwritten/bigquery/test/bigquery.ts Outdated

danieljbruce added 4 commits February 10, 2026 16:00

JS doc conversions

b508781

remove only

2e5188d

Remove bail: true

02a1eab

removed todo

f0ec49b

danieljbruce requested a review from alvarowolfx February 10, 2026 21:11

danieljbruce added 2 commits February 10, 2026 17:23

remove error in name

d4c401b

remove error from tests

94aa537

alvarowolfx reviewed Feb 11, 2026

View reviewed changes

alvarowolfx approved these changes Feb 11, 2026

View reviewed changes

danieljbruce merged commit bea42b2 into main Feb 11, 2026
15 of 17 checks passed

danieljbruce deleted the big-query-query-changes branch February 11, 2026 16:23

danieljbruce mentioned this pull request Feb 12, 2026

test: Finish tests for named parameters on query calls #7245

Merged

ldetmer mentioned this pull request Feb 12, 2026

chore: release main #7090

Merged

danieljbruce mentioned this pull request Feb 25, 2026

test: trim the unit tests to only capture important cases #7303

Merged

danieljbruce mentioned this pull request Mar 31, 2026

fix: Unblock new Node Bigquery releases #7931

Closed

Conversation

danieljbruce commented Feb 5, 2026

Description

Impact

Testing

Additional Information

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

danieljbruce left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

danieljbruce left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!