Skip to content

Reuse object-path indexing when exporting facts - #14829

Open
tamird wants to merge 1 commit into
google:masterfrom
tamird:nogo-objectpath-encoder
Open

tamird wants to merge 1 commit into
google:masterfrom
tamird:nogo-objectpath-encoder

Conversation

@tamird

@tamird tamird commented Sep 18, 2026 •

Copy link
Copy Markdown
Contributor

Nogo calls objectpath.For separately for each object whose facts it serializes, creating a fresh encoder each time. Reuse one encoder per package and update x/tools to v0.45.0, which includes the upstream fix for quadratic object-path searches 1 2. Update the required Go dependencies, their Bazel declarations, and their license audit records together.

Reset the encoder after a recovered panic so an incomplete index cannot affect subsequent objects. Add coverage for serialization of package and exported-object facts, plus a benchmark over increasing numbers of fields.

goos: linux
goarch: arm64
                │    before    │                after                 │
                │    sec/op    │    sec/op     vs base                │
Serialize/64-4    1455.1µ ± 2%   273.6µ ±  2%  -81.20% (p=0.000 n=10)
Serialize/256-4   20.749m ± 3%   1.167m ± 16%  -94.37% (p=0.000 n=10)
geomean            5.495m        565.1µ        -89.72%

                │    before     │                after                 │
                │     B/op      │     B/op      vs base                │
Serialize/64-4     954.8Ki ± 0%   289.4Ki ± 0%  -69.69% (p=0.000 n=10)
Serialize/256-4   10.869Mi ± 0%   1.032Mi ± 0%  -90.50% (p=0.000 n=10)
geomean            3.183Mi        553.1Ki       -83.03%

                │    before    │                after                │
                │  allocs/op   │  allocs/op   vs base                │
Serialize/64-4     3.121k ± 0%   1.617k ± 0%  -48.19% (p=0.000 n=10)
Serialize/256-4   12.347k ± 0%   6.250k ± 0%  -49.38% (p=0.000 n=10)
geomean            6.208k        3.179k       -48.79%

Assisted-by: Codex

@tamird

tamird commented Sep 18, 2026

Copy link
Copy Markdown
Contributor Author

Based on golang/go#78893 this might address internal issue b/498089396 - though I don't remember enough about how internal stuff works so maybe the internal stuff already gets the benefit of this optimization.

@milantracy or @konstantin-s-bogom this might be relevant to your interests.

@tamird

tamird commented Sep 23, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @kerumeto! Could you please run the import?

@kerumeto

Copy link
Copy Markdown
Contributor

Apologies, was waiting for a second reviewer but will get that later

@tamird

tamird commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

Looks like there's a busted license check in the import #14941? Anything I can do?

@EtiennePerot

Copy link
Copy Markdown
Collaborator

Yes, see the error message on BuildKite:

golang.org/x/crypto was audited at "v0.49.0", but is now "v0.51.0"
golang.org/x/mod was audited at "v0.34.0", but is now "v0.36.0"
golang.org/x/net was audited at "v0.52.0", but is now "v0.54.0"
golang.org/x/sys was audited at "v0.43.0", but is now "v0.44.0"
golang.org/x/telemetry was audited at "v0.0.0-20250908211612-aef8a434d053", but is now "v0.0.0-20260508192327-42602be52be6"
golang.org/x/term was audited at "v0.42.0", but is now "v0.43.0"
golang.org/x/text was audited at "v0.36.0", but is now "v0.37.0"
golang.org/x/tools was audited at "v0.43.0", but is now "v0.45.0"
licensecheck: 8 problems; regenerate tools/licensecheck/dependencies.yaml with licensecheck --mode=fetch, or amend governance/licensing.yaml

Package.Serialize calls objectpath.For separately for each object, which
creates a fresh encoder each time. Reuse one encoder for the package and
update x/tools to v0.45.0, which includes the upstream fix for quadratic
object-path searches [1, 2].

Update the affected Go dependencies, their Bazel declarations, and their
license audit records together. Preserve package facts and exported
object facts through a serialization round trip, and add a benchmark of
the serializer over increasing numbers of fields. Reset the encoder
after a recovered panic so a failed lookup cannot leave an incomplete
index for subsequent objects.

[1]: golang/go#78893
[2]: golang/tools@093c2dd

Assisted-by: Codex
@tamird
tamird force-pushed the nogo-objectpath-encoder branch from 7d8a944 to 63a6d5e Compare September 24, 2026 12:43
@tamird

tamird commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

Updated!

@tamird

tamird commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

@kerumeto could I get a fresh import please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants