Conversation
Refs Filimoa#1 (TextSpan parsing performance item) Co-Authored-By: Aiden <aiden@weco.ai>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Refs #1 (the parsing performance item).
The PDFMiner path builds a Pydantic
CharElementfor every glyph before grouping text into spans. This change streams the same fields into span grouping and reuses the rounded size for consecutive glyphs with the same raw size. Annotation inheritance, whitespace compression, style boundaries and finalTextSpanvalidation are preserved.On the repository's 171-page
meta-2022-10k.pdf, defaultDocumentParser().parse()took a median 27.57s versus 28.54s at the base revision: 0.97s less (3.4%). These are three fresh-process samples per revision on the same shared CPU host with Python 3.12.13. Each process parsed the document twice; all six final outputs matched the baseline after normalizing generated IDs and access dates. This is one workload, not a general speedup estimate.Validation:
conftest.pychanges into a missing directory, so the targeted run used--noconftestwith the upstream image fixture in an isolated overlay; the full suite was not run.subtypeon both revisions; this patch adds no lint findings.Optimization run: seven distinct valid candidates, with the baseline and development control separate. One rejected proposal is retained without a score. The selected source was replayed from this exact commit.
Developed and reviewed with AI assistance (GPT-6 Sol implementation and Astra review); no human review is claimed.