Skip to content

perf: avoid per-character models in PDFMiner parsing - #101

Open
dexhunter wants to merge 1 commit into
Filimoa:mainfrom
dexhunter:perf/textspan-parsing
Open

dexhunter wants to merge 1 commit into
Filimoa:mainfrom
dexhunter:perf/textspan-parsing

Conversation

@dexhunter

Copy link
Copy Markdown

Refs #1 (the parsing performance item).

The PDFMiner path builds a Pydantic CharElement for every glyph before grouping text into spans. This change streams the same fields into span grouping and reuses the rounded size for consecutive glyphs with the same raw size. Annotation inheritance, whitespace compression, style boundaries and final TextSpan validation are preserved.

On the repository's 171-page meta-2022-10k.pdf, default DocumentParser().parse() took a median 27.57s versus 28.54s at the base revision: 0.97s less (3.4%). These are three fresh-process samples per revision on the same shared CPU host with Python 3.12.13. Each process parsed the document twice; all six final outputs matched the baseline after normalizing generated IDs and access dates. This is one workload, not a general speedup estimate.

Validation:

  • Four existing PDFMiner tests passed. The existing conftest.py changes into a missing directory, so the targeted run used --noconftest with the upstream image fixture in an isolated overlay; the full suite was not run.
  • Native glyph, annotation and italic/bold-italic checks matched the baseline.
  • Scoped Ruff reports the same pre-existing F841 for subtype on both revisions; this patch adds no lint findings.

Optimization run: seven distinct valid candidates, with the baseline and development control separate. One rejected proposal is retained without a score. The selected source was replayed from this exact commit.

Developed and reviewed with AI assistance (GPT-6 Sol implementation and Astra review); no human review is claimed.

Refs Filimoa#1 (TextSpan parsing performance item)

Co-Authored-By: Aiden <aiden@weco.ai>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant