Skip to content

PDF text increases prompt usage but not count_tokens on gemini-4-hydrogen #3379

Open

Description

@equialgo

Description

Adding embedded text to a 10-page PDF substantially increases gemini-4-hydrogen's reported input usage, but its count_tokens estimate is unchanged. The comparison below uses a blank PDF with the same page count and dimensions as a control.

Observed through google-genai; I have not isolated whether the cause is in the SDK or backend, or tested raw REST. Please redirect me if another tracker is more appropriate.

Environment details

  • Python 3.12.13
  • macOS 26.5.2 (arm64)
  • google-genai==3.3.0, Gemini Developer API
  • thinking_level=LOW; no system instruction, tools, or response schema

Steps to reproduce

  1. Extract the attached pdf-token-repro.zip.
  2. Set GEMINI_API_KEY, then run:
pip install google-genai==3.3.0
python pdf_token_repro.py --pages 10

The script generates both synthetic PDFs and calls count_tokens and generate_content with identical contents for each model. No real documents or sensitive data are included.

Actual result

One run on 2026-11-16:

PDF Model count_tokens prompt_token_count TEXT
text.pdf gemini-3.8-flash 5,609 5,329 9
text.pdf gemini-4-hydrogen 5,638 14,642 9,322
blank.pdf gemini-3.8-flash 5,609 5,329 9
blank.pdf gemini-4-hydrogen 5,638 5,704 384

All four requests report 5,320 IMAGE tokens. On Hydrogen, adding PDF text increases reported usage by 8,938 TEXT tokens without changing the estimate.

Expected behavior

count_tokens should account for PDF text included in prompt usage, or document the difference and provide a supported way to estimate input usage before generation.

Earlier report with additional Gemini API and Vertex AI observations.

Activity

  1. added
    priority: p2Moderately-important priority. Fix may not be included in next release.
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    on Nov 17, 2026
  2. Venkaiahbabuneelam commented on Nov 17, 2026

    @Venkaiahbabuneelam

    Hi @equialgo,
    Thank you for the clear report and reproduction steps! We have confirmed that this discrepancy occurs on the backend (where generate_content includes extracted PDF TEXT tokens in prompt_token_count while count_tokens only accounts for page IMAGE tokens) rather than in the google-genai Python SDK, we will file a bug internally.

    Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

priority: p2Moderately-important priority. Fix may not be included in next release.type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions