Skip to content

Live Translate 3.5: same-language speech with echo_target_language=False (official SDK 3.6.1) #3402

Open

Description

@GrayWei444

Summary

gemini-3.5-live-translate-preview intermittently returns same-language speech with an actual outbound echoTargetLanguage:false setup, using the unmodified official Python SDK 3.6.1, explicit Developer API v1beta, and synthetic PCM. This reproduces without our application, a microphone, playback, caption-to-TTS conversion, or SDK/transport patches.

Please help triage or route this to the Live Translate service/model team. The serializer demonstrably sends the requested false value; we are not asserting an SDK serialization bug or a confirmed internal Google cause.

Reproduction and captured evidence

Full English report, standalone official-SDK reproducer, and synthetic evidence ZIP encoded as base64: https://gist.github.com/GrayWei444/cceb59e2e0f76b194f28a1aa6696820b

The ZIP SHA-256 is c5bb6625a61ca9402de6726b68c369ec0c6ec60c2ae9e645a3d13042f69e2d89. Download package.zip.base64 from that gist, decode it to package.zip, extract in a new directory, and check manifest.json. The gist also exposes report.txt and reproduce.py directly. Retained original frame payloads are unchanged; usage metadata and session-resumption handles were omitted. No keys, account/project identifiers, app source or personal recordings are included.

from pathlib import Path
import base64
Path("package.zip").write_bytes(base64.b64decode(Path("package.zip.base64").read_bytes()))

Gemini Live Translate same-language speech with echo_target_language False
Technical report to Google | 2026-11-19 UTC

Summary and request

We reproduced intermittent same-language speech from gemini-3.5-live-translate-preview with the unmodified official Python SDK 3.6.1 and an actual outbound echoTargetLanguage:false configuration. English audio sent to an English target can produce spoken English; Mandarin audio sent to a Chinese target can produce spoken Chinese. These are model audio bytes received from the API, independent of our Android/iOS application, caption rendering, TTS, microphone, headphones, or acoustic feedback.

Please confirm whether suppression of input already in the target language is a strict supported behavior or best effort, whether a supported configuration can make it reliable, and whether this can be routed to the Live Translate backend/model team for investigation. We are reporting behavior observed through the SDK, not asserting that the SDK serializer is at fault. The internal Google model/service cause remains unconfirmed.

Our product will temporarily accept this behavior. We are not deploying a client-side filter as a purported fix.

Expected behavior

The Live Translate guide describes echo_target_language=False as suppressing speech already in the target language. A source utterance in another language should still be translated. The baseline below uses a fixed target, explicit false, both transcription requests, and no system instruction. The current guide itself uses both transcription requests in its SDK examples.

https://aistudio.google.com/docs/live-api/live-translate

Environment and method

Model: gemini-3.5-live-translate-preview
API: Gemini Developer API, explicit v1beta, vertexai=False
Endpoint host: generativelanguage.googleapis.com
SDK: google-genai==3.6.1; websockets==17.4; Python 3.12.13
Host: Linux VPS; sequential independent sessions, maximum concurrency one
Input: pre-generated synthetic speech, mono signed 16-bit little-endian PCM at 16 kHz, sent in 100 ms chunks at real time; eight seconds of digital silence appended, then one audio_stream_end and four seconds of receive drain
Output: modelTurn.parts[].inlineData PCM at 24 kHz; no synthesized speech from text
Instrumentation: standard websockets Frame logging only; no connect/send/recv override, SDK patch, or application import
SDK integrity: 676 installed SDK files verified against distribution RECORD before and after the isolation tests

Synthetic source A: I would like a small cup of coffee without sugar.
Synthetic source B: 請幫我訂一張去曼谷的車票。
Voices: macOS say Samantha and Tingting, generated earlier to files; no personal recordings.

Actual captured baseline setup

{"setup":{"model":"models/gemini-3.5-live-translate-preview","generationConfig":{"responseModalities":["AUDIO"],"translationConfig":{"echoTargetLanguage":false,"targetLanguageCode":"en"}},"inputAudioTranscription":{},"outputAudioTranscription":{}}}

Strong counterexamples and a quiet control

  1. English source A -> target en, explicit false, no system instruction.
    The returned PCM contains an English source fragment supported independently by SenseVoice (c of coffee without sugar) and Whisper (Cop of coffee without sugar). This is not a claim of exact full-sentence transcription. The English input captions identify English; output captions have no text in this run, yet audio is present. Speech starts before audio_stream_end.
    Actual setup packet SHA-256:
    0b1d38f2f02605c58095c4167154d818d592d11530115334fd46c6ad86072b96
    Source-only PCM SHA-256:
    fea9104b9b4a490e30c68881de893ab7d10b58b8fda95ef8e66c31ad559c5e85
    Complete transmitted PCM including silence SHA-256:
    168ebcac0116910b85a984fed3b35d92bda33176503dc95262ee3350434c75ff
    Returned PCM SHA-256:
    f7e7cab85dac2cf515226c48df19cf5ab1737cea24bc1f8a9eeb1dadbff76b76

  2. Another independent English session has byte-identical setup and complete input PCM, but returns only low-level noise with no samples exceeding an absolute amplitude of 64. Its returned PCM SHA-256 is:
    50a77e74239a558ae9833e346ce5c36a8f2874ac6e04fa64c247595b25354d4b
    This demonstrates session variability, not an estimated failure rate.

  3. Mandarin source B -> target zh-Hant, explicit false, BOTH transcription requests omitted, no system instruction.
    Returned PCM still contains Chinese speech. SenseVoice recognizes 请帮我订一张去曼谷的车票; Whisper supports an incomplete Chinese fragment, 请我一张去的票. Captured model audio reconstructs exactly to the saved PCM. Thus caption-to-TTS duplication cannot explain this example. The service still emits transcription events despite the request omissions; omission is not evidence that the server disabled transcriptions.
    Complete transmitted PCM SHA-256:
    ff528bbdddc83ace8533ac35c7950e7c5239d00b0a11b95785513aacac9016f7
    Returned PCM SHA-256:
    b6596c1a531592ccb079c1199697f2388f05eedea82ef972491e97d51121fc82

The attached selected original frames preserve actual payload bytes for setup, input audio and serverContent. Usage metadata and session-resumption handles are omitted. Reconstructing all retained input/output audio matches the source plus silence and saved output exactly in all three attached cases.

Settings cross-check

After 12 isolation/control sessions, we ran a separate 25-session settings comparison. All 25 reached setup ready, sent complete planned input, and closed normally with code 1000. These are test-attempt counts, not 37 quality passes or a population failure rate. Every proposed remedy listed below has at least one same-language speech counterexample in these limited tests; quiet outcomes in other runs are retained.

Setting Representative observed result
Explicit false, fixed en, both captions English source fragment, as above
Remove input transcription request Chinese source-language speech
Remove both transcription requests Chinese source-language speech, as above
Strict unidirectional system instruction English coffee with; Chinese speech also observed
Minimal terminology system instruction Chinese source-language speech
Input transcription language_codes hints Chinese source-language speech
Omit echo field, relying on default English coffee without sugar and Chinese speech
One correct actual-source context sentence Chinese source-language speech
Target en-US instead of en English coffee fragment
Target zh instead of zh-Hant Chinese fragment; partial ASR agreement only

settings-counterexamples.json provides the exact actual setup, PCM hashes and both recognizer outputs for each representative setting. The hints are present on the wire as language_codes; they were not silently dropped by the SDK. Both supported ProtoJSON field-name spellings were accepted by our corrected audit.

The strict English instruction tested was: You are a strict unidirectional voice translator (zh-Hant to en). Output ONLY the translated English audio/text. Never repeat or summarize the source Mandarin.
We tested its Chinese-target mirror as well. This is a diagnostic comparison, not a recommended configuration: the current guide warns against output-format instructions and instructions conflicting with the configured target.

We also tested the November 18 developer suggestion with the correct actual source context: The speaker is speaking English. / The speaker is speaking Mandarin Chinese. The Chinese case still returned Chinese. This is a diagnostic use of known synthetic source language, not a deployable assumption about arbitrary live input or proof that every context instruction is ineffective.

Cross-language controls produced recognizable translated content, including Chinese -> English and English -> Chinese. An English echo=true control returned the source; a Chinese echo=true control produced only a small artifact and is not counted as a successful positive control. We do not claim complete translation-quality acceptance from these controls.

Integrity limitations and preserved test-tool failures

One of the 25 settings runs, a Chinese -> en-US cross-language control, originally saved 12,000 fewer bytes than appeared in raw frames. The missing final 250 ms was entirely zero-valued PCM. Our helper had cancelled its receiver before the SDK session context finished closing. That run is excluded from complete-capture/quality acceptance, with its original discrepancy retained privately. We corrected only the helper shutdown order and verified raw-to-saved equality in the next five sessions. No SDK or product code was patched. The other 24 settings runs, including the cited same-language counterexamples, have exact raw-to-saved PCM equality. A missing zero tail cannot generate speech already present in the original server frames.

Two independent offline recognizers support the cited language/phrase presence; we have not claimed human listening, exact whole-sentence equality, or ASR-derived replay counts. One recognizer hallucinated multiple text repetitions on a short crop in another run; those counts are not used here. Tiny ambiguous artifacts alone are not treated as decisive same-language speech evidence.

Reproduction package

Install requirements.txt in an isolated Python 3.12 environment. Supply your own GEMINI_API_KEY through the environment. No credential is included.

python reproduce.py input-en.wav --target en --output new-en-run
python reproduce.py input-zh.wav --target zh-Hant --captions none --output new-zh-run

Each invocation makes one API connection and may incur API charges. It records selected original frames, output.wav, output.pcm, configuration, hashes and errors without microphone or playback. Behavior is intermittent, so a quiet session does not invalidate the captured counterexample. The distributable adaptation was syntax/configuration/fixture checked offline; it was not used to manufacture the attached historical results or run extra cloud tests for this report.

Related first-hand reports

https://discuss.ai.google.dev/t/gemini-3-5-live-translate-preview-intermittently-outputs-english-instead-of-configured-target-language/174290
A different developer reported source-English leakage with a Spanish target, using JavaScript and v1alpha constrained ephemeral tokens. This differs from our v1beta/API-key reproduction and does not establish an identical internal cause. The thread was automatically closed, not publicly marked repaired.

https://discuss.ai.google.dev/t/gemini-3-5-live-translate-preview-systeminstruction-is-accepted-and-silently-ignored-plus-polarity-reversals-on-long-sentences/179912
The November 18 reply motivates our source-context comparison. It concerns broader wrong-language behavior; it is not a verified solution for same-target repetition. Please clarify current supported context-instruction behavior if relevant.

Requested follow-up

Please confirm the intended echo=false contract, identify any supported corrective configuration, and provide a backend issue reference or status if this needs service/model investigation. If this SDK tracker is not the owning queue, please route or identify the appropriate Live Translate team. We can respond with additional sanitized evidence; all shared audio here is synthetic. We have not established Google's internal root cause or asserted that all client-side issues in our product are excluded.

Activity

  1. added
    priority: p2Moderately-important priority. Fix may not be included in next release.
    type: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.
    on Nov 19, 2026
  2. Venkaiahbabuneelam commented on Nov 19, 2026

    @Venkaiahbabuneelam

    Hi @GrayWei444,

    In gemini-3.5-live-translate-preview, echo_target_language: false relies on dynamic streaming language identification on incoming audio chunks, which is currently best-effort rather than deterministic and can intermittently output same-language speech on short or boundary-adjacent utterances. Because the SDK correctly serializes and transmits echoTargetLanguage: false, this behavior originates directly within the model's streaming translation runtime.

    Adding conflicting system instructions can further interfere with the internal translation classifier, as outlined in the Gemini Live API documentation. If you require strict suppression while this model is in preview, you can conditionally discard inbound inline_data audio chunks whenever your source transcription matches the target language:

    async for response in session.receive():
        if response.server_content and response.server_content.model_turn:
            for part in response.server_content.model_turn.parts:
                if part.inline_data:
                    # Discard or forward audio based on upstream language classification
                    process_audio_chunk(part.inline_data.data)

    Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

priority: p2Moderately-important priority. Fix may not be included in next release.status:awaiting user responsetype: bugError or flaw in code with unintended results or allowing sub-optimal usage patterns.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions