feat(genai): surface audio token counts in UsageMetadata for TTS models #1821

Problem

ChatGoogleGenerativeAI does not surface audio token counts in UsageMetadata for TTS models (e.g. gemini-2.5-flash-tts). The raw API response includes per-modality token breakdowns via usage_metadata.prompt_tokens_details and usage_metadata.candidates_tokens_details, which contain entries with modality = "AUDIO" and their respective token_count.

langchain-core's UsageMetadata already has the right shape for this:

• InputTokenDetails.audio — audio input tokens
• OutputTokenDetails.audio — audio output tokens

Currently _get_usage_metadata (inlined in the response conversion function) only reads prompt_token_count, candidates_token_count, thoughts_token_count, total_token_count, and cached_content_token_count. The modality detail lists are ignored.

Expected Behaviour

For a TTS response, message.usage_metadata should include:

{
    "input_tokens": 10,
    "output_tokens": 0,
    "total_tokens": 10,
    "input_token_details": {"cache_read": 0},
    "output_token_details": {"audio": <n>},  # ← currently missing
}

Suggested Fix

After computing output_tokens, derive audio token counts from the modality detail lists:

audio_input_tokens = next(
    (e.token_count for e in (response.usage_metadata.prompt_tokens_details or []) if e.modality == "AUDIO"),
    0,
)
audio_output_tokens = next(
    (e.token_count for e in (response.usage_metadata.candidates_tokens_details or []) if e.modality == "AUDIO"),
    0,
)

Then include them in UsageMetadata:

input_token_details = {"cache_read": cache_read_tokens}
if audio_input_tokens:
    input_token_details["audio"] = audio_input_tokens

output_token_details = {}
if thought_tokens:
    output_token_details["reasoning"] = thought_tokens
if audio_output_tokens:
    output_token_details["audio"] = audio_output_tokens

Context

Discovered while experimenting with TTS model support (gemini-2.5-flash-tts) and ChatGoogleGenerativeAI. The candidates_tokens_details list is populated for TTS responses and is the only way to know how many audio tokens were generated.