GigaChat Audio: Time-aware Large Audio Language Model

Existing audio-conditioned large language models struggle with temporal grounding, especially in long recordings such as meetings, podcasts, and lectures. These models often fail to reliably answer "when" an event occurred, producing non-parseable, coarse, or inaccurate timestamps. This limitation hinders their use in applications where verifiability is critical, as users cannot easily navigate to the specific audio segment supporting a model's claim.

The model, GigaChat Audio, is built by attaching an audio front-end to a 10B-A1.8B Mixture-of-Experts text LLM. To enable temporal awareness, the model's input stream interleaves continuous audio tokens with periodic time markers (inter-timings). The model was fine-tuned on a new 10k+ hour dataset of synthetic supervision generated from timestamped transcripts using a text-only LLM. The data generation pipeline uses transcript slicing to mitigate front-loading bias and a verifier to ensure global consistency. The model's performance was evaluated on temporal grounding, fragment description, and timed summarization tasks across various audio durations.

Main Results:

GigaChat Audio demonstrates strong temporal grounding on both short and long audio, in contrast to baselines that degrade significantly with increased duration. The inclusion of periodic inter-timings is shown to be critical for long-form performance.

Model	AudioGrounding (7-10s, mIoU ↑)	AMI (15-50m, MAE ↓)	TGr (20-40m, mIoU ↑)	Summarization (20-40m, Tm ↑)
Qwen3-Omni-30B	50.8	290.5s	3.6	43.7
TimeAudio	58.8	–	–	–
Gemini 3 Flash	41.7	1.00s	56.1	73.6
Ours (inter=60s)	45.1	3.50s	53.8	76.7
Ours (inter=7s)	39.7	1.50s	65.2	79.0
Ours (w/o inter-timings)	24.5	66.0s	14.2	69.2

• The ablation without inter-timings shows a collapse in long-form temporal grounding accuracy, with mIoU on the 20-40 minute task dropping from 53.8 to 14.2.
• The model was evaluated on several benchmarks: AudioGrounding (AGr), AMI Meeting Corpus, a time-aware subset of DCASE Audio QA (DAQA), and custom benchmarks for Temporal Grounding (TGr), Fragment Descriptions, and Timed Summarization.
• For timed summarization, metrics included a judge-based timing structure score (Tm), an aggregate content score (AES), and the share of round-duration segments (Rd). The main model achieved Tm=76.7, AES=88.1, and Rd=16.9.

Length Extrapolation:

• Experiments show asymmetric generalization across audio durations. Models trained only on short audio fail to generalize to long recordings, while models trained only on long audio show degraded performance on short-audio tasks.
• A model trained on a mixture of all durations outperformed specialized models across nearly all evaluation lengths up to 120 minutes.

Ablation Studies on Inter-timings:

The frequency and format of temporal anchors were studied to find a balance between accuracy and computational cost.

Inter-timing Frequency Trade-off

Frequency	mIoU (↑)	MAE (s) (↓)	Added Token Ratio
7s	63.0	1.5	16.0%
15s	59.9	2.0	7.5%
30s	55.5	2.5	3.7%
60s	50.9	3.0	1.9%
120s	41.3	5.0	0.9%
240s	31.0	8.5	0.5%

• Even sparse anchors (e.g., every 60s) allow the model to interpolate second-level precision, achieving a median error of 3.0 seconds.
• Different text formats for the 60s markers were tested. The canonical hh:mm:ss format achieved 50.9 mIoU, while shorter formats like m:0 (47.1 mIoU) and m: (43.8 mIoU) offered comparable performance with fewer tokens. A pure seconds format (sec) degraded performance significantly to 20.9 mIoU.
• Using special, dedicated timing tokens instead of plain-text digits required a much higher proportion of temporal grounding data in the training mixture to match the performance of plain-text timestamps.

This work introduces a method for robust temporal grounding in very long audio recordings (up to 120 minutes) by interleaving periodic time markers with audio tokens. The authors release GigaChat Audio, an open-weight model specialized for this task, and a new 10k+ hour synthetic dataset with a novel generation pipeline designed to produce supervision for long-form temporal question answering and summarization.

The evaluation of fragment descriptions and timed summaries relies on an LLM-as-a-judge protocol, where a separate large language model scores the generated text against a reference. The authors note that such automatic metrics are known to correlate imperfectly with human judgments.