header-langage
简体中文
繁體中文
English
Tiếng Việt
한국어
日本語
ภาษาไทย
Türkçe
Scan to Download the APP
Partnered With:

Doubao Audio-Video Call Model Opens API: AI Can See, Hear, and Speak Simultaneously, and Even Initiates Conversations

Beating AI News, November 19 — Volcano Engine has officially opened the API for its Doubao real-time audio-video model SeedRealtime 1.0. The model was already launched on the Doubao App in August this year, and now third-party developers can also integrate this real-time video call capability into their own products.


SeedRealtime puts camera visuals, sound, and temporal order into a single end-to-end model. The AI can observe and listen to the user speak at the same time, while judging when to speak, reducing the latency and information loss caused by the traditional serial connection of multiple modules such as speech recognition, visual understanding, and speech synthesis.


In the official demo, when a user pours whole coffee beans into the coffee machine's portafilter, the AI proactively reminds them to grind first without waiting for a question. When visiting a museum, the user only needs to specify in advance which artifact to look for, and the model can continuously observe the camera feed and proactively alert when the target appears. It can also distinguish multi-person conversations and avoid being falsely triggered by nearby chatter in noisy environments.


ByteDance's human evaluation published in August this year showed that, compared with solutions that chain multiple models together, SeedRealtime reduced audio-video conversation rhythm issues by about half, with notable reductions in interruptions, delayed responses, and false triggers from background noise.

举报 Correction/Report
Correction/Report
Submit
Add Library
Visible to myself only
Public
Save
Choose Library
Add Library
Cancel
Finish