Google researchers, working with Stanford, Oxford, and several U.S. universities, have introduced StreamVibe, a video agent designed to decide when to speak up while a person works. The paper says the goal is an assistant that watches someone cook or assemble furniture and offers help only when needed.
The researchers say existing video assistants handle clear commands, such as "Let me know when the water starts boiling," but rarely pick up on unspoken intentions. They also struggle to track which steps are already done, which can lead to repeated advice or late responses. When users change their requests partway through, the assistants often keep following the original instructions.
To measure this, the team built a benchmark of about 4,100 examples drawn from more than 183 hours of everyday life and software recordings. Gemini 2.5 Flash suggested when to offer help and what to say, and four human evaluators had to unanimously approve each example. The benchmark covers clear instructions, vague goals, and requests that change over time.
The team then trained StreamVibe-VL, which the article says is apparently based on Alibaba's Qwen3.5-9B. The model decides every second whether to stay quiet or speak. A second training stage uses reinforcement learning to make it more reserved so it does not interrupt