Real-time AI has traditionally been built as a pipeline. A user speaks, a speech recognition system converts the audio into text, a language model processes the request, and a speech synthesis system generates a response. Additional components handle turn detection, interruptions, background noise, and dialogue state.

This architecture works, but it also imposes a fundamental limitation: the model itself is usually not responsible for the interaction. It receives a completed turn, processes it, and generates a response. The surrounding software determines when the user has finished speaking and when the model should start.

Thinking Machines Lab is pursuing a different approach. Its research preview, TML-Interaction-Small, is based on what the company calls an interaction model: a model designed from the beginning to perceive and respond continuously rather than operating through rigid conversational turns.

The distinction is more important than it may initially appear. A system that can listen while speaking, react to visual events without waiting for a prompt, track timing directly, and continue working on complex tasks in the background requires a different model architecture and training strategy.

TML-Interaction-Small is an early attempt to build such a system. It is a 276-billion-parameter Mixture-of-Experts model with approximately 12 billion active parameters. It processes audio, video, and text as continuous streams divided into 200-millisecond micro-turns. Its architecture combines a fast interaction model with a separate background model responsible for more demanding reasoning and tool-based tasks.

The result is an approach that prioritizes a different dimension of AI performance: not simply how intelligent a model is, but how naturally it can participate in an ongoing interaction.

Why Conventional Real-Time AI Systems Struggle With Interaction

Most voice AI systems are built around a sequence of discrete events.

The user speaks. The system detects the end of the utterance. Audio is processed. The model generates a response. The system begins speaking. The next user turn starts afterward.

This design creates a clear boundary between listening and responding. It is convenient for engineering, but human conversation rarely works this way.

People interrupt one another. They correct themselves halfway through a sentence. They use short acknowledgements such as “right” or “uh-huh” while listening. They react to visual information without explicitly describing it. They start speaking before another person has completely finished. They also change their minds while talking.

A conventional voice model must usually rely on external software to approximate these behaviors. Voice activity detection determines when a person is speaking. Turn-taking logic decides when the model should respond. Dialogue managers handle interruptions. Separate vision systems process video when it is available.

This collection of components can produce a convincing experience, but the interaction is still assembled from multiple systems with different objectives and timing assumptions.

The central idea behind an interaction model is to move more of that behavior into the model itself. Instead of treating a conversation as a sequence of completed user inputs and completed model outputs, the model processes the interaction as a continuous stream.

In TML-Interaction-Small, input and output are handled in short, interleaved 200-millisecond intervals. The model does not necessarily wait for a complete sentence before beginning to interpret the incoming information or prepare a response.

This enables several capabilities that are difficult to reproduce with a strictly turn-based architecture:

• speaking while continuing to listen;
• responding to interruptions without losing context;
• reacting to visual events as they happen;
• recognizing when speech is directed toward the model or another person;
• tracking elapsed time as part of the interaction;
• continuing complex reasoning in the background while remaining available for conversation.

These capabilities are not simply improvements to speech synthesis or latency. They reflect a different way of modeling the interaction itself.

What Is TML-Interaction-Small?

TML-Interaction-Small is the first public research implementation of Thinking Machines Lab’s interaction model concept.

The model is a large Mixture-of-Experts system with 276 billion total parameters and approximately 12 billion active parameters for a given computation. It was trained to process multiple modalities, including audio, video, and text, as continuous streams rather than as isolated, completed requests.

The model’s basic operating unit is a 200-millisecond micro-turn. During each interval, the system can receive new input and produce output. This gives it the ability to maintain an active interaction instead of switching between two separate states: listening and speaking.

The architecture also includes a background model. This separation is important because real-time interaction and complex reasoning have conflicting requirements.

A conversational model must respond quickly. A reasoning system may need considerably more time to plan, call tools, inspect information, or complete a multi-step operation. If the same model has to perform both tasks sequentially, the user may be forced to wait in silence while the system works.

TML-Interaction-Small attempts to avoid this problem by dividing the responsibilities. The interaction model maintains the live conversation, while the background model handles more demanding tasks asynchronously. When the background process produces a result, the interaction model can incorporate it into the ongoing conversation.

In theory, this allows a user to continue talking, ask follow-up questions, or change topics while a longer task is still running.

That is a significant architectural difference from systems that effectively pause the conversation until a tool call or reasoning process has finished.

The Four Capabilities That Define the Model

Speaking and listening at the same time

The most obvious consequence of continuous processing is that the model does not have to stop listening simply because it is currently speaking.

This makes possible interaction patterns that are difficult for conventional voice systems. A user can interrupt the model, correct a statement, or provide additional information while the response is still being generated.

The same mechanism can also support simultaneous interpretation. A person can begin speaking in one language, and the system can begin translating before the entire sentence has been completed.

The model can also provide conversational feedback while the user continues speaking. Short responses such as acknowledgements or prompts to continue become part of the interaction rather than separate turns.

In practical terms, the system is no longer limited to the pattern:

User speaks -> model waits -> model responds -> user speaks again.

Instead, both sides can remain active at the same time.

Reacting to video without an explicit request

TML-Interaction-Small is designed to process video alongside audio and text. This gives it another capability that conventional voice-only systems do not possess: proactive reactions to visual events.

For example, a model observing a person exercising could count repetitions as they happen. It could notice a change in posture, identify an object entering the camera frame, or react when a visual event meets a predefined condition.

The important distinction is that the user does not necessarily have to describe what is happening first.

This type of behavior changes the role of a multimodal model. Instead of functioning as a system that analyzes visual input only after receiving a question, it can potentially monitor a stream and decide when an event is relevant enough to mention.

The technology is still far from perfect. The reported RepCount-A results indicate that visual counting remains a difficult problem, with only 33.4 percent of cases falling within one repetition of the reference count. However, the underlying capability is technically significant because it represents a different interaction pattern from standard conversational AI.

Handling interruptions and self-corrections

Human speech is full of incomplete sentences, corrections, hesitation, and overlapping conversation.

A person may say, “Schedule the meeting for Thursday, no, actually Friday morning.” A conventional turn-based system may process the first statement before the correction arrives, depending on how its interruption logic works.

An interaction model can theoretically continue processing the stream and update its interpretation as new information arrives.

The same applies to backchanneling. When a user says “yes,” “right,” or “I see” while the model is speaking, the system should ideally understand whether the user is interrupting, acknowledging the response, or speaking to somebody else nearby.

These distinctions are difficult because they depend not only on the words themselves but also on timing and context.

This is one of the areas where the interaction model concept is particularly interesting. Timing is not merely an engineering detail. It becomes part of the information available to the model.

Performing complex work without disappearing from the conversation

Long-running tasks are one of the most frustrating aspects of AI assistants.

A user asks the system to perform a multi-step operation. The assistant begins working, calls several tools, processes the results, and remains silent until everything is complete.

The alternative implemented in TML-Interaction-Small is to keep the real-time interaction active while a separate background process handles the complex task.

A user can ask a question, receive a quick response, add more information, or change the subject. Once the background operation is complete, the system can bring the result back into the conversation.

This architecture resembles the way a human assistant might work. A person can continue a conversation while simultaneously asking a colleague to investigate something or waiting for a separate process to finish.

The challenge is not merely technical execution. The model must also decide when and how to introduce the result without disrupting the current conversation.

How the Model Performs

Thinking Machines Lab evaluates TML-Interaction-Small using two broad categories of benchmarks.

The first measures interaction quality. These tests focus on interruptions, backchanneling, background speech, turn timing, and related behaviors.

The second evaluates more conventional model capabilities, including instruction following, reasoning, response quality, and tool use.

This distinction is important because the model’s main architectural advantage is expected to appear in the first category.

Interactivity and interruptions

On FD-bench v1.5, which evaluates behavior involving interruptions, backchanneling, conversations with other people, and background speech, TML-Interaction-Small reportedly achieved a score of 77.8.

For comparison, GPT-Realtime-2.0 scored 46.8 under minimal reasoning settings and 47.8 at its highest tested reasoning configuration. Gemini 3.1 Flash Live Preview achieved 54.3 under the configuration cited in the comparison.

The difference is substantial.

However, interpreting the result requires some caution. A benchmark specifically designed around the architectural goals of one model family may not capture every aspect of real-world conversational quality. Independent testing across a wider range of environments will be needed to determine how well these results translate into practical applications.

Still, the result supports the central claim behind the interaction model architecture: continuous interaction can provide advantages that are difficult to obtain by simply adding more logic around a conventional model.

Turn-taking latency

TML-Interaction-Small reportedly achieves approximately 0.40 seconds of turn-taking latency on FD-bench v1.

Gemini 3.1 Flash Live Preview is reported at approximately 0.57 seconds, while GPT-Realtime-2.0 reaches approximately 1.18 seconds under minimal reasoning settings and 1.63 seconds at the highest tested reasoning level.

Latency is particularly important in voice applications.

A delay of a few hundred milliseconds can feel natural. A delay approaching one or two seconds can make a conversation feel artificial, especially when the user is expecting an immediate response.

This creates an important design trade-off. More reasoning can improve the quality of an answer, but additional computation can also make the interaction feel slower.

An architecture that separates fast interaction from deeper background processing attempts to address both requirements simultaneously.

General intelligence and instruction following

On Audio MultiChallenge, TML-Interaction-Small reportedly achieved an accuracy of 43.4 percent.

GPT-Realtime-2.0 at its highest reasoning configuration scored 48.5 percent on the same benchmark, placing it ahead on this particular measure.

This illustrates an important point. TML-Interaction-Small is not simply a universal replacement for every other real-time model.

The model’s strongest advantage appears to be interaction quality and responsiveness. On some traditional intelligence benchmarks, more heavily reasoning-oriented systems remain ahead.

The difference is therefore not necessarily about one model being categorically better. The more useful question is which type of performance matters most for a given application.

A customer service assistant that needs to maintain a natural conversation may benefit more from low latency and robust interruption handling. A research assistant performing difficult multi-step reasoning may prioritize accuracy over immediate response time.

Response quality and tool use

FD-bench v3 evaluates response quality and tool usage in audio-based scenarios.

With the background agent enabled, TML-Interaction-Small reportedly achieved 82.8 percent response quality and a 68.0 percent pass@1 score for tool-dependent tasks.

GPT-Realtime-2.0 achieved 80.0 percent and 52.0 percent under minimal reasoning settings, and 81.0 percent and 58.0 percent at the higher reasoning level.

The tool-use results are particularly interesting. They suggest that the separation between real-time interaction and background task execution may be useful for more than latency alone.

The conversational layer can remain responsive while another component performs the planning and execution required to complete a complex task.

New Benchmarks for Proactive Interaction

One of the difficulties in evaluating interaction models is that conventional language model benchmarks are not designed to measure continuous behavior.

A model can achieve a high score on a question-answering test while still being poor at handling interruptions or reacting to events at the correct moment.

To address this problem, Thinking Machines Lab reports results on several benchmarks focused specifically on timing and proactive behavior.

These include:

• TimeSpeak, which evaluates whether the model initiates speech at the correct time;
• CueSpeak, which measures responses to verbal cues;
• RepCount-A, which evaluates visual action counting;
• ProactiveVideoQA, which tests whether the model initiates speech in response to visual events;
• Charades temporal localization, which measures whether the model recognizes when an action begins and ends.

The reported results include 64.7 percent macro-accuracy on TimeSpeak, 81.7 percent on CueSpeak, 33.4 percent off-by-one accuracy on RepCount-A, a 31.5 PAUC result on ProactiveVideoQA, and 30.4 mIoU on Charades temporal localization.

These numbers should be interpreted carefully. Several of these tests are relatively new and do not yet have the same level of independent validation as established language and reasoning benchmarks.

Nevertheless, they address a genuine gap in existing evaluation methods. If AI systems are expected to interact with people continuously, measuring only the correctness of their final answers is not enough.

A system can provide a correct answer at the wrong time and still deliver a poor user experience.

TML-Interaction-Small vs GPT-Realtime-2

The comparison between TML-Interaction-Small and GPT-Realtime-2 illustrates two different approaches to real-time AI.

GPT-Realtime-2 is designed as a highly capable real-time model with strong reasoning performance. TML-Interaction-Small focuses more heavily on the mechanics of continuous interaction.

On interactivity-oriented benchmarks, TML-Interaction-Small has a substantial reported advantage. Its turn-taking latency is also significantly lower.

GPT-Realtime-2, particularly at higher reasoning settings, performs better on some conventional intelligence benchmarks.

A simplified comparison looks like this:

Benchmark	TML-Interaction-Small	GPT-Realtime-2.0, minimal	GPT-Realtime-2.0, xhigh	Gemini 3.1 Flash Live, minimal
FD-bench v1 latency	0.40 s	1.18 s	1.63 s	0.57 s
FD-bench v1.5 average	77.8	46.8	47.8	54.3
FD-bench v3 response quality	82.8%*	80.0%	81.0%	68.5%
Audio MultiChallenge	43.4%	37.6%	48.5%	26.8%
IFEval, VoiceBench	82.1%	81.7%	83.2%	67.6%
IFEval, text	89.7%	89.6%	95.2%	85.8%

*With the background agent enabled.

The broader pattern is clear. TML-Interaction-Small is optimized around the idea that a conversational system should remain continuously present. GPT-Realtime-2 places greater emphasis on the depth of reasoning available within the model.

The best choice depends heavily on the application.

The Architecture May Be More Important Than the Benchmark Scores

The most interesting aspect of TML-Interaction-Small may not be any individual benchmark result.

The more important question is whether interaction should be treated as a property of the model itself.

For years, AI systems have been built as collections of specialized components. A speech recognizer handles audio. A language model handles reasoning. A speech synthesizer produces output. Additional software manages turn-taking, interruptions, tools, and context.

This modular approach has obvious advantages. Individual components can be upgraded independently, systems are easier to debug, and engineering teams can combine specialized technologies.

However, some aspects of human interaction are difficult to reproduce by simply connecting independent modules.

Timing, overlapping speech, visual attention, and conversational context are closely related. A model that processes all of these signals together may be able to learn relationships that are difficult to reconstruct through external logic.

This is the bet behind interaction models.

The approach is not necessarily a replacement for modular systems. In fact, TML-Interaction-Small itself uses multiple components, including a background model. The difference is that the model architecture is designed around continuous interaction rather than treating interaction as a wrapper around a conventional language model.

Potential Applications

If the approach scales, interaction models could be useful in several areas.

Simultaneous translation

A system could begin translating while a person is still speaking instead of waiting for the complete sentence.

This could reduce latency and make conversations between speakers of different languages feel more natural.

AI assistants

An assistant that can listen continuously, recognize interruptions, monitor visual context, and perform background tasks could behave more like an active collaborator than a conventional chatbot.

Education and tutoring

A tutoring system could observe a student’s work, detect hesitation or confusion, and intervene at the appropriate moment without waiting for an explicit question.

Accessibility

Continuous interaction could make voice interfaces more useful for people who rely on speech, audio, or visual assistance.

Robotics

Robots operating in real environments need to process multiple streams of information simultaneously. They must react to speech, visual changes, movement, and timing while continuing to perform tasks.

An architecture designed around continuous multimodal interaction could be a natural fit for these systems.

Collaborative work

A future AI system could participate in meetings, monitor discussions, retrieve information in the background, and contribute at appropriate moments rather than waiting for someone to explicitly address it.

The Remaining Challenges

The interaction model concept is promising, but many practical challenges remain.

The first is computational cost. Processing continuous audio and video streams requires significantly more resources than processing occasional text requests.

The second is reliability. A system that proactively speaks must make correct decisions about when speech is useful. An assistant that constantly interrupts the user could be more annoying than one that waits too long.

The third is privacy. A continuously listening and potentially continuously watching system raises obvious questions about data retention, local processing, consent, and security.

There is also a fundamental evaluation problem. Human interaction is difficult to reduce to a single score. A model may perform well on a benchmark but still feel unnatural in a real conversation.

Finally, the architecture must deal with conflicting priorities. A model may need to respond immediately while also reasoning about a complex task. The background-agent approach helps, but coordinating multiple processes without creating inconsistent answers remains a difficult engineering problem.

Availability and Future Development

TML-Interaction-Small is currently presented as a limited research preview. Wider access is expected to follow, while pricing and commercial availability are expected to be announced separately.

The “Small” suffix also suggests that the model is intended to be the beginning of a broader product family.

Larger models could potentially improve reasoning and multimodal understanding, but they would also introduce a significant serving challenge. A model that processes continuous streams must be fast enough to respond in real time, regardless of its theoretical intelligence.

This makes model efficiency particularly important. Mixture-of-Experts architectures, specialized hardware, caching, streaming inference, and asynchronous execution are likely to become increasingly important as interaction models grow in capability.

Conclusion

TML-Interaction-Small represents an interesting shift in how real-time AI systems can be designed.

The central idea is straightforward: interaction should not necessarily be treated as a software layer surrounding an intelligent model. It can be part of the model’s fundamental architecture and training process.

That distinction enables capabilities such as simultaneous listening and speaking, proactive reactions to visual events, more natural interruption handling, and background execution of complex tasks without forcing the conversation to stop.

The model is not universally superior to existing real-time systems. Its reported results suggest a clear advantage in responsiveness and interactive behavior, while more reasoning-intensive models can still lead on some conventional intelligence benchmarks.

That trade-off may eventually become less important as architectures improve.

The more significant development is conceptual. Real-time AI may be moving away from the idea of a chatbot that waits for a complete request and toward systems that remain continuously engaged with their environment.

If that direction succeeds, the next generation of AI assistants may not feel like applications that users call when they need something. They may behave more like persistent collaborators that listen, observe, reason, and act continuously.

That is the real promise of interaction models.