Cognitive3D and HarmonEyes Bring Cognitive State Measurement into XR Session Analytics

A Clearer View of Performance Inside XR

As XR becomes more widely used for training, simulation, defense, research, and product validation, teams need to understand not only what users did, but what state they were in while they were doing it. A missed step, delayed response, or poor decision can look simple in a completion report, but the underlying cause may involve cognitive load, fatigue, readiness, attention, or the way the environment shaped the user’s focus.

That is why Cognitive3D is partnering with HarmonEyes, a division of RightEye, to bring real-time cognitive performance intelligence into XR session analytics. Through this partnership, HarmonEyes’ Theia™ eye-tracking AI models will be integrated into the Cognitive3D platform, adding cognitive-state outputs such as Mental Workload, Fatigue, Mental Readiness, and Attention to the spatial behaviour and performance data already captured inside Cognitive3D.

The result is a richer session record for XR teams that need to understand how people perform inside immersive environments. Instead of reviewing session behaviour and cognitive-state indicators separately, teams can evaluate them together in the full 3D context of the experience, helping them connect moments of strain, readiness, focus, or fatigue to the decisions and outcomes that matter.

Understanding the State Behind the Behaviour

Cognitive3D already helps teams build a complete record of XR sessions by capturing scenes, session activity, gaze, objectives, inputs, device data, and performance context. That data can then be explored through 3D session replay, session details, objective analysis, dashboards, exports, and AI-ready workflows.  

The HarmonEyes integration adds another dimension to that record by bringing cognitive-state outputs directly into the session analytics workflow. With Theia’s models flowing into Cognitive3D, teams can review signals such as workload, fatigue, readiness, and attention alongside the spatial behaviour, gaze, task progress, and event data that already show what happened during the experience.

This matters because behaviour alone does not always explain performance. A trainee may miss a required step because the instruction was unclear, because the interface pulled attention away from the cue, because the scenario introduced too much complexity at once, or because fatigue began to affect response quality. By connecting cognitive-state outputs to the exact moment and environment where performance changed, teams gain a clearer way to investigate why an outcome occurred.

For example, a training team reviewing a complex simulation might not only see that a trainee missed a required step. They may be able to examine whether that miss happened during a period of elevated workload, narrowing attention, or declining readiness. A research team may be able to align cognitive-state signals with spatial behaviour, gaze, and task flow. A product or UX team may be able to identify moments where cognitive strain appears to coincide with confusing instructions, interface friction, or poor scene design.

What the Integration Adds to Cognitive3D

The partnership brings HarmonEyes’ Theia™ models into the Cognitive3D session analytics workflow, adding cognitive-state context to the behavioural, spatial, gaze, objective, and performance data already captured in XR sessions. These outputs help teams move beyond simply reviewing what happened and begin understanding the human-state factors that may have influenced performance in the moment.

• Mental Workload The Mental Workload model tracks mental effort in real time, helping teams identify moments when a user may be approaching high cognitive load. In an XR training or simulation environment, this can help reveal where a task, interface, scenario, or decision point may be demanding too much cognitive effort. When viewed alongside Cognitive3D session replay and objectives, workload signals can help teams understand whether performance issues were tied to task complexity, environmental design, instruction clarity, or pressure within the scenario.

• Fatigue
  The Fatigue model tracks alertness in real time and helps indicate when fatigue may begin to affect processing speed, reaction time, or situational awareness. This is especially useful in longer training sessions, repeated trials, or high-pressure simulations where performance may decline over time. By aligning fatigue outputs with session activity, teams can better understand whether a missed cue, delayed response, or change in behaviour may have occurred during a period of reduced alertness.

• Mental Readiness
  Mental Readiness measures remaining cognitive capacity and helps teams understand when a user’s reserve may be depleting. This gives training and research teams a way to examine whether a participant had the capacity to take on additional complexity, respond to changing conditions, or make effective decisions during a scenario. In Cognitive3D, these outputs can be reviewed in context with user movement, gaze, objective progress, and event timelines to show how readiness changed across the full experience.

• Attention
  The Attention model interprets attention in the context of the user’s environment, including what they are looking at and whether their attention is broad or narrow. This can help teams understand whether users are focused on the right cues, scanning the environment effectively, or narrowing their attention at critical moments. When paired with Cognitive3D’s gaze, fixation, object interaction, and spatial replay data, attention outputs can provide a more complete view of how users perceive and respond to immersive content.

Together, these model outputs are designed to appear alongside the existing Cognitive3D session record, giving teams a more complete way to evaluate performance inside immersive environments. Instead of reviewing cognitive signals in isolation, teams can connect them to the exact scenes, tasks, decisions, and interactions that shaped the user’s experience.

Giving High-Stakes XR Teams Stronger Performance Evidence

In high-stakes XR, the question is rarely just whether the user completed the task. Teams also need to know whether the user completed it confidently, whether they noticed the right cues, whether they were overloaded at a critical decision point, and whether fatigue or attention may have affected the outcome. Those questions are difficult to answer with completion data alone.

This is especially important in defense, simulation, research, and advanced training environments where XR is used to evaluate readiness, procedural performance, decision-making, and human response under pressure. In those settings, teams need objective evidence that helps them understand not only the final result, but the conditions that shaped that result throughout the experience.

By combining Cognitive3D’s spatial analytics with HarmonEyes’ cognitive-state models, teams gain a more defensible way to connect user state with behaviour, decisions, and outcomes. The value is not just more data. It is a more complete record of performance that can help teams validate training effectiveness, refine scenario design, improve research workflows, and better understand how people respond inside immersive environments.

Built Directly into the XR Analytics Workflow

The integration is delivered through native support in the Cognitive3D SDK, with initial availability through the Cognitive3D Unity SDK. Theia’s real-time human-state outputs are customizable via Theia Agent, and the resulting model outputs flow into the Cognitive3D session record while eye-tracking data stays on-device.

For teams already using Cognitive3D, that means cognitive-state outputs can be reviewed in the same environment where they already analyze session replay, objectives, gaze, events, and aggregate dashboards. This keeps the workflow centered on the session itself, rather than forcing teams to move between disconnected tools or interpret cognitive signals without the spatial and behavioural context that gives them meaning.

That context is what makes the combined workflow powerful. A spike in workload becomes more meaningful when teams can see the task being attempted, the objects involved, and the environment surrounding the user. A fatigue signal becomes more useful when it is aligned with timing, movement, and decision points. An attention signal becomes more actionable when it is connected to the scene, object, interface, or training cue the user was engaging with at that moment.

A Stronger Foundation for XR Validation

As XR becomes more central to training, research, and operational decision-making, teams need stronger evidence from immersive sessions. They need to know what users did, where users looked, how users moved through the environment, whether objectives were completed, and how performance changed across users, cohorts, scenarios, and versions.

With HarmonEyes, Cognitive3D customers can add cognitive-state context to that record. That creates a more complete foundation for understanding performance in XR, especially when teams need to evaluate how users respond under complexity, pressure, fatigue, or shifting attention.

This partnership is a step toward making cognitive performance intelligence a standard part of XR analytics. For organizations using XR to train, test, research, or validate human performance, it offers a way to connect behaviour and cognitive state inside the full context of the immersive experience.

Cognitive3D and HarmonEyes will be demonstrating the joint workflow at AWE USA in Long Beach, with Cognitive3D at Booth 929 and HarmonEyes at Booth S13. To learn more about enabling HarmonEyes models inside Cognitive3D, visit Cognitive3D or HarmonEyes.

Interested in partnering with Cognitive3D? We work with platforms, studios, and tools across the XR ecosystem to bring spatial analytics into the workflows teams already use. If you're building something where measurement would add value for your users and customers, we'd like to hear from you. Find out more here.