Collect human demonstration data for robotics and physical AI
What your robotics data program has to prove
Collecting more hours of demonstration data is not enough. To build a useful dataset, you need to know what happened in each task, whether it was completed, and which episodes are ready to use. These are the four things the record can show.
What each operator demonstrated
Capture head and hand movement, controller inputs, object interactions, and timing to show how each task was performed.
Whether the task ran as designed
Track each task, its steps, and the outcome of every attempt to show whether it ran as intended and where it broke down.
Which sessions need review
Review recordings for completeness and capture issues before your team decides what belongs in the dataset.
What the program produced
See how much data was collected and kept, the tasks and outcomes it covers, and the consent and provenance recorded alongside it.
Turn collection runs into episodes your pipeline can use
Track Demonstration Sessions
Build a complete record of each demonstration. Record app activity and capture device inputs so teams can replay, validate, and analyze with confidence.
Learn MoreExplore Operator Behaviour
Organize and evaluate how individual operators move, act, and perform by accessing episode data, replaying sessions, or connecting records to AI systems.
Learn MoreAnalyze Episodes at Scale
See how collection performs across the program. Monitor trends, compare demonstrations, and prepare aggregate data for pipelines, reporting, and AI analysis.
Learn MoreCapture what actually happened in the episode
Record Complete Demonstration Episodes
Every session is captured from start to finish: movement, interaction, timing, and where time was spent, saved as one continuous record ready for 3D playback and comparison.
Define Tasks, Goals & Events
Define tasks, steps, events, and success conditions, and configured events can mark attempts and outcomes for review: the task labels curation depends on.
Log Operator Input & Motion
Controller inputs, hand tracking, movement, rotation, and interaction timing are recorded together, tied to task events and attempt progress.
Capture the Operator's Perspective
Headset position and rotation are tracked throughout the task, so each episode keeps the egocentric viewpoint it was demonstrated from.
Capture Detailed Device Data
Headset model, OS, thermal status, play space, and participant identifiers are captured automatically. When episodes differ across setups or sites, the context is already there.
Trusted by teams measuring spatial behaviour at scale
Used by teams turning human sessions, operator behaviour, and 3D replay into structured spatial data.
Adding Cognitive3D to your capture workflow
- Engines and devices
- Pre-built SDKs for Unity, Unreal, and C++ (plus visionOS, AndroidXR, and WebXR)
- Consumer and enterprise headsets and capture devices
- Most tracking runs through the engine your team already uses
- 40+ engine, tool, and platform integrations
- Setup
- Add the SDK to the capture application you already run
- Define the tasks, objects, and events that matter
- Docs, guides, samples, and friendly support make setup fast and reliable
- Integration requirements and performance depend on your application and SDK setup.
- Data and export
- Episode and session data exportable at any time
- CSV, JSON, REST API, and MCP Server access
- Regional and deployment options on request
Replay the episode before it enters the dataset
Replay Episodes in 3D
Movement, gaze, actions, timing, and task events replay together in 3D, from any angle. Jump straight to the moments that decide whether an episode is kept, relabelled, or cut.
See Where the Operator Looked
Gaze paths, fixation points, and heatmaps from supported eye-tracking headsets show whether the operator saw what the task required.
Access Episode Records & Export to Your Pipeline
Find the episode you need without digging: profiles, session details, tags, filters, and exports in one place. Export as CSV or JSON, or connect the REST API and MCP Server to the pipeline your team owns.
“Our replay engine shows exactly how a user moved, where they looked, and what they did. It's not a simulation. It's a synced 3D recording teams can trust to find clear, actionable insights.”
How the three steps work on an illustrative collection run
Track, explore, and analyze, applied to a teleoperated bin-picking task your team could be collecting tomorrow.
Tracked. An operator runs a set of pick attempts through a teleoperation setup. Each episode records pose, hand inputs, movement, and the task events the team defined.
Explored. Replaying the batch in 3D, an engineer finds attempts where the operator's view was blocked at the grasp. The task completed; the episode is misleading.
Analyzed. Segmenting the week by outcome shows a recurring blocked-view pattern in failed grasps, concentrated at one station.
Reported. The affected episodes are excluded, the protocol gets a camera-position note, and the cleaned set moves to the robot learning team. The kept-and-cut record goes into the program review.
Make the collection program easier to justify
Compare Demonstrations in 3D
View multiple episodes together in 3D and compare movement, interaction, and task flow across operators, setups, and conditions. Coverage gaps, repeats, and failure episodes worth keeping become evidence for dataset reviews and scale-up decisions.
Segment & Query the Collection
Slice the collection by task, outcome, operator, or event to build the subset the pipeline needs. Every cut keeps its query, so the selection stays defensible when the dataset gets challenged.
Walk into the next program review with evidence
Your robotics program already produces demonstrations, in teleoperation runs and simulated task scenarios. Bring a collection run and see what the episodes show.