5 Common Mistakes You're Probably Making When Training in VR (And How to Avoid Them)

There’s a big gap in knowledge when it comes to VR.

It’s especially pervasive when using VR for training.

People who understand a few key concepts are able to increase training effectiveness and lower costs. Those who don’t, will struggle to see the results they’re looking for.

We’ll share 5 common mistakes we see companies make when implementing VR training and how you can avoid them. Avoiding these mistakes will significantly improve your training efforts.

1. Thinking VR is the Same as Previous Platforms

Image via: CIO

VR is not the web.

VR is not mobile.

Successfully implementing VR into your business requires a thoughtful and unique approach.

There isn’t a simple turnkey solution for all your needs. You should work with multiple vendors to build the simulation, manage your LMS (Learning Management System) and capture the right data.

All of these pieces require time and investment. But you’ll be ahead of the curve if you take the time to understand a few concepts.

First, context is everything in VR.

Thousands of data points are captured for every minute that takes place in VR. So if you don’t know what to look for, it’s difficult to make sense of it.

Every single action, position, eye movement and biometric response can be captured in VR. It provides unprecedented amounts of behavioral data so you can find out how employees react to scenarios.

Measuring success isn’t as straightforward as keeping track of a few KPIs.

A common question we hear is, “What metric should I record to measure ROI?”

There is no simple answer to this. Success means something different for every company.

It’s necessary to define it based on your goals. Are you looking to reduce task completion time? Or are you preparing employees for critical safety procedures?

The metrics for these two learning outcomes are different.

By leveraging behavioral data for your needs, you’ll receive deeper insights into what works and doesn’t.

Some questions to consider:

• What is your end goal for training with VR?
• What does success look like?
• What are your expected learning outcomes?
• What is the expected ROI for the project?

2. Relying On Superficial Metrics

To transfer skills from VR to the workplace, you must be able to directly measure learning outcomes.

A learning outcome is a statement describing the skills an employee should be able to demonstrate after finishing the course. It provides clear direction on what they can take away.

Most importantly, you must directly measure how VR improves learning outcomes instead of resorting to guesswork.

A common mistake is relying on out-of-the box analytics. These analytics solutions only provide superficial metrics and not a lot of business value.

Examples of superficial metrics:

• Monthly Active Users
• Session Length
• Hardware Information
• Play Space
• User Height

The purpose of the metrics above is to give game developers information about their users. They aren’t designed to provide insight into enterprise use cases like training.

It also distracts you from what you should be paying attention to.

Digging beyond superficial metrics requires a bit of work but it’ll better inform future decisions and provide true value.

3. Missing the Quantitative

Not all quantitative metrics are superficial, often they simply require more context.

The problem is when companies rely exclusively on either qualitative or quantitative data to measure results. This is a common mistake from companies that don’t have the ability to capture deeper data.

Image via: Totango

At Cognitive3D, we help companies collect data from VR to enable real world insights. Our mission is to develop a new language for quantifying user behavior. Read more

Some companies measure success based on how much employees enjoy VR. For example they may use surveys to find out:

• Did they feel more engaged in VR?
• Did they find VR helpful?
• Would they recommend this method of training?

These questions do provide value, but they shouldn’t be an indicator for any business value.

Paying attention to surface level insights will lead you to inaccurate conclusions and can cost tens of thousands of dollars.

Check out this case study from Intel to see how VR provided an estimated five-year ROI of 300% due to a reduction in incidents and improvement in training.

Examples of metrics to capture:

• Task Completion Rate
• Knowledge Retention Rate
• Compliance Rate
• Biometric data such as heart rate, EEG, ECG
• Eye Fixation Time

The ways employees react to specific situations will help you figure where the blind spots are. Are they noticing the right hazards? How did they react after noticing it? Did they perform actions in the right sequence?

These metrics give you an objective way to measure employee performance. Scores can be assigned based on this data and applied to outcomes in real world scenarios.

4. Not Taking Full Advantage of VR

When designing VR training content, it’s important to take advantage of all 3 dimensions.

Make an effort and create unforgettable experiences by leveraging the feeling of “presence.” “Presence” is the feeling people get in VR when they suspend disbelief because of how real everything is.

“The incredible thing about the technology is that you feel like you’re actually present in another place with other people. People who try it say it’s different from anything they’ve ever experienced in their lives.” Mark Zuckerberg

This is the feeling you want to deliver to employees. The more immersive the experience is, the closer it’ll reflect realistic reactions.

Take them out of their comfort zones to test their skills. The environment should be interactive, they should be able to pick up tools and explore.

This form of learning is called “Active Learning” and a number of studies have shown that people learn better this way.

Active Learning: A method of learning which involves people in the process.

In contrast, reading and watching videos are examples of “Passive Learning.”

Passive Learning: A form of learning where people receive information and internalize it.

By combining “presence” and “active learning” methods, your employees will be much better prepared for anything on the job.

According to the “Forgetting Curve,” the average person forgets 50% of new information within 1 hour and 70% of new information within 1 day.

From our own studies, we've found that training in VR increases knowledge retention by 59% compared to "Passive Learning" methods such as video and text.

5. Not Leveraging Data

Image via: Adobe

The next steps after finishing the POC should be to apply and scale your learnings.

Examine your data to figure out what the biggest wins were. Did VR help employees learn faster? Make less mistakes? Follow safety procedures?

Compare their results to employees that were trained through video or text. Once you understand what areas VR was the most effective, you can apply it to the rest of the program.

Use a Learning Management System (LMS) to track and report the progress of your employees. There should be a complete program for delivering training content based on qualifications. As employees progress, they should receive access to more difficult materials.

“Often using VR is new for people so supporting and reassuring learners should be well planned.” Alexa Fowden, Director PwC UK

Questions to consider:

• What are the standards for course advancement?
• How long does it take to prepare for real world tasks?
• What actions immediately disqualify employees from passing?

By understanding what actions are conducive to success, you can design programs that encourage that behavior. You can also isolate steps that employees are struggling with and find out why.

Establish a baseline for the amount of time it takes to prepare someone to perform real tasks. For example, it may take an average of X hours to train someone to operate heavy machinery in person.

There’s a wealth of data available from VR that isn’t possible through any other platforms. It’s your opportunity to leverage it and best prepare your employees for their job.

Getting Started

This blog covered 5 common VR training mistakes when training in VR. Proper analytics and measurement is a key component for success.

At Cognitive3D we make it easy for you to capture data.

We are headset agnostic and support a number of software engines such as Unity, Unreal, C++ and JS.

This means you can simply install our SDK and connect your headset to collect analytics.

---