The Army’s new $480 million heads-up display lets soldiers wage war in a mixed-reality space — here’s how it works
Category : entrepreneur
- Business Insider recently participated in a soldier touch point event, an exercise intended to inform the development of new technologies, at Fort Pickett in Virginia.
- During the technology demonstration, we saw how the Army’s new heads-up display — the Integrated Visual Augmentation System (IVAS) — can help soldiers train using synthetic training environments in digital space.
- We stormed a building, battling enemies invisible to anyone without the HUD, and then did an immediate after action review using augmented reality.
- “It is feedback like they haven’t had before,” Maj. Gen. Maria Gervais, director of the Synthetic Training Environment team, said. “[Soldiers] see this as a viable training methodology for training close combat. The [after action review] piece of this, they would take this right now the way it is.”
The US Army is working with industry partners, namely Microsoft, to develop a mixed reality heads-up display, one that can give soldiers and squads access to tools that will let them train for battle like never before.
It was former Secretary of Defense Jim Mattis who said that US troops need to fight “25 bloodless battles” before their first fight. Augmented reality is how the Army is getting after that.
The Integrated Visual Augmentation System (IVAS), a multi-purpose HUD, is a signature system project for the Soldier Lethality (SL) team at Army Futures Command, but the Synthetic Training Environment (STE) team is currently also exploring ways to use the technology to help soldiers train and rehearse for battle.
Microsoft received a $480 million contract last fall to develop the IVAS headset, and “the funding supported the Army’s ability to kick-start the development of the initial IVAS capability sets and the test bed to evaluate the critical capabilities and integration into the overarching Synthetic Training Environment,” an STE team spokeswoman told Business Insider.
At a recent IVAS soldier touch point event at Fort Pickett in Virginia, Business Insider experienced firsthand how IVAS will be used as a training tool for soldiers.
During the event, a soldier-centered exercise intended to inform technology development, a squad consisting of this reporter, another journalist, Gen. John Murray, head of Army Futures Command, Army Vice Chief of Staff Gen. Joseph Martin, and James McPherson, who is the under secretary of the Army, raided a completely empty building.
For those of us with the experimental IVAS headsets, HUDs based on Microsoft’s HoloLens, the building was far from empty.
There were enemy combatants programmed to fire on our position and take cover when fired upon, civilians that would respond to voice commands, and even animals, in our case, dogs and goats.
Armed with weapons modified with special trackers, we could fire bullets and throw grenades in digital space. After no more than about five minutes of instruction, we were in the fight, with the generals leading the charge and shouts of “frag out” and “got him” echoing through the halls.
You could see your muzzle flashes and the explosions of grenades. You could see bullet holes in the wall where a shot missed. You could hear the cries of fallen enemies.
While the simulated warfare was fascinating, much more interesting was the after-action review, which began just a few minutes after we finished the raid.
We gathered around an empty table, but for those of us still wearing the IVAS headsets, it wasn’t empty at all. There was a 3D augmented reality version of the building we just stormed. Inside were little people representing the members of our squad.
Able to play, pause, fast forward and rewind like a home movie, we watched our team work its way through the building. We could see which way we were facing when we turned a corner and whether or not our rifles were pointing the same way.
“If you are a squad leader, you can really understand how your squad is maneuvering through this space,” an Army official explained.
In that same digital space, there was also an augmented reality leaderboard where we could see how each squad member performed (how many enemies killed, how many civilians killed, whether or not they were hit, etc). You can also see how an enemy was killed, specifically who took the shot and where the bullet hit.
Virtual reality trainers have been around for years, but the current experimental setup is like nothing seen before. The Army has been chasing this capability for over a decade, but the technology just wasn’t available until now.
“It is feedback like they haven’t had before,” Maj. Gen. Maria Gervais, director of the STE cross-functional team, said. “[Soldiers] see this as a viable training methodology for training close combat. The [after-action review] piece of this, they would take this right now the way it is.”
As the Army works to further develop this technology, it intends to expand it to the company level, as well as push for increased realism, such as a materials mod so that thin plywood barriers will not serve as adequate protection from a grenade.
“This is cutting-edge technology,” Army Chief of Staff Gen. James McConville told lawmakers earlier this year, explaining that IVAS and augmented reality training environments are “going to transform the way we train soldiers and the way soldiers operate in combat.”
“What it’s going to allow our Soldiers to do is to go into [augmented] reality and train on a mission they’re about ready to accomplish,” the general added. “We’re excited about it.”