Varjo's technology is like a magic leap forward in augmented reality
See als
Every story about virtual reality starts with a demo.
You're in the National Gallery of Finland, a beautiful Beaux-Arts structure in downtown Helsinki. Against the dark hues of the museum walls, luminous works of art shimmer. Using a familiar VR controller, you leap across the room to a spot you've selected by pointing a green circle in front of you. Standing in the new spot, just inches from "The Fighting Capercaillies," an 1886 painting by Finnish artist Ferdinand von Wright, you are effectively pressed with your nose against the painting, one of the most famous Finnish works of art.
This would not be proper behavior in the actual museum, but in this virtual world, your temerity allows you to see something you've never seen in VR before: Detail. You can see the weave of the canvas, the layers of oil paint, all the things you should see as you get close to something.
This is virtual reality at a new level of precision, made possible by tiny startup Varjo. Up a quiet street in Helsinki, inside an old police building, the company's carpeted demo room allows you to walk around with a generous length of cable between the headset and the PC. You think to yourself: This is a hell of a lot better than most VR. But that's just the beginning.
"We were always focused on being a mixed-reality company," says Jussi Mäkinen, head of marketing at Varjo. Since its founding three years ago, Varjo has been on a quest not for VR per se, but for augmented reality, or what is now being referred to as mixed reality or "extended reality," where the actual surroundings merge with the synthetic creations in a way not possible before.
Also: World's first human eye-resolution VR headset (sorry gamers, for industrial use only)
Varjo's augmented reality headset, the "XR-1," which will go on sale later this year, feels like many virtual reality headsets, but the electronics inside create a new level of augmented reality that fuses one's surroundings with virtual objects.
And so, you strap on a second headset, the "XR-1." It has two cameras that each capture the world in 12-megapixel resolution. The confines of the demo room replace the museum scene, and suddenly, there are your companions, standing around the room, looking and moving exactly as they do in the real world, exactly where they should be. And in the middle of the room, between all of you, is something unreal, a life-size, four-door sedan. It has the gloss and gleam of a real car, its slick highlights and shadows move as you move your head, and as you arch your back and crouch, your view of the car shifts in proper proportion and aspect relative to your vantage point.
Your companions in the real world can be seen peeking over the roof of the sedan, or standing in front of the hood, from your point of view, in perfect alignment with the dimensions of the car. Everything seems to reinforce object permanence in a way it never has in AR demos. In one of those revealing moments, as the door of the car is made to swing open suddenly, you instinctively jump out of the way of its arc, even though it's not real, just because its movement in space is so well established.
Thanks to the unprecedented resolution of the display plus a lot of software brilliance, the real world is seen through the cameras of the XR-1 is merged just about perfectly with the synthetic world of the sedan. Sit down on a desk chair, and you can feel you're sitting in front of the dashboard of the car, with all the controls in proper resolution, so they're readable, as your colleagues continue to mill about outside the windows.
Someone flicks off the light switch in the real room, and not only does it grow dim in the view through your XR, but the light bouncing off the metal chassis of the sedan adjusts appropriately, growing dim, continuing the illusion.
The XR-1, which is not yet on sale, is the grand ambition of Varjo, and a big improvement from Microsoft's HoloLens, now in its second iteration, and the initial results from startup Magic Leap.
ZDNet was invited to try it out, so ZDNet went to Helsinki.
The story of Varjo is a story of a dream deferred but not defeated. Work began on virtual and augmented reality at Nokia back around 2007. Some of that work eventually found its way into HoloLens, as Microsoft took control of Nokia. But neither of those efforts realized the most ambitious goals of the Nokia engineers, to create a headset that would render the real and the synthetic imagery in a convincing union.
Also: Hi-res VR headsets reignite enterprise mixed reality battle
And so, three years ago, former Nokia executive Niko Eiden, a 14-year veteran of the phone maker, founded Varjo with former Nokia colleagues. Eiden is now the chief executive. In the spring of 2016, when the company first raised money, it was the height of VR enthusiasm. Three headsets were on the market, from Facebook's Oculus unit, Sony, and HTC. Hololens was on its way, setting up an interesting debate between VR and AR. Getting money was fairly easy back then, Eiden recalled during a chat in the demo room.
But then, disappointment set in. VR headsets turned out to be poor in quality, and nausea-inducing and all the hardware units met with terrible sales. Much like Google's failed Google Glass, there was a sense that augmented reality could still deliver a market if it were re-tuned business applications rather than consumer.
And so Microsoft pressed on with its HoloLens developer edition, and startups such as Vuzix offered less-ambitious, more-practical headsets.
Varjo's existing product, the "VR-1" virtual reality headset, which went on sale earlier this year, has a central area within the field of view that is much higher resolution than anything that's come before, allowing for business VR apps that require use of fine detail.
Varjo hung in there and made it through the years of leaner financing and today is one of the few survivors. With a total of $46 million in financing, Varjo has a lot less than Magic Leap, which has $2.5 billion from Google and others. Despite vastly smaller resources, Varjo this year was able to bring to market its virtual reality headset, the "VR-1," and it expects later this year to start selling the XR-1 that mixes the real world via cameras with the virtual world.
Much has been written about the technical aspects of the headsets this year, but all that doesn't capture exactly what is going on here. The whole is more than the specs.
To review what has been widely cited elsewhere, the VR-1 has two displays over each eye, sandwiched together, for four displays in total, forming a single field of view of 87 degrees. One type of screen is similar to other VR headsets, at a resolution that is 1,440 pixels by 1,600. The other type, what's called a "focus screen," is 1,920 by 1,080. The two are optically combined using a mirror placed between them. The focus screen is what provides the higher resolution that allows for fine details to be seen. It shows up in the middle of the field of view, occupying about a third of the total field of view, its edges barely detectable as a subtle ripple in the middle of the larger screen. The cruder display forms an outside ring or periphery at the edge of one's field of view.
Also: This startup is building game-changing image rendering tech for VR and AR
Must-see offers
The whole thing runs off an X86 PC with Nvidia GPUs. A minimum configuration is the Nvidia "GTX 1080" GPU, but the more powerful "RTX 2080" is recommended. (The system doesn't support AMD GPUs at this point.)
The approach of the XR-1 involves some hardware and software artistry. Smart software fuses the camera imagery and synthetic imagery to generate the combo of real and imagined. This is a fundamentally different approach from that of Magic Leap and Microsoft. Those devices render the synthetic objects onto a transparent set of goggles, which means that rendered objects never quite sit right in the user's view of the real world.
"It's not just the display, and it's not just the camera," says Marcus Olsson, who has one of the all-time greatest job titles, Mixed Reality Lead. "There is a tremendous amount of work that goes into making these things come together and do so with a minimal amount of latency so that it works. There's a lot of software work there to integrate those things and form a smooth loop between them."
Some of the Varjo team at the company's headquarters in Helsinki, from left, Marcus Olsson, Mixed Reality Lead, Jani Ylinen, Specialist in 3D and Photogrammetry, Annaleena Kuronen, Marketing & Communications Lead, Niko Eiden, Co-Founder & CEO, Jussi Mäkinen, Chief Marketing Officer.
Indeed, the key thing that comes across in demos is that the view of the surrounding world shows no lag or other effects that might break the feel of being in that world as one turns one's head. And a virtual object, in this case, the sedan, integrates very convincingly with that real view in terms of proportions in space.
Is it flawless? No. As you move around the virtual car, there are stuttering movements of the car as it re-renders at times with your shift in perspective. Olsson says that is an artifact that will be ironed out with continued software development before the release of the final product. Although the view of the real world from the camera feed is very usable, it still has a bit of a film over everything. You won't be tricked into thinking this is your normal view of the world. But it also doesn't have the terrible feeling of a grid of pixels that is so present in other VR systems. And at no point did it induce nausea during the demo.
Minor quibbles are likely to be even less important with Varjo's target audience, enterprise users who want augmented and mixed reality to help some critical business functions. The VR-1 sells for $5,995, plus an annual service fee of $995 (not including the cost of base stations to track your movement, and the controller device, which all are purchased separately.) The XR will be still costlier, says Varjo marketing lead Mäkinen, without disclosing specifics. That's nothing for an enterprise customer if it can save companies model by being able to simulate things they would ordinarily have to build.
Also: Varjo XR-1 is an AR headset so fast, Volvo test drives cars while wearing it
Primers
Carmaker Volvo, which has invested an undisclosed amount in Varjo, is using the technology already in its commercial car prototyping. The company can replace some use of physical prototypes with virtual versions, explains Mäkinen. "This can substantially reduce the amount by which they need to make scale mock-ups if they can walk around and see the car from all angles."
In the past year, Volvo has worked with Varjo conducting tests where they have a driver wearing the XR drive a real car on an actual road. It's a way to test out new kinds of "heads-up" displays that are being designed for the dashboard. By seeing the real road through the cameras, with a virtual heads-up display rendered on top of the real dashboard, Volvo can eliminate some of the cost of building each iteration of a heads-up display in a real car, which can be another substantial cost savings, says Mäkinen.
One can imagine various refinements. For example, it would be nice if the higher-resolution focus screen filled a larger area than just one-third of the total viewable area. That is not at this point economical or technically feasible, says Mäkinen. It would also be nice to ditch the cord to the PC. And the headset itself is as clunky and heavy as the conventional VR headsets.
But fixing things like that are not a priority, says Mäkinen, for Varjo's business audience. Other areas can be much more important to make the platform even more usable. For example, the demo includes a brief look at hand simulation. You can hold your hands in front of you, which could make it possible for you to manipulate virtual objects in a straightforward manner. The demo was a rough sketch, it needs more work, but it shows promise.
With focused, sensible uses of AR like this, it's conceivable Varjo can make back all of its financings to date just selling several thousand headsets, which is a very low bar for any technology company.
There is a giddiness in the offices in Helsinki, albeit modest and without swagger, a clear sense the team is survivors amidst a lot of roadkill in VR and AR. The company, just 150 people, is in hiring mode. They have three floors of the police building, with an option to take over the fourth floor. They also have some staff scattered around the Washington, D.C. area. Being closer to Magic Leap's headquarters in Plantation, Florida, the question arises, and ZDNet had to ask, Have they hired any Magic Leap staff?
"Not yet," says CEO Eiden with a smile and a twinkle in his eye.