Google Wants to Put a 3D Camera on Your Smartphone — Here’s Why
Google’s quest to map the entire world doesn’t end with the outdoors. Now, Google wants to create maps of indoor spaces too — and that’s a powerful idea.
The product of the Google Advanced Technologies and Projects (ATAP) division, Project Tango is a way of 3D-mapping indoor spaces using specialized sensors mounted to a mobile device. Not everything born in ATAP makes it to the real world, but Project Tango has recently moved out of ATAP and is on its way to becoming a real product.
Let’s dive in and see what it’s all about.
What is Project Tango?
It’s a smartphone, and it’s a tablet, but the most critical element is a time-of-flight depth camera, which is capable of determining the depth of pixels in an image via a suite of clever algorithms and hardware. Combining all of that into a smartphone or tablet form factor just makes it easier for people to carry around and interact with. Using this, Google hopes to make smart devices smarter, giving them a human-like understanding of the physical world.
Google’s first prototype was a smartphone, as shown in the video above. Project Tango launched in an early form about a year ago , and since then has been rapidly evolving. We listed it as one of the cool research projects that could change the world back in November, and it is still firmly on that path.
The second iteration of Project Tango, shown above, was a $1,024 developer-only model that is currently out of stock. Google’s hoping to get developers involved to tap into all the possibilities for this technology, some of which we’ll look at in the next section.
For a glimpse into the work that has gone into Project Tango, check out the video below from ATAP.
A lot of resources are being poured into this project, and it will be exciting to see where it all goes. At first glance, some of the technology in Tango may seem a little gimmicky, but the applications that developers have come up with so far are hugely exciting.
What Could It Be Used for?
As developers get ahold of the APIs for working with Project Tango, we’re sure to see an explosion of ideas for the platform. For now, here are a few that could come to fruition.
Search and Rescue
ATAP worked with many different companies and universities to produce Project Tango, but one university in particular is set on putting Tango to work in urban search and rescue missions. The University of Massachusetts Lowell Robotics Lab has big plans ahead [Broken URL Removed] for this application.
In many search and rescue scenarios, (picture a partially collapsed building) rescuers often have no idea what sort of conditions are like ahead of them. Project Tango, mounted on a small drone, would allow unstable spaces to be mapped before humans enter, identifying risks and obstacles and making the process faster and safer.
Navigation for the Visually Impaired
Google does a lot of great things for the world, and this use of Project Tango should probably be counted among those achievements. For those who are completely blind, visually impaired, or just have trouble getting around, Project Tango could view the environment around them, understand its geometry, and give them auditory hints about the locations of people, doors, and potential obstacles.
This could really change the lives of millions of the visually impaired all over the world. Hopefully this application of the technology is the first to reach consumers.
In the augmented reality vane, Target has already tricked out a few of their stores to sport a winter wonderland-style aesthetic when viewed through a Project Tango device. In the future, app developers could use Tango’s sensors and cameras to bring games into your environment. In the video below from Android Authority, you can see one game, Zombie Gunship Reality, already taking advantage of this.
Once Google can get more Tango devices into the hands of consumers, you can bet the game designers will follow. At that point, the kinds of games we’re likely to see are only limited by the creativity of the developers.
In the virtual reality sense, integration with the Oculus Rift , Samsung Gear VR , or Microsoft HoloLens could allow for an even more immersive experience. Not to say that we ever expect Google to play nicely with Microsoft . In particular, one of the major limitations to smartphone-based VR headsets like the Gear VR is the lack of support for positional tracking. Creating a similar, Oculus-based headset for Project Tango would allow developers to use the depth camera data to track the position of your hands and head.
This could even lead to your body being used a controller. Like the Microsoft Kinect, Project Tango can sense where bodies are in space, and deduce their pose. Train that Tango device on yourself, and your arms and legs could suddenly be in the game — a feature sadly lacking from current virtual reality games. Alternately, with the ability to capture depth video, you could use Project Tango to create VR video that allows you to fluidly move your head around and observe the scene from many angles.
Trying out Furniture
Trying to figure out if that couch will fit in your living room? Unsure if that bookshelf would look good across from your shark tank? With Project Tango, you don’t have to guess. The depth camera is a virtual tape measure and laser level rolled into one and can measure distances, sizes, and even project an image of a product into a virtual simulation of your room. Walk around your house once with your Tango tablet, and you’ll have a virtual replica of your house on call whenever you need it. Alternately, you could use the tablet to 3D scan people or objects, and then bring them into the virtual world, to be 3d-printed or used as a virtual prop.
Ever spend way too much time wandering around a Walmart or Costco looking for that one item? Project Tango, if adopted by major retailers, could be like Google Maps for the indoors. Aisle411 and Walgreens have already integrated Project Tango with one Walgreens location, allowing customers with a Tango device to get directions to the appropriate aisle, as shown in the video above.
It doesn’t only have to be used for retail shopping. Finding your way around inside large office buildings is one possible benefit, and in any other areas where GPS is not available, local mapping like this could help people find their way around.
That’s right; Project Tango devices could help astronauts accomplish a bunch of chores that would otherwise take them away from important tasks, by providing a new source of data for experimental autonomous robots called ‘spheres’ that fly around the ISS using cold-gas thrusters. The spheres are still in the prototyping stage, but eventually they could take over a number of simple functions, and project Tango could help make that happen.
And this application isn’t far off. In fact, as shown in the video above, Google engineers have already taken the Tango into a Zero-G flight to test it out. The possible use cases for NASA and space travel in general are huge.
When Can I Get It?
Google has a lot of great services just on the horizon , and Project Tango is just one of those. Back at Google I/O 2014, ATAP announced that they were working with LG on a consumer product to be released “next year”, which means that we should see a consumer-oriented product coming in 2015. The fact that Tango has graduated from ATAP is good evidence of that.
The first device is likely to be a tablet, but after that, we could see smartphones cropping up from different hardware manufacturers if it catches on — as with Android, Google will probably partner with any hardware manufacturers who are interested. That said, in the world of razor-thin phones and thinner margins, it may take a while to convince manufacturers of the value of adding bulky, expensive sensors to their devices.
Beyond that general time-frame, we don’t know enough to be more specific. No pricing, specs, or specific release date have been announced yet. Though you can be sure we’ll all be waiting with bated breath.
What’s Your Favorite ATAP Project?
Project Tango isn’t the only idea to have come out of the special division within Google. There’s also Project Ara, the modular smartphone , as well as driverless cars that are taking us into the future .
Be sure to let us know what your favorite ATAP project is in the comments below. And what do you think of Project Tango? It it a gimmick, or a real evolution in the way we use technology?