Deep Shot uses camera for transferring data to phone and vice-versa

Deep Shot System

It’s habitual for some of us to have our phone and PCs connected to each other for long as we transfer data to and fro. Now imagine doing all of that minus any cables or wireless connections. A new system called Deep Shot developed by Tsung-Hsiang Chang from MIT and Google’s Yang Li is what claims to make the task effortless. By simply pointing the phone’s camera at a computer screen, open applications can be transferred between the two devices.

The system is said to take advantage of the fact that a range of Web applications employ a standard format known as uniform resource identifier (URI). The latter helps describe the applications’ state. Based on this, the possibility of using a smartphone’s camera to capture a snapshot of a computer screen and have the phone automatically throw open the corresponding application in the same state is revealed. It will apparently also work in the reverse to move data from the phone to the PC.

“People are used to using heavy tools to transfer data or synchronize two devices,” Tsung-Hsiang Chang a graduate student in MIT’s Computer Science and Artificial Intelligence Laboratory mentions. “You have to plug in a USB cable and maybe open iTunes and synchronize a bunch of data at the same time. But sometimes you just want to send a tiny bit of information, or a single piece of information.”

Users would be required to install minimal software on the phone and the computer that’s to be interacted with to facilitate this process. Existing computer vision algorithms will be used by the system to recognize the open application on the screen when data is being uploaded to a phone. The installed software will allow extraction and transmission of the corresponding URI. A change in the photograph’s frame will also modify framing of the application window. The method works conversely when downloading data from the phone.

“I find it a really compelling use case, so I would really hope that companies like Microsoft would really consider adding it,” commented Jeffrey Nichols, a researcher at IBM’s Almaden research center who specializes in mobile devices. “I see it being much more likely to happen with websites than with desktop applications. On the other hand, to some extent, we’re moving away from desktop applications and moving more and more to the Web, so it’s not clear to me how important it is that we really bring all the native application developers into the fold.”

Chang and Li designed the system last summer when Chang was interning with Google. A paper on it was presented at the Association for Computing Machinery’s CHI conference on computer-human interaction in May. The system is said to currently function with an array of common Web applications like Google Maps and Yelp.

Google owns rights to Deep Shot since Chang was at the company when he worked on it. The system hasn’t yet been made publicly available though.