Streaming Sensor Data in order to Rapidly Prototype Augmented Reality Apps
tango_ros_bridge allows you to stream sensor data from a Google Tango smartphone directly to any computer running ROS. Streaming data in this manner allows you to program apps the utilize the rich sensor data of Tango (fisheye camera, 3D location tracking, and 3D point clouds) using any of the programming languages supported by ROS (Python and C++). Additionally, you will be able to leverage all of the powerful features of ROS (e.g., visualization tools, debugging tools, recording tools) from the comfort of your own laptop (or other computer).
ros_tango_bridge has two components you will need to configure: an Android app that runs on your Tango smartphone and a ROS package that should be installed on your ROS computer.
In order to use
tango_ros_streamer, you must have the following dependencies installed.
Obtaining the Code
To obtain the code clone the
ros_tango_bridge repository. To streamline the integration of the code from the repository with ROS, you should clone the repository into your catkin workspace
src directory. For instance, if your catkin workspace is installed in the typical location, you would use these commands to clone the repository.
$ cd ~/catkin_ws/src $ git clone https://github.com/occamLab/tango_ros_bridge
Building and Installing the Android App
An Android Studio project is included as part of the repository under the
android_code subdirectory. Use Android Studio to build and install the app to your Tango-equipped smartphone.
Installing the ROS package
As long as you cloned the repository into your catkin workspace, there is nothing you have to do in order to install the ROS package. You do, however, have to install one Python library. The
decorator library can be installed using
$ sudo pip install decorators
On the machine running ROS, launch the main
tango_ros_streamer launch file.
$ roslaunch tango_streamer stream.launch
On the smartphone, open up the
ROS Streamer app, enter the IP address of your ROS computer and click
To ensure that the connection has been made properly, on the computer running ROS print out the pose information from the phone.
$ rostopic echo /tango_pose
If all goes well, you should see output similar to the following.
--- header: seq: 619 stamp: secs: 1524579566 nsecs: 828507900 frame_id: "odom" pose: position: x: -0.0494725894225 y: 0.0452275977115 z: 0.173853426765 orientation: x: 0.0352902299281 y: 0.1361060735 z: -0.0701101320277 w: 0.987580025018 --- header: seq: 620 stamp: secs: 1524579566 nsecs: 853507995 frame_id: "odom" pose: position: x: -0.0495403503256 y: 0.0454651608819 z: 0.173958186925 orientation: x: 0.0350289289155 y: 0.136183373272 z: -0.0704992675044 w: 0.987550968945
tango_ros_bridge consists of two main components: an Android app and a ROS package.
The architecture diagram shows the interplay between the Android app and the ROS package. The two components communicate over an IP-based network using both UDP and TCP sockets. The ros package converts the raw sensor data to an appropriate ROS message and publishes it in on a topic so that it can be consumed by client ROS nodes.
Sample Use Cases
Here are some instances of
ros_tango_bridge in action.
Sparse 3D Mapping of a Building
William Derksen utilized this platform to build a 3D map of Olin’s academic center. The image below shows a side view of two paths through all four floors of the academic center. The red path is the raw sensor data as determined by the Tango phone and the blue path is a corrected version that utilized special optical landmarks for drift correction.
Simple Dense 3D Mapping
Outline of content to be generated:
- Create a screen capture of creating a rough 3D map using
rvizand raw point cloud data.
- The screen capture will show the phone moving around in space, with point clouds being projected using
rviz. By the end of the video, the structure of the room will be clearly visible.
This code was developed starting in 2015 with the aim of allowing Olin students to easily program systems based on the Tango platform. In particular,
ros_tango_bridge is designed to smooth the learning curve that would be needed to program directly on the phone itself (e.g., learning Java, learning Android Studio, etc.).
The initial version of the code supported the Yosemite development Kit supplied by Google. The current version works with the Lenovo Phab2 Pro (although, adapting it to other platforms can be done with a modest amount of work).
Copyright 2018 Paul Ruvolo
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.