Stream Real World VR Scenes to Your Headset

The RICOH THETA is a 360° camera that can live stream 4K video to a variety of VR headsets, such as HTC Vive, Oculus Go, Oculus Rift, and Google Daydream. This creates a telepresence experience using a live camera feed. When the user turns their head in the VR headset, the scene displayed in the headset shifts as well.

This is an exciting space with many real world applications. In the example below, robotics designer Hughes Perret has attached a RICOH THETA to a custom-built rover. The rover uses the RICOH THETA to identify damage to pipes. The rover goes into a pipe and broadcasts the scene in real-time over Wi-Fi. If the human operator sees something of interest, they can easily change the area they are looking at.

In another example of live streaming telepresence, the research group for the TwinCam Go projects has attached two RICOH THETA cameras to a Segway. They then transmit the scene over a computer network to a VR headset. The system also has haptic feedback. The chair rotates as the Segway turns.

Up until recently, most projects required a separate computer to process the video. In a typical project, the RICOH THETA is connected to a computer with a USB cable. The computer then retransmits the video stream to the headset.

However, you can now transmit 4K 360° video over the Internet direct from the RICOH THETA using Wi-Fi.

RICOH released a plug-in to their camera that allows streaming to YouTube and Facebook direct from the camera. The source code for the project is available on GitHub. As the RICOH THETA runs the Android OS inside of the camera, you can easily modify the plug-in for your own project using Android Studio. The code uses the Apache 2.0 license.

If you have RICOH THETA camera, you can install the completed plug-in from the THETA Store. You configure the plug-in through a mobile app or web browser.

Once the plug-in is configured with the YouTube or Facebook live event settings, you do not need to use your mobile phone. The THETA can stream untethered using Wi-Fi.

The video is transmitted using Real-Time Messaging Protocol (RTMP). This is a standard that Facebook, YouTube, and Periscope use for their live events. Once the video is streamed to YouTube, multiple people around the world can simultaneously share your experience using VR headset or web browsers.

To receive a guide with over 25 source code examples, including HDR2EXR, FastCV example, OpenCV example, and TensorFlow demos, sign up for the RICOH THETA Dream and Build Contest. In addition to the code examples, you’ll receive an asset pack of 360 video and images for you to experiment on. If you don’t have RICOH THETA, you can still experiment with the concepts in an Android Virtual Device (AVD).

The RICOH THETA Dream and Build Contest is currently live on Hackster. 10 participants who apply before April 16th will receive a RICOH THETA to develop with. Then you’ll have time and resources to build a customized plug-in to make the camera do what you want.

The friendly and active theta360.guide developer community is available to get assistance from other developers just like you.