How We Added Video Calls to a Unity-Powered VR Product in Just Five Hours

At Whereby, we’re always looking for new ways to innovate and test our product’s capabilities in the wild. So when 3D design platform, Gravity Sketch, challenged us to help integrate real-time video communications into their virtual reality environment, we jumped at an opportunity to really test the boundaries of our technology. Using our new “lower level” SDK, we collaborated with their team to build fully functioning, real-time communication inside their platform in less than a day.

Designing the future

Gravity Sketch offers a category-defining immersive VR design software that allows industrial designers and teams to create and iterate on 3D models at 1:1 scale. They work with some of the most prominent automotive, footwear, and large-scale industrial design firms on the planet. Using Gravity Sketch’s technology inside of a standard VR headset, teams can create designs in 3D space and at any level of scale, allowing designers to physically move around and through a design, and view work from all angles. Currently, in the industrial design space, a lot of design still happens using huge clay blocks, into which designs are carved by hand. Each design iteration can cost 10’s or even 100’s of thousands of dollars — quite the legacy to overcome. Through its collaborative design environment, Gravity Sketch enables teams to ideate and review designs together, in 3D and in real-time.

A journey to collaboration

The real-time communication Gravity Sketch provides within their VR environment solves multiple problems for their customers: designers working together remotely want to collaborate in real time, as if they’re in a design studio, while reviewers want to join a project from a screen to see the work and offer input.

But solving this is tricky: New UX patterns need to be invented to make it easy to join and participate in conversations. Screen sharing works completely differently, as it needs to support perspective views within the 3D environment. And, of course, representing video tiles for all participants in the conversation will look and feel different than a typical video conferencing experience.

The more we learned about their use cases, the more intrigued we became: Yes, Whereby was built for the web, and it’s simple to integrate, but how far could we push the technology? Gravity Sketch represented the perfect way to truly find out. Could we really use what we’ve built to put real-time video into a native, 3D environment running on some VR headsets?

We had to try it, so we got the Gravity Sketch and Whereby teams together for a five-hour hackathon, and the results far surpassed our expectations for the day.

Whereby, but make it VR

The release of our SDKs presented an exciting opportunity to integrate Whereby’s video call technology into their innovative design solution. Our SDK offers extensive flexibility in the kind of experiences that can be built to enable video calls in bespoke, creative ways.

In a little over 5 hours, 6 engineers from Whereby and Gravity Sketch were able to:

• Get fully-integrated, Whereby-powered conversations working inside of Gravity Sketch's "LandingPad" web application using our Browser SDK.

• Get the full Whereby UX working inside of the desktop and VR environments, bringing the Whereby room experience into VR.

• Build a customized UX inside the desktop and VR environments that allowed each video participant to be represented within the VR environment as a 3D "floating" video tile that could move around in and interact with the space.
  

Woah! How was that possible?

Working in person alongside the Gravity Sketch team was certainly an accelerant to achieving so much in such a short space of time. The teams were able to problem-solve in real-time as they built, collaborating incredibly closely throughout the day.

Also, we found that Unity's browser implementation is fully featured, so we didn't run into any fundamental blockers bringing some web technology inside the VR environment.

The power of our Browser SDK really came into its own and provided the flexibility needed without any real issues. Our team were able to build a new Whereby front-end and data API that elegantly solved the video tile integration in VR without needing to develop native components, nor introducing problematic performance issues on the Quest devices.

We’re excited to see where the Gravity Sketch team takes the project and for them to bring it to market for their customers soon. Our team truly had a blast collaborating and helping to take their exciting platform to the next level.

Get creative with our SDKs

We love seeing and hearing about all the inventive ways that our video call technology is being used. Have you created something really ace with our SDK? Share it with us in our Developer Community.

Want to test out our SDKs for yourself? Find out more in our Developer Docs and get started with Whereby Embedded here.