What is the physics behind the virtual reality

The working group of Prof. Reinhard Klein has been working on the key question "How can a real environment be transferred virtually to another user in real time?" With the SLAMCast software, this question could now be solved for quasi-static scenes and made usable for very different applications of so-called telepresence. The basic methodology of the SLAMCast software was described by Patrick Stotko, Stefan Krumpen, Matthias B. Hullin, Michael Weinmann and Reinhard Klein in a specialist article in the magazine in 2019 IEEE Transactions on Visualization and Computer Graphics.

The only requirement for real-time transmission with SLAMCast is the recording of a scene with an RGBD camera, as can already be found in some new smartphones. While the image sensor captures a conventional color image, the depth sensor measures the distance between the viewed objects and the camera. In this way, a room can be captured three-dimensionally. What is special now is the virtual presentation of the recorded environment to another person, e.g. the basement to a heating installer. In order for the whole thing to be possible in real time, a fast transmission over the Internet is necessary in order to enable the large amounts of data to describe the reality that is virtually building up for the viewer live and to visualize it with virtual reality glasses.

The SLAMCast software solves this challenge by combining efficient real-time 3D scanning and a compact, realistic 3D scene model. This model shows the recorded surfaces as precisely as possible but also compactly without losing important information about the scene. "With virtual reality glasses, the user gets a real live 3D feeling because he can explore the room himself on foot and, above all, independently," says Michael Weinmann, one of the researchers at the Institute for Computer Science. This is important for use in practice so that you don't always have to tell the person with the camera where to film. Another advantage of the compact model and the lean data transmission with the help of SLAMCast is that groups of more than 20 users can virtually participate in a scene simultaneously and independently of one another.

How the transfer to the economy works

The contract and the conditions between the university and DoubleMe were successfully negotiated by PROvendis GmbH as the IP service provider of the University of Bonn. “Both sides are more than satisfied with the license agreement,” says Salih Çakmak, Manager for Patents and Licenses at PROvendis. However, the transfer of software into the economy also harbors difficulties. "We have a lot of great research results in computer science, but it is not always easy to put them into practice," says Rüdiger Wolf from the enaCom Transfer Center at the University of Bonn. This is also confirmed by Salih Çakmak: “There is often the difficult initial situation that no patent has been applied for for software or that patent protection has not been granted for the software, which offers additional protection for the basic technology. However, copyright always applies. In addition, a team of developers often works on the source code, so that the rights are distributed among different people. ”That was also the case with the team behind SLAMCast.

According to Salih Çakmak, liability issues can be another pitfall: What happens, for example, if the software cannot be used properly because there are no technical requirements that have never been communicated? Do the developers provide ongoing technical support in the event of problems? In any case, these points must be negotiated individually by the parties - as is the case with the University of Bonn and DoubleMe.

And yet the numerous developments in software research are increasingly finding their way into practice. “Software is booming at the universities in North Rhine-Westphalia: At PROvendis, we receive numerous inquiries about the protection of software and the intellectual property of such computer-implemented inventions. We have set ourselves the goal of systematically utilizing these research results, too, ”says Salih Çakmak. In the meantime, a whole team from different disciplines at the IP service provider is focusing on processing software cases. As part of the joint project NRW Hochschul-IP, the former NRW patent association, university scientists are also specifically trained on the exploitation potential of so-called "other property rights". This also includes software as the interface between invention and copyright.

As part of the joint project, PROvendis takes on the exploitation of the technologies, but scientists can use their activity to acquire interested parties themselves and accelerate the success of the transfer: Michael Weinmann and his colleagues have spread the SLAMCast software on various platforms and held lectures - with videos that show that the software also works in practice. Different specialist communities and large conferences with industry participation were also considered. "This acquisition is a lot of work," says Michael Weinmann. But it was worth it for the Bonn-based company: A founder of DoubleMe found them on one of the platforms and contacted them.