Erleb-AR
Switzerland has a long and rich cultural heritage. Unfortunately, much of it has either disappeared or been reduced to ruins. New technology aims to bring the past back to life.
Factsheet
- Schools involved School of Engineering and Computer Science
- Institute(s) Institute for Human Centered Engineering (HUCE)
- Research unit(s) HUCE / Laboratory for Computer Perception and Virtual Reality
- Funding organisation Schweizerische Eidgenossenschaft (Bundesverwaltung)
- Duration (planned) 01.04.2019 - 30.06.2021
- Head of project Prof. Marcus Hudritsch
- Partner Bundesamt für Kultur BAK
- Keywords Augmented Reality, Virtual Reality
Situation
Much of Switzerland’s rich cultural heritage is only partially visible today, in the form of ruins or archaeological artefacts. Some of it has disappeared entirely. Often there are only basic information panels to explain a location’s historical significance. Lots of information is available at museums and local installations, but it is often difficult for us to vividly imagine what the past looked like. Emerging technologies make it possible to combine camera images with models of historical buildings, objects and people. Augmented reality means we no longer have to rely solely on the power of our imagination but can be transported back into the past at the scene of events. This enables knowledge to be conveyed more vividly than ever before, and Swiss cultural heritage to be quite literally brought back to life. Although augmented reality applications are already available, the high-precision merging of existing outdoor structures with 3D models remains extremely complex. Variations in weather and light conditions, the changing of the seasons and the movement of objects such as cars or pedestrians are just some of the issues that must be resolved to create a successful AR experience.
Course of action
In view of the exacting requirements in terms of precision and stability, the open-source project ORB-SLAM was selected as the basis for developing our own AR framework. Existing solutions from other providers were also evaluated, but either failed to meet our requirements or were not at a sufficiently mature stage in their development at the start of the project. Based on distinctive points in a camera image, ORB-SLAM creates a point cloud akin to a three-dimensional map. This can be used to precisely determine the camera’s position (‘relocalisation’), allowing a 3D model to be inserted into the camera image with great precision. As part of an iterative process where the existing solution was continually tested at various locations, our AR framework was enhanced with features that enable us to create and store point clouds and to fit 3D models into them. Unfortunately, we observed that it is not always possible to create a point cloud that consistently allows for relocalisation to take place. This is why we developed various fallback solutions, which also enable augmentation to be displayed when localisation fails, albeit with less precision and stability. These solutions are based on sensors installed in modern mobile phones (GPS and orientation sensors) and on fixed markers at the locations.
Result
An app has been created that is available as a free download from the Google Play Store and the Apple App Store. This app can currently be used at three locations to visualise historical buildings which have been reduced to ruins or have disappeared completely. At the former Roman settlement of Aventicum (Avenches), the amphitheatre, the Cigognier sanctuary and the theatre can be viewed. At its sister town of Augusta Raurica (Augst), the Schönbühl temple and the third theatre, of which much remains today, can also be recreated. Finally, the Christoffelturm tower, which was demolished in the mid-19th century, can be admired in its original setting on Bern’s Bahnhofsplatz.
Looking ahead
Additional locations of historical importance can be added to our app relatively easily where an AR scene is worthwhile. The models created for the project would also be ideal for use in virtual reality, where users are completely immersed in a different world. The expertise acquired means the HuCe-cpvrLab is now well placed to implement other AR/VR projects for the visualisation of structures that no longer exist or are yet to be built.
