GST[AR]

Within the last 6 months, GiGa has been hard at work to bring you a new way to experience and communicate 3d subsurface models.

We are very happy to present to you GST[AR].

GST[AR] is a free iOS/iPadOS app which allows you to browse, download, view, and share 3d geological models in augmented reality (AR). If you can’t wait to get your hands on this new experience, you can download the app straight away on any iPhone or iPad featuring at least iOS 13: https://apps.apple.com/de/app/gst-ar/id1519232314.

Here we are going to dive a bit into the history as well as the features of the app and the future.

History

When participating in the subsurface hackathon in Copenhagen, Denmark in 2018 (read more about it at https://agilescientific.com/blog/2018/7/27/visualization-in-copenhagen-part-2) , a team consisting of three GiGa members and two external participants set out to create a first prototype to display outcrops with the help of AR. The final result after less than 48 hours of intense work was already more than everyone had hoped for: two prototypical AR apps for iOS and Android and a companion web app. It was even possible to collaboratively look at the same scene in AR with multiple devices.

During the next two years, our team presented the idea of such an app on multiple occasions, spurred discussions, looked for new ideas and, ultimately, funding. Although funding was hard to come by, we found a lot of encouragement to pursue the idea of such an AR app. We are convinced that such an app holds enormous potential. One of our biggest goals at GiGa is to make 3d models accessible for a lot of people. Our web viewer enables people today to look at complex models within an ordinary web browser without the need of any expert software. An app that would be available on an App Store could reach even more people that simply possess smart devices and could open up a completely new point of view for them. This is also the reason why we are sticking to Augmented Reality. It’s an extremely accessible technology that is widely available through the spread of smart devices. Virtual reality in comparison does not have the same ease of use or low barrier of entry. For now, it remains a niche for enthusiasts. But even an AR app would have to be developed first.

At the beginning of 2020, swisstopo did a call for tenders asking for three AR showcases that would open up new ways on how to present and interact with their data. Naturally, we applied and were very happy when it was announced that we were able to take part in this project. The final result was an internal app for swisstopo that GiGa was allowed to use and develop further. Read more about the whole AR showcase at: https://www.swisstopo.admin.ch/de/home/meta/pub-caroussel/geodata-news.html.

After a couple minor changes, multiple TestFlight programs and some more polishing, we decided that the app was ready for the App Store, which finally brings us to the present.

Features

Ultimately, the app allows you to browse, view and share 3d geological models. We want to describe these features a bit more in-depth. Furthermore you are invited to take a look at our videos on Youtube:

• GST[AR] on the iPad: https://youtu.be/WaAxu0EbFEg
• GST[AR] on the iPhone: https://youtu.be/CXuDUNCyIPQ

Browse

Before you can view your model in AR, you will first have to make a selection of the features that you actually want to visualize. To do so, you can either select one of the preinstalled sources, which are

• GiGa’s demo featuring a Saxon dataset,
• swisstopo’s public models, and
• a model of the Ore Mountains (created at Freiberg university of mining and technology),

or you can select a pre-loaded model which works without an internet connection.

No matter which source you choose, after tapping it, you will be able to display all of the individual features that a model is made of. If you know GiGa’s web application, you will notice that they are grouped similarly to the model tree, which makes it easier to select a group of features that belong together. If you look for a specific feature in an incredibly long list, a search field is available at the top of the screen.

If you are happy with your selection, you can tap on the „AR“ button on the bottom of the screen. After the model is successfully loaded, you will be taken to the AR view.

View in AR

Before you will be able to view your model, you will first have to place it on a horizontal surface. If your device does not come with the handy LIDAR sensor yet, you might be asked to move it around a bit to give it a sense of its surroundings. An overlay will keep you informed about the state of this process and when you are ready to proceed. You will be able to place a model on the table, floor or wherever you like as soon as the overlay vanishes.

Now you can manipulate the scene via natural gestures for rotation (two finger rotation) and scaling (two finger pinch). If you need more granular control over the scene, you can use sliders that are hidden behind buttons in the toolbar on the bottom of the view. Here, you are also able to access superelevation of the model, which is only possible via slider. You can also move yourself and your device around the model to get a different perspective, be closer or further away from the model or even go inside of it.

You can identify individual features by tapping onto them which presents their name and color. If you want to highlight a feature, you can tap on it in the horizontal legend (if visible) or advanced legend (rightmost button) to make all other features transparent. Tapping anywhere else returns to the default state. Should your model end up in a bad location, you can simply place it somewhere else as well.

Share

As mentioned before, we at GiGa try to make 3d geological models as accessible as possible. The ultimate realization of the concept in AR is to share and spectate a scene with other peers, allowing for collaborative exploration and discussion. After placing a model in AR, GST[AR] allows you to share a model with up to 7 other people in the same room. To share a model, simply tap on the share button in the bottom left. Everyone else in the room will then be able to join the session with the GST[AR] app. This allows everyone to see the same immersive model on their own device at the same time, placed at the same location. Every manipulation of the scene (rotation, scale, superelevation, location) will be synchronized locally via a peer-to-peer network between the devices.

In a session, users have additional tools to communicate about the model. Long pressing on the model will place an individual marker that will show up on all other devices. This allows for easy discussion about individual parts of a model. Another long press on your marker lets it vanish again. Should all users focus on one individual feature, a user can select it and then share this selection with everyone else through the advanced legend. This will highlight this feature on all participating devices. But fear not, every user can just simply tap anywhere else and keep on exploring at their own pace.

Settings

You can customize your experience by changing a couple of settings. In the settings menu, you will have the possibility to change aspects of scaling, how high the models float above the surface or even enable such cool features like „people occlusion“ that make your AR scene even more immersive! Furthermore, if you want to access models from a GST Web instance that is not present in the app, you have the possibility to add endpoints of your own or just simply rearrange them. Small details like the color of your marker can be changed here as well.

Outlook

With this release we realized the basic functionality that we envision as a solid basis for future development and the integration of new ideas. Currently we are porting the app to iPad and a first glimpse of this functionality is available on the App Store today (it might just be a little rough around the edges). Support for iPad promises a more immersive experience and better view of the models thanks to the bigger display.

We are also looking forward to your valuable input and innovative ideas and are happy to discuss with you what the future could look like. If you like the app feel free to rate it accordingly.