Research Areas
Real-time graphics
This research focuses on scalable graphics algorithms for rendering large and complex scenes at real-time frame rates. We investigate novel approaches to techniques such as level of detail, image-based rendering, potentially visible sets, frame extrapolation and geometry compression. We are particularly interested in rendering methods suitable for streaming rendering and virtual reality displays.| <a href="/pdf/Schmalstieg_{"id"=>"447", "title"=>"Disocclusion Buffer"}.pdf"><img src="/img/Schmalstieg_{"id"=>"447", "title"=>"Disocclusion Buffer"}.jpg" width=200 alt="Disocclusion Buffer"></a> | <a href="/pdf/Schmalstieg_{"id"=>"446", "title"=>"NeuralPVS"}.pdf"><img src="/img/Schmalstieg_{"id"=>"446", "title"=>"NeuralPVS"}.jpg" width=200 alt="NeuralPVS"></a> | <a href="/pdf/Schmalstieg_{"id"=>"427", "title"=>"Laced Wires"}.pdf"><img src="/img/Schmalstieg_{"id"=>"427", "title"=>"Laced Wires"}.jpg" width=200 alt="Laced Wires"></a> |
| <a href="/pdf/Schmalstieg_{"id"=>"414", "title"=>"Trim Regions"}.pdf"><img src="/img/Schmalstieg_{"id"=>"414", "title"=>"Trim Regions"}.jpg" width=200 alt="Trim Regions"></a> | <a href="/pdf/Schmalstieg_{"id"=>"396", "title"=>"Temporally Adaptive Shading Atlas"}.pdf"><img src="/img/Schmalstieg_{"id"=>"396", "title"=>"Temporally Adaptive Shading Atlas"}.jpg" width=200 alt="Temporally Adaptive Shading Atlas"></a> | <a href="/pdf/Schmalstieg_{"id"=>"362", "title"=>"Shading Atlas Streaming"}.pdf"><img src="/img/Schmalstieg_{"id"=>"362", "title"=>"Shading Atlas Streaming"}.jpg" width=200 alt="Shading Atlas Streaming"></a> |
Situated visualizations
Situted visualization uses augmented reality displays to present information to a human user that is dynamically derived from the user's physical environment. It provides guidance, instructions and context in everyday situations. We investigate when, where and how to visualize such information, and what information to visualize.| <a href="/pdf/Schmalstieg_{"id"=>"440", "title"=>"CECILIA"}.pdf"><img src="/img/Schmalstieg_{"id"=>"440", "title"=>"CECILIA"}.jpg" width=200 alt="CECILIA"></a> | <a href="/pdf/Schmalstieg_{"id"=>"433", "title"=>"AR assembly guidance with error management"}.pdf"><img src="/img/Schmalstieg_{"id"=>"433", "title"=>"AR assembly guidance with error management"}.jpg" width=200 alt="AR assembly guidance with error management"></a> | <a href="/pdf/Schmalstieg_{"id"=>"416", "title"=>"guitARhero"}.pdf"><img src="/img/Schmalstieg_{"id"=>"416", "title"=>"guitARhero"}.jpg" width=200 alt="guitARhero"></a> |
| <a href="/pdf/Schmalstieg_{"id"=>"419", "title"=>"AR Visualization Patterns"}.pdf"><img src="/img/Schmalstieg_{"id"=>"419", "title"=>"AR Visualization Patterns"}.jpg" width=200 alt="AR Visualization Patterns"></a> | <a href="/pdf/Schmalstieg_{"id"=>"404", "title"=>"RagRug"}.pdf"><img src="/img/Schmalstieg_{"id"=>"404", "title"=>"RagRug"}.jpg" width=200 alt="RagRug"></a> |
Photorealistic augmented reality
Ideally, virtual objects displayed in augmented reality would be indistinguishable from real objects. This grand challenge requires capturing the entire image formation process in reality and simulating the virtual counterpart. It encompasses reality capture, light transport, camera simulation and much more.| <a href="/pdf/Schmalstieg_{"id"=>"428", "title"=>"Neural Bokeh"}.pdf"><img src="/img/Schmalstieg_{"id"=>"428", "title"=>"Neural Bokeh"}.jpg" width=200 alt="Neural Bokeh"></a> | <a href="/pdf/Schmalstieg_{"id"=>"337", "title"=>"Learning lightprobes"}.pdf"><img src="/img/Schmalstieg_{"id"=>"337", "title"=>"Learning lightprobes"}.jpg" width=200 alt="Learning lightprobes"></a> | <a href="/pdf/Schmalstieg_{"id"=>"316", "title"=>"Instant relighting"}.pdf"><img src="/img/Schmalstieg_{"id"=>"316", "title"=>"Instant relighting"}.jpg" width=200 alt="Instant relighting"></a> |
Image-based modeling and rendering
Capturing the geometry and appearance of the real world and novel view sythesis from images is a key technique for compelling extended reality techniques, such as telepresence or mediated reality. In this work, we investigate emerging scene representations, such as light fields and radiance fields.| <a href="/pdf/Schmalstieg_{"id"=>"448", "title"=>"Sorted Opacity Fields"}.pdf"><img src="/img/Schmalstieg_{"id"=>"448", "title"=>"Sorted Opacity Fields"}.jpg" width=200 alt="Sorted Opacity Fields"></a> | <a href="/pdf/Schmalstieg_{"id"=>"442", "title"=>"AAA Gaussians"}.pdf"><img src="/img/Schmalstieg_{"id"=>"442", "title"=>"AAA Gaussians"}.jpg" width=200 alt="AAA Gaussians"></a> | <a href="/pdf/Schmalstieg_{"id"=>"418", "title"=>"VR Photo Inpainting"}.pdf"><img src="/img/Schmalstieg_{"id"=>"418", "title"=>"VR Photo Inpainting"}.jpg" width=200 alt="VR Photo Inpainting"></a> |
| <a href="/pdf/Schmalstieg_{"id"=>"406", "title"=>"Good Keyframes to Inpaint"}.pdf"><img src="/img/Schmalstieg_{"id"=>"406", "title"=>"Good Keyframes to Inpaint"}.jpg" width=200 alt="Good Keyframes to Inpaint"></a> | <a href="/pdf/Schmalstieg_{"id"=>"380", "title"=>"Instant"}.pdf"><img src="/img/Schmalstieg_{"id"=>"380", "title"=>"Instant"}.jpg" width=200 alt="Instant"></a> | <a href="/pdf/Schmalstieg_{"id"=>"374", "title"=>"MR Light Fields"}.pdf"><img src="/img/Schmalstieg_{"id"=>"374", "title"=>"MR Light Fields"}.jpg" width=200 alt="MR Light Fields"></a> |
3D reconstruction and authoring
Augmented reality applications require new types of multimedia content to deliver convincing instructions and guidance. This content is generated using techniques in the area of semantic and parameterized reconstruction combined with procedural and computer-assisted 3D authoring.| <a href="/pdf/Schmalstieg_{"id"=>"429", "title"=>"Subsurface infrastructure reconstruction"}.pdf"><img src="/img/Schmalstieg_{"id"=>"429", "title"=>"Subsurface infrastructure reconstruction"}.jpg" width=200 alt="Subsurface infrastructure reconstruction"></a> | <a href="/pdf/Schmalstieg_{"id"=>"434", "title"=>"AuthXR"}.pdf"><img src="/img/Schmalstieg_{"id"=>"434", "title"=>"AuthXR"}.jpg" width=200 alt="AuthXR"></a> | <a href="/pdf/Schmalstieg_{"id"=>"444", "title"=>"IntelliCap"}.pdf"><img src="/img/Schmalstieg_{"id"=>"444", "title"=>"IntelliCap"}.jpg" width=200 alt="IntelliCap"></a> |
| <a href="/pdf/Schmalstieg_{"id"=>"408", "title"=>"Model-free AR instructions"}.pdf"><img src="/img/Schmalstieg_{"id"=>"408", "title"=>"Model-free AR instructions"}.jpg" width=200 alt="Model-free AR instructions"></a> | <a href="/pdf/Schmalstieg_{"id"=>"330", "title"=>"Authoring of AR surface instructions"}.pdf"><img src="/img/Schmalstieg_{"id"=>"330", "title"=>"Authoring of AR surface instructions"}.jpg" width=200 alt="Authoring of AR surface instructions"></a> | <a href="/pdf/Schmalstieg_{"id"=>"288", "title"=>"Technical Document Retargeting to AR"}.pdf"><img src="/img/Schmalstieg_{"id"=>"288", "title"=>"Technical Document Retargeting to AR"}.jpg" width=200 alt="Technical Document Retargeting to AR"></a> |
Localization and tracking
Wide-area localization and tracking is key enabling technology for augmented reality. Our research investigates scalable scene descriptors, sensor fusion, and multimodal localization techniques.| <a href="/pdf/Schmalstieg_{"id"=>"417", "title"=>"Bag of Wor(l)d Anchors"}.pdf"><img src="/img/Schmalstieg_{"id"=>"417", "title"=>"Bag of Wor(l)d Anchors"}.jpg" width=200 alt="Bag of Wor(l)d Anchors"></a> | <a href="/pdf/Schmalstieg_{"id"=>"407", "title"=>"Compact World Anchors"}.pdf"><img src="/img/Schmalstieg_{"id"=>"407", "title"=>"Compact World Anchors"}.jpg" width=200 alt="Compact World Anchors"></a> | <a href="/pdf/Schmalstieg_{"id"=>"402", "title"=>"HoloLens stereo tracking"}.pdf"><img src="/img/Schmalstieg_{"id"=>"402", "title"=>"HoloLens stereo tracking"}.jpg" width=200 alt="HoloLens stereo tracking"></a> |
| <a href="/pdf/Schmalstieg_{"id"=>"366", "title"=>"TrackCap"}.pdf"><img src="/img/Schmalstieg_{"id"=>"366", "title"=>"TrackCap"}.jpg" width=200 alt="TrackCap"></a> | <a href="/pdf/Schmalstieg_{"id"=>"364", "title"=>"VR Upper Body Pose"}.pdf"><img src="/img/Schmalstieg_{"id"=>"364", "title"=>"VR Upper Body Pose"}.jpg" width=200 alt="VR Upper Body Pose"></a> | <a href="/pdf/Schmalstieg_{"id"=>"295", "title"=>"2.5D Streetmap Localization"}.pdf"><img src="/img/Schmalstieg_{"id"=>"295", "title"=>"2.5D Streetmap Localization"}.jpg" width=200 alt="2.5D Streetmap Localization"></a> |
Immersive displays
Conventional XR displays generate only certain types of depth cues and fall short of a hypothetical ultimate display. Our research on immersive display technology improves some of the limitations of existing XR display technology using light field approximation and focal cue synthesis.| <a href="/pdf/Schmalstieg_{"id"=>"431", "title"=>"Gaze-Contingent Layered Optical See Through"}.pdf"><img src="/img/Schmalstieg_{"id"=>"431", "title"=>"Gaze-Contingent Layered Optical See Through"}.jpg" width=200 alt="Gaze-Contingent Layered Optical See Through"></a> | <a href="/pdf/Schmalstieg_{"id"=>"412", "title"=>"Off-Axis Layered See-Through"}.pdf"><img src="/img/Schmalstieg_{"id"=>"412", "title"=>"Off-Axis Layered See-Through"}.jpg" width=200 alt="Off-Axis Layered See-Through"></a> | <a href="/pdf/Schmalstieg_{"id"=>"403", "title"=>"Video See Through with Focal Cues"}.pdf"><img src="/img/Schmalstieg_{"id"=>"403", "title"=>"Video See Through with Focal Cues"}.jpg" width=200 alt="Video See Through with Focal Cues"></a> |