Punctum is a mobile app that uses the gesture of pointing out objects in the physical environment to interact with their digital information. By overlaying three-dimensional virtual models with their related elements in the real world, it's possible to browse buildings and other volumetric elements in a seamless experience.
It was developed by Sergio Boiteux in 2014 as result of his master's research on Interaction Design at ESDI/UERJ.
The use of geotags in digital assets and the popularization of mobile computing established a new model of information-based interaction.
Many Location-Based Services (LBS) and Augmented Reality apps treat geo-data as single coordinates to present georeferenced files, to evaluate distances or get relative direction from user. They rely on screen mediation and end up constraining the relation between the user and the environment. It interferes with the interactive experience by adding icons and symbols built upon users field of view, already framed and flattend by the screen.
As an outcome of the research, a mobile application was developed that spares the use of displays when selecting points of interest in the physical environment. A tridimensional approach to geo-data (width, depth and height) can recreate the surrounding and reproduce its occlusions and perspective. That way, without the interference of screens or immersive devices, the environment itself is presented as an interface and make technology more subtle in its daily use.
Punctum reproduces a popular technique for object selection in virtual environments called ray-casting to interact with real world objects. It evaluates the nearest virtual model that intersects the vector established by user’s position and his device orientation.
When the user selects a point of interest with his device, an online request is generated with his GPS position (latitude, longitude and elevation) and inertial sensors data (pan, tilt and roll). On server-side, a virtual environment is recreated based on the coordinates, and the object is selected by evaluating the nearest face intersecting the orientation vector.
On server-side, Punctum integrates OpenStreetMap's volumetric georeferenced data and Wikipedia's complementary information, so, after identifying the object of interest in the real world, all public and colaborative information about it is sent back to the app and displayed to the user.
Sergio Boiteux is an interactive designer and founder of Plano B, a design studio based in Rio de Janeiro / Brazil.
Check out the volumetric data available in OpenStreetMap through OSM Builder script.