Metri XR
Service
Mixed Reality (MR)
Industry
Education
Year
2020
Metri XR is an extension project of Do You Speak Holo?, an application for Microsoft HoloLens 1 co-designed with linguistics experts. In project Metri XR we port the MR application for HoloLens 1 to a VR application for Oculus Quest The main idea of these two projects is to introduce a new therapeutical way of learning a foreign language through MR and VR.
Output 1 💡 | Port the MR app for Hololens 1 to a VR app for Oculus Quest
Output 1 💡
Port the MR app for Hololens 1 to a VR app for Oculus Quest
HoloLens 1
Oculus Quest
Output 2 💡
Web-based platform for activities, customization & monitoring
Project content 🎯
Problems, audience and main challenge
Main Target:
People with language disorder
Educational Context:
E.g. School, Therapy
Needs:
Training in a foreign language
Constraints:
Difficulty performing testing
Children comprehension ability
Goals:
Learning process trough
MR application
Scenarios 🎮
Three playable learning experiences
Each activity seeks to promote the learning of a particular set of words (vocabulary) and morpho-syntactic structures of the English language such as functional words (possessives, prepositions of place, comparatives, and superlatives)in a context-oriented activity using MR. We designed and developed three scenarios, of which we provide some examples taken from the practice phases.
Possessive adjectives: the virtual teacher asks the user to move the holographic items into baskets belonging to different characters. The digital content is based on the vocabulary of fruits and vegetables, for instance: “This is her apple”
Comparatives and superlatives: in this scenario, we focus on the vocabulary of animals. The virtual teacher asks the user to identify an animal according to a comparative/superlative relationship with the other ones
and to move it toward a holographic representation of Noah’s ark. For instance: “Move the animal that is bigger than the rabbit but smaller than the rhino”.
Prepositions of place: the vocabulary involves school/office items. The user is asked Possessive to tidy up
a real desk (recognized by the device thanks to spatial mapping capabilities), moving the virtual objects in some specific spots, according to the prepositions of place identified by the virtual teacher. For instance:
“Pick the rubber behind the ruler and put it on the book”.
Storyboard 🎯
Therapist & patient session
Architecture map 🗺
Tablet app structure
Prototype 🎬
The web-based application has been developed and teachers can now monitor their patient sessions live during therapy. When the session is over, they can write notes about patient behaviors and track patient grammatical errors derived from a misunderstanding of grammar rules.
With this app, therapists can keep track of all patient data. It helps them track the long record of patients' behaviors so they can more easily treat them for their exact language disorder.
The main errors that can be tracked are: Grammatical Errors & Usability Errors
Grammatical Errors - derived from a misunderstanding of grammar rules, e.g the child is asked to put a fruit
in the basket of the male character, but he drops the fruit in the female character’s basket. A therapist can then
deduce that the child does not understand proper possessive adjectives.
Usability Errors - derived from the wrong usage of the devices, e.g the child grabs digital content with
an Oculus controller but accidentally drops the picked-up digital object. E.g (HoloLens) the child is not able
to perform an air-tap gesture to pick up holograms (or is not able to maintain the air-tap gesture)
The team 🫶
Dream combo!
Bogdan Novakovic
Designer
Ibrahim El Shemy
Developer
Franca Garzotto
Mentor