elkraneo’s Twitter Archive—№ 4,387

I got to talk about Augmented Reality at BorderlessEng. The talk included some developer tips and an overview of how the Apple AR platform has changed, but the best part was the section on making Augmented Reality Applications Accessible 🧵1/24 #Accessibility #AR
On twitter.com ♻️ 7 Retweets ❤️ 16 Favorites 2022 Sep 22 Mood +4 🙂

…in reply to @elkraneo
I began with my currently preferred definitions of Augmented Reality and Disability Augmented Reality: understood as a “Combination of digital and physical worlds” with “Real-time interactions” and a degree of “Identification/segmentation of virtual and real elements” 🧵2/24
Permalink On twitter.com ❤️ 2 Favorites 2022 Sep 22 Mood -2 🙁

…in reply to @elkraneo
Disability: Defined as the “The results of the interaction between individuals with a health condition and the features of their environment” 🧵3/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood -2 🙁

…in reply to @elkraneo
Also mention, the responsibility of fighting the limited social support and assuming that almost everyone will temporarily or permanently experience disability at some point in their life This is my interpretation of the current WHO definition web.archive.org/web/20220920025713/https://www.who.int/health-topics/disability 🧵4/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood -2 🙁

…in reply to @elkraneo
Then continued presenting the accessibility capabilities included in Reality Composer developer.apple.com/augmented-reality/tools/ First labeling a scene and then doing the same for an imported 3D model 🧵5/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood +1 🙂

…in reply to @elkraneo
(The presentation didn't mention labeling "Behaviors") 🧵6/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood 0

…in reply to @elkraneo
After demonstrating the developer perspective including metadata, I wanted to show a subjective experience from someone who wanted to use it but couldn't see the screen 🧵7/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood 0

…in reply to @elkraneo
And then, from the perspective of somebody using Voice Control. The fact that so much can be achieved with so little effort is impressive. Accessibility Services apple.com/accessibility/ are excellent at working with 'less than ideal contextual information' scenarios.👏 🧵8/24
Permalink On twitter.com ♻️ 2 Retweets ❤️ 2 Favorites 2022 Sep 22 Mood +8 🙂

…in reply to @elkraneo
But we can do better. Assistive technologies have been developed to work on graphical user interfaces (GUIs) by identifying common tasks and then creating accessible alternatives. Therefore it should be possible to apply the same principle to Augmented Reality interfaces 🧵9/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood +3 🙂

…in reply to @elkraneo
And that is exactly what Making Mobile Augmented Reality Applications Accessible arxiv.org/abs/2010.06035 is about 🧵10/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood +1 🙂

…in reply to @elkraneo
This paper analyzed several AR apps and identified common tasks. Then, prototypes proposing accessible alternatives were constructed and tested with a group of blind participants. From the dataset, a clear picture can be seen of where AR is being used 🧵11/24
Permalink On twitter.com ❤️ 2 Favorites 2022 Sep 22 Mood +1 🙂

…in reply to @elkraneo
The analysis shows that there are 5 basic tasks required to interact with an AR app: - Observing AR Content 👓 - Establishing Physical/Virtual Correspondence ⚓️ - Creating Virtual Content 🎨 - Transforming Virtual Content 🕹 - Activating Virtual Content ▶️ 🧵12/24
Permalink On twitter.com ❤️ 2 Favorites 2022 Sep 22 Mood +2 🙂

…in reply to @elkraneo
👓 Observing AR Content: The explorative task, look around, perceive the size, shape, color, style of a virtual object. Relationships between objects can also be checked either between virtual and virtual or between physical and virtual 🧵13/30
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood +1 🙂

…in reply to @elkraneo
⚓️ Establishing Physical/Virtual Correspondence: It's used for finding where to anchor virtual content. This task is present when scanning for surfaces, images, objects, or marking a location in space 🧵14/30
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood 0

…in reply to @elkraneo
🎨 Creating Virtual Content: The constructive task. After the scanning process is complete, the next most common task is to place an object at a defined point 🧵15/30
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood +1 🙂

…in reply to @elkraneo
🕹 Transforming Virtual Content “Traditional” 3D manipulations. Position, Orientation, Scale, and Deletion. Each manipulations' goal is highly dependent on the context 🧵16/30
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood 0

…in reply to @elkraneo
▶️ Activating Virtual Content The selection task. Trigger sound, narration, animation, or physics-based motion. This task is by far the most diverse 🧵17/30
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood 0

…in reply to @elkraneo
With this list of basic tasks, it is now possible to make them accessible to everyone 🧵18/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood +1 🙂

…in reply to @elkraneo
First, accessibility services must be made aware of virtual content, and then a way must be provided for developers to feed metadata to it You can see this concept applied on the Reality Composer accessibility options in combination with the AR views. 🧵19/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood 0

…in reply to @elkraneo
Then, it is important to know that simply making accessibility features available for developers to add to their apps might not be enough Also, more advances will need to be made in 3D space semantic description and language processing 🧵20/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood +2 🙂

…in reply to @elkraneo
And finally, consider the benefits of creating virtual versions of the surroundings with the aim of using them with augmented reality content 🧵21/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood +2 🙂

…in reply to @elkraneo
This mixed-reality scenario relies on the fact that Virtual Reality is easier in some ways to make accessible because the system knows (in theory) about all content in the virtual world 🧵22/24
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 22 Mood +1 🙂

…in reply to @elkraneo
This presentation made me realize that AR is capable of also augment accessibility and become a valuable technology for people with disabilities 🧵23/24
Permalink On twitter.com ❤️ 2 Favorites 2022 Sep 22 Mood -1 🙁

…in reply to @elkraneo
Special Thanks and References craft.do/s/K3k6ACmoawMe7W ABaillyLloyd AtarayoSD basthomas dadederk heckj lennartstolz MarcoInEnglish marius_const matkoin MaxxFrazer romanroibu RoxanaJula Sommer StrangeNative Tgo1014
Permalink On twitter.com ♻️ 2 Retweets ❤️ 10 Favorites 2022 Sep 22 Mood +2 🙂

…in reply to @elkraneo
There is a video now available with subtitles on English and Spanish (generated with whisper) youtu.be/ev0k0kS34BQ
Permalink On twitter.com ❤️ 2 Favorites 2022 Sep 29 Mood 0

…in reply to @elkraneo
If you feel interested in making augmented reality accessible, then you have to check out the work done by jayhersk, AnhongGuo and the HAIL lab. Can't be more grateful 🙇
Permalink On twitter.com ❤️ 1 Favorite 2022 Sep 29 Mood +8 🙂