Apple talks about new iPhone features like door detection, live captioning

Global Accessibility Awareness Day is Thursday, so this week Apple announced several major new accessibility features for iPhone, Apple Watch, iPad, and Mac via its News Blog this week.

One of the most widely used will probably be Live Captions coming to iPhone, Mac and iPad. This feature shows AI-controlled, real-time-updated captions for speech coming from any audio source on the phone, whether the user is “on the phone or in FaceTime, using a video conferencing or social networking app, streaming multimedia content, or having talking to someone next to them.”

There is also door detection. Unfortunately, it will only work on iPhones and iPads with a lidar sensor (i.e. iPhone 12 Pro, iPhone 13 Pro, or the latest iPad Pro models), but sounds useful for those who are blind or visually impaired. It uses the iPhone’s camera and augmented reality sensors, combined with machine learning, to identify doors and alert users audibly about where the door is, whether it’s open or closed, how it can be opened, and what writing or markings it might have.

Door detection will combine people detection and image descriptions in a new “detection mode”designed for blind or visually impaired users on iOS and iPadOS. However, the Apple blog post does not say when the feature will launch.

Other accessibility features Apple says are on the way include 20 new Voice Over languages, new hand gestures on the Apple Watch, and a feature that allows gamers to get help from a “buddy”with a different game controller without turning off their own. In addition, there are new Siri and Apple Books settings designed to improve accessibility for people with disabilities, audio recognition settings, and Apple Watch screen mirroring on iPhone, giving Watch users access to many special features available on iPhone but not on watch.

Tech enthusiasts often lament that smartphones (and personal technology in general) have become stagnant, without many exciting new developments. But that couldn’t be further from the truth for many people with disabilities. Google, Apple, and numerous researchers and startups have made significant strides in bringing powerful new accessibility features to mobile devices.

Listed image from Apple