Last Saturday Patently Apple posted a report titled "The Website of the Year interviewed Apple's Head of Accessibility who explained how the company is driven to Think Holistically" when designing accessibility software and hardware.

Sarah Herrlinger, Apple’s global head of accessibility sat down with UK's website of the year the "Evening Standard" for an interview.  In the big picture, Herrlinger notes that "One of the great things about doing this kind of work at Apple is that the company controls its hardware, software and operating systems. We can infuse accessibility [features] across everything we do and do it in a really holistic way."

Yesterday Apple was granted an in-depth patent titled "Interface scanning for disabled users." The patent generally relates to electronic devices like a MacBook, iPad or iPhone for physically-impaired users and, more specifically, to electronic devices that allow a motor-impaired user to navigate a computer interface.

Apple notes at one point that a computing device like an iPhone could be further configured to select any of the elements in response to an input received by input device while the particular element is being scanned, thereby allowing a user to interact with any of the elements in the user interface using one or more inputs (e.g., button press, mouse click, tap on a touch sensitive surface, voice command, movement of a part of the users body, or the like). This is particularly useful for disabled users that are unable to operate input devices requiring motion and careful coordination, such as a keyboard, mouse, or touch sensitive display.

Apple's video below highlights what the patent described above. The use of their assistive technology called 'Switch Control' allows Ian MacKay to use his iPhone take a photo without ever having to touch the display.  

Apple's patent further covers systems and processes for scanning a user interface. One process can include scanning multiple elements within a user interface by highlighting the elements. The process can further include receiving a selection while one of the elements is highlighted and performing an action on the element that was highlighted when the selection was received.

The action can include scanning the contents of the selected element or performing an action associated with the selected element. The process can be used to navigate an array of application icons, a menu of options, a standard desktop or laptop operating system interface, or the like. The process can also be used to perform gestures on a touch-sensitive device or mouse and track pad gestures (e.g., flick, tap, or freehand gestures).

Apple's engineering team has creatively worked to allow new kinds of user input to control an iPhone or iPad's interface for the disabled. Examples include using a breath sensor to measure air pressure blown into a straw to assist users select an item on screen; also an image sensor configured to detect movement of a user such a blinking or certain movements of the user's head to control a UI element; an IR proximity sensor configured to detect an object within a threshold distance from the sensor; and an audio sensor configured to detect a sound having a predetermined characteristic (e.g., volume, pitch, or the like), and more.

Apple's patent FIG. 5 below illustrates an exemplary process for defining and grouping elements of a user interface (including element menus); FIG. 6 illustrates an exemplary process for scanning elements of a user interface.

Apple's patent FIG. 19 above covers the use of process #600 of patent FIG. 6 that presents users with a gesture menu where they can select whether to use taps, flicks, pinches, pan, double tap, drag, freehand and adjust the number of fingers used to perform a gesture; FIG. 31 illustrates an exemplary interface #3100 that illustrates the operation of process #600 to perform freehand gestures from a menu; FIG. 51 illustrates an interface covering a virtual keyboard and menu for choosing alternative keys.  

If this subject matter is of interest to you, then be sure to check out the full patent and its 54 patent figures here. Also be sure to check out Apple's Accessibility web page here for more on what areas that Apple is delivering solutions for those with disabilities.

Some of Apple's Inventors

Eric Seymour: Director, Accessibility Software Engineering

Darren Minifie: Engineering Manager, iOS Vision Accessibility

Patti Hoa: Software Engineering Manager

Clare Kasemset: Software Engineer: develops accessibility iOS solutions 

Gregory Hughes, PhD: Software Engineering Manager, Accessibility Products

Ian Fisch: iOS Accessibility Software Engineer

Chris fleizach: Human (that's it)