AlternativeInput

Most computers support a limited range of input devices, namely mouse and keyboard. The basic operation of these is built into the Operating System (e.g. Windows, Linux OS-X) and well behaved programs expand on this for their user interaction. A fairly restricted choice is available for users who are unable to use these devices, or those that want alternative. While alternatives and adaptations are available for those with Severe Physical Difficulties they are not always widely supported. Joysticks and game consoles have basic operating system support but few programs other than games support them. Much the same situation exists with support for alternative output such as speech or braille.

Assistive Technology and Accessing Applications

Providing ad-hock support for other devices in each program raises several problems for the users such as inconsistent or patchy support. A better solution is that devices and associated software are used to mediate between programs and users, giving control and monitoring of the programs through the alternative devices. A user can then use one set of devices to interact with many programs. This type of Assistive Technology or AT is also called Electronic AT or EAT. This provides users with a standard interface to many programs.

In order that AT can work consistently with as many programs as possible the programs need to provide standard mechanisms for access and control. These important standards are the AccessibilityAPIs. These API's work by exposing an application's user interface to programs and thus are slightly removed from the actual application state and actions. This is also a slightly fragile situation as User Interfaces often change.

Gestures

A useful abstraction of user interaction is the concept of gestures made through input devices. For example the Firefox Mouse Gestures add-on allows invocation of firefox actions with just a mouse. Key presses can also be thought of as gestures. Thinking in terms of gestures as an abstraction offers more flexibility than specific actions such as click, double-click or Alt+key.

Capability Palette

Some users can only invoke a limited set of gestures and often even they vary from day to day. We can make life easier or them by allowing the selection of gestures that they are capable of through a combination of devices, and assign them to actions in applications. The users preferred gestures then form a capability palette from which they choose how to interact. This puts the user in charge of how they interact with programs. The current GUI interaction model expects each program's design to determine the gestures that are available to users. For example a file could be opened by Mouse:click File, click Open; Mouse:Click Open Toolbar button or Key:Alt+F, Alt+O. The associate user gestures are Mouse:Click and Key:Press. A capability palette allows the user to select their gestures. When this becomes part of an AssistiveTechnology working via AccessibilityAPIs the user can use their preferred gestures with any program.

This requires a the ability to invoke abstract actions that are explicitly exposed by the application, perhaps via a registry service. If all else fails keyboard shortcuts could be used to invoke most actions (if properly implemented by the application). It also requires a separate alternative input component to manage user gestures across devices.

This is idea explored in more detail in CapabilityPalletDesign and by Thorwil.

Alternative Physical Devices

A variety of alternative input devices are available:

  • Switch, including breath and sound.
  • Joystick (switch or analogue).
  • Trackerballs (basically an upside down mouse).
  • Head pointers or head mice.
  • Eye trackers.
  • Brain wave.
  • Touch devices.
  • Speech.

Various alternative hardware devices provide aternative access methods for use with the various AlternativeInputProjectsAndProducts that exist.

Input Modes

In general devices can be categorised into 2 main modes of user interaction:

  • Pointing to any area of the screen (as is done with a mouse) for example with a head mouse, eye tracking or a joystick. This random access allows direct operation programs UI and text entry via an On Screen Keyboard. Additional features such as screen magnification or special tremor reducing hardware or drivers can be useful. However, for text input this is still much slower than typing, espeically for users with major difficulties like poor pointing accuracy or problems with selections and need for dwell-select functionality. These devices often have drivers that simulate mice. Pointing implies vision so alternatives are needed for users with limited vision.
  • Switch access with very limited gestures (compared with a keyboard) such pressing one or two buttons. This often uses a mechanism called "scanning" that steps between options allow selection and activation one at a time. Variations include manual or automatic movement of the selection. Scanning is extremely tedious, for example when an option is missed, the user must cycle back to the start. Optimisations exist such as arranging ordering the items according or grouping items to allow 'homing in' from a large group. Prediction plays an import role. Selection and activation may not be visually cued and can use sound or haptics.

There is some overlap as a group of switches can be used to indicate direction and thus provide pointing.

We could possibly add more complex encoded input such as chorded keyboards. These allow for one of several values to input in one event, but less than a full keyboard.

Switch access is relatively tedious due to the limited gestures available to users and exacerbated by frequent difficulties with accuracy and consistency. It is thus an area that is ripe for innovation and users could benefit from using several devices and modes together according to their abilities and choice.

There is an online demo of various scanning modes (needs windows flash plugin) and the Ace Centre have a comprehensive introduction to Switch Access.

Device Connection and Operation

Alternative Pointers usually emulate a mouse. Thus there is no special support required by the operating system or programs. The pointer device driver will often have special options to meet the needs of users such as shake eliminatio n.

Historically (on Windows at least) there have a number of ways of connecting switches to PCs as no standard interface existed. These include keyboard or mouse emulation, SerialKeys and games port (via a sound card).

Switch access to applications is usually provided either via

  • specific support in the application itself (e.g. Clicker, E-Scape)
  • device drivers that emulate other input devices (e.g. Sensory Software driver)
  • a general purpose intermediate scanning overlay program (e.g. GOK, SAW or The Grid).

USB Joy Sticks and Games Controllers

The USB standard has created the ability to support a large range of input devices, or Human Interface Device (HID). The popularity of Games controllers for PCs has meant these that these devices are well supported on Windows XP due to the large games market. Devices may be plugged in and will work immediately through plug-and-play technology. They appear as standard devices (in Control Panel -> Games Contollers on Windows) and support a large number of switches (as well as analogue joysticks).

Thus USB Games Controllers provide a good mechanism to connect switches as shown by the availability of devices such as Sensory Software's Joy Cable and Joy Box.