Game Accessibility Suite

Introduction and Requirements

for GASPilot 1.0

Draft 1 by Michael Ellison

"Video games offer people with disabilities the opportunity to reconnect with their peers and abilities that have been lost or never had." - Robert Florio

Computer Accessibility & Games

Games are an important part of life for people of all ages. From crosswords to Hide & Go Seek, and from card games to the latest versions of Doom; games provide a source for escape, relaxation, and mental stimulation that we as humans need to balance our lives.

In cases of severe disability, many of these pursuits are blocked from the very people who need them the most. With the advance of technology and the proliferation of personal computing power, we currently have the tools to provide the disabled access to a far broader variety of pursuits that ever before.

Unfortunately, the current state of the computer accessibility software industry is pretty dismal. While there are a few notable exceptions (especially for the Macintosh computer), the majority of the companies in the field produce high priced but relatively low quality software – much of which appears not to have been updated substantially since the mid 90's.

This seems to be the result of the wide diversity of disabilities and their requirements for interaction with computers coupled with the very low volumes of sales for products that address any one disability. The fact that the manufacturers can count on insurance companies to purchase at least minimal assistance at high prices may also contribute.

Similar reasons appear to be preventing the game industry from filling that void. The amount of research and development required to make games accessible to disabled gamers is hard to justify against the low numbers of additional sales it would generally be expected to create. Features that affect the majority of gamers almost always outweigh those that would allow disabled gamers to participate.

The Game Accessibility Suite will bridge the gap between disabled gamers and modern computer games, finally providing convenient access to the same games everyone else is playing.

Introduction to the Game Accessibility Suite

The Game Accessibility Suite, or GAS, is an open source code library and toolbox to research and enhance video game accessibility. GAS is currently in the design and prototyping phase of development. It is released under an MIT-style license.

GAS Goals

GASPilot 1.0

GASPilot 1.0, the first application of the Game Accessibility Suite, aims to provide crucial accessibility features for existing games.

The planned accessibility features of GASPilot 1.0 are:

  1. To provide control of the speed of games to suit the needs of the gamer.
  2. To provide support for a wide variety of commercial and custom controller types and allow gamers to map them in a usable fashion to the inputs required by individual games.
  3. To provide a scanning switch interface manipulated by one or more simple switches to control games.
  4. To provide support for closed-captioning in games that do not natively support it.
  5. To provide control of game-specific actions using vocal commands.
  6. To provide the insertion of Microsoft Active Accessibility support into games to allow standard accessibility tools to manipulate the game as they do other applications.
  7. To provide spoken feedback for actions and menus within games for gamers with poor vision.
  8. To provide in-game color adjustment to increase contrast and reduce the ambiguity of objects for gamers with low vision and color vision impairments.

GASPilot 1.0 Current Status

Currently, GASPilot 1.0 is under active development and projected to be released by Q1 2009. There is currently a full project roadmap available on the Game Accessibility Suite website. A list of the project's projected expenses is also available.

If you would like to participate in the development of GASPilot 1.0 or donate money or hardware to the cause, please contact Michael Ellison. GAS is currently investigating affiliation with non-profit groups, but at this time has not formed a non-profit nor has it associated officially with one. Thus, donations are specifically not tax-deductible at this time.

Additional GASPilot Requirements

The following requirements for GASPilot 1.0 are in addition to the target accessibility features listed above and described in detail later in this document.

Speed Control

The Problem:

Disabled gamers often have to deal with control systems and methods that are more awkward and slower to use than their standard counterparts. As a result, normal game play may be too fast in existing games for the gamer to survive.

The Solution:

GASPilot currently (as of 0.2) includes a feature that provides the real time control of game speeds down to about 25% of original in most Windows-based games. This is achieved by changing the response of system calls querying the time to report that a shorter period of time has elapsed than has actually occurred since previous calls.

While the technique is fully successful in many popular modern games such as Half-Life 2, Doom 3, and Far Cry – additional work needs to be done to fix audio playback and similar issues experienced by other games. Emulated games, such as MAME and many of those found in GameTap, are particularly affected.

Controller Mapping

The Problem:

Video games use a wide variety of input designs, generally requiring some combination of keyboard, mouse, and/or gamepad input to play. While they often support some kind of configuration of these inputs, the support of the features needed for disabled gamers is almost unheard of in modern games.

The Solution:

GASPilot 0.2 provides prototype support for the QuadControl to play Half-Life 2, Doom 3, and similar games. The goal of the project is to model a wide variety of games and controller types and provide a simple configuration between them. These configuration files will then be able to be exported and traded between users to expand the supported games for a variety of controller types and usages.

GASPilot 1.0 will focus on low cost off-the-shelf consumer controllers, with additional support for affordable custom controllers for disabled gamers. This will include trackballs, mice, joysticks, custom gamepad modifications, Wii remotes, and more.

Some specific issues that GASPilot aims to solve when mapping a controller to a game include:

Switch Control

The Problem:

Some disabled gamers have only a switch or two to control their computer with. Switches for this purpose include Sip and Puff systems, Tilt switches, big buttons, and a wide variety of custom and homebrew solutions. While scanning switch software exists, there is not currently a solution for playing modern games via switch on Windows systems (although a decent application does appear to exist on the Macintosh in SwitchXS , which is used by Macintosh gamers to play games like World of Warcraft and others).

The Solution:

GASPilot 1.0 will provide scanning switch interfaces compatible with in-game use and customizable for each game genre and specific games using its existing game-overlay techniques and skinning framework.

Closed Captioning

The Problem:

Closed Captioning provides translations of sounds and speech into text that is accessible to deaf gamers. While subtitles of speech have become more prevalent in the top games, only a small handful of games implement full closed captioning with support for audio queues beyond just dialog. This is especially a problem as games become more advanced and audible cues become critical to survival.

The Solution:

GASPilot 1.0 aims to provide closed captioning in games that do not internally provide it. The interface for closed captioning will be based off of the work of Reid Kimball and his team at Games[CC] . It will include a 3D audio radar to provide directional cues in addition to the text.

Initial research has shown that while external captioning may not be feasible in some games (at least without the assistance of their developers), many modern games that use hardware acceleration for 3D audio through common interfaces will be possible to caption and provide an audio radar for. This is done by hooking into the 3D audio libraries (such as OpenAL and DirectSound's 3D audio interface) and monitoring their use.

While the technology is sound, there will still be quite a bit of effort required to implement full closed-captioning on games. The GAS project will work to form a community to caption highly demanded games and redistribute captioning information, as well as building a complimentary toolset to those provided by Games[CC].

Speech Control

The Problem:

Speech Recognition is an increasingly popular tool for many people. The technology has improved over the years to where it's excellent for a limited set of commands, and often very useful for full dictation. Vocal commands can provide a primary or secondary interface to a game that's quite a bit more flexible in terms of number of actions than traditional controllers.

Unfortunately, modern games currently provide neither native speech control nor the accessibility framework that most normal applications provide to enable speech control.

The Solution:

GASPilot 1.0 will allow configurable commands to be linked to game inputs, just like a button or any other controller input. This will be done by generating a custom grammar for a game from simple XML and receiving commands from Microsoft's SAPI interface – the standard used by professional voice recognition software such as Dragon Naturally Speaking and IBM's ViaVoice.

Microsoft's Active Accessibility

The Problem:

Microsoft Active Accessibility is Microsoft's name for a suite of accessibility features built into the Windows system and used by a wide range of accessibility tools.

For most applications, these features are automatically enabled by using Windows standard controls such as menus, buttons, and combo boxes. Games, however, must reinvent such controls - and I have yet to find a mainstream video game that supports active accessibility.

The Solution:

GASPilot 1.0 will provide standard accessibility features for games using genre- and game-specific configurations to expose the game's inputs and controls to Windows. WM_GETOBJECT messages sent to the game's primary window will be intercepted by the GASPilotDLL, and an IAccessible object will be returned. Additional accessible objects will be returned for any inputs designated in the game configuration.

Spoken Feedback

The Problem:

In many cases, text and icons may be too small in a game for a gamer with low vision to easily read or recognize them. Even GASPilot's own in-game overlays may present difficulties to users with low vision.

The Solution:

When possible, GAS plans on providing optional spoken feedback for such items through Microsoft's Text-To-Speech APIs, which will use internal voices or those provided by a professional Text-To-Speech package (such as Dragon Naturally Speaking).

Additionally, spoken feedback will be available for GASPilot itself and any interfaces it injects into video games.

Color Adjustment

The Problem:

In many games, the colors used are too dark or similar for gamers with low vision to easily discern important features. Additionally, colors that are ambiguous to color blind gamers are often used as the sole distinguishing feature between safety and danger, friend and foe, and similar characteristics.

The Solution:

GAS will provide a real-time color filter for video games based off the technology in a previous test application called CodeBlind. This will allow for both color modification to improve game playability and simulation modes for game developers to test their games to see if they contain issues that would affect gamers with various vision disabilities.

GASPilot's Technical Features

The technical basis behind GASPilot is its ability to inject a DLL into the target game application and take control of the aspects of the game that need to be modified to provide accessibility. The techniques used are largely built upon the writings of Jeffrey Richter, Feng Yuan, and John Robbins. Additionally, GASPilot has the ability to draw its interfaces over games (or modify the game's drawing entirely) using techniques similar to Fraps and XFIRE. These techniques have been coded up from scratch using the ideas from the above and are currently included as a part of the Common Assistive Technology library (CAT) at the core of the Game Accessibility Suite. Two prototypes have been released proving the effectiveness of the code, although additional improvements to the techniques are planned for the future. The accessibility features of GASPilot are almost all built upon this core.

GASPilot 1.0 will also make liberal use of XML-based skinning techniques. The high-level API for this is provided by the CAT library, with various technologies coming from the following third party libraries:

The licenses for the above libraries, as well as that of GAS, are available here and in redistributions of GASPilot and its associated libraries and source code.