Introduction and Requirements
for GASPilot 1.0
Draft 1 by Michael Ellison
"Video games offer people with disabilities the opportunity to reconnect with their peers and abilities that have been lost or never had." - Robert Florio
Games are an important part of life for people of all ages. From crosswords to Hide & Go Seek, and from card games to the latest versions of Doom; games provide a source for escape, relaxation, and mental stimulation that we as humans need to balance our lives.
In cases of severe disability, many of these pursuits are blocked from the very people who need them the most. With the advance of technology and the proliferation of personal computing power, we currently have the tools to provide the disabled access to a far broader variety of pursuits that ever before.
Unfortunately, the current state of the computer accessibility software industry is pretty dismal. While there are a few notable exceptions (especially for the Macintosh computer), the majority of the companies in the field produce high priced but relatively low quality software – much of which appears not to have been updated substantially since the mid 90's.
This seems to be the result of the wide diversity of disabilities and their requirements for interaction with computers coupled with the very low volumes of sales for products that address any one disability. The fact that the manufacturers can count on insurance companies to purchase at least minimal assistance at high prices may also contribute.
Similar reasons appear to be preventing the game industry from filling that void. The amount of research and development required to make games accessible to disabled gamers is hard to justify against the low numbers of additional sales it would generally be expected to create. Features that affect the majority of gamers almost always outweigh those that would allow disabled gamers to participate.
The Game Accessibility Suite will bridge the gap between disabled gamers and modern computer games, finally providing convenient access to the same games everyone else is playing.
The Game Accessibility Suite, or GAS, is an open source code library and toolbox to research and enhance video game accessibility. GAS is currently in the design and prototyping phase of development. It is released under an MIT-style license.
To make the resulting techniques visible to the game development community and promote incorporation of accessibility features in games.
GASPilot 1.0, the first application of the Game Accessibility Suite, aims to provide crucial accessibility features for existing games.
The planned accessibility features of GASPilot 1.0 are:
To provide in-game color adjustment to increase contrast and reduce the ambiguity of objects for gamers with low vision and color vision impairments.
Currently, GASPilot 1.0 is under active development and projected to be released by Q1 2009. There is currently a full project roadmap available on the Game Accessibility Suite website. A list of the project's projected expenses is also available.
If you would like to participate in the development of GASPilot 1.0 or donate money or hardware to the cause, please contact Michael Ellison. GAS is currently investigating affiliation with non-profit groups, but at this time has not formed a non-profit nor has it associated officially with one. Thus, donations are specifically not tax-deductible at this time.
The following requirements for GASPilot 1.0 are in addition to the target accessibility features listed above and described in detail later in this document.
Must run on Windows XP SP2/32-bit
Vista and other operating systems may be targeted later. Currently, most games – even new ones – run better on Windows XP SP2/32.
Must be usable by its target market
Obvious, but often forgotten in software and features built for accessibility. We'll need help testing by anyone qualified or willing to do so.
Must support popular modern games
Many game accessibility projects are targeted at simple games or even build disability-specific games. GASPilot aims to make the currently popular games accessible.
Speed
ControlDisabled gamers often have to deal with control systems and methods that are more awkward and slower to use than their standard counterparts. As a result, normal game play may be too fast in existing games for the gamer to survive.
GASPilot currently (as of 0.2) includes a feature that provides the real time control of game speeds down to about 25% of original in most Windows-based games. This is achieved by changing the response of system calls querying the time to report that a shorter period of time has elapsed than has actually occurred since previous calls.
While the technique is fully successful in many popular modern games such as Half-Life 2, Doom 3, and Far Cry – additional work needs to be done to fix audio playback and similar issues experienced by other games. Emulated games, such as MAME and many of those found in GameTap, are particularly affected.
Controller
Mapping
Video games use a wide variety of input designs, generally requiring some combination of keyboard, mouse, and/or gamepad input to play. While they often support some kind of configuration of these inputs, the support of the features needed for disabled gamers is almost unheard of in modern games.
GASPilot 0.2 provides prototype support for the QuadControl to play Half-Life 2, Doom 3, and similar games. The goal of the project is to model a wide variety of games and controller types and provide a simple configuration between them. These configuration files will then be able to be exported and traded between users to expand the supported games for a variety of controller types and usages.
GASPilot 1.0 will focus on low cost off-the-shelf consumer controllers, with additional support for affordable custom controllers for disabled gamers. This will include trackballs, mice, joysticks, custom gamepad modifications, Wii remotes, and more.
Some specific issues that GASPilot aims to solve when mapping a controller to a game include:
The game presents an inconsistent interface.
Even if a game natively supports joystick or gamepad input, games commonly require a mouse for menu selections to start and/or configure them. With GASPilot 0.2, joystick emulation of a mouse is provided for the QuadController to allow navigation of these menus.
Digital vs. Analog control
In many games, the camera is controlled by the mouse. There are often configurable keys provided to look up and down and turn. Unfortunately, it seems to be very popular to make these turn too fast to be useful. The same issue becomes apparent when mapping a digital joystick to camera controls. GASPilot 0.2 currently provides filtered movement for turning and looking around with the QuadControl. The current techniques work, and will be improved as development continues.
Press-and-hold buttons
There are often buttons that must be held down in games to perform an action while manipulating other controls. For many gamers with disabilities, this can be difficult. For a mouth controller where you have to suck or blow on a tube, it's undoable for long periods of time and impossible to do while trying to shoot by blowing in another tube. GASPilot will make such controls toggles. When the button is pressed, it just toggles the state and frees the gamer to perform other actions. For mice, similar toggles may be implemented via radial menus and other overlays.
Switch
Control
Some disabled gamers have only a switch or two to control their computer with. Switches for this purpose include Sip and Puff systems, Tilt switches, big buttons, and a wide variety of custom and homebrew solutions. While scanning switch software exists, there is not currently a solution for playing modern games via switch on Windows systems (although a decent application does appear to exist on the Macintosh in SwitchXS , which is used by Macintosh gamers to play games like World of Warcraft and others).
GASPilot 1.0 will provide scanning switch interfaces compatible with in-game use and customizable for each game genre and specific games using its existing game-overlay techniques and skinning framework.
Closed
Captioning
Closed Captioning provides translations of sounds and speech into text that is accessible to deaf gamers. While subtitles of speech have become more prevalent in the top games, only a small handful of games implement full closed captioning with support for audio queues beyond just dialog. This is especially a problem as games become more advanced and audible cues become critical to survival.
GASPilot 1.0 aims to provide closed captioning in games that do not internally provide it. The interface for closed captioning will be based off of the work of Reid Kimball and his team at Games[CC] . It will include a 3D audio radar to provide directional cues in addition to the text.
Initial research has shown that while external captioning may not be feasible in some games (at least without the assistance of their developers), many modern games that use hardware acceleration for 3D audio through common interfaces will be possible to caption and provide an audio radar for. This is done by hooking into the 3D audio libraries (such as OpenAL and DirectSound's 3D audio interface) and monitoring their use.
While the technology is sound, there will still be quite a bit of effort required to implement full closed-captioning on games. The GAS project will work to form a community to caption highly demanded games and redistribute captioning information, as well as building a complimentary toolset to those provided by Games[CC].
Speech
Control
Speech Recognition is an increasingly popular tool for many people. The technology has improved over the years to where it's excellent for a limited set of commands, and often very useful for full dictation. Vocal commands can provide a primary or secondary interface to a game that's quite a bit more flexible in terms of number of actions than traditional controllers.
Unfortunately, modern games currently provide neither native speech control nor the accessibility framework that most normal applications provide to enable speech control.
GASPilot 1.0 will allow configurable commands to be linked to game inputs, just like a button or any other controller input. This will be done by generating a custom grammar for a game from simple XML and receiving commands from Microsoft's SAPI interface – the standard used by professional voice recognition software such as Dragon Naturally Speaking and IBM's ViaVoice.
Microsoft Active Accessibility is Microsoft's name for a suite of accessibility features built into the Windows system and used by a wide range of accessibility tools.
For most applications, these features are automatically enabled by using Windows standard controls such as menus, buttons, and combo boxes. Games, however, must reinvent such controls - and I have yet to find a mainstream video game that supports active accessibility.
GASPilot 1.0 will provide standard accessibility features for games using genre- and game-specific configurations to expose the game's inputs and controls to Windows. WM_GETOBJECT messages sent to the game's primary window will be intercepted by the GASPilotDLL, and an IAccessible object will be returned. Additional accessible objects will be returned for any inputs designated in the game configuration.
Spoken
Feedback
In many cases, text and icons may be too small in a game for a gamer with low vision to easily read or recognize them. Even GASPilot's own in-game overlays may present difficulties to users with low vision.
Color
Adjustment
In many games, the colors used are too dark or similar for gamers with low vision to easily discern important features. Additionally, colors that are ambiguous to color blind gamers are often used as the sole distinguishing feature between safety and danger, friend and foe, and similar characteristics.
GAS will provide a real-time color filter for video games based off the technology in a previous test application called CodeBlind. This will allow for both color modification to improve game playability and simulation modes for game developers to test their games to see if they contain issues that would affect gamers with various vision disabilities.
The technical basis behind GASPilot is its ability to inject a DLL into the target game application and take control of the aspects of the game that need to be modified to provide accessibility. The techniques used are largely built upon the writings of Jeffrey Richter, Feng Yuan, and John Robbins. Additionally, GASPilot has the ability to draw its interfaces over games (or modify the game's drawing entirely) using techniques similar to Fraps and XFIRE. These techniques have been coded up from scratch using the ideas from the above and are currently included as a part of the Common Assistive Technology library (CAT) at the core of the Game Accessibility Suite. Two prototypes have been released proving the effectiveness of the code, although additional improvements to the techniques are planned for the future. The accessibility features of GASPilot are almost all built upon this core.
GASPilot 1.0 will also make liberal use of XML-based skinning techniques. The high-level API for this is provided by the CAT library, with various technologies coming from the following third party libraries:
The licenses for the above libraries, as well as that of GAS, are available here and in redistributions of GASPilot and its associated libraries and source code.