• Eye-tracking
• C#
• EyeX
• Rx

Eye tracking for disabled persons

My father-in-law lives since 10 years with an Amyotrophic Lateral Sclerosis (ALS). This disease left him totally disabled, unable to move anything below the neck.

For a long time I was looking for a way to allow him being autonomous on a computer. In 2014, I acquired a cheap eye tracking device, Tobii’s EyeX, initially made to augment games.

My idea was to make him control the mouse pointer with his gaze, and use his voice to issue some commands (‘click !’, ‘scroll down !’…). Alfred was my first attempt, but it did not went very far, due to a lack of time.

The Optikey project

In september 2015, thanks to a colleague, I heard about Julius Sweetland’s Optikey. Julius has done an humongous work during 3 years to make barely the thing I dreamed of.

When Optikey is running, a visual keyboard takes your screen’s top, allowing to write words simply by looking to keys. A dictionnary is used to make word suggestions, and the typed sentences will appear into any fields that previously had focus. Words can also be spoken using Windows text-to-speech feature.

In addition to the classical letters, diacritics and numerical symbols, Optikey includes mouse controls, to click, double click, scroll, drag’n drop…

Localization and voice commands

I immediately got in touch with Julius, offering to merge my work within Optikey. The usage of voice to trigger commands was in its roadmap, so he accepted my proposal.

But my first work was to entirely localize sources, and share with the growing OptiKey community on transifex online localization tool. After two weeks of intense work, the whole application got English and French localization, and German came few days after.

A work in progress

Adding the voice commands is still a work in progress. Optikey is C# program using Windows Presentation foundation, NuGet package manager and Reactive eXtention for .Net on .Net 4.5 runtime. Until now, it’s a Windows-only application, so I’m using the builtin speech recognition engine.

Users are now able to affect a custom spoken word on every possible command (depending on their chosen locale), and the command will apply once the word is recognized.

Source repository

My fork of Optikey is on github, and the documentation comes as a Wiki.