Craig Occupational Therapist and Assistive Technology Specialist Erin Muston-Firsch was recently invited to Israel to participate in Microsoft’s annual, worldwide OneWeek Hackathon and visit ALYN Hospital to share her expertise in assistive technology and adaptive gaming. ALYN, a rehabilitation center for children, adolescents and young adults with physical challenges and disabilities, was looking to expand opportunities for their patients to enjoy video gaming.
In collaboration with Microsoft, Erin traveled to Jerusalem earlier this year to participate in the hackathon, give a workshop to ALYN staffers on gaming, and fit a console, adaptive controller and switches for an ALYN resident. Erin and other Craig occupational therapists supported Microsoft in the development of an adaptive Xbox controller in 2018-2019, so it was gratifying to have the opportunity to bring that experience to a new audience.
The hackathon, called Hack4Good, was part of Microsoft’s annual OneWeek Hackathon that is held at Microsoft locations around the world. Gabi Michel, Senior Hardware Program Manager at Microsoft in Redmond, joined Erin on the trip and provided insight for the Hack team members from her past adaptive design experience. “Accessibility is the most critical part of product development,” Gabi says. “When you do not deliberately and intentionally include, you will unintentionally exclude. I love working with and connecting our partners as we all continue the efforts to make the world more accessible.” Gabi and Erin gave their support to the hackathon participants as they worked on a project to make the world more accessible: the Pointer project.
The Pointer project grew out of a need experienced by many young residents at ALYN who are just learning how to communicate with words: they rely on augmentative or alternative communication (AAC) devices to speak, but they lack the motor skills to look around and point to items in their environment for which they don’t know a name. The Pointer project integrates artificial intelligence (AI), a web camera and communication software so the children can communicate about their environment. Here’s how it works: the web camera captures images of the world that appear on the computer screen. The user then uses eye control to point to an object in the captured image, and image recognition AI names that object. There’s also a user interface that allows for basic commands to pair with the object notification, helping the user generate speech related to the object.
Next on the agenda was a special visit to an ALYN resident, Eli. Dubbed the “Controller-thon,” Erin and Gabi visited Eli to provide him with an Xbox, adaptive controller and switches. Erin worked with Eli’s occupational therapist, Naomi, to fit the equipment so Eli and his friend Danny would be able to play. The room was also packed with many other ALYN residents eagerly watching to see how things would come together. “It was such a great example of how gaming can connect people and transcend barriers like language and ability,” Erin says.
Since ALYN would like to increase gaming overall at the hospital, Erin and Gabi also led workshops, lectures and a brainstorming session to expand staff knowledge and understanding about the use of video gaming in rehabilitation therapy. Erin was able to exchange innovative ideas with ALYN therapists and recognize the similarities between their work despite being in different parts of the world. “One of the best things I learned is that, no matter where you are in the world, occupational therapists all speak the same language,” Erin says.
Gaming continues to be a priority at Craig in neurorehabilitation, and Erin and others are continually advancing the field of adaptive gaming with pioneering solutions for assistive technology. You can check out gaming resources and read more about the impact of the program for Craig patients.