About this Project
This project was my team’s submission to the HackUMass VI Hardware & Software Hackathon. Working in a team of five, we were inspired by one team member’s family member who is a budding programmer with Carpal Tunnel Syndrome to build a better solution for people with limited use of their hands to be able to program. This project won the Hackathon Grand Prize and Prize for Best Documentation.
Project Pitch
Currently, the programming community does a very poor job of catering to those who have limited or no use of their hands. Whether it be due to repeated stress injury or other medical reasons, many programmers are hindered from their craft due to this issue. This is why we created CodeAbility.
We found speech recognition to be a natural solution to this problem. However, current implementations of this method fail to give their users an ideal experience because they fall into one of two extremes; they either are too syntactically oriented, forcing you to dictate every bracket, space, and semicolon, or they use a complex set of keywords that create a steep learning curve and make the language impossible to understand. We wanted to find a happy balance between these two extremes in order to provide the best user experience possible.
So, we created a naturally flowing English-like language which is parsed into python. This allows you to dictate an algorithm or complex programmatic phrase just as if you were speaking pseudo-code to a friend.
In addition to this natural language input, we implemented vim-based voice keywords that allow great flexibility and ability to spot-edit previously written code. Last, a set of custom build and completely configurable foot pedals allow users to navigate the page and perform quick actions just as easily as they would with a mouse.
Overall, CodeAbility implements a unique combination of hardware and software to provide an experience that is both efficient and easy to use for people who program without their hands.
Read more technical detail on GitHub.
Or, read more about our story on the project submission page.
Demo Video
Here’s a complete demo video of the project in action!