Our Motivation

We want to make a positive impact in the lives of more than 20 million visually impaired children worldwide. We have been working diligently with many experts to develop high value applications for them.

Brief description of our FLL team

Our team consists of three boys and four girls ages 9-11. We started the team because of our interest in robot building, programming, research, and just plain fun. We enjoy being together and always have fun at our team meetings. We play games and enjoy developing and performing skits as a team. We enjoy different sports, soccer, basketball, tennis, etc. We conduct science experiments and participate in science fairs. Members of our team had fought through learning disabilities in their early childhood. We were empathetic towards children with disabilities and wanted to do something to help them learn STEM skills.

We chose this problem because we saw the struggles of visually impaired children to visualize advanced math topics such as graphs, curves, etc. The current methods are inaccurate and frustrating. We developed an application that provides haptic feedback on touch screen devices. We demonstrated it to several visually impaired students and learning specialists. We got very enthusiastic response and they asked us to expand it to other areas, e.g. shapes, letters (to learn how sign the name), maps, slope fields, etc. We have set up an application platform that is based on open source, game development platform. We have set up a non-profit organization and are expanding the application portfolio to address the needs of visually impaired.

Currently, visually impaired students use special graph paper with raised lines to visualize graphs. They use push pins, rubber bands and pipe cleaners. It is slow, inaccurate and frustrating. Several students complain of headaches after the math lessons. The teacher has to work with each student individually to help them with the graphs. The graphing calculators available in the market (e.g. Orion TI-84 Plus) cost $599 and do not have touchscreen, making it hard to trace and visualize.

Our invention is an application platform called Haptic Vision, to build fun and engaging apps for visually impaired. The platform provides haptic feedback system by taking advantage of the haptic sensors and actuators in touchscreen devices. It provides several reusable libraries to provide haptic feedback functions (vibration of the devise, variation sound pitch based on the location of the target, etc.). Our initial app focused on the helping students learn graphs and curves. The touchscreen has a grid. Instead of raised lines, the lines on the screen provide haptic feedback in the form of vibration and sound. The screen vibrates when the students follow the curve and starts to beep as they wander off course. The pitch varies based on the distance from the target. The app was developed on an open source platform called COCOS2D. It supports multiple platforms (Android, iOS) from a single code base. It is powerful, easy to use and has many game like features.

Our solution improves the speed, accuracy and affordability. By introducing game like features it makes learning math fun for visually impaired. The teacher can broadcast the curve/shape to all the students’ devices instantly. All the students can follow along at the same time. The shape of the curve, e.g. parabola can be followed more accurately on a touchscreen with haptic feedback, compared to a graph sheet with pushpins, rubber bands/pipe cleaners. Currently, there are no apps in Apple Appstore or Google Play that provide Haptic feedback for tracing graphs, curves, etc. Haptic Vision apps can downloaded on any touchscreen device. We found some Android tablets with 7 inch touchscreens that are selling for $36 on Amazon.com. As the apps will be free to download, the solution very affordable compared to talking calculators such as Orion TI-84 Plus that priced at $599.

The initial application was developed for graphing with the help of volunteer app developer within a month’s time. The actual effort was around 40 hours. The COCOS2D platform has an enthusiastic community of game developers with wide following amongst the high school and college students. We set up a non-profit organization and set up the code base in Github. Volunteer developers can contribute by adding new apps/features. We are actively working several professionals involved in teaching blind/visually impaired students. We got a list of high priority apps and feature that will have the biggest impact. We are working with a professional architect to break it up into small tasks that can be coded in 10-20 hours of effort. We plan to conduct a Hapathon (Hackathon for Haptic Apps) at the local high school and university. We are confident that we can make this platform grow through contributions of caring people throughout the world.