Speech-to-sign-language software wins Microsoft dev contest

Some of the winners of the Imagine Cup could become commercial products, a Microsoft official said

Speech-to-sign language translation software written by a team from Thailand won the main category of a Microsoft-sponsored software development contest for students, the Imagine Cup, on Thursday.

The four students from Kasetsart University in Bangkok won the software design category, ahead of six other finalists. They worked 12-hour days for six months to complete their entry, relying on strict planning to get them through logistical problems caused by riots and political instability in their home country.

Imagine Cup entrants were required to use Microsoft software development platforms to further one or more of the United Nations' Millennium Goals, which include halving extreme poverty, halting the spread of HIV/AIDS, and providing universal primary education, all by 2015.

The competition's US$25,000 first prize attracted entries from 325,000 students, according to Microsoft. Of those, 400 made it to the final round, which offered total prize money of $240,000 across five competition categories and six awards.

The Thai team's software, EyeFeel, combines speech recognition, face recognition and sign-language animation to offer real time translation for people with hearing problems.

"The voice recognition module captures the speech and converts the sentence so it fits the grammar of sign language", Team Skeek captain Pichai Sodsai told IDG News Service. "The sign language is then animated on the screen, while face recognition is used to distinguish different speakers." The software also puts in text balloons much like a comic book, all in real time.

In order to make the software viable, the team needed to make efficient use of resources. "We make extensive use of multithreading code to combine all parts of the software into one, functioning whole", said Sodsai. "Each part needs to run next to the other. It could not be done without that."

Even now, there are some issues the team needs to deal with. Sign language has problems keeping up with spoken language, and during the demo the animation and the written text lagged significantly.

"It is a huge challenge to deal with," said Sodsai. "We deal with most of the problem by letting the software automatically shorten the sentence." Each sentence is brought back to its core meaning, making it easier for the software to keep up. Another limitation is that the software currently only supports American Sign Language and English. "Switching languages is still a huge task," Sodsai said. "We would need to write a whole module for that."

It will take the team members another year to perfect the system, according to Sodsai. Until then, they keep on looking for potential buyers.

"The design is very modular, so we can add and remove features when needed," he said. "We are planning to use a licensing scheme based on free core components and paid extra options."

Team Skeek estimated that there are around 350 million people worldwide with hearing impairments. The main market would be in education, where the software can be used by deaf and nearly deaf people to follow lectures and presentations.

EyeFeel came ahead of a Serbian team that developed a hands-free communication and Web-browsing interface for people with severe forms of paralysis. Third place was taken by New Zealand, with an application for converting files to sounds that can be broadcast over long distances using the FM and AM bands to places where there is little or no Internet connectivity. OLPC (One Laptop Per Child) laptops can receive the files through a connected radio, and convert them back to their original form.

Taiwanese team SmarterME won the Embedded Development competition category with a power meter that tracks the power consumption of individual devices and household appliances. The Game Design category was won by a team from the Philippines.

Not all the projects will have a commercial future, according to Jon Perera, General Manager of the Microsoft Education Group.

"In terms of real world applicability, some of the software [has the potential to] land in the real world market and will have the impact the students want it to have. And, certainly, we see a lot of software that won't necessarily become a commercial or a long term viable solution in the market," he said.

The exploration itself is a starting point for discussions about the role of technology in solving problems, Perera said.

"Some solutions are already being used by governments. The team from Jordan for example built a 'desertification tracking system,' which is a self-powered embedded chipset board that tracks the growth of the desert over time," he said. "The government of Jordan is partnering with them and is sponsoring the team to have several sites tracked."

Join the PC World newsletter!

Error: Please check your email address.

Tags application developmentMicrosoftsoftware

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Michiel van Blommestein

IDG News Service
Show Comments

Most Popular Reviews

Latest News Articles


PCW Evaluation Team

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Aysha Strobbe

Windows 10 / HP Spectre x360

Ultimately, I think the Windows 10 environment is excellent for me as it caters for so many different uses. The inclusion of the Xbox app is also great for when you need some downtime too!

Mark Escubio

Windows 10 / Lenovo Yoga 910

For me, the Xbox Play Anywhere is a great new feature as it allows you to play your current Xbox games with higher resolutions and better graphics without forking out extra cash for another copy. Although available titles are still scarce, but I’m sure it will grow in time.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?