IBM: Watson could boost confidence of financial industry

Open source software such as Lucene and Linux played a crucial role in IBM's Jeopardy computer

The software that helped IBM's Watson computer reign victorious on the Jeopardy game show in February could also help the financial industry assess risk more effectively, a pair of IBM executives stated on Monday at a high-performance computing conference.

"The financial community is always interested in reducing risk and being more assured of their decisions. The analytics behind Watson can help with some of that," said Jean Staten, IBM's director of cross-company Linux usage, in an interview with IDG.

Staten, along with Eddie Epstein, who led the technical team behind the IBM Watson computer, both spoke Monday at the High Performance Computing Linux Financial Markets conference, held in New York.

Watson's ability to judge the confidence of an answer it provides to a user query could help further speed split-second decision-making process in the financial community, the two noted.

In February, IBM's Watson system beat two previous champions at answering cleverly worded trivia questions on the Jeopardy television game show.

In each episode, after host Alex Trebek asked the contestants a question, Watson's best three answers would be posted on the bottom of the screen, along with a self-assessed confidence rating -- ranging from 0 per cent to 100 per cent -- how likely each answer was correct. The greater the confidence level, the more likely the given answer would actually be correct, according to the machine.

That ability to rank the probable validity of computer-generated answers could be valuable to the financial community, Staten noted. Using an organization's internally collected data, such a feature would also allow hedge fund managers, for instance, to quickly make more informed decisions about investments to make, or to avoid.

The Watson system, when asked a question, will seek to find as many probable answers for that question as it can, explained Epstein. Each answer it produces will contain a confidence level, or an assessment of whether the answer is the correct one, based on analysis of the source of the data, the structure of the question, and other factors.

This approach is "representative of really of what we do in everyday life. When we make decision, we have level of confidence of these decisions," Staten said, adding that we rarely know with absolute certainty whether our answer to some question is the correct one.

In Jeopardy, Watson would submit an answer if it had a confidence rating of around 80 per cent or higher. Within organizations, the threshold of what constitutes a correct or a reliable answer could be calculated by using a set of training data, Staten said.

The Watson system used a variety of open source technologies to help generate answers, each with its own set of algorithms for deriving answers. The Apache Hadoop data processing framework was used for preprocessing the vast amounts of unstructured data that IBM had collected for the challenge. The Apache UIMA (Unstructured Information Management Architecture) provided the framework for deploying search engines, such as Apache Lucene and the Indri inference engine, that would figure out the best answers from a variety of perspectives.

Another potentially appealing aspect to Watson is its speed, Staten said. The Watson system ran over 2,800 processors across two racks of Power7 servers, all of which were configured to run multiple searches in parallel. This ability to carry out many search tasks in parallel allowed the system to provide answers to Trebek within three seconds.

"Speed is more important than ever. We will only have more and more unstructured data that we have to deal with. This is not going away. The technology that Watson is based on provides the framework within which many different industries could benefit," Staten said.

Staten said that IBM is still determining how to bring the Watson technology to the financial service industry. It may not offer a Watson-branded solution per se, though bits of the technology could come embedded within future IBM systems.

Joab Jackson covers enterprise software and general technology breaking news for The IDG News Service. Follow Joab on Twitter at @Joab_Jackson. Joab's e-mail address is

Join the PC World newsletter!

Error: Please check your email address.

Tags open sourceLinuxIBMfinancesoftwareoperating systemsindustry verticalsnon-Windows

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Joab Jackson

IDG News Service
Show Comments

Cool Tech

Lexar® Professional 1000x microSDHC™/microSDXC™ UHS-II cards

Learn more >

D-Link TAIPAN AC3200 Ultra Wi-Fi Modem Router (DSL-4320L)

Learn more >

Xiro Drone Xplorer V -3 Axis Gimbal & 1080p Full HD 14MP Camera

Learn more >

ASUS ROG Swift PG279Q – Reign beyond virtual world

Learn more >

D-Link PowerLine AV2 2000 Gigabit Network Kit

Learn more >

Crucial® BX200 SATA 2.5” 7mm (with 9.5mm adapter) Internal Solid State Drive

Learn more >

Gadgets & Things

Lexar Professional 2000x SDHC™/SDXC™ UHS-II cards

Learn more >

Lexar® Professional 1000x microSDHC™/microSDXC™ UHS-II cards

Learn more >


Learn more >

Family Friendly

Lexar Professional 2000x SDHC™/SDXC™ UHS-II cards

Learn more >

ASUS VivoPC VM62 - Incredibly Powerful, Unbelievably Small

Learn more >

Lexar® Professional 1000x microSDHC™/microSDXC™ UHS-II cards

Learn more >

Stocking Stuffer

Lexar Professional 2000x SDHC™/SDXC™ UHS-II cards

Learn more >

Lexar® Professional 1000x microSDHC™/microSDXC™ UHS-II cards

Learn more >

Christmas Gift Guide

Click for more ›

Most Popular Reviews

Best Deals on PC World

Latest News Articles


GGG Evaluation Team

Kathy Cassidy


First impression on unpacking the Q702 test unit was the solid feel and clean, minimalist styling.

Anthony Grifoni


For work use, Microsoft Word and Excel programs pre-installed on the device are adequate for preparing short documents.

Steph Mundell


The Fujitsu LifeBook UH574 allowed for great mobility without being obnoxiously heavy or clunky. Its twelve hours of battery life did not disappoint.

Andrew Mitsi


The screen was particularly good. It is bright and visible from most angles, however heat is an issue, particularly around the Windows button on the front, and on the back where the battery housing is located.

Simon Harriott


My first impression after unboxing the Q702 is that it is a nice looking unit. Styling is somewhat minimalist but very effective. The tablet part, once detached, has a nice weight, and no buttons or switches are located in awkward or intrusive positions.


Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?