“Jeopardy!” player Ken Jennings, who won a record 74 consecutive games, refers to his opponent, an IBM computer called “Watson,” after a practice round of the quiz show in Yorktown Heights, New York, in 2011. (AP Photo/Seth Wenig, File)

“Jeopardy!” player Ken Jennings, who won a record 74 consecutive games, refers to his opponent, an IBM computer called “Watson,” after a practice round of the quiz show in Yorktown Heights, New York, in 2011. (AP Photo/Seth Wenig, File)

AI can read! Tech firms race to smarten up thinking machines

“A long way from computers being able to read … general text in the same way that humans can.”

  • By MATT O’BRIEN AP Technology Writer
  • Tuesday, January 23, 2018 3:07pm
  • Business

By Matt O’Brien / Associated Press

PROVIDENCE, R.I. — Seven years ago, a computer beat two human quizmasters on a “Jeopardy” challenge. Ever since, the tech industry has been training its machines to make them even better at amassing knowledge and answering questions.

And it’s worked, at least up to a point. Just don’t expect artificial intelligence to spit out a literary analysis of Leo Tolstoy’s “War and Peace” any time soon.

Research teams at Microsoft and Chinese tech company Alibaba reached what they described as a milestone earlier this month when their AI systems outperformed the estimated human score on a reading comprehension test. It was the latest demonstration of rapid advances that have improved search engines and voice assistants and that are finding broader applications in health care and other fields.

The answers they got wrong — and the test itself — also highlight the limitations of computer intelligence and the difficulty of comparing it directly to human intelligence.

Error! error!

“We are still a long way from computers being able to read and comprehend general text in the same way that humans can,” said Kevin Scott, Microsoft’s chief technology officer, in a LinkedIn post that also commended the achievement by the company’s Beijing-based researchers.

The test developed at Stanford University demonstrated that, in at least some circumstances, computers can beat humans at quickly “reading” hundreds of Wikipedia entries and coming up with accurate answers to questions about Genghis Khan’s reign or the Apollo space program.

The computers, however, also made mistakes that many people wouldn’t have.

Microsoft, for instance, fumbled an easy football question about which member of the NFL’s Carolina Panthers got the most interceptions in the 2015 season (the correct answer was Kurt Coleman, not Josh Norman). A person’s careful reading of the Wikipedia passage would have discovered the right answer, but the computer tripped up on the word “most” and didn’t understand that seven is bigger than four.

“You need some very simple reasoning here, but the machine cannot get it,” said Jianfeng Gao, of Microsoft’s AI research division.

Human vs. machine

It’s not uncommon for machine-learning competitions to pit the cognitive abilities of computers against humans. Machines first bested people in an image-recognition competition in 2015 and a speech recognition competition last year, although they’re still easily tricked. Computers have also vanquished humans at chess, Pac-Man and the strategy game Go.

And since IBM’s “Jeopardy” victory in 2011, the tech industry has shifted its efforts to data-intensive methods that seek to not just find factoids, but better comprehend the meaning of multi-sentence passages.

Like the other tests, the Stanford Question Answering Dataset, nicknamed Squad, attracted a rivalry among research institutions and tech firms — with Google, Facebook, Tencent, Samsung and Salesforce also giving it a try.

“Academics love competitions,” said Pranav Rajpurkar, the Stanford doctoral student who helped develop the test. “All these companies and institutions are trying to establish themselves as the leader in AI.”

Limits of understanding

The tech industry’s collection and digitization of huge troves of data, combined with new sets of algorithms and more powerful computing, has helped inject new energy into a machine-learning field that’s been around for more than half a century. But computers are still “far off” from truly understanding what they’re reading, said Michael Littman, a Brown University computer science professor who has tasked computers to solve crossword puzzles.

Computers are getting better at the statistical intuition that allows them to scan text and find what seems relevant, but they still struggle with the logical reasoning that comes naturally to people. (And they are often hopeless when it comes to deciphering the subtle wink-and-nod trickery of a clever puzzle.) Many of the common ways of measuring artificial intelligence are in some ways teaching to the test, Littman said.

“It strikes me for the kind of problem that they’re solving that it’s not possible to do better than people, because people are defining what’s correct,” Littman said of the Stanford benchmark. “The impressive thing here is they met human performance, not that they’ve exceeded it.”

Talk to us

> Give us your news tips.

> Send us a letter to the editor.

> More Herald contact information.

More in Business

Lily Lamoureux stacks Weebly Funko toys in preparation for Funko Friday at Funko Field in Everett on July 12, 2019.  Kevin Clark / The Herald)
Everett-based Funko ousts its CEO after 14 months

The company, known for its toy figures based on pop culture, named Michael Lunsford as its interim CEO.

The livery on a Boeing plane. (Christopher Pike / Bloomberg)
Former Lockheed Martin CFO joins Boeing as top financial officer

Boeing’s Chief Financial Officer is being replaced by a former CFO at… Continue reading

Izaac Escalante-Alvarez unpacks a new milling machine at the new Boeing machinists union’s apprentice training center on Friday, June 6, 2025 in Everett, Washington. (Olivia Vanni / The Herald)
Boeing Machinists union training center opens in Everett

The new center aims to give workers an inside track at Boeing jobs.

Some SnoCo stores see shortages after cyberattack on grocery supplier

Some stores, such as Whole Foods and US Foods CHEF’STORE, informed customers that some items may be temporarily unavailable.

People take photos and videos as the first Frontier Arlines flight arrives at Paine Field Airport under a water cannon salute on Monday, June 2, 2025 in Everett, Washington. (Olivia Vanni / The Herald)
Water cannons salute Frontier on its first day at Paine Field

Frontier Airlines joins Alaska Airlines in offering service Snohomish County passengers.

Amit B. Singh, president of Edmonds Community College. 201008
Edmonds College and schools continue diversity programs

Educational diversity programs are alive and well in Snohomish County.

A standard jet fuel, left, burns with extensive smoke output while a 50 percent SAF drop-in jet fuel, right, puts off less smoke during a demonstration of the difference in fuel emissions on Tuesday, March 28, 2023 in Everett, Washington. (Olivia Vanni / The Herald)
Sustainable aviation fuel center gets funding boost

A planned research and development center focused on sustainable aviation… Continue reading

Helion's 6th fusion prototype, Trenta, on display on Tuesday, July 9, 2024 in Everett, Washington. (Olivia Vanni / The Herald)
Helion celebrates smoother path to fusion energy site approval

Helion CEO applauds legislation signed by Gov. Bob Ferguson expected to streamline site selection process.

Britney Barber, owner of Everett Improv. Barber performs a shows based on cuttings from The Everett Herald. Photographed in Everett, Washington on May 16, 2022. (Kevin Clark / The Herald)
August 9 will be the last comedy show at Everett Improv

Everett improv club closing after six years in business.

Pharmacist John Sontra and other employees work on calling customers to get their prescriptions transferred to other stores from the Bartell Drugs Pharmacy on Hoyt Avenue on Wednesday, July 2, 2025 in Everett, Washington. (Olivia Vanni / The Herald)
Bartell Drugs location shutters doors in Everett

John Sontra, a pharmacist at the Hoyt Avenue address for 46 years, said Monday’s closure was emotional.

Support local journalism

If you value local news, make a gift now to support the trusted journalism you get in The Daily Herald. Donations processed in this system are not tax deductible.