When it’s stats vs. pundits, bet on the math

WASHINGTON — Forget political pundits, gut instincts and psychics. The mightier-than-ever silicon chip seems to reveal the future.

In just two weeks this fall, computer models displayed an impressive prediction prowess.

It started when the first computer model alerted meteorologists to the pre-Halloween disaster headed for the Northeast from a bunch of clouds in the Caribbean. Nearly a week later, that weather system became Hurricane Sandy and grew into a superstorm after taking a once-in-a-century sharp turn into New Jersey.

Then, statistician and blogger Nate Silver correctly forecast on his beat-up laptop how all 50 states would vote for president. He even predicted a tie in Florida and projected it eventually would tip to President Barack Obama, which is the equivalent of predicting a coin landing on its side. He did it by taking polling data, weighing it for past accuracy and running 40,000 computer simulations at a time.

He then gave his forecast in terms of percentages, saying that Obama had a 91 percent chance of being re-elected.

In the case of Sandy, lives were at stake. With the election, reputations were on the line and some pundits were dismissive of the computer modeling. Bets were made. Challenges issued.

The math majors came out on top thanks to better and more accessible data and rapidly increasing computer power.

“In this particular case, rationality scored a win,” said Princeton University neuroscientist Sam Wang, who since 2004 has been using mathematical formulas and polling data to predict elections for the Princeton Election Consortium. Wang predicted a “100 percent chance” of an Obama victory, but missed Florida, giving it to Republican Mitt Romney. For the record, Wang notes that he beat Silver at accurate Senate race predictions.

Computers soon should be able to tell health officials where the next food poisoning outbreak will spread, a U.S. government lab predicts.

Tom Mitchell, head of the Machine Learning Department at Carnegie Mellon University, called computer model predictions based on historical evidence “one of more positive trends we’re going to see this century. We’re just beginning.”

Take a look at baseball, where Silver got his start as a stats geek. The Oakland A’s, a team that famously uses computer statistics in selecting players, surprised everyone by getting into the playoffs despite one of the lowest payrolls in baseball.

Computer modeling tells the government what happens when a nuclear bomb explodes, helped Goodyear make a better tire and helped the makers of Pringles figure out how to keep the potato crisps from breaking in the can, said Bill Tang, program director for the Princeton Plasma Physics Laboratory simulation program.

Every time you swipe a credit card, a computer is using predictive models based on past evidence to determine if it’s really you or if it is fraud, Mitchell added.

For about 40 years, climate scientists have used computer models to predict what global warming will look like with dead-on accuracy, said climate computer modeler Andrew Weaver of the University of Victoria in British Columbia.

For computer models to make predictions, three things are needed: computer power, mathematical formulas designed to mirror real world cause-and-effects and current conditions converted into numbers that can be used in formulas.

Experts input the data of current conditions into the formulas that say if X and Y happen, then it will produce Z. Then the computers run those what-if simulations over and over again, with slight variations changing the end results. These scenarios are run tens of thousands of times, giving a whole range of outcomes.

The key is seeing what happens most often and why. It’s not a dead-on prediction, but breaks down the future into probabilities.

“It’s essentially solving equations that are too extensive to solve with pencil and paper,” Weaver explained.

It all comes down to collecting data, crunching it and spitting out probabilities. It’s evidence turned into numbers. It’s math.

Experts believe it’s the future.

Silver said what he did with the election was nothing compared with what meteorologists did with Sandy, which was a matter of “real life-and-death consequences.”

The National Weather Service forecast an extremely rare due-west turn by the storm into southern New Jersey, he said. “It’s astounding. That’s a huge win for computer modeling.”

Silver’s bold predictions that Obama would win upset some political pundits who predicted a Romney victory, based on what they perceived as momentum, the enthusiasm of crowds, gut instinct and partisanship.

But Silver was right, besting his 2008 record of getting 49 of 50 states right for president.

“This is a victory for the stuff (computer modeling) in politics,” he said Thursday in a telephone interview. “It doesn’t mean we’re going to solve world peace with a computer. It doesn’t mean we’re going to be able predict earthquakes … but we can chip away at the margins.”

One of the next fields Silver said he’d like to get into is education because he feels that all the data being generated “is not being used in the best way.”

“I hope that people focus not on me personally, but what I’m trying to do,” Silver said.

What he and his colleagues are trying to do is take a chaotic world and make sense of it, turn events into equations to be solved.

More than anything statistics are tools for understanding, like a wrench for an auto mechanic, said Bill James, the godfather of modern baseball statistics and a colleague of Silver’s.

James said in an email that contemplating what will happen in the future is something that “we all do every day, without really thinking about it. It is a necessary and relevant process. Thus, it is something that is worthy of our best analytical efforts.”

———

On the Net

Nate Silver’s election blog: http://fivethirtyeight.blogs.nytimes.com

Princeton Election Consortium: http://election.princeton.edu

National Hurricane Center’s computer models: http://www.nhc.noaa.gov/modelsummary.shtml

Talk to us

> Give us your news tips.

> Send us a letter to the editor.

> More Herald contact information.

More in Local News

A firefighter stands in silence before a panel bearing the names of L. John Regelbrugge and Kris Regelbrugge during the ten-year remembrance of the Oso landslide on Friday, March 22, 2024, at the Oso Landslide Memorial in Oso, Washington. (Ryan Berry / The Herald)
‘Flood of emotions’ as Oso Landslide Memorial opens on 10th anniversary

Friends, family and first responders held a moment of silence at 10:37 a.m. at the new 2-acre memorial off Highway 530.

Julie Petersen poses for a photo with images of her sister Christina Jefferds and Jefferds’ grand daughter Sanoah Violet Huestis next to a memorial for Sanoah at her home on March 20, 2024 in Arlington, Washington. Peterson wears her sister’s favorite color and one of her bangles. (Annie Barker / The Herald)
‘It just all came down’: An oral history of the Oso mudslide

Ten years later, The Daily Herald spoke with dozens of people — first responders, family, survivors — touched by the deadliest slide in U.S. history.

Victims of the Oso mudslide on March 22, 2014. (Courtesy photos)
Remembering the 43 lives lost in the Oso mudslide

The slide wiped out a neighborhood along Highway 530 in 2014. “Even though you feel like you’re alone in your grief, you’re really not.”

Director Lucia Schmit, right, and Deputy Director Dara Salmon inside the Snohomish County Department of Emergency Management on Friday, March 8, 2024, in Everett, Washington. (Ryan Berry / The Herald)
How Oso slide changed local emergency response ‘on virtually every level’

“In a decade, we have just really, really advanced,” through hard-earned lessons applied to the pandemic, floods and opioids.

Ron and Gail Thompson at their home on Monday, March 4, 2024 in Oso, Washington. (Olivia Vanni / The Herald)
In shadow of scarred Oso hillside, mudslide’s wounds still feel fresh

Locals reflected on living with grief and finding meaning in the wake of a catastrophe “nothing like you can ever imagine” in 2014.

Kelli Littlejohn, who was 11 when her older sister Melissa Lee was murdered, speaks to a group of investigators and deputies to thank them for bringing closure to her family after over 30 years on Thursday, March 28, 2024, at Snohomish County Superior Court in Everett, Washington. (Ryan Berry / The Herald)
‘She can rest in peace’: Jury convicts Bothell man in 1993 killing

Even after police arrested Alan Dean in 2020, it was unclear if he would stand trial. He was convicted Thursday in the murder of Melissa Lee, 15.

Ariel Garcia, 4, was last seen Wednesday morning in an apartment in the 4800 block of Vesper Dr. (Photo provided by Everett Police)
Everett police searching for missing child, 4

Ariel Garcia was last seen Wednesday at an apartment in the 4800 block of Vesper Drive. The child was missing under “suspicious circumstances.”

The rezoned property, seen here from the Hillside Vista luxury development, is surrounded on two sides by modern neighborhoods Monday, March 25, 2024, in Lake Stevens, Washington. (Ryan Berry / The Herald)
Despite petition, Lake Stevens OKs rezone for new 96-home development

The change faced resistance from some residents, who worried about the effects of more density in the neighborhood.

Rep. Suzan DelBene, left, introduces Xichitl Torres Small, center, Undersecretary for Rural Development with the U.S. Department of Agriculture during a talk at Thomas Family Farms on Monday, April 3, 2023, in Snohomish, Washington. (Olivia Vanni / The Herald)
Under new federal program, Washingtonians can file taxes for free

At a press conference Wednesday, U.S. Rep. Suzan DelBene called the Direct File program safe, easy and secure.

Former Snohomish County sheriff’s deputy Jeremie Zeller appears in court for sentencing on multiple counts of misdemeanor theft Wednesday, March 27, 2024, at Snohomish County Superior Court in Everett, Washington. (Ryan Berry / The Herald)
Ex-sheriff’s deputy sentenced to 1 week of jail time for hardware theft

Jeremie Zeller, 47, stole merchandise from Home Depot in south Everett, where he worked overtime as a security guard.

Everett
11 months later, Lake Stevens man charged in fatal Casino Road shooting

Malik Fulson is accused of shooting Joseph Haderlie to death in the parking lot at the Crystal Springs Apartments last April.

T.J. Peters testifies during the murder trial of Alan Dean at the Snohomish County Courthouse on Tuesday, March 26, 2024 in Everett, Washington. (Olivia Vanni / The Herald)
Bothell cold case trial now in jury’s hands

In court this week, the ex-boyfriend of Melissa Lee denied any role in her death. The defendant, Alan Dean, didn’t testify.

Support local journalism

If you value local news, make a gift now to support the trusted journalism you get in The Daily Herald. Donations processed in this system are not tax deductible.