Zimmer was fired as the company's executive chairman last week. On Monday he submitted a letter resigning from the board.
Zimmer says in the letter that it's clear from his firing that the board is determined to avoid addressing his growing concerns with recent board decisions and the company's direction.
Zimmer, 64, built Men's Wearhouse Inc. from one small Texas store using a cigar box as a cash register to one of North America's largest men's clothing sellers with 1,143 locations.
He also became one of TV's most recognizable pitchmen, with the slogan "You're going to like the way you look. I guarantee it."
Men's Wearhouse fell 62 cents to $35.20 in afternoon trading.
MORE HBJ HEADLINES
Payoff plan for college loans comes in several forms Putting Tubman on $20 bill doesn’t erase earning gaps More brands get serious about plus-size teen fashions Friendship not required, but workers must get along Treatment for double chins hits the market this fall Experts expect banks to start pruning their branches