Zimmer was fired as the company's executive chairman last week. On Monday he submitted a letter resigning from the board.
Zimmer says in the letter that it's clear from his firing that the board is determined to avoid addressing his growing concerns with recent board decisions and the company's direction.
Zimmer, 64, built Men's Wearhouse Inc. from one small Texas store using a cigar box as a cash register to one of North America's largest men's clothing sellers with 1,143 locations.
He also became one of TV's most recognizable pitchmen, with the slogan "You're going to like the way you look. I guarantee it."
Men's Wearhouse fell 62 cents to $35.20 in afternoon trading.
MORE HBJ HEADLINES
Playfulness, fun showcased at Zinc Oil sands restart plans thwarted as wildfire threat returns 9:44 a.m. Nokia plans return to mobile market with cellphones, tablets 9:41 a.m. Overtime pay may become reality for more U.S. workers 9:38 a.m. NY comptroller: Best Buy, Nordstrom to pursue renewables 9:42 a.m. Briefs: Peoples Bank announces new real estate team