WASHINGTON — A top lawmaker briefed on the investigation into a Secret Service prostitution scandal predicted more firings would follow the forced ouster of three agency employees.

“I wouldn’t be surprised if you saw more dismissals and more being forced out sooner rather than later,” Rep. Peter King, R-N.Y., said Thursday. King is being updated on the investigation by Secret Service Director Mark Sullivan.

The Secret Service is moving quickly to quell the scandal that erupted late last week, when at least some of 11 agency employees implicated in the incident brought prostitutes back to their hotel in Cartagena, Colombia, where they were setting up security for a visit by President Barack Obama.

So far, three people involved have lost their jobs. The service said Wednesday that one supervisor was allowed to retire, and another will be fired for cause. A third employee, who was not a supervisor, has resigned.

The two supervisors are in the agency’s uniformed division; one is a sergeant, according to a person familiar with Secret Service operations and refused to be identified because he was not authorized to discuss the matter.

The team under investigation includes members of the agency’s “jump teams,” which are sent to sites ahead of the president’s arrival to set up security. Others involved are on counter-assault and counter-sniper teams. The majority of those involved are believed to be based in the Washington area.

Eight other Secret Service employees remain on administrative leave and have had their top-secret clearance revoked.

Sullivan has offered the agents under investigation the opportunity to take a polygraph test, though the agents can refuse.

The scandal also involved about 10 military service members and as many as 20 women.

King said agency investigators in Colombia still have not been able to talk to the women who were brought back to the hotel. The investigators do, however, have the names, addresses and pictures of the women, said King, the chairman of the House Homeland Security Committee, which oversees the Secret Service.

In Washington and Colombia, separate U.S. government investigations are already under way. In addition to the Secret Service investigators in Colombia, King said he has assigned four congressional investigators to the probe. The House Committee on Oversight and Government Reform, led by Rep. Darrell Issa, R-Calif., sought details of the Secret Service investigation, including the disciplinary histories of the agents involved.

In a letter to the Secret Service director, Issa and Rep. Elijah Cummings of Maryland, the committee’s senior Democrat, said the agents “brought foreign nationals in contact with sensitive security information.” The lawmakers have demanded that Sullivan provide them by May 1 with detailed information about the incident, including a full timeline of the events that unfolded in Colombia and assurances that none of the women involved were under the age of 18.

King said Sullivan took employment action against “the three people he believes the case was clearest against.” The lawmaker said the agency was “reasonably confident” that drug use was not an issue with the three agents who have been forced out. But he said Secret Service investigators would continue to look into whether drugs played a role in the incident as they talk to the other eight agents involved.

Hotel workers told Secret Service investigators they found no drugs or drug paraphernalia in the rooms where the agents stayed, according to a person familiar with the investigation. The person was not authorized to discuss the probe publicly and spoke only on condition of anonymity.

Sen. Chuck Grassley, the senior Republican on the Senate Judiciary Committee, said news of the three agents leaving the Secret Service was a positive development.

“I’ve always said that if heads don’t roll, the culture in a federal agency will never change,” the Iowa lawmaker said in a statement.

The episode took a sharp political turn Wednesday when presumptive Republican presidential nominee Mitt Romney said he would fire the agents involved.

Romney told radio host Laura Ingraham that he’d “clean house” at the Secret Service.

“The right thing to do is to remove people who have violated the public trust and have put their play time and their personal interests ahead of the interests of the nation,” Romney said.

While Romney suggested to Ingraham that a leadership problem led to the scandal, he told a Columbus, Ohio, radio station earlier that he has confidence in Sullivan, the head of the agency.

At least 10 military personnel who were staying at the same hotel are also being investigated for misconduct. The troops are suspected of violating curfews set by their commanders.

Two U.S. military officials have said they include five Army Green Berets. One of the officials said the group also includes two Navy Explosive Ordinance Disposal technicians, two Marine dog handlers and an Air Force airman. The officials spoke on condition of anonymity because the investigation is still under way.

The Secret Service’s Office of Professional Responsibility, which handles that agency’s internal affairs, is investigating, and the Homeland Security Department’s inspector general also has been notified.

Sullivan, who this week has briefed lawmakers behind closed doors, said he has referred to the case to an independent government investigator.