CBS 58-- Owners of Cedarburg restaurant at odds with the community are disputing allegations they are violating the city’s liquor license laws.

We first told you about the controversy surrounding August Weber Haus in February. After sexual assault allegations against the owner’s son, who worked as a bartender, surfaced, the restaurant stopped keeping regular hours and serving food. Citizens filed a complaint saying it violated a city ordinance.

After protests and a formal complaint to the city about their business practices, the owners' of August Weber Haus, Steven and Diane Banas, are fighting back.

“We didn't want to leave all of these unfounded challenges, out there,” said Tim Algiers, the Banas’ attorney. “We wanted to respond.”

Algiers, on behalf of Steve and Diane Banas, sent the city of Cedarburg a lengthy response to claims the restaurant is not open the required number of hours.

“We never thought that my clients were out of compliance,” Algiers said.

The original complaint was filed by the Committee for a Safe Cedarburg, a group of citizens. They say August Weber Haus keeps irregular hours, takes pictures of customers, and even calls police when people go in to the restaurant.

But the Banas' argue they only take photos of members of the Safe Cedarburg group and called police because they felt Intimidated and harassed.

“Basically there's a group out there that's driving customers away,” Algiers said.

The Banas' did make some changes in response to the complaint. Their Facebook page used to say they \"serve lunch and dinner.\" It now says they serve \"drinks.\"

“Serving food is not a requirement of a liquor license but the attacks were there,” Algiers said. “And what they are doing is trying to get food back on the menu.”

August Weber Haus stopped being a full service restaurant about two years ago, when the Banas' son, Jacob, was accused drugging and sexually assaulting women while working as a bartender. The Banas' say they \"slowed their business\" in the wake of that investigation, which is ongoing but no charges have not been filed.

“Our biggest concern and the thing that really kicked off the entire campaign months ago and galvanized this community is peoples concern about their safety,” said Antonio Ramirez, a member of the Committee for a Safe Cedarburg.

And while the city sent a warning letter to the Banas' about their hours, the city also told the Committee for a Safe Cedarburg they would have to file another complaint before the issue could be brought to the common council. The common council could then revoke or suspend the liquor license. The safe Cedarburg group plans to continue monitoring the restaurant.

“There are a lot of members of our group that are concerned about the August Weber Haus' irregular hours and the fact that they are using one of the rare and valuable Class B liquor licenses,” Ramirez said.

But the Banas' don't have any plans to back down either.

“They love this business and you know, what? They love and cherish their reputation,” Algiers said.

Police say the investigation in to the sexual assault allegations is open and active, and the safe Cedarburg group says they want to know why it’s taking so long.

Meanwhile Jacob Banas has filed a defamation lawsuit against the women who accused him.