HENDERSONVILLE, N.C. — When Representative Madison Cawthorn’s name comes up in this city of 14,000, where he was born and raised and it is not difficult to bump into someone who knew him from his home-schooling days, there tends to be a visceral reaction.

There are sighs from Republicans who elected him to his first term in November 2020 and met his meteoric rise in Washington with the praise and excitement reserved for a hometown hero — only to be disappointed by his behavior and bad press ever since.

There are groans and looks of utter disgust from people with Democratic and independent leanings — some of whom have chosen to cast a ballot in a Republican primary for the first time in hopes of removing him from office.

And there are eye-rolls and shrugs from his die-hard supporters, “America First” conservatives after the fashion of Donald J. Trump, who chalk up Mr. Cawthorn’s controversies to youthful indiscretion and instead reserve their opprobrium for the liberal media, Democrats, his Republican opponents and political groups with deep pockets.

“I don’t care what he’s done,” said Moiena Gilbert, 77, a retired certified nursing assistant who pulled up in an old Ford pickup to cast an early vote this week at Henderson County’s Board of Elections. “I am going to vote for the man.”

What there is not a lot of is indifference. In this southwestern corner of the state, a largely working-class and Republican stronghold set against the Blue Ridge and Smoky Mountains, it seems as if nearly everyone has made up his or her mind on the young firebrand once seen as the future of the Republican Party.

In interviews with more than 30 voters in Mr. Cawthorn’s 11th Congressional District, including nearly two dozen registered Republicans, it was clear that his support had weakened, even among hard-right Trump followers who said Mr. Cawthorn’s immaturity and lack of focus on his constituents had led them to disregard his endorsement by the former president and give one of his rivals their vote.

Mr. Cawthorn needs to garner only 30 percent of the vote on Tuesday to avoid a runoff in a crowded field split among seven other challengers. They are led by Chuck Edwards, a state senator who has the endorsements of most members of the Legislature from his district, and Michele Woodhouse, the elected Republican chair of Mr. Cawthorn’s district who once was among his staunch supporters.

Whether Mr. Cawthorn can dodge a runoff has been a constant source of debate in his hometown among friends, co-workers and in Christian circles.

“I think there is a lot of support for Madison — they just may be afraid to tell you,” said one Baptist deacon leaving the Bethany Bible Church after a Wednesday night Bible study.

Chip Worrell, 62, a charter member of the same church and a woodworker who helped erect its building, disagreed.

“I don’t think he is going to be re-elected,” he said.

Mr. Cawthorn, 26, who was injured in a car crash at 18, has seldom been out of the headlines since making his first run for Congress in 2020, when it emerged that he had made up parts of his autobiography. He falsely claimed his injuries had kept him from attending the Naval Academy, but admitted in court that it had already rejected him. Young women at the conservative Christian college he attended before dropping out accused him of sexual harassment.

Elected in 2020 as the youngest member ever to serve in the House, he helped spread Mr. Trump’s stolen-election lies and aligned himself with other incendiary far-right representatives, Marjorie Taylor Greene of Georgia and Lauren Boebert of Colorado.

But his re-election campaign has been marred by a seemingly endless series of embarrassing reports — beginning when he claimed that people he “looked up to” in Washington had invited him to orgies and used cocaine. (The remark drew a scolding from the House Republican leader, Kevin McCarthy.)

The revelations ranged from traffic violations, like driving with a revoked license, to two incidents in which he brought a loaded gun to an airport. Politico published photos of Mr. Cawthorn in lingerie. The Washington Examiner reported his involvement in a cryptocurrency scheme and suggested it may have violated federal insider trading laws. And nude photos and videos have circulated showing him in sexually suggestive antics, in what appeared to be attempts to raise questions about Mr. Cawthorn’s sexuality.

Mr. Cawthorn’s campaign did not respond to requests for comment. Writing on Twitter, he told supporters that he and a friend had simply been joking around crassly.

“I told you there would be a drip drip campaign,” he wrote. “Blackmail won’t win. We will.”

Democrats have criticized some of the attacks for stirring homophobia. Supporters in Mr. Cawthorn’s district see the leaks as the work of his opponents or of G.O.P. leaders like Mr. McCarthy.

But a super PAC created to oust Mr. Cawthorn, which has held itself out as a clearinghouse of damaging information about him, said the tips it has received have largely come from Mr. Cawthorn’s former aides and supporters.

“From the very start, we have been focused on firing Cawthorn, but firing him in a way that was factual and honest,” said David Wheeler, a Democrat who co-founded the group, American Muckrakers Inc., with Mr. Cawthorn’s 2020 Democratic opponent, Moe Davis.

In Henderson, Transylvania and Haywood counties, many voters recalled how Mr. Cawthorn won the seat — replacing Mark Meadows, who became chief of staff in the Trump White House — by modeling himself after Mr. Trump.

Many compared his brashness to Mr. Trump’s and brushed away the photos of him partying or goofing off as the digressions of a young man. Some believed them to be fake.

“If I was a young kid with a cellphone, I wouldn’t have a job either,” said David Roberts, 33, an engineer and unaffiliated voter in Hendersonville who planned to cast a ballot for Mr. Cawthorn on Tuesday. “I am not voting for him to be my best friend or date my daughter.”

Less easily brushed away were Mr. Cawthorn’s attempts to bring guns through airport security and his traffic violations, which many saw as irresponsible considering the crash that left him in a wheelchair. “Disgrace,” “immature” and “embarrassment” were common refrains.

“He’s broken the law. He hasn’t really done anything for this district that I can think of,” said Scott Tekavec, 59, a maintenance technician who said he did not usually vote Republican but decided to cast a ballot for Mr. Edwards as an expression of his disdain for Mr. Cawthorn.

Perhaps the most frequently cited objections to Mr. Cawthorn, however, were his track record of missing important votes in Congress and reports that he had moved into a newly-drawn conservative district nearby before deciding to run for re-election to his seat in the 11th District.

“He isn’t doing his job,” Lynn Cagle, 47, a truck driver in Haywood County, said of Mr. Cawthorn as he left a senior center after voting for Mr. Edwards.

Mr. Cawthorn’s opponents lack his ability to draw attention, but they see an opening nonetheless. At a Hendersonville rally, Ms. Woodhouse presented herself as a true “America First” conservative and Mr. Cawthorn as unelectable.

And Rodd Honeycutt, a retired Army colonel, said he had voted for Mr. Cawthorn in 2020 but felt the need to challenge him this year over his lack of leadership.

“There is a trend line of missteps and indiscipline,” Mr. Honeycutt said, adding: “It’s really a distraction right now when we should be focused on kitchen-table issues like the cost of gas, or inflation, or what is going on with the war in Ukraine.”

At Bethany Bible Church, Christine Tuttle, 61, a bookkeeper, and her daughter, Lizzie, 20, said they remembered Mr. Cawthorn as respected, outgoing and popular among the home-school families.

They said their image of him was tainted when young women came forward with accusations that he had forcibly kissed them.

Mrs. Tuttle said she still voted for him in 2020. “He had so much promise,” she said.

She and her daughter said they would not be voting for Mr. Cawthorn this time. But they said they knew plenty of people who would.