(CNN) -- Rain was still coming down Sunday in Colorado, preventing aerial efforts to search for those missing in a devastating flood, authorities said.
The National Guard grounded all helicopters in the state.
"It's unlikely at this point that we'll be able to reach those who are stranded in the hard-to-reach areas," said Kim Kobel, a spokesperson for Boulder's Office of Emergency Management.
But rescuers continued their ground efforts, searching for what could be hundreds of people unaccounted for.
A tearful Larimer County sheriff told reporters that what he's seen, even in the most devastated areas, has restored his hope.
Sheriff Justin Smith visited areas "somewhat cut off from the rest of the world," he said.
The roads and homes might be gone, but Smith said "inch by inch, mile by mile, community by community, they are taking this stuff back."
Smith spoke of firefighters who pulled signs out of the mud and residents using their ATVs to rescue neighbors.
Now his family is beginning the slow work of cleaning up. But they won't do it alone. Koplen says his phone has been ringing non-stop with offers of food and assistance. Even the gawkers who come to see the storm damage are kind, he marveled.
But Koplen knows there's much to do. In places roads and bridges are just gone. Even getting out of his neighborhood is tricky.
Authorities acknowledge that such issues abound. Smith said he couldn't begin to estimate the scope of the damage. "I've known these areas for 25 years," he said "I don't recognize some of them."
He and another official cautioned that the death toll would almost certainly rise.
It may already be as high as six.
Previously, four deaths were blamed on the flooding and a fifth person was presumed dead.
On Sunday, authorities announced another resident presumed dead -- an 80-year-old woman who suffered from injuries and was unable to leave her home.
Another 482 people remain unaccounted for, but authorities said that number could go down throughout the day.
Residents are still keeping a wary eye on the sky, as more rain is expected -- and could be enough to halt rescue efforts.
"We're going to be in for some steady rain over the next 12 hours," said Kobel. It shouldn't total more than 1 to 2 inches though. "So that's the good news."
Still, authorities worry that any additional water on ground that's already soaked by up to 15 inches of rain will cause more flooding and dislodge mud and debris.
In recent days, rescues in hard to reach areas have taken extraordinary measures. On Saturday a Chinook helicopter piloted by the Colorado National Guard picked up 78 children who got stranded while on a field trip.
Smith spoke of "hearty people" who didn't wait for officials to reach them but who are "finding roads out." He said he'd heard reports of people hiking out of canyons.
The four confirmed deaths included a man and a woman, both 19, who were swept away after leaving their car Thursday in Boulder County. Authorities said the woman left the car first, and the man jumped out to try to save her. Authorities recovered both bodies.
Another body was found in a collapsed home in Jamestown in the same county. Rescuers recovered another body on a roadway in Colorado Springs in El Paso County.
The other person presumed dead is a 60-year-old woman. Larimer County officials said witnesses saw her swept away by floodwater that demolished her home.
Neighbors tried unsuccessfully to rescue the woman, said Nick Christensen, executive officer of the sheriff's office.
Her body has not been recovered.
Officials plan recovery
Gov. John Hickenlooper said he spoke by phone with U.S. Transportation Secretary Anthony Foxx, who "was adamant that the $5 million that was released Friday was just the beginning" of federal assistance.
"We're going to come back and rebuild better than it was before," the governor said.
Hickenlooper said experts from Vermont will arrive this week to share lessons about improved road-building learned in the wake of Hurricane Irene.
Damage worth millions
Boulder County alone will need an estimated $150 million to repair 100 to 150 miles of roadway and 20 to 30 bridges, county transportation director George Gerstle said. The repair bill will be "10 to 15 times our annual budget," he said.
A helicopter surveillance mission Saturday carrying Hickenlooper and members of Colorado's congressional delegation was diverted twice to pick up people waving to be rescued.
After the officials' delayed arrival at a Boulder airport, U.S. Sen. Mark Udall promised a bipartisan push in Congress for federal aid for flood recovery.
President Barack Obama signed a major disaster declaration for Colorado on Sunday and ordered federal aid to supplement state and local recovery efforts in Boulder County.
Already some are doing their own math. Barb Vacek doesn't know the dollar amount but she's taken stock of the emotional cost.
Her family is exhausted, their home and many of their mementos, gone. "I did lose my family slides from my parents who are deceased," she said. "I was the person entrusted with them, so my childhood is wiped out."
Right now, she says, her guilt outweighs her grief.