The researchers said piles of plastic on beaches create physical barriers and "deadly traps" for the crabs.
The study looked at strawberry hermit crab populations on two remote tropical island locations.
The scientists say more research is needed into how plastic pollution is affecting wildlife populations worldwide, especially on land.
"The potential for plastics on beaches and in other terrestrial ecosystems to cause harm is under-acknowledged," said co-author Alex Bond, a senior curator in the department of life sciences at the Natural History Museum in London.
He says plastic in the ocean entangles and is ingested by wildlife, but on land it acts as a trap and a barrier to species going about their daily lives.
The researchers surveyed sites on the Cocos (Keeling) Islands in the Indian Ocean and Henderson Island in the South Pacific. They say both locations are littered with millions of pieces of plastic.
They say crabs had crawled into plastic containers and were unable to get out, eventually dying. The containers had openings that allowed the crabs to enter, but were positioned with the opening facing an upward angle, so that the crabs would have difficulty crawling back out.
The researchers counted how many hazardous containers there were and how many contained trapped crabs, and extrapolated their findings to estimate totals for the islands.
"These results are shocking but perhaps not surprising," said lead researcher Jennifer Lavers from the institute for marine and Antarctic studies at the University of Tasmania in Australia.
"It is inevitable that these creatures will interact with and be affected by plastic pollution," she said.
The problem is worsened by the fact that hermit crabs don't have a shell of their own. As they grow, they need to move into larger shells. When one crab dies, it emits a smell that tells another crab a new shell is available.
Meaning, "the very mechanism that evolved to ensure hermit crabs could replace their shells, has resulted in a lethal lure," according to the paper.
In one container, researchers found 526 hermit crabs. They also found containers with both dead and living crabs, the latter presumably drawn in by the former.
The authors say hermit crabs play an important role in the ecosystem. They fertilise and aerate soil, and disperse seeds. They also play a role in tourism - an important source of employment on the islands - by giving visitors a chance to observe native wildlife.
Though the study was conducted on remote islands, Mr Bond says plastic pollution is global, and that this is likely to be an issue anywhere hermit crabs live alongside debris.
