Paranoia may be a response to uncertainty in general, and social interactions can be particularly complex and difficult to predict, Reed added.
Paranoia is a key symptom of serious mental illness, marked by the belief that other people have malicious intentions. But it also manifests in varying degrees in the general population.
"When our world changes unexpectedly, we want to blame that volatility on somebody, to make sense of it, and perhaps neutralize it," study senior author Philip Corlett from the Yale University (US), said in a paper published in the journal eLife.
"Historically in times of upheaval, such as the great fire of ancient Rome in 64 C.E. or the 9/11 terrorist attacks, paranoia and conspiratorial thinking increased," Corlett added.
The prevailing theory is that paranoia stems from an inability to accurately assess social threats.
But the research team hypothesized that paranoia is instead rooted in a more basic learning mechanism that is triggered by uncertainty, even in the absence of social threat.
"We think of the brain as a prediction machine; unexpected change, whether social or not, may constitute a type of threat -- it limits the brain's ability to make predictions," said study researcher Erin Reed.
"Paranoia may be a response to uncertainty in general, and social interactions can be particularly complex and difficult to predict," Reed added.
In a series of experiments, the research team asked participants with different degrees of paranoia to play a card game in which the best choices for success were changed secretly.
People with little or no paranoia were slow to assume that the best choice had changed.
However, those with paranoia expected even more volatility in the game. They changed their choices capriciously -- even after a win.
The researchers then increased the levels of uncertainty by changing the chances of winning halfway through the game without telling the participants.
This sudden change made even the low-paranoia participants behave like those with paranoia, learning less from the consequences of their choices.
In a related experiment, the research team trained rats, a relatively asocial species, to complete a similar task where the best choices of success changed.
Rats who were administered methamphetamine -- known to induce paranoia in humans -- behaved just like paranoid humans.
They, too, anticipated high volatility and relied more on their expectations than learning from the task.
The research team then used a mathematical model to compare choices made by rats and humans while performing these similar tasks.
The results from the rats that received methamphetamine resembled those of humans with paranoia, researchers found.