Everything we see is a mash-up of the brain's last 15 seconds of visual information

For the brain, this is no easy feat. On the one hand, the visual world alters continuously because of changes in light, viewpoint and other factors. On the other, our visual input constantly changes due to blinking and the fact that our eyes, head and body are frequently in motion.

To get an idea of the noisiness of this visual input, place a phone in front of your eyes and record a live video while you are walking around and looking at different things.

The jittery, messy result is exactly what your brain deals with in every moment of your visual experience. This can be seen also in the video below. The white circle on the right shows potential eye movements, and the blurry blob on the left reveals the jumpy visual input in every moment.

Yet, seeing never feels like work for us. Rather than perceiving the fluctuations and visual noise that a video might record, we perceive a consistently stable environment. So how does our brain create this illusion of stability? This process has fascinated scientists for centuries and it is one of the fundamental questions in vision science.

The time machine brain

In our latest research, we discovered a new mechanism that, among others, can explain this illusory stability. The brain automatically smoothes our visual input over time. Instead of analysing every single visual snapshot, we perceive in a given moment an average of what we saw in the past 15 seconds.

So, by pulling together objects to appear more similar to each other, our brain tricks us into perceiving a stable environment. Living in the past can explain why we do not notice subtle changes that occur over time.

In other words, the brain is like a time machine which keeps sending us back in time. It's like an app that consolidates our visual input every 15 seconds into one impression so that we can handle everyday life. If our brains were always updating in real time, the world would feel like a chaotic place with constant fluctuations in light, shadow and movement. We would feel like we were hallucinating all the time.

We created an illusion to illustrate how this stabilisation mechanism works. Looking at the video below, the face on the left side slowly ages for 30 seconds, and yet, it is very difficult to notice the full extent of the change in age. In fact, observers perceive the face as ageing more slowly than it actually is.

To test this illusion we recruited hundreds of participants and asked them to view close-ups of faces morphing chronologically in age in 30-second timelapse videos. When asked to tell the age of the face at the very end of the video, the participants almost consistently reported the age of the face that was presented 15 seconds before.

As we watch the video, we are continuously biased towards the past and so the brain constantly sends us back to the previous ten to 15 seconds (where the face was younger).

Instead of seeing the latest image in real time, humans actually see earlier versions because our brain's refresh time is about 15 seconds. So this illusion demonstrates that visual smoothing over time can help stabilise perception.

What the brain is essentially doing is procrastinating. It's too much work to constantly deal with every single snapshot it receives, so the brain sticks to the past because the past is a good predictor of the present. Basically we recycle information from the past because it's more efficient, faster and less work.

This idea which is also supported by other results of mechanisms within the brain that continuously bias our visual perception towards our past visual experience is known as continuity fields.

Our visual system sometimes sacrifices accuracy for the sake of a smooth visual experience of the world around us. This can explain why, for example, when watching a film we don't notice subtle changes that occur over time, such as the difference between actors and their stunt doubles.

Repercussions

There are positive and negative implications to our brain operating with this slight lag when processing our visual world. The delay is great for preventing us from feeling bombarded by visual input every day, but it can also risk life-or-death consequences when absolute precision is needed.

For example, radiologists examine hundreds of images in batches, seeing several related images one after the other. When looking at an X-ray, clinicians are typically asked to identify any abnormalities and then classify them.

During this visual search and recognition task, researchers have found that radiologists' decisions were based not only on the present image, but also on images they had previously seen, which could have grave consequences for patients.

Our visual system's sluggishness to update can make us blind to immediate changes because it grabs on to our first impression and pulls us toward the past. Ultimately, though, continuity fields promote our experience of a stable world. At the same time, it's important to remember that the judgements we make every day are not totally based on the present, but strongly depend on what we have seen in the past.