The image that lands at the back of each eye is flat, like a photograph printed on a curved screen.1PubMed. Retinal curvature and geometry of image formation Yet what you experience is a world with depth, because the brain turns those flat images into 3D. It is solving a puzzle with missing pieces, so it leans on assumptions about how the world usually looks, which is also why it can sometimes be fooled.2PubMed. Perception viewed as an inverse problem
Why a Flat Image Is Not Enough on Its Own
Depth is lost the moment light hits the retina, and every sighted animal faces the same task of getting information about a 3D world from 2D images there.3PubMed Central. Stereopsis in animals: evolution, function and mechanisms A small nearby object and a large faraway one can cast the same-sized image. So the flat image alone can’t say which scene you’re looking at. Shape is ambiguous too: a slanted rectangle can cast the same image as a trapezoid seen face-on. In principle, endlessly many arrangements of objects could produce an identical retinal image, so the brain can’t simply read depth off the picture. It treats seeing as working backward from the image to the most likely scene.4PubMed. Perception viewed as an inverse problem
The brain fills the gap with built-in assumptions. It relies on prior expectations, such as that objects tend to be symmetrical, to pick one 3D interpretation.5ACO Center @ UCI. Solving an ill-posed inverse problem of three-dimensional (3D) vision That works because almost all natural objects are symmetrical or nearly so.6ACO Center @ UCI. Solving an ill-posed inverse problem of three-dimensional (3D) vision Other everyday assumptions include that objects don’t change size when they move, that the ground is mostly flat and below you, and that surfaces are solid and continuous. They’re tuned by experience of ordinary settings, so they can misfire in rare setups built to break them. A mistaken guess feels just as convincing as a correct one, since you only experience the finished interpretation, never the alternatives that were rejected.
How Two Eyes Create 3D Vision
Your eyes sit a few centimetres apart, so each sees a slightly different view. The brain compares views from two locations to work out depth, a method like triangulation.7PubMed Central. Stereopsis in animals: evolution, function and mechanisms This 3D vision from two eyes is called stereopsis, a word you may meet in eye exams.
You can try it yourself. Hold a finger up and close one eye, then the other. The finger seems to jump because each eye sees it from a different angle. That jump is the difference the brain uses to judge distance, and nearer objects jump more than distant ones. 3D movies use this: the glasses send a slightly different image to each eye, and the screen seems to have depth.
The difference between the two images is called disparity: the same object lands at slightly different positions on each retina. A point at the distance you are looking at lands on matching spots in both eyes, while nearer or farther points land on mismatched spots. The size and direction of the mismatch tell the brain whether something is nearer or farther than what you’re looking at, and by roughly how much. Before it can compare, the brain must work out which spot in one image matches which spot in the other, which is hard in busy scenes with repeating patterns.
Single cells in the first visual area of the brain respond to the difference between the eyes’ images, but they can’t by themselves create the sense of depth.8PubMed. Responses of primary visual cortical neurons to binocular disparity without depth perception Depth from two eyes comes from many cells working together.9PubMed. Responses of primary visual cortical neurons to binocular disparity without depth perception Regions toward the top of the brain are strongly engaged by 3D information and help build the shape of surfaces you see.10PubMed. The Human Brain in Depth: How We See in 3D
Two-eye depth is most precise within arm’s reach or a few metres, which suits pouring, threading a needle, or catching. Because your eyes are only a few centimetres apart, views of far objects are nearly identical, so the difference shrinks with distance. For mountains, the brain leans mostly on other clues, so the two-eye clue adds little there.
Depth Clues That Work With One Eye
Clues such as perspective, where parallel lines meet in the distance, and shading come from a single image. These picture-based clues affect how far people reach even when looking with one eye.11PubMed. Pictorial depth cues always influence reaching distance
You use several of these constantly:
- Relative size: when you know roughly how large cars or people are, a smaller image reads as farther away.
- Overlap: an object that partly blocks another is seen as nearer, and this needs no knowledge of size at all.
- Texture gradient: paving stones or grass blades look finer and more tightly packed with distance.
- Atmospheric haze: distant hills look paler and bluer because of the air between you and them.
- Height in the visual field: on level ground, objects higher up toward the horizon read as farther.
Movement adds another clue. When you move, near objects slide past faster than far ones, a strong clue to the layout of a scene.12PubMed Central. The neural basis of depth perception from motion parallax Out a car window, roadside posts zip by while distant hills barely move, and your brain reads that speed difference as distance.
The eye’s focusing effort can also contribute to judging distance.13PubMed Central. (Disparity-Driven) Accommodation Response Contributes to Perceived Depth The lens changes shape to focus and the brain can sense that effort, though it’s a weak clue, mainly useful up close. The brain doesn’t rely on any one clue. It combines them, trusting the more reliable ones in a given scene and the weaker ones less when they disagree.
That’s why closing one eye doesn’t flatten the world. You lose only the two-eye clue, while perspective, shading and motion remain. Threading a needle gets harder, but rooms still look deep.
Why Illusions and Pictures Fool Us
A hollow mask seen from inside looks like a normal outward face.14PubMed. The hollow-face illusion: object-specific knowledge, general assumptions or properties of the stimulus? The brain prefers to see surfaces bulging outward when the image is ambiguous.15PubMed. The hollow-face illusion: object-specific knowledge, general assumptions or properties of the stimulus? Lighting plays a part as well: a face lit from above that bulges out looks the same as a hollow one lit from below, and we expect light from above.16brainfacts.org. Depth Perception and the Hollow Face Illusion
A hollow face is a rare real-world shape, so the brain’s strong expectation about faces overrides the real depth information. The effect tends to be weaker when the mask is close, since a short viewing distance gives the two-eye clue more weight. Rotating mask displays online show it well. When the mask turns, the hollow side seems to move in the wrong direction, and even knowing it’s hollow, you still see it bulge.
The same light-from-above assumption makes a dent in a photograph look like a bump if you turn the picture upside down, since the shading now implies light from below. Moon craters in photos can flip into domes when the lighting is reversed.
Pictures are double. Looking at one, you see both the depth in the scene and the flat surface it’s on at once.17PubMed Central. From pictures to reality: modelling the phenomenology and psychophysics of 3D perception So paintings feel deep but never fully real.
Why 3D Screens and VR Can Tire Your Eyes
In real life your eyes turn inward and focus at the same distance together. When you look at something near, both adjustments happen automatically, and the brain is used to them agreeing. But most 3D screens show images on one fixed focal plane.18Results in Optics. Vergence-accommodation conflict in optical see-through display: review and prospect In a headset, a virtual object can seem to float close to you while the screen stays at the same optical distance, so your eyes turn for near but focus far. This is called the focus mismatch, or vergence-accommodation conflict.19PubMed Central. Effects of virtual target size, position, and parallax on vergence-accommodation conflict as estimated by actual gaze
It can make the two images harder to merge and cause discomfort and fatigue.20PubMed Central. Vergence–accommodation conflicts hinder visual performance and cause visual fatigue Objects may look doubled or blurred for a moment. People report tired or sore eyes, headaches, and blurry vision after prolonged use; these are usually signs of effort rather than eye damage.
Taking regular breaks from headsets and 3D films helps, as does avoiding long sessions with objects popping very close. Stop if you feel a headache or blurred vision. Looking at something far away, such as across a room or out a window, lets focus and eye direction relax together. Content where objects stay at a comfortable distance is easier than scenes with things lunging toward your face. If you wear glasses, wear them with the headset as intended, since uncorrected vision adds its own focusing effort. If symptoms are frequent or don’t go away after rest, an eye exam is reasonable, as an unrelated eye problem may be the cause.
Living Without Two-Eye Depth and Getting It Back
Two-eye depth vision switches on at around 4 months after birth, and its development depends heavily on visual experience.21PubMed Central. Early-onset binocularity in preterm infants reveals experience-dependent visual development in humans So abnormal two-eye vision in early childhood can badly disrupt it, with sensitivity lasting to at least about 4.6 years.22PubMed. The critical period for susceptibility of human stereopsis That’s why misaligned or lazy eyes should be checked early. Treatment works best in childhood, and a parent who notices a turned eye should book an exam.
Losing this signal doesn’t make the world flat. People without two-eye depth still see rooms, faces, and streets as three-dimensional, because perspective, shading, overlap and motion still work. People with misaligned eyes (strabismus) judge depth from perspective about as well as others.23PubMed Central. Unimpaired perception of relative depth from perspective cues in strabismus Many such people drive, play sports, and work normally, though fine nearby tasks like threading a needle can be harder. Some only find out when an eye test or 3D film shows it, since daily life never signalled anything was missing. The same goes for someone who loses the use of one eye later in life: the two-eye clue goes, but what the brain learned about the other clues stays.
Possible signs include 3D movies that look flat or uncomfortable, and trouble catching balls or pouring, though these have other causes too. An eye exam can check.
Recovery is sometimes possible. Some adults with poor 3D vision recovered it partly after thousands of practice trials.24PubMed Central. Recovery of stereopsis through perceptual learning in human adults with abnormal binocular vision Training also helped some adults with a lazy eye, or amblyopia, regain depth vision.25PubMed Central. Recovery of stereo acuity in adults with amblyopia The studies used repeated depth-judging tasks, often as practice games, in which people had to pick out which object was nearer. Improvement was partial and varied between people, not a full return to typical depth vision for everyone. Because the benefit depends on the cause, an eye doctor first needs to work out whether it comes from misaligned eyes, lazy eye, or another condition. And adult recovery doesn’t make early treatment unnecessary, since children’s brains respond far more readily.