A depth map is an image where each pixel stores a distance value instead of a color, representing how far every point in a scene is from the camera. In its simplest form, it looks like a grayscale photo: bright pixels represent objects at one distance extreme and dark pixels represent the other. This single, flat image encodes a full layer of 3D spatial information that cameras, phones, robots, and game engines all rely on.
How a Depth Map Represents Distance
A standard photograph records color at each pixel. A depth map replaces that color with a number representing distance from the sensor to whatever surface that pixel “sees.” When visualized, these distances are typically shown as a gradient from black to white. A common convention maps white to the closest objects and black to the farthest, though some systems reverse this.
The precision of those distance values depends on how the data is stored. A basic 8-bit grayscale image can only distinguish 256 levels of depth, which is enough for a rough visualization but too coarse for engineering work. Professional pipelines store depth in high-dynamic-range formats like OpenEXR, which supports 16-bit, 32-bit, and 64-bit floating-point values per pixel. That extra precision lets software distinguish surfaces separated by fractions of a millimeter.
Hardware That Captures Depth
Several sensor technologies produce depth maps directly, each with different strengths.
Time-of-Flight (ToF) sensors emit pulses of infrared light and measure how long each pulse takes to bounce back. The round-trip time at every pixel is converted into a distance. ToF cameras are compact enough to fit inside smartphones and tablets. Their raw accuracy is in the low-millimeter range: one study measured a root mean square error of about 4.4 mm from a ToF sensor, which dropped to roughly 3.6 mm after correction with a high-resolution color camera. That’s precise enough for augmented reality but not for industrial metrology.
LiDAR works on the same principle at a larger scale. A laser fires thousands of pulses per second, measuring the two-way travel time of light to map terrain, buildings, or vegetation across hundreds of meters. The result is a dense set of distance measurements that can be organized into a depth map or a 3D point cloud. LiDAR is standard equipment for land surveying, forestry mapping, and the perception systems in self-driving cars.
Structured light sensors project a known pattern of dots or lines onto a scene and watch how that pattern deforms across surfaces. A flat wall leaves the pattern unchanged; a curved face warps it. The sensor reads those distortions to calculate depth at each pixel. This is the technology behind face-unlock systems and some 3D scanners.
Depth From Stereo Cameras
You don’t always need a special sensor. Two ordinary cameras mounted side by side can produce a depth map through the same principle your eyes use: triangulation. Each camera captures the scene from a slightly different angle, and software matches corresponding points between the two images. The horizontal shift between matching points is called disparity.
The math is straightforward. Depth at any pixel is inversely proportional to its disparity. If you know the distance between the two cameras and the focal length of the lenses (both fixed, known values), you can convert every disparity measurement into a real-world distance. Objects close to the cameras shift a lot between views and have high disparity; distant objects barely shift at all. OpenCV, a widely used computer vision library, has built-in tools that compute a full disparity-based depth map from a pair of stereo images.
AI-Generated Depth From a Single Photo
One of the more remarkable recent developments is monocular depth estimation: predicting a depth map from a single, ordinary photograph. There’s no second camera, no infrared sensor, just one flat image. Neural networks trained on millions of photos with known depth learn to read visual cues like object size, texture gradients, occlusion, and perspective lines to estimate how far away each pixel is.
These models typically use an architecture with two parts. One network estimates the structure of the scene (which surfaces are closer, which are farther), while the other figures out the camera’s position and orientation. Together they produce a plausible depth map. The results aren’t as metrically accurate as a LiDAR scan, but they’re useful for visual effects, content creation, and any situation where dedicated depth hardware isn’t available.
Depth Buffers in 3D Graphics
If you’ve played a video game or watched a 3D-rendered movie, depth maps were involved behind the scenes. Game engines and rendering software maintain a depth buffer (also called a Z-buffer) for every frame. As the GPU draws each triangle in a scene, it records the distance of that surface from the virtual camera at every pixel. When two surfaces overlap on screen, the depth buffer determines which one is in front and which is hidden.
GPU depth buffers don’t store distance linearly, which surprises a lot of people encountering them for the first time. Instead, the stored value is proportional to the reciprocal of the actual distance (1/z rather than z). This fits naturally into the math of perspective projection and, conveniently, varies linearly across the screen. The tradeoff is that precision is concentrated near the camera and thins out toward the horizon. Modern rendering techniques combat this by using a floating-point depth buffer with a reversed mapping, where the near plane is set to 1 and the far plane to 0. This arrangement counteracts the nonlinearity and dramatically reduces depth-sorting errors in distant geometry.
Post-processing effects like depth of field blur, fog, and ambient occlusion all sample the depth buffer to figure out how far each pixel is from the camera, then apply their effect accordingly.
Everyday Uses You’ve Already Seen
Portrait mode on smartphones is probably the most familiar application of depth maps. When you take a portrait photo, the phone generates a depth map to figure out which pixels belong to you (the foreground) and which belong to the background. The software then creates a binary mask separating the two regions. The background pixels get blurred with a filter while the foreground stays sharp. The two layers are combined into the final image, mimicking the shallow depth of field you’d get from a large-aperture camera lens. The quality of the blur depends almost entirely on how accurate the depth map is, which is why portrait mode sometimes struggles with fine details like stray hairs or transparent objects.
Robotics, Navigation, and Autonomous Vehicles
Robots and self-driving vehicles use depth maps to understand the physical space around them in real time. A stereo camera pair or a LiDAR sensor generates depth data for every frame, and onboard software analyzes that data to detect obstacles, measure distances, and plan safe paths. In one research system, a stereo depth map and an AI-based scene understanding model worked together: the depth map provided distance to every visible surface, the AI labeled what those surfaces were (road, pedestrian, curb), and a control algorithm combined both outputs to steer a two-wheeled robot around obstacles.
This same pairing of depth and recognition powers augmented reality headsets, warehouse robots, drone navigation, and the spatial mapping that lets AR apps place virtual furniture in your living room.
Building 3D Models From Depth Maps
A single depth map gives you the shape of a scene from one viewpoint. Combine depth maps from multiple viewpoints and you can reconstruct a full 3D model. This is a core step in photogrammetry, the process of turning photographs into 3D geometry. Each depth map contributes a set of 3D points (one per pixel, positioned using the depth value and the camera’s known location). Merged together, these points form a dense point cloud, which can then be converted into a solid mesh with a surface you can texture and render.
Depth maps are especially valuable in this pipeline because they resolve a fundamental problem: a 2D image throws away the depth dimension during capture, and reconstructing it from color alone is ambiguous. Feeding explicit depth data into the pipeline provides geometric cues that help algorithms produce more uniform point distributions and preserve fine surface details. Recent frameworks combine depth maps from pre-trained estimation networks with initial point clouds as geometric scaffolding, generating high-fidelity 3D reconstructions even from a single input image.