Ray casting is a technique where you shoot an imaginary line (a “ray”) from a point in a specific direction and figure out what it hits. It’s one of the most fundamental operations in computer graphics, used to render 3D scenes, detect collisions in video games, select objects in virtual reality, and even visualize medical scans. The core idea is simple: define a starting point, define a direction, and find the first object the ray intersects.
How Ray Casting Works
A ray is just a half-infinite line. It starts at a point and extends in one direction forever. In practice, the computer defines the ray using two pieces of information: a starting position (like the location of a camera or a player’s eyes) and a direction vector (where the ray is pointing). From there, the algorithm checks every object in the scene to find which one the ray hits first.
The “first intersection” part matters. If a ray passes through multiple objects, only the closest one is relevant for most purposes. That’s the object you’d actually see from the camera’s perspective, or the object a player is pointing at. The algorithm sorts all intersections by distance from the starting point and picks the nearest one.
For simple shapes, finding intersections is pure algebra. A ray hitting a sphere, for example, reduces to solving a quadratic equation. You plug the ray’s direction and the sphere’s position into a formula, check whether the result has real solutions (meaning the ray does hit the sphere), and calculate the exact point of contact. More complex shapes require more involved math, but the principle stays the same: test the ray against each object’s geometry and find where they meet.
The Wolfenstein 3D Era
Ray casting became famous in the early 1990s through games like Wolfenstein 3D. The engine worked by casting one ray for every vertical column of pixels on the screen. Each ray shot outward from the player’s viewpoint, and when it hit a wall, the engine calculated how far away that wall was and drew the column at the appropriate height. Closer walls got taller columns, farther walls got shorter ones. This created a convincing 3D perspective from what was really a flat 2D map.
The technique used an algorithm called Digital Differential Analysis (DDA), which efficiently steps through a grid of squares to find which ones a line passes through. Instead of checking every tiny increment along the ray (which risks missing thin walls entirely), DDA jumps from grid square to grid square, guaranteeing it catches every wall the ray crosses. This made it fast enough to run on computers as old as the Intel 286.
The tradeoff was severe constraints on level design. All walls had to be the same height, perfectly vertical, and aligned to a grid. No stairs, no jumping, no height differences. Later games like Doom and Duke Nukem 3D used more advanced ray casting engines that allowed sloped surfaces, varying heights, textured floors and ceilings, and even transparent walls, but the core column-by-column approach remained.
Ray Casting vs. Ray Tracing
People often confuse ray casting with ray tracing, and the terms do overlap. Ray casting refers specifically to the act of sending a ray and finding what it hits. Ray tracing takes that further: after the initial ray hits a surface, it spawns additional rays to simulate how light bounces, refracts, and creates shadows. A ray tracer might cast a ray from the camera to a shiny surface, then cast a reflected ray from that surface into the scene, then cast yet another ray toward each light source to check for shadows.
Ray casting is one step in the ray tracing process. You can use ray casting on its own for fast, simple rendering or interaction detection, but ray tracing chains multiple ray casts together to produce physically accurate lighting, reflections, and transparency. This recursive bouncing is what makes ray tracing computationally expensive and visually stunning.
Performance and Optimization
The naive approach to ray casting checks every object in the scene for each ray, giving it a time complexity of O(N), where N is the number of objects. In a scene with a million polygons, that’s a million checks per ray, and you might be casting millions of rays per frame. This gets expensive fast.
Acceleration structures solve this problem by organizing objects spatially so the algorithm can skip large groups of them at once. Techniques like spatial subdivision (dividing the scene into a grid of cells) or tree structures (like octrees, which recursively divide 3D space into smaller boxes) let the algorithm quickly narrow down which objects are even near the ray’s path. There’s empirical evidence that these acceleration methods achieve sub-linear time complexity, meaning performance scales much better than checking every object. In theory, certain spatial data structures can bring the per-ray cost down to O(log N), though extreme memory requirements make the most aggressive optimizations impractical for real-time use.
For voxel-based scenes (where the world is made of small cubes, like in Minecraft), specialized ray casting methods can handle fully dynamic environments where every voxel potentially changes every frame, without needing precomputed spatial data.
Medical Imaging
Ray casting isn’t just for games. It’s a core technique in medical visualization, where doctors need to view 3D data from CT or MRI scans as a 2D image on a screen. The method is called volume ray casting: imaginary rays are sent through the 3D scan data (one ray per pixel in the final image), and at short, equal intervals along each ray, the algorithm collects samples of the tissue density or signal intensity at that point. These samples are then blended together from front to back to produce a final image that reveals internal structures.
This approach produces high-quality images but is computationally demanding, especially when multiple overlapping datasets are involved, such as combining an MRI scan with a real-time ultrasound during surgery. GPU-accelerated versions of this technique have been integrated into surgical navigation systems, proving clinically useful during procedures like ultrasound-guided brain tumor removal. The ability to composite multiple volumes lets surgeons see, for example, a pre-operative scan overlaid with live imaging data in a single view.
Object Selection in VR
In virtual reality, ray casting is how you point at and select things. A ray extends from your hand controller into the virtual scene, and whatever the ray hits first becomes the selected object. It’s essentially the same algorithm used in rendering, repurposed for interaction.
The challenge in VR is precision. Pointing a ray exactly through a small button or distant object is difficult with handheld controllers, so researchers have developed techniques to make selection more forgiving. One approach, inspired by a concept called the bubble cursor, doesn’t require the ray to pass directly through an object. Instead, it identifies whichever target is nearest to the ray, even if the ray doesn’t quite hit it. The system can measure “nearest” either by straight-line distance to the object’s boundary or by the angle between the ray and the object as seen from the user’s hand. When the ray does pass through multiple objects, the first one it intersects takes priority. A curved line, rendered as a Bezier curve, often connects the controller to the selected target as a visual indicator.