Audio signal processing is the manipulation of sound signals to analyze, modify, or enhance them. It covers everything from converting sound waves into digital data your computer can work with, to applying reverb on a vocal track, to the noise cancellation happening inside your earbuds right now. Whether you’re recording music, making a phone call, or listening to a podcast, audio signal processing is running behind the scenes.
How Sound Becomes Data
Sound in the real world is a continuous wave of air pressure changes. Your microphone converts that into a continuous electrical voltage. But computers can’t work with continuous signals directly. They need discrete numbers. So the first step in digital audio signal processing is converting that analog signal into digital form using an analog-to-digital converter (ADC).
This conversion involves two approximations. First, the continuous range of voltages gets divided into a finite set of values, a process called amplitude quantizing. Second, the signal gets measured at regular intervals rather than continuously, which is called sampling. A periodic timer triggers the ADC to capture a snapshot of the signal’s voltage at each interval. The result is a long sequence of numbers that represents the original sound wave. When it’s time to play the audio back, a digital-to-analog converter (DAC) reverses the process, turning those numbers back into an electrical signal that drives your speakers or headphones.
Sampling Rate and Why 44.1 kHz Exists
The sampling rate is how many times per second the ADC captures a measurement. The Nyquist theorem tells us that to accurately capture a sound, you need to sample at least twice the frequency of the highest pitch in that sound. Human hearing tops out around 20,000 Hz, so the minimum sampling rate to capture everything we can hear is 40,000 samples per second. Standard CD audio uses 44,100 Hz (44.1 kHz), which provides a comfortable margin above that minimum. Professional recording often uses 48 kHz, 96 kHz, or even 192 kHz for additional headroom during editing and processing.
Bit Depth and Dynamic Range
If sampling rate determines how often the signal is measured, bit depth determines how precisely each measurement is recorded. More bits means more possible values for each sample, which translates directly into a wider dynamic range: the gap between the quietest and loudest sounds the system can represent.
A 16-bit signal (the CD standard) offers about 96 dB of dynamic range across 65,536 possible values per sample. That’s enough to cover most listening situations comfortably. Professional audio typically uses 24-bit depth, which expands the range to roughly 144 dB across over 16 million possible values. In practice, even the best microphones rarely exceed 130 dB of dynamic range, so 24-bit captures more detail than most recording hardware can actually deliver. The 32-bit floating-point format used in modern recording software pushes the theoretical ceiling to 192 dB, which is less about capturing real-world sound and more about giving you enormous headroom during mixing so that calculations inside your software don’t introduce rounding errors.
Viewing Sound in the Frequency Domain
One of the most powerful tools in audio signal processing is the Fast Fourier Transform, or FFT. In its raw form, a digital audio signal is just a series of amplitude values over time. The FFT takes a section of that signal and breaks it apart into its individual frequency components, each with its own amplitude and phase. Think of it like taking a smoothie and identifying every fruit that went into it.
This frequency-domain view is what powers the spectrum analyzers you see in recording software, showing you exactly which frequencies are present and how loud they are at any moment. But FFT isn’t just for visualization. Equalizers, noise reduction tools, and pitch correction all rely on frequency-domain analysis to isolate and manipulate specific parts of the sound spectrum. Without this ability to decompose a complex sound into its frequency ingredients, most modern audio processing wouldn’t be possible.
Dynamic Range Processing
Compression is one of the most common forms of audio signal processing. A compressor automatically reduces the volume of sounds that exceed a set level, narrowing the gap between quiet and loud moments. This is why a podcast host’s voice stays at a consistent level even when they shift between speaking softly and laughing loudly.
Five parameters control how a compressor behaves. The threshold sets the volume level at which compression kicks in. The ratio determines how aggressively the volume gets reduced once that threshold is crossed. At a 4:1 ratio, for every 4 dB a signal goes above the threshold, only 1 dB makes it through to the output. The attack controls how quickly the compressor responds once the signal crosses the threshold. A fast attack clamps down on sharp peaks almost instantly, while a slow attack lets the initial punch of a drum hit or vocal consonant pass through before compressing. The release controls how quickly the compressor lets go after the signal drops back below the threshold. And makeup gain boosts the overall level back up after compression has reduced it, since squashing the loud parts naturally makes the whole signal quieter.
Time-Based Effects
Reverb and delay are time-based effects that simulate or create the impression of physical space. Delay repeats a signal after a set time interval. Reverb simulates the complex pattern of reflections that happen when sound bounces around a room, hall, or cathedral.
There are two fundamentally different approaches to digital reverb. Convolution reverb uses an impulse response: a recording of how a real space responds to a short burst of sound (like a starter pistol or a sine sweep). That recording captures the reflections, frequency character, and decay of the environment. The plugin then performs a mathematical operation called convolution that essentially stamps those spatial characteristics onto your dry audio. The result can sound remarkably realistic, because it’s based on an actual place.
Algorithmic reverb takes a different approach, building reverb from scratch using networks of digital delays, filters, and feedback loops. Early designs combined comb filters and all-pass filters to build up dense patterns of reflections. Modern versions use feedback delay networks where multiple delays feed into each other through a carefully tuned matrix. Algorithmic reverbs can’t perfectly replicate a specific room, but they offer far more flexibility to sculpt imaginary spaces and tend to use less processing power.
Noise Cancellation
Active noise cancellation (ANC) in headphones is a real-time application of audio signal processing. Sound is a pressure wave with compression and rarefaction phases. ANC works by analyzing incoming noise with a microphone, then generating a sound wave with the same amplitude but inverted phase. When the original noise wave and the inverted wave meet, they cancel each other out through destructive interference, reducing the perceived volume of the noise.
This happens through adaptive algorithms that continuously analyze the background noise waveform and adjust the cancellation signal in real time. The system has to be fast enough that the anti-noise wave lines up precisely with the incoming sound. This is why ANC works best on steady, low-frequency sounds like airplane engine hum or air conditioning, and struggles more with sudden, unpredictable noises.
Hardware DSP vs. Software Processing
Audio signal processing can run on dedicated hardware chips or on a general-purpose computer CPU. Dedicated digital signal processors (DSPs) are chips designed specifically for the math involved in audio processing. They offer extremely low latency (the delay between input and output) and high reliability, because they aren’t sharing resources with an operating system, a web browser, or anything else. This makes them essential for live sound, where even a few extra milliseconds of delay can cause problems for performers.
Software-based processing running on a standard computer CPU is more flexible. You can run any plugin, swap effects freely, and take advantage of ever-increasing computer power. The tradeoff is higher latency, potential instability from driver conflicts or software crashes, and the reality that your CPU is juggling audio alongside everything else the computer is doing. Some audio interfaces split the difference by including their own DSP chips that handle certain effects, offloading that work from your computer’s processor while still letting you use standard recording software.
AI-Powered Audio Processing
Neural networks have opened up capabilities that traditional signal processing couldn’t achieve. The most visible example is source separation: isolating individual instruments or voices from a mixed recording. Tools built on deep learning architectures can now extract a vocal track, a drum part, or a bass line from a finished song with impressive accuracy. These systems use architectures that model both short-term patterns (the shape of individual notes and syllables) and long-term patterns (the structure of phrases and musical passages) to distinguish overlapping sources.
The same underlying technology powers modern noise reduction in video calls, where the system learns to distinguish speech from background sounds like keyboard clicks, barking dogs, or construction noise. Rather than relying on fixed frequency filters, these AI models adapt to the specific characteristics of the noise and the speaker’s voice in real time.