Element-wise multiplication is an operation where two arrays or matrices of the same shape are multiplied together, one matching pair of elements at a time. If you have two 3×3 matrices, the element in row 1, column 1 of the first matrix gets multiplied by the element in row 1, column 1 of the second matrix, and so on for every position. The result is a new matrix of the same shape, where each entry is the product of the two corresponding inputs.
How It Works
Say you have two simple lists of numbers: [2, 4, 6] and [3, 5, 7]. Element-wise multiplication pairs them up by position and multiplies: 2×3, 4×5, 6×7, giving you [6, 20, 42]. The same logic scales to two-dimensional matrices, three-dimensional arrays, or any number of dimensions, as long as both inputs share the same shape.
In formal mathematics, this operation on matrices is called the Hadamard product, often written with a circle-dot symbol (⊙). For two matrices A and B of size N×M, the result C is defined so that each element C(i,j) = A(i,j) × B(i,j). Because you’re performing one multiplication per element, the computational cost scales linearly with the total number of elements. A 100×100 matrix pair requires exactly 10,000 multiplications.
How It Differs From Matrix Multiplication
This is the distinction that trips most people up. Standard matrix multiplication (the dot product) is a fundamentally different operation. It multiplies rows against columns, sums the results, and often produces an output with a completely different shape than the inputs. For two vectors, the dot product returns a single number. Element-wise multiplication keeps every individual product separate and always returns an output the same size as the inputs.
- Output shape: Element-wise multiplication produces a vector or matrix the same shape as the inputs. The dot product typically produces a scalar or a matrix with different dimensions.
- What it computes: Element-wise multiplication multiplies corresponding elements directly. The dot product multiplies corresponding elements and then sums them.
- Dimensional requirements: Element-wise multiplication needs both inputs to be the same shape (with some exceptions for broadcasting). Matrix multiplication needs the inner dimensions to match, meaning the number of columns in the first matrix must equal the number of rows in the second.
A quick example makes this concrete. Take vectors [1, 2, 3] and [4, 5, 6]. Element-wise multiplication gives [4, 10, 18]. The dot product gives 4 + 10 + 18 = 32.
Broadcasting: When Shapes Don’t Match
In practice, you don’t always need both arrays to be exactly the same shape. Most numerical libraries support “broadcasting,” a set of rules that automatically stretch smaller arrays to match larger ones during element-wise operations. The simplest case is multiplying an array by a single number: [1, 2, 3] × 2 gives [2, 4, 6]. The scalar 2 is conceptually expanded into [2, 2, 2] before the element-wise multiplication happens.
Broadcasting follows specific rules. The library compares dimensions starting from the rightmost side and working left. Two dimensions are compatible when they’re equal or when one of them is 1. A dimension of size 1 gets stretched to match the other. If neither condition is met, the operation throws an error. This means you can multiply a 4×3 matrix by a 1×3 row, and the row will automatically apply to every row of the matrix. Missing dimensions are treated as size 1, so a one-dimensional array of length 3 also broadcasts against a 4×3 matrix without issue.
How to Do It in Code
In Python’s NumPy library, the standard * operator performs element-wise multiplication on arrays. So a * b where both are NumPy arrays gives the Hadamard product, not a dot product. You can also use the explicit function np.multiply(a, b), which does exactly the same thing.
PyTorch follows the same convention: the * operator between two tensors is element-wise, and torch.mul(a, b) is the explicit equivalent. In MATLAB, the syntax uses a dot prefix: A .* B performs element-wise multiplication, while A * B is reserved for standard matrix multiplication. This MATLAB convention is worth remembering, since mixing them up is a common source of bugs.
For matrix (dot product) multiplication, you’d use np.matmul(a, b) or the @ operator in NumPy, torch.matmul() in PyTorch, and plain * in MATLAB.
Why It Matters in Machine Learning
Element-wise multiplication is everywhere in modern neural networks. Its most prominent role is in gating mechanisms, where one array of values controls how much of another array passes through. Long Short-Term Memory (LSTM) networks and Gated Recurrent Units (GRUs) rely on this heavily. These architectures use element-wise multiplication to decide which pieces of information to remember and which to forget at each time step. A gate produces values between 0 and 1 for each element, and multiplying that gate against the data effectively scales each feature independently, letting some through fully and blocking others entirely.
Transformer models, the architecture behind most large language models, also use element-wise multiplication when modulating attention scores and context vectors. In multimodal systems that combine different types of data (like images and text), element-wise multiplication provides a computationally efficient way to blend features from different sources, approximating complex interactions without the cost of full matrix operations.
The reason element-wise multiplication is so useful in these contexts comes down to two properties: it preserves the shape of the data, and it operates on each feature independently. This makes it a natural fit for any situation where you want to selectively scale, mask, or weight individual features rather than mix them all together the way a dot product would.
Common Use Cases Outside Neural Networks
Image processing relies on element-wise multiplication for applying filters and masks. If you want to darken a specific region of an image, you multiply the pixel array by a mask array that has values of 1.0 where you want the image unchanged and smaller values where you want it dimmed. Signal processing uses it similarly, multiplying a signal by a window function to isolate a segment in time.
In data science, element-wise multiplication is the default way to apply weights to features, normalize data by scaling factors, or compute things like weighted averages. Any time you need to apply a per-element transformation that depends on another array of values, you’re reaching for element-wise multiplication.