The sigmoid activation function is the right choice when you need outputs that represent probabilities, specifically values between 0 and 1. Its primary modern uses are binary classification output layers, multi-label classification outputs, and gating mechanisms inside recurrent neural networks. For hidden layers in deep networks, sigmoid has largely been replaced by faster alternatives.
What the Sigmoid Function Does
The sigmoid function takes any real number as input and maps it to a value between 0 and 1, following an S-shaped curve. Large positive inputs push the output toward 1, large negative inputs push it toward 0, and an input of zero returns exactly 0.5. The formula is straightforward: divide 1 by 1 plus Euler’s number raised to the negative input value.
This 0-to-1 output range is what makes sigmoid useful. It naturally represents probability: the output can be read as “how likely is this event?” That probabilistic interpretation is the core reason sigmoid persists in modern deep learning, even as other activation functions have taken over most other roles.
Binary Classification Output Layers
Sigmoid is the standard output activation function for binary classification. When your network needs to answer a yes-or-no question (is this email spam, does this image contain a tumor, will this customer churn), you place a single neuron with sigmoid activation as your final layer. The output directly represents the predicted probability of the positive class.
This connection runs deep. Logistic regression, one of the oldest and most widely used classification methods, works by computing a weighted sum of input features and then passing that sum through the sigmoid function. The weighted sum is technically the “log-odds,” the logarithm of the ratio between the probability of the event happening and it not happening. Sigmoid converts those log-odds into an actual probability. A neural network with a sigmoid output layer is doing the same thing, just with learned features instead of hand-picked ones.
Once you have that probability, you can either use it directly (this patient has a 73% chance of readmission) or convert it to a binary prediction by choosing a threshold, typically 0.5. Medical image segmentation models also use sigmoid outputs when classifying each pixel as belonging to one of two classes.
Multi-Label Classification
When each input can belong to multiple categories simultaneously, sigmoid is again the right choice for the output layer. Think of tagging a news article that could be about both “politics” and “economics” at the same time, or an image that contains both a dog and a cat. Here, each output neuron gets its own sigmoid activation and independently predicts the probability for its category.
This is different from multi-class classification (where only one category is correct), which uses softmax instead. The key distinction: softmax forces all output probabilities to sum to 1, creating competition between classes. Sigmoid treats each output independently, so multiple classes can all have high probabilities at once.
Gating in Recurrent Networks
Inside LSTM and GRU architectures, sigmoid plays a structural role that has nothing to do with final predictions. These networks use “gates” to control how information flows through a sequence, and those gates rely on sigmoid activations.
In a GRU, for example, the update gate uses sigmoid to produce values between 0 and 1 for each element of the hidden state. When the gate output is close to 1, the network retains the old state and effectively ignores the current input, which helps capture long-term dependencies in sequences. When the gate output is close to 0, the network replaces the old state with new information. The reset gate works similarly: values near 1 preserve previous context, while values near 0 wipe the slate clean, helping capture short-term dependencies.
Sigmoid is ideal for this job because the 0-to-1 range acts as a dial. A value of 0.8 means “keep 80% of this information.” No other common activation function provides this clean, bounded range that naturally represents a proportion. LSTMs use the same principle across three gates: the forget gate, input gate, and output gate.
Why Not to Use Sigmoid in Hidden Layers
For hidden layers in deep networks, sigmoid is generally a poor choice. Two problems explain why ReLU and its variants have almost entirely replaced it.
The first is the vanishing gradient problem. The sigmoid derivative equals the sigmoid output multiplied by one minus the sigmoid output. This derivative reaches its maximum value of 0.25 when the input is zero, and it drops toward zero as inputs become large in either direction. During backpropagation, gradients pass through each layer by multiplication. If every layer shrinks the gradient by a factor of 0.25 or less, the signal reaching early layers in a deep network becomes vanishingly small. Those layers essentially stop learning. The network “saturates” near the boundaries of the S-curve, where the function is nearly flat and gradients are almost zero.
The second issue is that sigmoid outputs are not centered around zero. All outputs fall between 0 and 1, meaning they’re always positive. When every input to the next layer is positive, the gradients for all weights in that layer move in the same direction during each update. This creates a zigzagging optimization path that slows down training convergence.
ReLU avoids both problems. It outputs zero for negative inputs and passes positive inputs through unchanged, so its gradient is either 0 or 1. There’s no compression, no saturation for positive values, and it’s computationally cheaper to calculate. Google’s machine learning documentation notes that ReLU is less susceptible to vanishing gradients and significantly easier to compute than sigmoid or tanh.
Quick Decision Guide
- Binary classification output layer: Use sigmoid. One neuron, one probability.
- Multi-label classification output layer: Use sigmoid on each output neuron independently.
- Multi-class classification output layer: Use softmax, not sigmoid.
- Gating mechanisms in LSTMs or GRUs: Sigmoid is built into the architecture by design.
- Hidden layers in feedforward or convolutional networks: Use ReLU or a variant (Leaky ReLU, GELU, Swish) instead.
- Shallow networks or small models: Sigmoid can work in hidden layers when vanishing gradients aren’t a concern, such as networks with only one or two hidden layers.
- Any layer where you need bounded 0-to-1 output: Sigmoid is a natural fit, for instance in attention mechanisms that produce weights or any custom architecture requiring a “proportion” value.
Sigmoid in Composed Activation Functions
Recent research explores composing multiple activation functions together, combining, for example, sigmoid’s bounded squashing behavior with ReLU’s sparsity-inducing properties. The idea is that different functions capture different types of nonlinearity, and combining them lets a network learn a wider variety of patterns. Swish (the input multiplied by its own sigmoid) is already a practical example of this: it’s used in many modern architectures and outperforms plain ReLU in certain deep networks. So while sigmoid has largely left hidden layers as a standalone function, its mathematical properties continue to influence newer activation designs.