Softsign bounded activation

Read the equation

Dividing x by 1+|x| preserves its sign and maps finite real inputs into (−1,1). Near zero it is approximately x; large magnitudes approach saturation gradually. It has a continuous first derivative, but its second derivative changes discontinuously at zero, whereas tanh is analytic.

Terms & definitions

How it works

Read the definition from its right-hand expression to the quantity on the left. Its surrounding source records the intended input, state, or parameter context. Where the source leaves an operation unspecified, this page preserves that gap.

Assumptions & limits

• The source’s claim that it is smoother than tanh at the origin is not mathematically correct.

Original context

## 12.1 Softsign

```
softsign(x) = x / (1 + |x|)
```

**Range**: (-1, 1)

**Explanation**: Bounded activation that preserves sign and approaches ±1 asymptotically. Smoother than tanh at origin.

## 12.2 Sigmoid

```
σ(x) = 1 / (1 + e^{-x})
```

**Range**: (0, 1)

**Explanation**: Classic S-curve activation for probability-like outputs.

## 12.3 Clamp

```
clamp₀₁(x) = max(0, min(1, x))

softsign(x) = x / (1 + |x|)

Related builds