Reciprocal rescaling leaves the rank-one operator unchanged

Read the equation

Multiplying one factor by a nonzero scalar and dividing the other by that scalar cancels in their outer product. The individual vectors can therefore move in magnitude while the effective weight update remains identical. This is why an operator-based distance avoids a basic ambiguity in factor-only measurements.

Terms & definitions

How it works

Read the definition from its right-hand expression to the quantity on the left. Its surrounding source records the intended input, state, or parameter context. Where the source leaves an operation unspecified, this page preserves that gap.

Assumptions & limits

• The scalar must be nonzero.

Original context

(4.1)

The adapted layer applies (W0+Ut)h to input h, where W0 is its frozen base weight. A change in one factor need not change that operator. For any nonzero scalar λt,

(λtbt)(at/λt)T=btatT.

(4.2)

This includes simultaneous reversal of both signs. I compare both the factors and their product to distinguish an operator change from an equivalent factorization. That comparison does not presume an error in the original study.

The required matrix quantities reduce to vector operations:

({\lambda }_{t}{b}_{t}){({a}_{t}/{\lambda }_{t})}^{T}={b}_{t}{a}_{t}^{T}.

Related builds