Skip to main content

rms_norm

Function rms_norm 

Source
pub fn rms_norm(
    x: impl AsRef<Array>,
    weight: Option<&Array>,
    eps: f32,
) -> Result<Array>
Expand description

Root Mean Square normalization (RMS norm).

The normalization is with respect to the last axis of the input x.

ยงParams

  • x: input array
  • weight: An optional multiplicative weight. When present, it must be one-dimensional with the same size as the last axis of x. When absent, only normalization is applied.
  • eps: A small additive constant for numerical stability