Struct AdaDelta 

pub struct AdaDelta {
    pub lr: Array,
    pub rho: Array,
    pub eps: Array,
    pub state: State<(Array, Array)>,
}

The AdaDelta optimizer with a learning rate.

The default rho is 0.9, matching Python MLX. Earlier mlx-rs releases used 0.99.

Please refer to the original paper for more details:

[1]: Zeiler, M.D., 2012. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701.

Fields

lr: Array

The learning rate

rho: Array

The coefficient used for computing a running average of squared gradients. Defaults to AdaDelta::DEFAULT_RHO (0.9, matching Python MLX).

eps: Array

The epsilon added to the denominator to improve numerical stability. Default to AdaDelta::DEFAULT_EPS.

state: State<(Array, Array)>

Inner state

Implementations

impl AdaDelta

pub fn new( lr: impl Into<f32>, ) -> Result<Self, <<Self as Buildable>::Builder as Builder<Self>>::Error>

Creates a new instance of AdaDelta.

impl AdaDelta

pub const DEFAULT_RHO: f32 = 0.9

Default value for rho, matching Python MLX.

pub const DEFAULT_EPS: f32 = 1e-6

Default value for eps

Trait Implementations

impl Buildable for AdaDelta

type Builder = AdaDeltaBuilder

The builder type for this buildable type

impl Builder<AdaDelta> for AdaDeltaBuilder

type Error = AdaDeltaBuildError

Error with building

fn build(self) -> Result<AdaDelta, Self::Error>

Build the type

impl Clone for AdaDelta

fn clone(&self) -> AdaDelta

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AdaDelta

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Optimizer for AdaDelta

type State = HashMap<Rc<str>, (Array, Array)>

State of the optimizer.

fn state(&self) -> &Self::State

Get the state of the optimizer.

fn state_mut(&mut self) -> &mut Self::State

Get the mutable state of the optimizer.

fn update_single( &mut self, key: &Rc<str>, gradient: &Array, parameter: &mut Array, ) -> Result<()>

Update a single parameter with the given gradient.

fn update<M>( &mut self, model: &mut M, gradients: impl Borrow<FlattenedModuleParam>, ) -> Result<()>

where M: ModuleParameters,

Apply the gradients to the parameters of the model and update the model with the new parameters.

impl Updatable for AdaDelta

fn state_projection( &mut self, ) -> Result<StateProjection<'_>, StateProjectionError>

Declare all required and optional state slots.

impl Updatable for &mut AdaDelta

fn state_projection( &mut self, ) -> Result<StateProjection<'_>, StateProjectionError>

Declare all required and optional state slots.