SGD

Type Members

1. case class Default() extends LearningRateSchedule with Product with Serializable

   It is the default learning rate schedule.

2. case class EpochDecay(decayType: (Int) ⇒ Double) extends LearningRateSchedule with Product with Serializable

   It is an epoch decay learning rate schedule The learning rate decays through a function argument on number of run epochs

3. case class EpochDecayWithWarmUp(warmUpIteration: Int, warmUpDelta: Double, decayType: (Int) ⇒ Double) extends LearningRateSchedule with Product with Serializable

   Learning rate schedule based on warm up Iterations

4. case class EpochSchedule(regimes: Array[Regime]) extends LearningRateSchedule with Product with Serializable

   EpochSchedule is a learning rate schedule which configure the learning rate according to some pre-defined Regime.

5. case class EpochStep(stepSize: Int, gamma: Double) extends LearningRateSchedule with Product with Serializable

   EpochStep is a learning rate schedule, which rescale the learning rate by gamma for each stepSize epochs.

6. case class Exponential(decayStep: Int, decayRate: Double, stairCase: Boolean = false) extends LearningRateSchedule with Product with Serializable

   Exponential is a learning rate schedule, which rescale the learning rate by lr_{n + 1} = lr * decayRate ^ (iter / decayStep)

7. trait LearningRateSchedule extends AnyRef

   Hyper parameter schedule for SGD

8. case class MultiStep(stepSizes: Array[Int], gamma: Double) extends LearningRateSchedule with Product with Serializable

   similar to step but it allows non uniform steps defined by stepSizes

9. case class NaturalExp(decay_step: Int, gamma: Double) extends LearningRateSchedule with Product with Serializable

   NaturalExp is a learning rate schedule, which rescale the learning rate by exp ( -decay_rate * iter / decay_step ) referring to tensorflow's learning rate decay # natural_exp_decay

10. case class Plateau(monitor: String, factor: Float = 0.1, patience: Int = 10, mode: String = "min", epsilon: Float = 1.0E-4, cooldown: Int = 0, minLr: Float = 0) extends LearningRateSchedule with Product with Serializable

    Plateau is the learning rate schedule when a metric has stopped improving.

11. case class Poly(power: Double, maxIteration: Int) extends LearningRateSchedule with Product with Serializable

    A learning rate decay policy, where the effective learning rate follows a polynomial decay, to be zero by the max_iteration.

12. case class Regime(startEpoch: Int, endEpoch: Int, config: Table) extends Product with Serializable

    A structure to specify hyper parameters by start epoch and end epoch.

13. case class SequentialSchedule(iterationPerEpoch: Int) extends LearningRateSchedule with Product with Serializable

    Stack several learning rate schedulers.

14. case class Step(stepSize: Int, gamma: Double) extends LearningRateSchedule with Product with Serializable

    A learning rate decay policy, where the effective learning rate is calculated as base_lr * gamma ^ (floor(iter / stepSize))

15. case class Warmup(delta: Double) extends LearningRateSchedule with Product with Serializable

    A learning rate gradual increase policy, where the effective learning rate increase delta after each iteration.