MsraFiller

A Filler based on the paper [He, Zhang, Ren and Sun 2015]: Specifically accounts for ReLU nonlinearities.

Aside: for another perspective on the scaling factor, see the derivation of [Saxe, McClelland, and Ganguli 2013 (v3)].

It fills the incoming matrix by randomly sampling Gaussian data with std = sqrt(2 / n) where n is the fanIn, fanOut, or their average, depending on the varianceNormAverage parameter.

varianceNormAverage: VarianceNorm use average of (fanIn + fanOut) or just fanOut

Linear Supertypes

Serializable, Serializable, Product, Equals, InitializationMethod, AnyRef, Any

Instance Constructors

new MsraFiller(varianceNormAverage: Boolean = true)

varianceNormAverage
VarianceNorm use average of (fanIn + fanOut) or just fanOut

Type Members

type Shape = Array[Int]

Definition Classes
InitializationMethod

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def init[T](variable: Tensor[T], dataFormat: VariableFormat)(implicit ev: TensorNumeric[T]): Unit

Initialize the given weight and bias.
Initialize the given weight and bias.
variable
the weight to initialize
dataFormat
the data format of weight indicating the dimension order of the weight. "output_first" means output is in the lower dimension "input_first" means input is in the lower dimension.

Definition Classes
MsraFiller → InitializationMethod
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
val varianceNormAverage: Boolean

VarianceNorm use average of (fanIn + fanOut) or just fanOut
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )

Related Doc: package nn

case class MsraFiller(varianceNormAverage: Boolean = true) extends InitializationMethod with Product with Serializable

Instance Constructors

new MsraFiller(varianceNormAverage: Boolean = true)

Type Members

type Shape = Array[Int]

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def clone(): AnyRef

final def eq(arg0: AnyRef): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def init[T](variable: Tensor[T], dataFormat: VariableFormat)(implicit ev: TensorNumeric[T]): Unit

final def isInstanceOf[T0]: Boolean

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

final def synchronized[T0](arg0: ⇒ T0): T0

val varianceNormAverage: Boolean

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from InitializationMethod

Inherited from AnyRef

Inherited from Any

Ungrouped