# Simple Layers

## Linear

Scala:

``````val module = Linear(
inputSize,
outputSize,
withBias = true,
wRegularizer = null,
bRegularizer = null,
initWeight = null,
initBias = null,
``````

Python:

``````module = Linear(
input_size,
output_size,
init_method="default",
with_bias=True,
wRegularizer=None,
bRegularizer=None,
init_weight=None,
init_bias=None,
``````

The `Linear` module applies a linear transformation to the input data, i.e. `y = Wx + b`. The `input` given in `forward(input)` must be either a vector (1D tensor) or matrix (2D tensor). If the input is a vector, it must have the size of `inputSize`. If it is a matrix, then each row is assumed to be an input sample of given batch (the number of rows means the batch size and the number of columns should be equal to the `inputSize`).

Scala example:

``````import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._

val module = Linear(3, 5)

println(module.forward(Tensor.range(1, 3, 1)))
``````

Gives the output,

``````com.intel.analytics.bigdl.tensor.Tensor[Float] =
0.79338956
-2.3417668
-2.7557678
-0.07507719
-1.009765
[com.intel.analytics.bigdl.tensor.DenseTensor of size 5]
``````

Python example:

``````from bigdl.nn.layer import *
import numpy as np

module = Linear(3, 5)

print(module.forward(np.arange(1, 4, 1)))
``````

Gives the output,

``````[array([ 0.31657887, -1.11062765, -1.16235781, -0.67723978,  0.74650359], dtype=float32)]
``````

## Reverse

Scala:

``````val m = Reverse(dim = 1, isInplace = false)
``````

Python:

``````m = Reverse(dimension=1)
``````

Reverse the input w.r.t given dimension. The input can be a Tensor or Table. `Dimension` is one-based index.

Scala example:

``````import com.intel.analytics.bigdl.utils._
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat

def randomn(): Double = RandomGenerator.RNG.uniform(0, 1)
val input = Tensor(2, 3)
input.apply1(x => randomn().toFloat)
println("input:")
println(input)
val layer = new Reverse(1)
println("output:")
println(layer.forward(input))
``````
``````input:
0.17271264898590744 0.019822501810267568    0.18107921979390085
0.4003877849318087  0.5567442716564983  0.14120339532382786
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x3]
output:
0.4003877849318087  0.5567442716564983  0.14120339532382786
0.17271264898590744 0.019822501810267568    0.18107921979390085
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x3]

``````

Python example:

``````input = np.random.random((2,3))
layer = Reverse(1)
print("input:")
print(input)
print("output:")
print(layer.forward(input))
``````
``````creating: createReverse
input:
[[ 0.89089717  0.07629756  0.30863782]
[ 0.16066851  0.06421963  0.96719367]]
output:
[[ 0.16066851  0.06421963  0.96719366]
[ 0.89089715  0.07629756  0.30863783]]

``````

## Reshape

Scala:

``````val reshape = Reshape(size, batchMode)
``````

Python:

``````reshape = Reshape(size, batch_mode)
``````

The `forward(input)` reshape the input tensor into `size(0) * size(1) * ...` tensor, taking the elements row-wise.

parameters: `size` the size after reshape `batchMode` It is an optional argument. If it is set to `Some(true)`, the first dimension of input is considered as batch dimension, and thus keep this dimension size fixed. This is necessary when dealing with batch sizes of one. When set to `Some(false)`, it forces the entire input (including the first dimension) to be reshaped to the input size. Default is `None`, which means the module considers inputs with more elements than the product of provided sizes (```size(0) * size(1) * ..```) to be batches, otherwise in no batch mode.

Scala example:

``````import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.utils.T
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat

val reshape = Reshape(Array(3, 2))
val input = Tensor(2, 2, 3).rand()
val output = reshape.forward(input)
-> print(output.size().toList)
List(2, 3, 2)
``````

Python example:

``````from bigdl.nn.layer import *
from bigdl.nn.criterion import *
import numpy as np
reshape =  Reshape([3, 2])
input = np.random.rand(2, 2, 3)
output = reshape.forward(input)
-> print output[0].shape
(2, 3, 2)
``````

## Index

Scala:

``````val model = Index(dimension)
``````

Python:

``````model = Index(dimension)
``````

Applies the Tensor index operation along the given dimension.

Scala example:

``````import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.utils.T

val input1 = Tensor(3).rand()
val input2 = Tensor(4)
input2(Array(1)) = 1.0f
input2(Array(2)) = 2.0f
input2(Array(3)) = 2.0f
input2(Array(4)) = 3.0f

val input = T(input1, input2)
val model = Index(1)
val output = model.forward(input)

scala> print(input)
{
2: 1.0
2.0
2.0
3.0
[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 4]
1: 0.124325536
0.8768922
0.6378146
[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 3]
}
scala> print(output)
0.124325536
0.8768922
0.8768922
0.6378146
[com.intel.analytics.bigdl.tensor.DenseTensor of size 4]
``````

Python example:

``````from bigdl.nn.layer import *
import numpy as np

input1 = np.random.randn(3)
input2 = np.array([1, 2, 2, 3])
input = [input1, input2]

model = Index(1)
output = model.forward(input)

>>> print(input)
[array([-0.45804847, -0.20176707,  0.50963248]), array([1, 2, 2, 3])]

>>> print(output)
[-0.45804846 -0.20176707 -0.20176707  0.50963247]
``````

## Identity

Scala:

``````val identity = Identity()
``````

Python:

``````identity = Identity()
``````

Identity just return input as the output which is useful in same parallel container to get an origin input

Scala example:

``````import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._
val identity = Identity()

val input = Tensor(3, 3).rand()
> print(input)
0.043098174 0.1035049   0.7522675
0.9999951   0.794151    0.18344955
0.9419861   0.02398399  0.6228095
[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 3x3]

> print(identity.forward(input))
0.043098174 0.1035049   0.7522675
0.9999951   0.794151    0.18344955
0.9419861   0.02398399  0.6228095
[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 3x3]

``````

Python example:

``````from bigdl.nn.layer import *
identity = Identity()
>  identity.forward(np.array([[1, 2, 3], [4, 5, 6]]))
[array([[ 1.,  2.,  3.],
[ 4.,  5.,  6.]], dtype=float32)]

``````

## Narrow

Scala:

``````val layer = Narrow(dimension, offset, length = 1)
``````

Python:

``````layer = Narrow(dimension, offset, length=1)
``````

Narrow is an application of narrow operation in a module. The module further supports a negative length in order to handle inputs with an unknown size.

Parameters: `dimension` narrow along this dimension `offset` the start index on the given dimension * `length` length to narrow, default value is 1

Scala Example

``````import com.intel.analytics.bigdl.nn.Narrow
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.numeric.NumericFloat
import com.intel.analytics.bigdl.utils.T

val layer = Narrow(2, 2)
val input = Tensor(T(
T(-1f, 2f, 3f),
T(-2f, 3f, 4f),
T(-3f, 4f, 5f)
))

val gradOutput = Tensor(T(3f, 4f, 5f))

val output = layer.forward(input)
2.0
3.0
4.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3x1]

0.0 3.0 0.0
0.0 4.0 0.0
0.0 5.0 0.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3x3]
``````

Python Example

``````layer = Narrow(2, 2)
input = np.array([
[-1.0, 2.0, 3.0],
[-2.0, 3.0, 4.0],
[-3.0, 4.0, 5.0]
])

output = layer.forward(input)

print output
[[ 2.]
[ 3.]
[ 4.]]

[[ 0.  3.  0.]
[ 0.  4.  0.]
[ 0.  5.  0.]]
``````

## Unsqueeze

Scala:

``````val layer = Unsqueeze(dim)
``````

Python:

``````layer = Unsqueeze(dim)
``````

Insert singleton dim (i.e., dimension 1) at position pos. For an input with `dim = input.dim()`, there are `dim + 1` possible positions to insert the singleton dimension. The dim starts from 1.

Scala example:

``````import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._

val layer = Unsqueeze(2)
val input = Tensor(2, 2, 2).rand
val gradOutput = Tensor(2, 1, 2, 2).rand
val output = layer.forward(input)

> println(input.size)
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x2x2]

[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x1x2x2]

> println(output.size)
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x1x2x2]

[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x2x2]

``````

Python example:

``````from bigdl.nn.layer import *
import numpy as np

layer = Unsqueeze(2)
input = np.random.uniform(0, 1, (2, 2, 2)).astype("float32")
gradOutput = np.random.uniform(0, 1, (2, 1, 2, 2)).astype("float32")

output = layer.forward(input)

> output
[array([[[[ 0.97488612,  0.43463323],
[ 0.39069486,  0.0949123 ]]],

[[[ 0.19310953,  0.73574477],
[ 0.95347691,  0.37380624]]]], dtype=float32)]
[array([[[ 0.9995622 ,  0.69787127],
[ 0.65975296,  0.87002522]],

[[ 0.76349133,  0.96734989],
[ 0.88068211,  0.07284366]]], dtype=float32)]
``````

## Squeeze

Scala:

``````val module = Squeeze(dims=null, numInputDims=Int.MinValue)
``````

Python:

``````module = Squeeze(dims, numInputDims=-2147483648)
``````

Delete all singleton dimensions or a specific singleton dimension.

• `dims` Optional. If this dimension is singleton dimension, it will be deleted. The first index starts from 1. Default: delete all dimensions.
• `num_input_dims` Optional. If in a batch model, set to the inputDims.

Scala example:

``````import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat

val layer = Squeeze(2)
> print(layer.forward(Tensor(2, 1, 3).rand()))
0.43709445  0.42752415  0.43069172
0.67029667  0.95641375  0.28823504
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x3]
``````

Python example:

``````from bigdl.nn.layer import *

layer = Squeeze(2)
>layer.forward(np.array([[[1, 2, 3]], [[1, 2, 3]]]))
out: array([[ 1.,  2.,  3.],
[ 1.,  2.,  3.]], dtype=float32)

``````

## Select

Scala:

``````val layer = Select(dim, index)
``````

Python:

``````layer = Select(dim, index)
``````

A Simple layer selecting an index of the input tensor in the given dimension. Please note that the index and dimension start from 1. In collaborative filtering, it can used together with LookupTable to create embeddings for users or items.

Scala example:

``````import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.utils.T
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat

val layer = Select(1, 2)
layer.forward(Tensor(T(
T(1.0f, 2.0f, 3.0f),
T(4.0f, 5.0f, 6.0f),
T(7.0f, 8.0f, 9.0f)
)))

layer.backward(Tensor(T(
T(1.0f, 2.0f, 3.0f),
T(4.0f, 5.0f, 6.0f),
T(7.0f, 8.0f, 9.0f)
)), Tensor(T(0.1f, 0.2f, 0.3f)))
``````

Gives the output,

``````4.0
5.0
6.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]

0.0     0.0     0.0
0.1     0.2     0.3
0.0     0.0     0.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3x3]
``````

Python example:

``````from bigdl.nn.layer import Select
import numpy as np

layer = Select(1, 2)
layer.forward(np.array([
[1.0, 2.0, 3.0],
[4.0, 5.0, 6.0],
[7.0, 8.0, 9.0]
]))
layer.backward(np.array([
[1.0, 2.0, 3.0],
[4.0, 5.0, 6.0],
[7.0, 8.0, 9.0]
]), np.array([0.1, 0.2, 0.3]))
``````

Gives the output,

``````array([ 4.,  5.,  6.], dtype=float32)

array([[ 0.        ,  0.        ,  0.        ],
[ 0.1       ,  0.2       ,  0.30000001],
[ 0.        ,  0.        ,  0.        ]], dtype=float32)
``````

Scala:

``````val module = MaskedSelect()
``````

Python:

``````module = MaskedSelect()
``````

Performs a torch.MaskedSelect on a Tensor. The mask is supplied as a tabular argument with the input on the forward and backward passes.

Scala example:

``````import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.utils.T
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import scala.util.Random

val input1 = Tensor(2, 2).apply1(e => Random.nextFloat())
val input = T()
input(1.0) = input1
> print(layer.forward(input))
0.2577119
0.5061479
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2]
``````

Python example:

``````from bigdl.nn.layer import *

input1 = np.random.rand(2,2)
array([ 0.1525335 ,  0.05474588], dtype=float32)
``````

## Transpose

Scala:

``````val module = Transpose(permutations)
``````

Python:

``````module = Transpose(permutations)
``````

Concat is a layer who transpose input along specified dimensions. permutations are dimension pairs that need to swap.

Scala example:

``````import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat

val input = Tensor(2, 3).rand()
val layer = Transpose(Array((1, 2)))
val output = layer.forward(input)

> input
0.6653826   0.25350887  0.33434764
0.9618287   0.5484164   0.64844745
[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 2x3]

> output
0.6653826   0.9618287
0.25350887  0.5484164
0.33434764  0.64844745
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3x2]

``````

Python example:

``````from bigdl.nn.layer import *
import numpy as np

layer = Transpose([(1,2)])
input = np.array([[0.6653826, 0.25350887, 0.33434764], [0.9618287, 0.5484164, 0.64844745]])
output = layer.forward(input)

> output
[array([[ 0.66538262,  0.96182871],
[ 0.25350887,  0.54841638],
[ 0.33434764,  0.64844745]], dtype=float32)]

``````

## InferReshape

Scala:

``````val layer = InferReshape(size, batchMode = false)
``````

Python:

``````layer = InferReshape(size, batch_mode=False)
``````

Reshape the input tensor with automatic size inference support. Positive numbers in the `size` argument are used to reshape the input to the corresponding dimension size.

There are also two special values allowed in `size`:

1. `0` means keep the corresponding dimension size of the input unchanged. i.e., if the 1st dimension size of the input is 2, the 1st dimension size of output will be set as 2 as well.
2. `-1` means infer this dimension size from other dimensions. This dimension size is calculated by keeping the amount of output elements consistent with the input. Only one `-1` is allowable in `size`.

For example,

``````   Input tensor with size: (4, 5, 6, 7)
-> InferReshape(Array(4, 0, 3, -1))
Output tensor with size: (4, 5, 3, 14)
``````

The 1st and 3rd dim are set to given sizes, keep the 2nd dim unchanged, and inferred the last dim as 14.

Parameters: `size` the target tensor size `batchMode` whether in batch mode

Scala example:

``````import com.intel.analytics.bigdl.nn.InferReshape
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.numeric.NumericFloat

val layer = InferReshape(Array(0, 3, -1))
val input = Tensor(1, 2, 3).rand()
val gradOutput = Tensor(1, 3, 2).rand()

val output = layer.forward(input)

println(output)
(1,.,.) =
0.8170822   0.40073588
0.49389255  0.3782435
0.42660004  0.5917206

[com.intel.analytics.bigdl.tensor.DenseTensor of size 1x3x2]

(1,.,.) =
0.8294597   0.57101834  0.90910035
0.32783163  0.30494633  0.7339092

[com.intel.analytics.bigdl.tensor.DenseTensor of size 1x2x3]
``````

Python example:

``````layer = InferReshape([0, 3, -1])
input = np.random.rand(1, 2, 3)

output = layer.forward(input)

print output
[[[ 0.68635464  0.21277553]
[ 0.13390459  0.65662414]
[ 0.1021723   0.92319047]]]

[[[ 0.84927064  0.55205333  0.25077972]
[ 0.76105869  0.30828172  0.1237276 ]]]
``````

## Replicate

Scala:

``````val module = Replicate(
nFeatures,
dim = 1,
nDim = Int.MaxValue)
``````

Python:

``````module = Replicate(
n_features,
dim=1,
n_dim=INTMAX)
``````

Replicate repeats input `nFeatures` times along its `dim` dimension

Notice: No memory copy, it set the stride along the `dim`-th dimension to zero.

Scala example:

``````import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._

val module = Replicate(4, 1, 2)

println(module.forward(Tensor.range(1, 6, 1).resize(1, 2, 3)))
``````

Gives the output,

``````com.intel.analytics.bigdl.tensor.Tensor[Float] =
(1,1,.,.) =
1.0 2.0 3.0
4.0 5.0 6.0

(1,2,.,.) =
1.0 2.0 3.0
4.0 5.0 6.0

(1,3,.,.) =
1.0 2.0 3.0
4.0 5.0 6.0

(1,4,.,.) =
1.0 2.0 3.0
4.0 5.0 6.0

[com.intel.analytics.bigdl.tensor.DenseTensor of size 1x4x2x3]
``````

Python example:

``````from bigdl.nn.layer import *
import numpy as np

module = Replicate(4, 1, 2)

print(module.forward(np.arange(1, 7, 1).reshape(1, 2, 3)))
``````

Gives the output,

``````[array([[[[ 1.,  2.,  3.],
[ 4.,  5.,  6.]],

[[ 1.,  2.,  3.],
[ 4.,  5.,  6.]],

[[ 1.,  2.,  3.],
[ 4.,  5.,  6.]],

[[ 1.,  2.,  3.],
[ 4.,  5.,  6.]]]], dtype=float32)]
``````

## View

Scala:

``````val view = View(2, 8)
``````

or

``````val view = View(Array(2, 8))
``````

Python:

``````view = View([2, 8])
``````

This module creates a new view of the input tensor using the sizes passed to the constructor. The method setNumInputDims() allows to specify the expected number of dimensions of the inputs of the modules. This makes it possible to use minibatch inputs when using a size -1 for one of the dimensions.

Scala example:

``````import com.intel.analytics.bigdl.nn.View
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat

val view = View(2, 8)

val input = Tensor(4, 4).randn()

val output = view.forward(input)
``````

Gives the output,

``````output: com.intel.analytics.bigdl.tensor.Tensor[Float] =
-0.43037438     1.2982363       -1.4723133      -0.2602826      0.7178128       -1.8763185      0.88629466      0.8346704
0.20963766      -0.9349786      1.0376515       1.3153045       1.5450214       1.084113        -0.29929757     -0.18356979
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x8]
``````

``````gradInput: com.intel.analytics.bigdl.tensor.Tensor[Float] =
0.7360089       0.9133299       0.40443268      -0.94965595
0.80520976      -0.09671917     -0.5498001      -0.098691925
-2.3119886      -0.8455147      0.75891125      1.2985301
0.5023749       1.4983269       0.42038065      -1.7002305
``````

Python example:

``````from bigdl.nn.layer import *
from bigdl.nn.criterion import *
from bigdl.optim.optimizer import *
from bigdl.util.common import *

view = View([2, 8])

input = np.random.uniform(0, 1, [4, 4]).astype("float32")
gradOutput = np.random.uniform(0, 1, [2, 8]).astype("float32")

output = view.forward(input)

print output
``````

## Contiguous

Be used to make input, gradOutput both contiguous

Scala:

``````val contiguous = Contiguous()
``````

Python:

``````contiguous = Contiguous()
``````

Used to make input, gradOutput both contiguous

Scala example:

``````import com.intel.analytics.bigdl.nn.Contiguous
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat

val input = Tensor(5).range(1, 5, 1)
val contiguous = new Contiguous()
val output = contiguous.forward(input)
println(output)

val gradOutput = Tensor(5).range(2, 6, 1)
``````

Gives the output,

``````output: com.intel.analytics.bigdl.tensor.Tensor[Float] =
1.0
2.0
3.0
4.0
5.0
[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 5]
``````

``````gradInput: com.intel.analytics.bigdl.tensor.Tensor[Float] =
2.0
3.0
4.0
5.0
6.0
[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 5]
``````

Python example:

``````from bigdl.nn.layer import *
from bigdl.nn.criterion import *
from bigdl.optim.optimizer import *
from bigdl.util.common import *

contiguous = Contiguous()

input = np.arange(1, 6, 1).astype("float32")
input = input.reshape(1, 5)

output = contiguous.forward(input)
print output

``````

Gives the output,

``````[array([[ 1.,  2.,  3.,  4.,  5.]], dtype=float32)]
``````

``````[array([[ 2.,  3.,  4.,  5.,  6.]], dtype=float32)]
``````

## GaussianSampler

Takes {mean, log_variance} as input and samples from the Gaussian distribution

Scala:

``````val sampler = GaussianSampler()
``````

Python:

``````sampler = GaussianSampler()
``````

Scala example:

``````import com.intel.analytics.bigdl.nn.GaussianSampler
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.utils.RandomGenerator._
import com.intel.analytics.bigdl.utils.T

val input1 = Tensor[Float](2, 3).range(1, 6, 1)
val input2 = Tensor[Float](2, 3).range(1, 12, 2)
val input = T(input1, input2)

val gradOutput = Tensor[Float](2, 3).range(2, 13, 2)

val sampler = new GaussianSampler()
val output = sampler.forward(input)
println(output)

``````

Gives the output,

``````output: com.intel.analytics.bigdl.tensor.Tensor[Float] =
4.507061    9.247583    -14.053247
34.783264   -70.69336   -333.97656
[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 2x3]
``````

``````gradInput: com.intel.analytics.bigdl.utils.Table =
{
1: 2.0  4.0     6.0
8.0  10.0    12.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x3]
2: 3.5070612    14.495168   -51.159744
123.13305    -378.4668   -2039.8594
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x3]
}
``````

Python example:

``````from bigdl.nn.layer import *
from bigdl.nn.criterion import *
from bigdl.optim.optimizer import *
from bigdl.util.common import *
import numpy as np

sampler = GaussianSampler()

input1 = np.arange(1, 7, 1).astype("float32")
input2 = np.arange(1, 12, 2).astype("float32")
input2 = input1.reshape(2, 3)
input2 = input2.reshape(2, 3)
input = [input1, input2]

output = sampler.forward(input)

``````

Gives the output,

``````>>> print output
[[ 1.73362803  2.99371576  0.44359136]
[ 0.04700017  2.85183263  3.04418468]]
``````

``````>>> print gradInput
[array([[  2.,   4.,   6.],
[  8.,  10.,  12.]], dtype=float32), array([[  0.73362803,   1.98743176,  -7.66922569],
[-15.81199932, -10.7408371 , -17.73489189]], dtype=float32)]
``````

Use a mask value to skip timesteps for a sequence

Scala:

``````val mask = Masking(0.0)
``````

Python:

``````mask = Masking(0.0)
``````

Scala example:

``````import com.intel.analytics.bigdl.nn.Masking
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.utils.T

val batchSize = 3
val times = 5
val features = 2
val inputData = Array[Double](1.0, 1, 2, 2, 3, 3, 4, 4, 5, 5, -1, 1, 2, 2, 3, 3, 4, 4, 5, 5,
1, 1, -1, -1, 3, 3, 4, 4, 5, 5)
val input = Tensor[Double](inputData, Array(batchSize, times, features))
val gradOutput = Tensor[Double](Array(batchSize, times, features)).fill(1.0)

println(output)

``````

Gives the output,

``````output: =
(1,.,.) =
1.0 1.0
2.0 2.0
3.0 3.0
4.0 4.0
5.0 5.0

(2,.,.) =
-1.0    1.0
2.0 2.0
3.0 3.0
4.0 4.0
5.0 5.0

(3,.,.) =
1.0 1.0
0.0 0.0
3.0 3.0
4.0 4.0
5.0 5.0

[com.intel.analytics.bigdl.tensor.DenseTensor\$mcD\$sp of size 3x5x2]
``````

``````gradInput:
(1,.,.) =
1.0 1.0
1.0 1.0
1.0 1.0
1.0 1.0
1.0 1.0

(2,.,.) =
1.0 1.0
1.0 1.0
1.0 1.0
1.0 1.0
1.0 1.0

(3,.,.) =
1.0 1.0
0.0 0.0
1.0 1.0
1.0 1.0
1.0 1.0

[com.intel.analytics.bigdl.tensor.DenseTensor\$mcD\$sp of size 3x5x2]
``````

Python example:

``````from bigdl.nn.layer import *
from bigdl.util.common import *
import numpy as np

n_samples = 3
n_timesteps = 7
n_features = 2
input = np.array([1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7, -1, 1, 2, 2, 3, 3, 4, 4, 5, 5, 6, 6, 7, 7,
1, 1, 2, 2, 3, 3, -1, -1, 5, 5, 6, 6, 7, 7]).reshape(n_samples, n_timesteps, n_features)
model = Sequential()

output = model.forward(input)

``````

Gives the output,

``````>>> print output
[[[ 1.  1.]
[ 2.  2.]
[ 3.  3.]
[ 4.  4.]
[ 5.  5.]
[ 6.  6.]
[ 7.  7.]]

[[-1.  1.]
[ 2.  2.]
[ 3.  3.]
[ 4.  4.]
[ 5.  5.]
[ 6.  6.]
[ 7.  7.]]

[[ 1.  1.]
[ 2.  2.]
[ 3.  3.]
[ 0.  0.]
[ 5.  5.]
[ 6.  6.]
[ 7.  7.]]]
``````

``````>>> print gradInput
[[[ 1.  1.]
[ 1.  1.]
[ 1.  1.]
[ 1.  1.]
[ 1.  1.]
[ 1.  1.]
[ 1.  1.]]

[[ 1.  1.]
[ 1.  1.]
[ 1.  1.]
[ 1.  1.]
[ 1.  1.]
[ 1.  1.]
[ 1.  1.]]

[[ 1.  1.]
[ 1.  1.]
[ 1.  1.]
[ 0.  0.]
[ 1.  1.]
[ 1.  1.]
[ 1.  1.]]]
``````

## Maxout ##

Scala:

``````val maxout = Maxout(2, 5, 3,
withBias = true,
wRegularizer = null,
bRegularizer = null,
initWeight = null,
initBias = null)
``````

Python:

``````maxout = Maxout(2, 5, 3,
with_bias = True,
w_regularizer=None,
b_regularizer=None,
init_weight=None,
init_bias=None)
``````

Maxout layer select the element-wise maximum value of maxoutNumber Linear(inputSize, outputSize) layers

parameters: `inputSize` the size the each input sample `outputSize` the size of the module output of each sample `maxoutNumber` number of Linear layers to use `withBias` whether use bias in Linear `wRegularizer` instance of [Regularizer], applied to the input weights matrices. `bRegularizer` instance of [[Regularizer]] applied to the bias. `initWeight` initial weight `initBias` initial bias

Scala example:

``````import com.intel.analytics.bigdl.nn.Maxout
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.utils.T

val input_size = 2
val batch_size = 3
val output_size = 5
val maxout_number = 3

val input = Tensor[Float](batch_size, input_size).rand()
val layer = Maxout[Float](input_size, output_size, maxout_number)
val output = layer.forward(input)
``````

Gives the output,

``````0.19078568  0.94480306  0.25038794  0.8114594   0.7753764
0.2822805   0.9095781   0.2815394   0.82958585  0.784589
0.35188058  0.7629706   0.18096384  0.7100433   0.6680352
[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 3x5]
``````

``````gradInput:
-0.18932924 0.9426162
-0.3118648  0.67255044
-0.31795382 1.944398
[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 3x2]
``````

Python example:

``````from bigdl.nn.layer import *
from bigdl.util.common import *
import numpy as np

val input_size = 2
val batch_size = 3
val output_size = 5
val maxout_number = 3

val input = Tensor[Float](batch_size, input_size).rand()
val layer = Maxout[Float](input_size, output_size, maxout_number)
val output = layer.forward(input)

``````

Gives the output,

``````>>> print output
[[ 0.12344513  0.19081372  0.15130989  0.6341747   0.70982581]
[ 0.04154952 -0.13281995  0.2648508   0.36793122  0.67043799]
[ 0.41355255  0.17691913  0.15496807  0.5880245   0.74583203]]
``````

``````>>> print gradInput
[[ 0.53398496  0.01809531]
[-0.20667852  0.4962275 ]
[ 0.37912956  0.08742841]]
``````

## Cropping2D

Scala:

``````val module = Cropping2D(heightCrop, widthCrop, dataFormat=DataFormat.NCHW)
``````

Python:

``````m = Cropping2D(heightCrop, widthCrop, data_format="NCHW")
``````

Cropping layer for 2D input (e.g. picture). It crops along spatial dimensions, i.e. width and height. # Arguments heightCrop: Array of length 2. How many units should be trimmed off at the beginning and end of the height dimension. widthCrop: Array of length 2. How many units should be trimmed off at the beginning and end of the width dimension dataFormat: DataFormat.NCHW or DataFormat.NHWC. # Input shape 4D tensor with shape: `(samples, depth, first_axis_to_crop, second_axis_to_crop)` # Output shape 4D tensor with shape: `(samples, depth, first_cropped_axis, second_cropped_axis)`

Scala example:

``````
scala >
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._
import com.intel.analytics.bigdl.tensor.Storage

val module = Cropping2D(Array(1, 1), Array(1, 1))
val input = Tensor(2, 1, 3, 3).rand()
val output = module.forward(input)

> input
(1,1,.,.) =
0.024445634 0.73160243  0.1408418
0.95527077  0.51474196  0.89850646
0.3730063   0.40874788  0.7043526

(2,1,.,.) =
0.8549189   0.5019415   0.96255547
0.83960533  0.3738476   0.12785637
0.08048103  0.6209139   0.6762928

[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 2x1x3x3]

> output
(1,1,.,.) =
0.51474196

(2,1,.,.) =
0.3738476

[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x1x1x1]
``````

Python example:

``````from bigdl.nn.layer import *
import numpy as np

input = np.random.rand(2, 1, 3, 3)
print ("input is :",input)

m = Cropping2D([1, 1], [1, 1])
out = m.forward(input)
print ("output m is :",out)
``````

Gives the output,

``````input is : [[[[ 0.54167415  0.59110695  0.583436  ]
[ 0.7401184   0.93070248  0.88311626]
[ 0.08472445  0.90583803  0.83751593]]]

[[[ 0.98047837  0.13156681  0.73104089]
[ 0.15081809  0.1791556   0.18849927]
[ 0.12054713  0.75931796  0.40090047]]]]
creating: createCropping2D
output m is : [[[[ 0.93070251]]]

[[[ 0.1791556 ]]]]
``````

## Cropping3D

Scala:

``````val module = Cropping3D(dim1Crop, dim2Crop, dim3Crop, dataFormat="channel_first")
``````

Python:

``````m = Cropping3D(dim1Crop, dim2Crop, dim3Crop, dataFormat="channel_first")
``````

Cropping layer for 3D data (e.g. spatial or spatio-temporal). # Arguments dim1Crop, dim2Crop, dim3Crop: each is an Array of two int, specifies how many units should be trimmed off at the beginning and end of the 3 cropping dimensions. dataFormat: Cropping3D.CHANNEL_FIRST or Cropping3D.CHANNEL_LAST

Scala example:

``````
scala >
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._
import com.intel.analytics.bigdl.tensor.Storage

val module = Cropping3D(Array(1, 1), Array(1, 1), Array(1, 1))
val input = Tensor(2, 1, 3, 3, 3).rand()
val output = module.forward(input)

> input
(1,1,1,.,.) =
0.33822843  0.83652526  0.6983564
0.40552914  0.50253755  0.26770833
0.12843947  0.7388038   0.8611642

(1,1,2,.,.) =
0.52169484  0.98340595  0.37585744
0.47124776  0.1858571   0.20025288
0.24735944  0.68807006  0.12379094

(1,1,3,.,.) =
0.3149784   0.43712634  0.9625379
0.37466723  0.8551855   0.7831635
0.979082    0.6115703   0.09862939

(2,1,1,.,.) =
0.8603551   0.64941335  0.382916
0.9402129   0.83625364  0.41554055
0.9974375   0.7845985   0.4631692

(2,1,2,.,.) =
0.41448194  0.06975327  0.68035746
0.6495608   0.95513606  0.5103921
0.4187052   0.676009    0.00466285

(2,1,3,.,.) =
0.043842442 0.9419528   0.9560404
0.8702963   0.4117603   0.91820705
0.39294028  0.010171742 0.23027366

[com.intel.analytics.bigdl.tensor.DenseTensor\$mcF\$sp of size 2x1x3x3x3]

> output
(1,1,1,.,.) =
0.1858571

(2,1,1,.,.) =
0.95513606

[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x1x1x1x1]

``````

Python example:

``````from bigdl.nn.layer import *
import numpy as np

input = np.random.rand(2, 1, 3, 3, 3)
print ("input is :",input)

m = Cropping3D([1, 1], [1, 1], [1, 1])
out = m.forward(input)
print ("output m is :",out)
``````

Gives the output,

``````input is : [[[[[ 0.00484727  0.64335228  0.21672991]
[ 0.6667991   0.90280284  0.17537352]
[ 0.17573056  0.51962225  0.7946977 ]]

[[ 0.54374072  0.02084648  0.817017  ]
[ 0.10707117  0.96247797  0.97634706]
[ 0.23012049  0.7498735   0.67309293]]

[[ 0.22704888  0.31254715  0.59703825]
[ 0.61084924  0.55686219  0.55321829]
[ 0.75911533  0.00731942  0.20643018]]]]

[[[[ 0.89015703  0.28932907  0.80356569]
[ 0.55100695  0.66712567  0.00770912]
[ 0.91482596  0.43556021  0.96402856]]

[[ 0.36694364  0.27634374  0.52885899]
[ 0.40754185  0.79033726  0.42423772]
[ 0.20636923  0.72467024  0.80372414]]

[[ 0.50318154  0.54954067  0.71939314]
[ 0.52834256  0.26762247  0.32269808]
[ 0.53824181  0.42523858  0.95246198]]]]]
creating: createCropping3D
output m is : [[[[[ 0.96247798]]]]

[[[[ 0.79033726]]]]]
``````