Merge/Split Layers
Pack
Scala:
val module = Pack(dim)
Python:
module = Pack(dim)
Pack is used to stack a list of n-dimensional tensors into one (n+1)-dimensional tensor.
Scala example:
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.utils.T
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
val module = Pack(2)
val input1 = Tensor(2, 2).randn()
val input2 = Tensor(2, 2).randn()
val input = T()
input(1) = input1
input(2) = input2
val output = module.forward(input)
> input
{
2: -0.8737048 -0.7337217
0.7268678 -0.53470045
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 2x2]
1: -1.3062215 -0.58756566
0.8921608 -1.8087773
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 2x2]
}
> output
(1,.,.) =
-1.3062215 -0.58756566
-0.8737048 -0.7337217
(2,.,.) =
0.8921608 -1.8087773
0.7268678 -0.53470045
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x2x2]
Python example:
from bigdl.nn.layer import *
import numpy as np
module = Pack(2)
input1 = np.random.randn(2, 2)
input2 = np.random.randn(2, 2)
input = [input1, input2]
output = module.forward(input)
> input
[array([[ 0.92741416, -3.29826586],
[-0.03147819, -0.10049306]]), array([[-0.27146461, -0.25729802],
[ 0.1316149 , 1.27620145]])]
> output
array([[[ 0.92741418, -3.29826593],
[-0.27146462, -0.25729802]],
[[-0.03147819, -0.10049306],
[ 0.13161489, 1.27620149]]], dtype=float32)
MM
Scala:
val m = MM(transA=false,transB=false)
Python:
m = MM(trans_a=False,trans_b=False)
MM is a module that performs matrix multiplication on two mini-batch inputs, producing one mini-batch.
Scala example:
scala>
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._
import com.intel.analytics.bigdl.utils.T
val input = T(1 -> Tensor(3, 3).randn(), 2 -> Tensor(3, 3).randn())
val m1 = MM()
val output1 = m1.forward(input)
val m2 = MM(true,true)
val output2 = m2.forward(input)
scala> print(input)
{
2: -0.62020904 -0.18690863 0.34132162
-0.5359324 -0.09937895 0.86147165
-2.6607985 -1.426654 2.3428898
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3x3]
1: -1.3087689 0.048720464 0.69583243
-0.52055264 -1.5275089 -1.1569321
0.28093573 -0.29353273 -0.9505267
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3x3]
}
scala> print(output1)
-1.0658705 -0.7529337 1.225519
4.2198563 1.8996398 -4.204146
2.512235 1.3327343 -2.38396
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3x3]
scala> print(output2)
1.0048954 0.99516183 4.8832207
0.15509865 -0.12717877 1.3618765
-0.5397563 -1.0767963 -2.4279075
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3x3]
Python example:
from bigdl.nn.layer import *
import numpy as np
input1=np.random.rand(3,3)
input2=np.random.rand(3,3)
input = [input1,input2]
print "input is :",input
out = MM().forward(input)
print "output is :",out
produces output:
input is : [array([[ 0.13696046, 0.92653165, 0.73585328],
[ 0.28167852, 0.06431783, 0.15710073],
[ 0.21896166, 0.00780161, 0.25780671]]), array([[ 0.11232797, 0.17023931, 0.92430042],
[ 0.86629537, 0.07630215, 0.08584417],
[ 0.47087278, 0.22992833, 0.59257503]])]
creating: createMM
output is : [array([[ 1.16452789, 0.26320592, 0.64217824],
[ 0.16133308, 0.08898225, 0.35897085],
[ 0.15274818, 0.09714822, 0.3558259 ]], dtype=float32)]
CMaxTable
Scala:
val m = CMaxTable()
Python:
m = CMaxTable()
CMaxTable is a module that takes a table of Tensors and outputs the max of all of them.
Scala example:
scala>
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._
import com.intel.analytics.bigdl.utils.T
val input1 = Tensor(3).randn()
val input2 = Tensor(3).randn()
val input = T(input1, input2)
val m = CMaxTable()
val output = m.forward(input)
val gradOut = Tensor(3).randn()
val gradIn = m.backward(input,gradOut)
scala> print(input)
{
2: -0.38613814
0.74074316
-1.753783
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3]
1: -1.6037064
-2.3297918
-0.7160026
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3]
}
scala> print(output)
-0.38613814
0.74074316
-0.7160026
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
scala> print(gradOut)
-1.4526331
0.7070323
0.29294914
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3]
scala> print(gradIn)
{
2: -1.4526331
0.7070323
0.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
1: 0.0
0.0
0.29294914
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
}
Python example:
from bigdl.nn.layer import *
import numpy as np
input1 = np.random.rand(3)
input2 = np.random.rand(3)
print "input is :",input1,input2
m = CMaxTable()
out = m.forward([input1,input2])
print "output of m is :",out
grad_out = np.random.rand(3)
grad_in = m.backward([input1, input2],grad_out)
print "grad input of m is :",grad_in
Gives the output,
input is : [ 0.48649797 0.22131348 0.45667796] [ 0.73207053 0.74290136 0.03169769]
creating: createCMaxTable
output of m is : [array([ 0.73207051, 0.74290138, 0.45667794], dtype=float32)]
grad input of m is : [array([ 0. , 0. , 0.86938971], dtype=float32), array([ 0.04140199, 0.4787094 , 0. ], dtype=float32)]
SplitTable
Scala:
val layer = SplitTable(dim)
Python:
layer = SplitTable(dim)
SplitTable takes a Tensor as input and outputs several tables,
splitting the Tensor along the specified dimension dimension. Please note
the dimension starts from 1.
The input to this layer is expected to be a tensor, or a batch of tensors;
when using mini-batch, a batch of sample tensors will be passed to the layer and
the user needs to specify the number of dimensions of each sample tensor in a
batch using nInputDims.
+----------+ +-----------+
| input[1] +---------> {member1, |
+----------+-+ | |
| input[2] +-----------> member2, |
+----------+-+ | |
| input[3] +-------------> member3} |
+----------+ +-----------+
Scala example:
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.utils.T
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
val layer = SplitTable(2)
layer.forward(Tensor(T(
T(1.0f, 2.0f, 3.0f),
T(4.0f, 5.0f, 6.0f),
T(7.0f, 8.0f, 9.0f)
)))
layer.backward(Tensor(T(
T(1.0f, 2.0f, 3.0f),
T(4.0f, 5.0f, 6.0f),
T(7.0f, 8.0f, 9.0f)
)), T(
Tensor(T(0.1f, 0.2f, 0.3f)),
Tensor(T(0.4f, 0.5f, 0.6f)),
Tensor(T(0.7f, 0.8f, 0.9f))
))
Gives the output,
{
2: 2.0
5.0
8.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
1: 1.0
4.0
7.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
3: 3.0
6.0
9.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
}
0.1 0.4 0.7
0.2 0.5 0.8
0.3 0.6 0.9
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3x3]
Python example:
from bigdl.nn.layer import SplitTable
import numpy as np
layer = SplitTable(2)
layer.forward(np.array([
[1.0, 2.0, 3.0],
[4.0, 5.0, 6.0],
[7.0, 8.0, 9.0]
]))
layer.backward(np.array([
[1.0, 2.0, 3.0],
[4.0, 5.0, 6.0],
[7.0, 8.0, 9.0]
]), [
np.array([0.1, 0.2, 0.3]),
np.array([0.4, 0.5, 0.6]),
np.array([0.7, 0.8, 0.9])
])
Gives the output,
[
array([ 1., 4., 7.], dtype=float32),
array([ 2., 5., 8.], dtype=float32),
array([ 3., 6., 9.], dtype=float32)
]
array([[ 0.1 , 0.40000001, 0.69999999],
[ 0.2 , 0.5 , 0.80000001],
[ 0.30000001, 0.60000002, 0.89999998]], dtype=float32)
DotProduct
Scala:
val m = DotProduct()
Python:
m = DotProduct()
Outputs the dot product (similarity) between inputs
Scala example:
import com.intel.analytics.bigdl.utils._
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.utils.{T, Table}
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
val mlp = DotProduct()
val x = Tensor(3).fill(1f)
val y = Tensor(3).fill(2f)
println("input:")
println(x)
println(y)
println("output:")
println(mlp.forward(T(x, y)))
input:
1.0
1.0
1.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
2.0
2.0
2.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
output:
6.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 1]
Python example:
import numpy as np
from bigdl.nn.layer import *
mlp = DotProduct()
x = np.array([1, 1, 1])
y = np.array([2, 2, 2])
print("input:")
print(x)
print(y)
print("output:")
print(mlp.forward([x, y]))
creating: createDotProduct
input:
[1 1 1]
[2 2 2]
output:
[ 6.]
CSubTable
Scala:
val model = CSubTable()
Python:
model = CSubTable()
Takes a sequence with two Tensor and returns the component-wise subtraction between them.
Scala example:
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.utils.T
val model = CSubTable()
val input1 = Tensor(5).rand()
val input2 = Tensor(5).rand()
val input = T(input1, input2)
val output = model.forward(input)
scala> print(input)
{
2: 0.29122078
0.17347474
0.14127742
0.2249051
0.12171601
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 5]
1: 0.6202152
0.70417005
0.21334995
0.05191216
0.4209623
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 5]
scala> print(output)
0.3289944
0.5306953
0.072072536
-0.17299294
0.2992463
[com.intel.analytics.bigdl.tensor.DenseTensor of size 5]
Python example:
model = CSubTable()
input1 = np.random.randn(5)
input2 = np.random.randn(5)
input = [input1, input2]
output = model.forward(input)
Gives the output,
array([-1.15087152, 0.6169951 , 2.41840839, 1.34374809, 1.39436531], dtype=float32)
CDivTable
Scala:
val module = CDivTable()
Python:
module = CDivTable()
Takes a table with two Tensor and returns the component-wise division between them.
Scala example:
val module = CDivTable()
val input = T(1 -> Tensor(2,3).rand(), 2 -> Tensor(2,3).rand())
input: com.intel.analytics.bigdl.utils.Table =
{
2: 0.802295 0.7113872 0.29395157
0.6562403 0.06519115 0.20099664
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 2x3]
1: 0.7435388 0.59126955 0.10225375
0.46819785 0.10572237 0.9861797
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 2x3]
}
module.forward(input)
res6: com.intel.analytics.bigdl.tensor.Tensor[Float] =
0.9267648 0.8311501 0.34785917
0.7134549 1.6217289 4.906449
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x3]
Python example:
module = CDivTable()
input = [np.array([[1, 2, 3],[4, 5, 6]]), np.array([[1, 4, 9],[6, 10, 3]])]
module.forward(input)
[array([
[ 1., 0.5 , 0.33333334],
[ 0.66666669, 0.5 , 2. ]], dtype=float32)]
JoinTable
Scala:
val layer = JoinTable(dimension, nInputDims)
Python:
layer = JoinTable(dimension, n_input_dims)
It is a table module which takes a table of Tensors as input and
outputs a Tensor by joining them together along the dimension dimension.
The input to this layer is expected to be a tensor, or a batch of tensors;
when using mini-batch, a batch of sample tensors will be passed to the layer and
the user need to specify the number of dimensions of each sample tensor in the
batch using nInputDims.
Parameters:
dimension to be join in this dimension
nInputDims specify the number of dimensions that this module will receiveIf it is more than the dimension of input tensors, the first dimension would be considered as batch size
+----------+ +-----------+
| {input1, +-------------> output[1] |
| | +-----------+-+
| input2, +-----------> output[2] |
| | +-----------+-+
| input3} +---------> output[3] |
+----------+ +-----------+
Scala example:
import com.intel.analytics.bigdl.nn.JoinTable
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.numeric.NumericFloat
import com.intel.analytics.bigdl.utils.T
val layer = JoinTable(2, 2)
val input1 = Tensor(T(
T(
T(1f, 2f, 3f),
T(2f, 3f, 4f),
T(3f, 4f, 5f))
))
val input2 = Tensor(T(
T(
T(3f, 4f, 5f),
T(2f, 3f, 4f),
T(1f, 2f, 3f))
))
val input = T(input1, input2)
val gradOutput = Tensor(T(
T(
T(1f, 2f, 3f, 3f, 4f, 5f),
T(2f, 3f, 4f, 2f, 3f, 4f),
T(3f, 4f, 5f, 1f, 2f, 3f)
)))
val output = layer.forward(input)
val grad = layer.backward(input, gradOutput)
println(output)
(1,.,.) =
1.0 2.0 3.0 3.0 4.0 5.0
2.0 3.0 4.0 2.0 3.0 4.0
3.0 4.0 5.0 1.0 2.0 3.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 1x3x6]
println(grad)
{
2: (1,.,.) =
3.0 4.0 5.0
2.0 3.0 4.0
1.0 2.0 3.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 1x3x3]
1: (1,.,.) =
1.0 2.0 3.0
2.0 3.0 4.0
3.0 4.0 5.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 1x3x3]
}
Python example:
layer = JoinTable(2, 2)
input1 = np.array([
[
[1.0, 2.0, 3.0],
[2.0, 3.0, 4.0],
[3.0, 4.0, 5.0]
]
])
input2 = np.array([
[
[3.0, 4.0, 5.0],
[2.0, 3.0, 4.0],
[1.0, 2.0, 3.0]
]
])
input = [input1, input2]
gradOutput = np.array([
[
[1.0, 2.0, 3.0, 3.0, 4.0, 5.0],
[2.0, 3.0, 4.0, 2.0, 3.0, 4.0],
[3.0, 4.0, 5.0, 1.0, 2.0, 3.0]
]
])
output = layer.forward(input)
grad = layer.backward(input, gradOutput)
print output
[[[ 1. 2. 3. 3. 4. 5.]
[ 2. 3. 4. 2. 3. 4.]
[ 3. 4. 5. 1. 2. 3.]]]
print grad
[array([[[ 1., 2., 3.],
[ 2., 3., 4.],
[ 3., 4., 5.]]], dtype=float32), array([[[ 3., 4., 5.],
[ 2., 3., 4.],
[ 1., 2., 3.]]], dtype=float32)]
SelectTable
Scala:
val m = SelectTable(index: Int)
Python:
m = SelectTable(dimension)
Select one element from a table by a given index. In Scala API, table is kind of like HashMap with one-base index as the key. In python, table is a just a list.
Scala example:
import com.intel.analytics.bigdl.utils._
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.utils.{T, Table}
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
val input = T(Tensor(2,3).randn(), Tensor(2,3).randn())
println("input: ")
println(input)
println("output:")
println(SelectTable(1).forward(input)) // Select and output the first element of the input which shape is (2, 3)
println(SelectTable(2).forward(input)) // Select and output the second element of the input which shape is (2, 3)
input:
{
2: 2.005436370849835 0.09670211785545313 1.186779895312918
2.238415300857082 0.241626512721254 0.15765709974113828
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x3]
1: 0.5668905654052705 -1.3205159007397167 -0.5431464848526197
-0.11582559521074104 0.7671830693813515 -0.39992781407893574
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x3]
}
output:
0.5668905654052705 -1.3205159007397167 -0.5431464848526197
-0.11582559521074104 0.7671830693813515 -0.39992781407893574
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x3]
2.005436370849835 0.09670211785545313 1.186779895312918
2.238415300857082 0.241626512721254 0.15765709974113828
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x3]
Python example:
import numpy as np
from bigdl.nn.layer import *
input = [np.random.random((2,3)), np.random.random((2, 1))]
print("input:")
print(input)
print("output:")
print(SelectTable(1).forward(input)) # Select and output the first element of the input which shape is (2, 3)
input:
[array([[ 0.07185111, 0.26140439, 0.9437582 ],
[ 0.50278191, 0.83923974, 0.06396735]]), array([[ 0.84955122],
[ 0.16053703]])]
output:
creating: createSelectTable
[[ 0.07185111 0.2614044 0.94375819]
[ 0.50278193 0.83923972 0.06396735]]
NarrowTable
Scala:
val narrowTable = NarrowTable(offset, length = 1)
Python:
narrowTable = NarrowTable(offset, length = 1)
NarrowTable takes a table as input and returns a subtable starting from index offset having length elements
Negative length means the last element is located at Abs|length| to the last element of input
Scala example:
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._
import com.intel.analytics.bigdl.utils.T
val narrowTable = NarrowTable(1, 1)
val input = T()
input(1.0) = Tensor(2, 2).rand()
input(2.0) = Tensor(2, 2).rand()
input(3.0) = Tensor(2, 2).rand()
> print(input)
{
2.0: 0.27686104 0.9040761
0.75969505 0.8008061
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 2x2]
1.0: 0.94122535 0.46173728
0.43302807 0.1670979
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 2x2]
3.0: 0.43944374 0.49336782
0.7274511 0.67777634
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 2x2]
}
> print(narrowTable.forward(input))
{
1: 0.94122535 0.46173728
0.43302807 0.1670979
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 2x2]
}
Python example:
from bigdl.nn.layer import *
narrowTable = NarrowTable(1, 1)
> narrowTable.forward([np.array([1, 2, 3]), np.array([4, 5, 6])])
[array([ 1., 2., 3.], dtype=float32)]
CAddTable
Scala:
val module = CAddTable(inplace = false)
Python:
module = CAddTable(inplace=False)
CAddTable merges the input tensors in the input table by element-wise adding. The input table is actually an array of tensor with same size.
Scala example:
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._
val mlp = Sequential()
mlp.add(ConcatTable().add(Identity()).add(Identity()))
mlp.add(CAddTable())
println(mlp.forward(Tensor.range(1, 3, 1)))
Gives the output,
com.intel.analytics.bigdl.nn.abstractnn.Activity =
2.0
4.0
6.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
Python example:
from bigdl.nn.layer import *
import numpy as np
mlp = Sequential()
mlp.add(ConcatTable().add(Identity()).add(Identity()))
mlp.add(CAddTable())
print(mlp.forward(np.arange(1, 4, 1)))
Gives the output,
[array([ 2., 4., 6.], dtype=float32)]
CAveTable
Scala:
val model = CAveTable(inplace=false)
Python:
model = CAveTable(inplace=False)
CAveTable merges the input tensors in the input table by element-wise taking the average. The input table is actually an array of tensor with same size.
Scala example:
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._
import com.intel.analytics.bigdl.utils.T
val model = CAveTable()
val input1 = Tensor(5).rand()
val input2 = Tensor(5).rand()
val input = T(input1, input2)
val output = model.forward(input)
Gives the output,
input1: com.intel.analytics.bigdl.tensor.Tensor[Float] =
0.6061657
0.55972266
0.972365
0.5624792
0.7495829
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 5]
input2: com.intel.analytics.bigdl.tensor.Tensor[Float] =
0.3897284
0.82165825
0.46275142
0.95935726
0.64157426
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 5]
output: com.intel.analytics.bigdl.tensor.Tensor[Float] =
0.49794704
0.69069046
0.7175582
0.76091826
0.6955786
[com.intel.analytics.bigdl.tensor.DenseTensor of size 5]
Python example:
from bigdl.nn.layer import *
import numpy as np
model = CAveTable()
input1 = np.random.rand(5)
input2 = np.random.rand(5)
input = [input1, input2]
output = model.forward(input)
print(input1)
print(input2)
print(output)
Gives the output,
[array([ 0.26202468 0.15868397 0.27812652 0.45931689 0.32100054], dtype=float32)]
[array([ 0.51839282 0.26194293 0.97608528 0.73281455 0.11527423], dtype=float32)]
[array([ 0.39020872 0.21031344 0.62710589 0.5960657 0.21813738], dtype=float32)]
CMulTable
Scala:
val model = CMulTable()
Python:
model = CMulTable()
Takes a sequence of Tensors and outputs the multiplication of all of them.
Scala example:
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.utils.T
val model = CMulTable()
val input1 = Tensor(5).rand()
val input2 = Tensor(5).rand()
val input = T(input1, input2)
val output = model.forward(input)
scala> print(input)
{
2: 0.13224044
0.5460452
0.33032498
0.6317603
0.6665052
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 5]
1: 0.28694472
0.45169437
0.36891535
0.9126049
0.41318864
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 5]
}
scala> print(output)
0.037945695
0.24664554
0.12186196
0.57654756
0.27539238
[com.intel.analytics.bigdl.tensor.DenseTensor of size 5]
Python example:
model = CMulTable()
input1 = np.random.randn(5)
input2 = np.random.randn(5)
input = [input1, input2]
output = model.forward(input)
>>> print(input)
[array([ 0.28183274, -0.6477487 , -0.21279841, 0.22725124, 0.54748552]), array([-0.78673028, -1.08337196, -0.62710066, 0.37332587, -1.40708162])]
>>> print(output)
[-0.22172636 0.70175284 0.13344601 0.08483877 -0.77035683]
MV
Scala:
val module = MV(trans = false)
Python:
module = MV(trans=False)
It is a module to perform matrix vector multiplication on two mini-batch inputs, producing a mini-batch.
trans means whether make matrix transpose before multiplication.
Scala example:
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor._
import com.intel.analytics.bigdl.utils.T
val module = MV()
println(module.forward(T(Tensor.range(1, 12, 1).resize(2, 2, 3), Tensor.range(1, 6, 1).resize(2, 3))))
Gives the output,
com.intel.analytics.bigdl.tensor.Tensor[Float] =
14.0 32.0
122.0 167.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 2x2]
Python example:
module = MV()
print(module.forward([np.arange(1, 13, 1).reshape(2, 2, 3), np.arange(1, 7, 1).reshape(2, 3)]))
Gives the output,
[array([ 0.31657887, -1.11062765, -1.16235781, -0.67723978, 0.74650359], dtype=float32)]
FlattenTable
Scala:
val module = FlattenTable()
Python:
module = FlattenTable()
FlattenTable takes an arbitrarily deep table of Tensors (potentially nested) as input and a table of Tensors without any nested table will be produced
Scala example:
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.utils.T
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
val module = FlattenTable()
val t1 = Tensor(3).randn()
val t2 = Tensor(3).randn()
val t3 = Tensor(3).randn()
val input = T(t1, T(t2, T(t3)))
val output = module.forward(input)
> input
{
2: {
2: {
1: 0.5521984
-0.4160644
-0.698762
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3]
}
1: -1.7380241
0.60336906
-0.8751049
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3]
}
1: 1.0529885
-0.792229
0.8395628
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3]
}
> output
{
2: -1.7380241
0.60336906
-0.8751049
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3]
1: 1.0529885
-0.792229
0.8395628
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3]
3: 0.5521984
-0.4160644
-0.698762
[com.intel.analytics.bigdl.tensor.DenseTensor$mcF$sp of size 3]
}
Python example:
from bigdl.nn.layer import *
import numpy as np
module = Sequential()
# this will create a nested table
nested = ConcatTable().add(Identity()).add(Identity())
module.add(nested).add(FlattenTable())
t1 = np.random.randn(3)
t2 = np.random.randn(3)
input = [t1, t2]
output = module.forward(input)
> input
[array([-2.21080689, -0.48928043, -0.26122161]), array([-0.8499716 , 1.63694575, -0.31109292])]
> output
[array([-2.21080685, -0.48928043, -0.26122162], dtype=float32),
array([-0.84997159, 1.63694572, -0.31109291], dtype=float32),
array([-2.21080685, -0.48928043, -0.26122162], dtype=float32),
array([-0.84997159, 1.63694572, -0.31109291], dtype=float32)]
CMinTable
Scala:
val layer = CMinTable()
Python:
layer = CMinTable()
CMinTable takes a bunch of tensors as inputs. These tensors must have same shape. This layer will merge them by doing an element-wise comparision and use the min value.
Scala example:
import com.intel.analytics.bigdl.nn._
import com.intel.analytics.bigdl.utils.T
import com.intel.analytics.bigdl.tensor.Tensor
import com.intel.analytics.bigdl.tensor.TensorNumericMath.TensorNumeric.NumericFloat
val layer = CMinTable()
layer.forward(T(
Tensor(T(1.0f, 5.0f, 2.0f)),
Tensor(T(3.0f, 4.0f, -1.0f)),
Tensor(T(5.0f, 7.0f, -5.0f))
))
layer.backward(T(
Tensor(T(1.0f, 5.0f, 2.0f)),
Tensor(T(3.0f, 4.0f, -1.0f)),
Tensor(T(5.0f, 7.0f, -5.0f))
), Tensor(T(0.1f, 0.2f, 0.3f)))
Gives the output,
1.0
4.0
-5.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
{
2: 0.0
0.2
0.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
1: 0.1
0.0
0.0
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
3: 0.0
0.0
0.3
[com.intel.analytics.bigdl.tensor.DenseTensor of size 3]
}
Python example:
from bigdl.nn.layer import CMinTable
import numpy as np
layer = CMinTable()
layer.forward([
np.array([1.0, 5.0, 2.0]),
np.array([3.0, 4.0, -1.0]),
np.array([5.0, 7.0, -5.0])
])
layer.backward([
np.array([1.0, 5.0, 2.0]),
np.array([3.0, 4.0, -1.0]),
np.array([5.0, 7.0, -5.0])
], np.array([0.1, 0.2, 0.3]))
Gives the output,
array([ 1., 4., -5.], dtype=float32)
[array([ 0.1, 0., 0.], dtype=float32),
array([ 0., 0.2, 0.], dtype=float32),
array([ 0., 0., 0.30000001], dtype=float32)]