dataset

Type Members

1. trait AbstractDataSet[D, DataSequence] extends AnyRef

   A set of data which is used in the model optimization process.

2. class ArraySample[T] extends Sample[T]

   A kind of sample who use only one array

3. case class ByteRecord(data: Array[Byte], label: Float) extends Product with Serializable

   A byte array and a label.

4. class CachedDistriDataSet[T] extends DistributedDataSet[T]

   Wrap a RDD as a DataSet.

5. class ChainedTransformer[A, B, C] extends Transformer[A, C]

   A transformer chain two transformer together.

6. class DefaultPadding extends PaddingStrategy

7. trait DistributedDataSet[T] extends AbstractDataSet[T, RDD[T]]

   Represent a distributed data.

8. case class FixedLength(fixedLength: Array[Int]) extends PaddingStrategy with Product with Serializable

   Set the first dimension's length to fixed length.

9. class Identity[A] extends Transformer[A, A]

   Just transform the input to output.

10. abstract class Image extends Serializable

    Represent an image

11. trait Label[T] extends AnyRef

    Represent a label

12. class LocalArrayDataSet[T] extends LocalDataSet[T]

    Wrap an array as a DataSet.

13. trait LocalDataSet[T] extends AbstractDataSet[T, Iterator[T]]

    Manage some 'local' data, e.

14. class LocalImagePath extends AnyRef

    Represent a local file path of an image file

15. case class LocalSeqFilePath(path: Path) extends Product with Serializable

    Represent a local file path of a hadoop sequence file

16. trait MiniBatch[T] extends Serializable

    A interface for MiniBatch.

17. case class PaddingLongest(paddingLength: Array[Int]) extends PaddingStrategy with Product with Serializable

    Add an constant length to longest feature in the first dimension

18. case class PaddingParam[T](paddingTensor: Option[Array[Tensor[T]]] = scala.None, paddingStrategy: PaddingStrategy = new DefaultPadding())(implicit evidence$14: ClassTag[T]) extends Serializable with Product

    Feature Padding param for MiniBatch.

19. abstract class PaddingStrategy extends Serializable

20. abstract class Sample[T] extends Serializable

    Class that represents the features and labels of a data sample.

21. class SampleToMiniBatch[T] extends Transformer[Sample[T], MiniBatch[T]]

    Convert a sequence of Sample to a sequence of MiniBatch through function toMiniBatch.

22. abstract class Sentence[T] extends Serializable

    Represent a sentence

23. class SparseMiniBatch[T] extends ArrayTensorMiniBatch[T]

    SparseMiniBatch is a MiniBatch type for TensorSample.

24. class TensorSample[T] extends Sample[T]

    A kind of Sample who hold both DenseTensor and SparseTensor as features.

25. trait Transformer[A, B] extends Serializable

    Transform a data stream of type A to type B.

26. class SampleToBatch[T] extends Transformer[Sample[T], MiniBatch[T]]

    Convert a sequence of single-feature and single-label Sample to a sequence of MiniBatch, optionally padding all the features (or labels) in the mini-batch to the same length