bigdl.util package

Submodules

bigdl.util.common module

class bigdl.util.common.JTensor(storage, shape, bigdl_type='float')

Bases: object

A wrapper to easy our work when need to pass or return Tensor to/from Scala.

>>> import numpy as np
>>> from bigdl.util.common import JTensor
>>> np.random.seed(123)
>>>

classmethod from_ndarray(a_ndarray, bigdl_type='float')

Convert a ndarray to Tensor which would be used in Java side.

>>> import numpy as np
>>> from bigdl.util.common import JTensor
>>> from bigdl.util.common import callBigDlFunc
>>> np.random.seed(123)
>>> data = np.random.uniform(0, 1, (2, 3)).astype("float32")
>>> result = JTensor.from_ndarray(data)
>>> data_back = result.to_ndarray()
>>> (data == data_back).all()
True
>>> tensor1 = callBigDlFunc("float", "testTensor", JTensor.from_ndarray(data))  # noqa
>>> array_from_tensor = tensor1.to_ndarray()
>>> (array_from_tensor == data).all()
True

to_ndarray()

class bigdl.util.common.JavaCreator(bigdl_type)

Bases: bigdl.util.common.SingletonMixin

classmethod get_creator_class()

classmethod set_creator_class(cclass)

class bigdl.util.common.JavaValue(jvalue, bigdl_type, *args)

Bases: object

jvm_class_constructor()

class bigdl.util.common.RNG(bigdl_type='float')

generate tensor data with seed

set_seed(seed)

uniform(a, b, size)

class bigdl.util.common.Sample(features, label, bigdl_type='float')

Bases: object

classmethod from_ndarray(features, label, bigdl_type='float')

Convert a ndarray of features and label to Sample, which would be used in Java side.

>>> import numpy as np
>>> from bigdl.util.common import callBigDlFunc
>>> from bigdl.util.common import Sample
>>> from numpy.testing import assert_allclose
>>> sample = Sample.from_ndarray(np.random.random((2,3)), np.random.random((2,3)))
>>> sample_back = callBigDlFunc("float", "testSample", sample)
>>> assert_allclose(sample.features.to_ndarray(), sample_back.features.to_ndarray())
>>> assert_allclose(sample.label.to_ndarray(), sample_back.label.to_ndarray())

class bigdl.util.common.SingletonMixin

Bases: object

classmethod instance(bigdl_type='float')

class bigdl.util.common.TestResult(result, total_num, method)

A testing result used to benchmark the model quality.

bigdl.util.common.callBigDlFunc(bigdl_type, name, *args)

Call API in PythonBigDL

bigdl.util.common.callJavaFunc(sc, func, *args)

Call Java Function

bigdl.util.common.create_spark_conf()

bigdl.util.common.get_bigdl_conf()

bigdl.util.common.get_dtype(bigdl_type)

bigdl.util.common.get_spark_context(conf=None)

Get the current active spark context and create one if no active instance :param conf: combining bigdl configs into spark conf :return: SparkContext

bigdl.util.common.get_spark_sql_context(sc)

bigdl.util.common.init_engine(bigdl_type='float')

bigdl.util.common.to_list(a)

bigdl.util.engine module

bigdl.util.engine.prepare_env()

bigdl.util.tf_utils module

Module contents