bigdl.util package

Submodules

bigdl.util.common module

class bigdl.util.common.JTensor(storage, shape, bigdl_type='float')

Bases: object

A wrapper to easy our work when need to pass or return Tensor to/from Scala.

>>> import numpy as np
>>> from bigdl.util.common import JTensor
>>> np.random.seed(123)
>>>

classmethod flatten_ndarray(a_ndarray)

Utility method to flatten a ndarray

Returns:(storage, shape)

>>> from bigdl.util.common import JTensor
>>> np.random.seed(123)
>>> data = np.random.uniform(0, 1, (2, 3))
>>> (storage, shape) = JTensor.flatten_ndarray(data)
>>> shape
[2, 3]
>>> (storage, shape) = JTensor.flatten_ndarray(np.array(2))
>>> shape
[1]

classmethod from_ndarray(a_ndarray, bigdl_type='float')

Convert a ndarray to Tensor which would be used in Java side.

>>> import numpy as np
>>> from bigdl.util.common import JTensor
>>> from bigdl.util.common import callBigDlFunc
>>> np.random.seed(123)
>>> data = np.random.uniform(0, 1, (2, 3)).astype("float32")
>>> result = JTensor.from_ndarray(data)
>>> data_back = result.to_ndarray()
>>> (data == data_back).all()
True
>>> tensor1 = callBigDlFunc("float", "testTensor", JTensor.from_ndarray(data))  # noqa
>>> array_from_tensor = tensor1.to_ndarray()
>>> (array_from_tensor == data).all()
True

to_ndarray()

class bigdl.util.common.JavaCreator(bigdl_type)

Bases: bigdl.util.common.SingletonMixin

classmethod get_creator_class()

classmethod set_creator_class(cclass)

class bigdl.util.common.JavaValue(jvalue, bigdl_type, *args)

Bases: object

jvm_class_constructor()

class bigdl.util.common.RNG(bigdl_type='float')

generate tensor data with seed

set_seed(seed)

uniform(a, b, size)

class bigdl.util.common.Sample(features, label, features_shape, label_shape, bigdl_type='float')

Bases: object

classmethod from_ndarray(features, label, bigdl_type='float')

class bigdl.util.common.SingletonMixin

Bases: object

classmethod instance(bigdl_type='float')

class bigdl.util.common.TestResult(result, total_num, method)

A testing result used to benchmark the model quality.

bigdl.util.common.callBigDlFunc(bigdl_type, name, *args)

Call API in PythonBigDL

bigdl.util.common.callJavaFunc(sc, func, *args)

Call Java Function

bigdl.util.common.create_spark_conf()

bigdl.util.common.get_bigdl_conf()

bigdl.util.common.get_spark_context(conf=None)

Get the current active spark context and create one if no active instance :param conf: combining bigdl configs into spark conf :return: SparkContext

bigdl.util.common.get_spark_sql_context(sc)

bigdl.util.common.init_engine(bigdl_type='float')

bigdl.util.common.to_list(a)

bigdl.util.engine module

bigdl.util.engine.prepare_env()

bigdl.util.tf_utils module

Module contents