For Python users, BigDL supports loading pre-defined Keras models.
The Keras version we support and test is Keras 1.2.2 with TensorFlow backend. Up to now, we have generally supported ALL its layers.
After loading a model into BigDL, you can train, evaluate or tune this model in a distributed manner. We have generally supported ALL the losses in Keras 1.2.2. See here to find the corresponding criterions in BigDL.
Load a Keras model into BigDL
A Keras model definition in JSON file can be loaded as a BigDL model. Saved weights in HDF5 file can also be loaded together with the architecture of a Keras model. See here on how to save a model in Keras.
You can directly call the API
Model.load_keras to load a Keras model into BigDL.
keras==1.2.2 is required beforehand. If you are to load a HDF5 file, you also need to install
h5py. These packages can be installed via
from bigdl.nn.layer import * bigdl_model = Model.load_keras(json_path=None, hdf5_path=None, by_name=False)
json_pathThe JSON file path containing the Keras model definition to be loaded. Default to be
Noneif you choose to load a Keras model from a HDF5 file.
hdf5_pathThe HDF5 file path containing the pre-trained weights with or without the model architecture. Please use weights from Keras 1.2.2 with
tensorflow backend. Default to be
Noneif you choose to only load the model definition from JSON but not to load weights. In this case, BigDL will use initialized weights for the model.
by_nameWhether to load the weights of layers by name. Use this option only when you provide a HDF5 file. Default to be
False, meaning that weights are loaded based on the network's execution order topology. Otherwise, if it is set to be
True, only those layers with the same name will be loaded with weights.
Please provide either
hdf5_pathwhen you call
Model.load_keras. You can provide
json_pathonly to just load the model definition. You can provide
hdf5_pathtogether if you have separate files for the model architecture and its pre-trained weights. Also, if you save the model architecture and its weights in a single HDF5 file, you can provide
JSON and HDF5 files can be loaded from any Hadoop-supported file system URI. For example,
# load from local file system bigdl_model = Model.load_keras(json_path="/tmp/model.json", hdf5_path="/tmp/weights.h5") # load from HDFS bigdl_model = Model.load_keras(hdf5_path="hdfs://model.h5") # load from S3 bigdl_model = Model.load_keras(hdf5_path="s3://model.h5")
Here we show a simple example on how to load a Keras model into BigDL. The model used in this example is a CNN from Keras 1.2.2.
# Define a CNN model in Keras 1.2.2 from keras.models import Sequential from keras.layers import Dense, Dropout, Activation, Flatten from keras.layers import Convolution2D, MaxPooling2D keras_model = Sequential() keras_model.add(Convolution2D(32, 3, 3, border_mode='valid', input_shape=(1, 28, 28))) keras_model.add(Activation('relu')) keras_model.add(Convolution2D(32, 3, 3)) keras_model.add(Activation('relu')) keras_model.add(MaxPooling2D(pool_size=(2, 2))) keras_model.add(Dropout(0.25)) keras_model.add(Flatten()) keras_model.add(Dense(128)) keras_model.add(Activation('relu')) keras_model.add(Dropout(0.5)) keras_model.add(Dense(10)) keras_model.add(Activation('softmax')) # Save the Keras model definition to JSON model_json = keras_model.to_json() path = "/tmp/lenet.json" with open(def_path, "w") as json_file: json_file.write(model_json) # Load the JSON file to a BigDL model from bigdl.nn.layer import * bigdl_model = Model.load_keras(json_path=path)
After loading the model into BigDL, you can train it with the MNIST dataset. See here for the full example code which includes the training and validation after model loading. After 12 epochs, accuracy >97% can be achieved.
You can find several more examples here to get familiar with loading a Keras model into BigDL. We will add more examples to this directory in the future.
However, there exist some arguments for Keras layers that are not supported in BigDL for now. Also, we haven't supported self-defined Keras layers, but one can still define your customized layer converter and weight converter methods for new layers if you wish. See here for the full list of unsupported layer arguments and some known issues we have found so far.
In our future work, we will continue to add functionality and better support running Keras on BigDL.