Keras Support
For Python users, BigDL supports loading pre-defined Keras models.
The Keras version we support and test is Keras 1.2.2 with TensorFlow backend. Up to now, we have generally supported ALL its layers.
After loading a model into BigDL, you can train, evaluate or tune this model in a distributed manner. We have generally supported ALL the losses in Keras 1.2.2. See here to find the corresponding criterions in BigDL.
If you haven't been familiar with BigDL yet, you may refer to Python User Guide on how to install and run BigDL for Python users before you start this page.
Load a Keras model into BigDL
A Keras model definition in JSON file can be loaded as a BigDL model. Saved weights in HDF5 file can also be loaded together with the architecture of a Keras model. See here on how to save a model in Keras.
You can directly call the API Model.load_keras
to load a Keras model into BigDL.
Remark: keras==1.2.2
is required beforehand. If you are to load a HDF5 file, you also need to install h5py
. These packages can be installed via pip
easily.
from bigdl.nn.layer import *
bigdl_model = Model.load_keras(json_path=None, hdf5_path=None, by_name=False)
Parameters:
json_path
The JSON file path containing the Keras model definition to be loaded. Default to beNone
if you choose to load a Keras model from a HDF5 file.hdf5_path
The HDF5 file path containing the pre-trained weights with or without the model architecture. Please use weights from Keras 1.2.2 withtensorflow backend
. Default to beNone
if you choose to only load the model definition from JSON but not to load weights. In this case, BigDL will use initialized weights for the model.by_name
Whether to load the weights of layers by name. Use this option only when you provide a HDF5 file. Default to beFalse
, meaning that weights are loaded based on the network's execution order topology. Otherwise, if it is set to beTrue
, only those layers with the same name will be loaded with weights.
NOTES:
-
Please provide either
json_path
orhdf5_path
when you callModel.load_keras
. You can providejson_path
only to just load the model definition. You can providejson_path
andhdf5_path
together if you have separate files for the model architecture and its pre-trained weights. Also, if you save the model architecture and its weights in a single HDF5 file, you can providehdf5_path
only. -
JSON and HDF5 files can be loaded from any Hadoop-supported file system URI. For example,
# load from local file system
bigdl_model = Model.load_keras(json_path="/tmp/model.json", hdf5_path="/tmp/weights.h5")
# load from HDFS
bigdl_model = Model.load_keras(hdf5_path="hdfs://model.h5")
# load from S3
bigdl_model = Model.load_keras(hdf5_path="s3://model.h5")
LeNet Example
Here we show a simple example on how to load a Keras model into BigDL. The model used in this example is a CNN from Keras 1.2.2.
# Define a CNN model in Keras 1.2.2
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Convolution2D, MaxPooling2D
keras_model = Sequential()
keras_model.add(Convolution2D(32, 3, 3, border_mode='valid',
input_shape=(1, 28, 28)))
keras_model.add(Activation('relu'))
keras_model.add(Convolution2D(32, 3, 3))
keras_model.add(Activation('relu'))
keras_model.add(MaxPooling2D(pool_size=(2, 2)))
keras_model.add(Dropout(0.25))
keras_model.add(Flatten())
keras_model.add(Dense(128))
keras_model.add(Activation('relu'))
keras_model.add(Dropout(0.5))
keras_model.add(Dense(10))
keras_model.add(Activation('softmax'))
# Save the Keras model definition to JSON
model_json = keras_model.to_json()
path = "/tmp/lenet.json"
with open(def_path, "w") as json_file:
json_file.write(model_json)
# Load the JSON file to a BigDL model
from bigdl.nn.layer import *
bigdl_model = Model.load_keras(json_path=path)
After loading the model into BigDL, you can train it with the MNIST dataset. See here for the full example code which includes the training and validation after model loading. After 12 epochs, accuracy >97% can be achieved.
You can find several more examples here to get familiar with loading a Keras model into BigDL. We will add more examples to this directory in the future.
Limitations
We have tested the model loading functionality with several standard Keras applications and examples.
However, there exist some arguments for Keras layers that are not supported in BigDL for now. Also, we haven't supported self-defined Keras layers, but one can still define your customized layer converter and weight converter methods for new layers if you wish. See here for the full list of unsupported layer arguments and some known issues we have found so far.
In our future work, we will continue to add functionality and better support running Keras on BigDL.