The TensorCraft is a HTTP server that serves Keras models using TensorFlow runtime.
Currently TensorCraft is in beta, client and server API may change in the future versions.
This server solves such problems as:
- Versioning of models.
- Warehousing of models.
- Enabling CI/CD for machine-learning models.
This is the recommended way to install tensorcraft. Simply run the following
command:
snap install tensorcraft --devmode --edge
snap start tensorcraftTensorCraft can be used as a Docker container. The major note on this approach is
that tensorflow library that is installed into the Docker image is not compiled
with support of AVX instructions or GPU.
docker pull netrack/tensorcraft:latestIn order to start the container, run the following command:
docker run -it -p 5678:5678/tcp netrack/tensorcraftYou can optinally specify volume to persist models between restarts of conatiner:
docker run -it -p 5678:5678/tcp -v tensorcraft:/var/run/tensorcraft netrack/tensorcraftInstall latest version from pypi repository.
pip install tensorcraftOne of the possible ways of using tensorcraft is publising model snapshots to
the server on each epoch end.
from keras.models import Sequential
from keras.layers import Dense, Activation
from tensorcraft.callbacks import ModelCheckpoint
model = keras.Sequential()
model.add(Dense(32, input_dim=784))
model.add(Activation('relu'))
model.compile(optimizer='sgd', loss='binary_crossentropy')
model.fit(x_train, y_train, callbacks=[ModelCheckpoint(verbose=1)], epochs=100)Currently, tensorcraft supports only models in the TensorFlow Saved Model, therefore
in order to publish Keras model, it must be saved as Saved Model at first.
Considering the following Keras model:
from tensorflow import keras
from tensorflow.keras import layers
inputs = keras.Input(shape=(8,), name='digits')
x = layers.Dense(4, activation='relu', name='dense_1')(inputs)
x = layers.Dense(4, activation='relu', name='dense_2')(x)
outputs = layers.Dense(2, activation='softmax', name='predictions')(x)
model = keras.Model(inputs=inputs, outputs=outputs, name='3_layer_mlp')Save it using the export_saved_model function from the 2.0 TensorFlow API:
keras.experimental.export_saved_model(model, "3_layer_mlp")To start server run server command:
sudo tensorcraft serverBy default it starts listening unsecured port on localhost at http://localhost:5678.
Default configuration saves models to /var/lib/tensorcraft directory. Apart of
that server requires access to /var/run directory in order to save pid file
there.
Note, both client and server of tensorcraft application share the same code
base. This implies the need to install a lot of server dependencies for a
client. This will be improved in uncoming versions.
Once model saved in directory, pack it using tar utility. For instance, this
is how it will look like for 3_layer_mlp model from the previous example:
tar -cf 3_layer_mlp.tar 3_layer_mlpNow the model packed into the archive can be pushed to the server under the arbitrary tag:
tensorcraft push --name 3_layer_mlp --tag 0.0.1 3_layer_mlp.tarYou can list all available models on the server using the following command:
tensorcraft listAfter the execution of list command you'll see to available models:
3_layer_mlp:0.0.1
3_layer_mlp:latestThis is the features of tensorcraft server, each published model name results in
creation of model group. Each model group has it's latest tag, that references
the latest pushed model.
Remove of the unused model can be performed in using remove command:
tensorcraft remove --name 3_layer_mlp --tag 0.0.1Execution of remove commands results in the remove of the model itself, and
the model group, when is is the last model in the group.
In order to use the pushed model, tensorcraft exposes REST API. An example query
to the server looks like this:
curl -X POST https://localhost:5678/models/3_layer_mlp/0.0.1/predict -d \
'{"x": [[1.0, 2.1, 1.43, 4.43, 12.1, 3.2, 1.44, 2.3]]}'The code and docs are released under the Apache 2.0 license.