Scikit-learn is a great python library for all sorts of machine learning
algorithms, and really well documented for the model development side of
things. But once you have a trained classifier and are ready to run it in
production, how do you go about doing this?
There’s a few managed services that will do it for you, but for my situation
these weren’t a good fit. We just wanted to deploy the model onto a modest
sized Digital Ocean instance, as a REST API that can be externally queried.
Challenges
- saving the model in a form that can be loaded onto a remote server
- wrapping up the classifier in an API
- installing a scikit-learn environment on a server
- finally, deploying the code onto the remote server
Model persistence
The scikit docs have a good section on this topic:
http://scikit-learn.org/stable/modules/model_persistence.html
I’d recommend using joblib to serialize your model:
from sklearn.externals import joblib
from sklearn.pipeline import make_pipeline
from sklearn.feature_extraction.text import HashingVectorizer
from sklearn.svm import LinearSVC
# code to load training data into X_train, y_train, split train/test set
vec = HashingVectorizer()
svc = LinearSVC()
clf = make_pipeline(vec, svc)
svc.fit(X_train, y_train)
joblib.dump({'class1': clf}, 'models', compress=9)
I’m saving a single model under the key class1, but there’s scope to
add several more classifiers as you go.
Also note, to keep the model size manageable even with a large number of
features (as would happen with word vectors) the HashingVectorizer is used,
because it is very memory efficient (constant with size of input) vectorizer.
In validation the accuracy was very close to using a TfIdfVectorizer.
REST API
For an API, I’ve used flask to wrap up classifier predict() as a simple HTTP service:
from flask import jsonify, request, Flask
from sklearn.externals import joblib
models = joblib.load('models')
app = Flask(__name__)
@app.route('/', methods=['POST'])
def predict():
text = request.form.get('text')
results = {}
for name, clf in models.iteritems():
results[name] = clf.predict([text])[0]
return jsonify(results)
if __name__ == '__main__':
app.run()
This receives a POST / with JSON {"text": "..."}, and returns the first
classification predicted from the model.
This is set out as a python a package (under mlapi/__init__.py), with a
corresponding setup.py:
from setuptools import setup
__version__ = '0.1.0'
setup(
name='mlapi',
version=__version__,
description='mlapi',
author='Your Name',
author_email='[email protected]',
packages=['mlapi'],
entry_points={},
)
Creating it as a python package means it can be built to a wheel (.whl) which makes
it very easy to distribute for deployment, next…
Deploying
I use fabric to deploy - it’s nice and simple to setup.
For python installation on the server I use the Anaconda distribution of
python miniconda. Miniconda removes
all the difficulties of installing scikit-learn and the complex dependencies
(numpy, scipy, etc.), taking the pain out of setting up the python environment
to run your app.
It’ll run on any version of Linux - in this case I was using Ubuntu 14.04 LTS.
The app service is started by Upstart, but equally Systemd can be used with
some adaptions if that’s your flavour.
Create a fabfile.py to configure fabric:
from fabric.api import env, local, run, sudo
from fabric.operations import put
import anaconda
env.roledefs = {
'prd': ['[email protected]'],
}
APP_PATH = '/apps/mlapi'
def setup():
# anaconda setup
anaconda.install()
run('mkdir -p %s/logs' % APP_PATH)
put('deploy/mlapi.conf', '/etc/init/mlapi.conf', use_sudo=True)
put('deploy/upstart_mlapi', '/etc/sudoers.d/mlapi', use_sudo=True)
sudo('chown root:root /etc/init/mlapi.conf /etc/sudoers.d/mlapi')
def deploy():
# ensure environment up to date
put('environment.yml', APP_PATH)
anaconda.create_env(APP_PATH+'/environment.yml')
# install egg
local('python setup.py bdist_wheel')
wheel = 'mlapi-0.1.0-py2-none-any.whl'
put('dist/'+wheel, APP_PATH)
with anaconda.env('ml'):
run('pip install -U %s/%s' % (APP_PATH, wheel))
# deploy models
put('models', APP_PATH)
# restart gunicorn
# stop, then start: ensure it succeeds first time
run('sudo /usr/sbin/service mlapi stop; sudo /usr/sbin/service mlapi start')
anaconda.py is an extra module for fabric to manage installing the miniconda
python distribution:
from fabric.api import cd, run
from fabric.contrib.files import exists
from fabric.context_managers import prefix
# defaults
CONDA_REPO = 'http://repo.continuum.io/miniconda/'
CONDA_VERS = 'Miniconda2-3.19.0-Linux-x86_64.sh'
def install(conda_repo=CONDA_REPO, conda_vers=CONDA_VERS, home='~'):
anaconda = home+'/anaconda'
if not exists(anaconda):
run('mkdir -p %s/downloads' % home)
with cd(home+'/downloads'):
run('wget -nv -N %s%s' % (conda_repo, conda_vers))
run('bash %s -b -p %s' % (conda_vers, anaconda))
def create_env(environment_yml, home='~'):
anaconda_bin = '%s/anaconda/bin' % home
with cd(anaconda_bin):
if exists(anaconda_bin):
run('./conda env update -f %s' % environment_yml)
else:
run('./conda env create -f %s' % environment_yml)
def env(name, home='~'):
"""Run with an anaconda environment"""
return prefix('source %s/anaconda/bin/activate %s' % (home, name))
The two configuration files installed in setup() are the
upstart configuration to /etc/init/mlapi.conf and a sudoers file
/etc/sudoers.d/mlapi to allow a normal user to stop and restart the service
without requiring root permissions.
mlapi.conf:
description "mlapi"
start on (filesystem)
stop on runlevel [016]
respawn
setuid deploy
setgid nogroup
chdir /apps/mlapi
exec /home/deploy/anaconda/envs/ml/bin/gunicorn mlapi:app --access-logfile /apps/mlapi/logs/access.log
upstart_mlapi:
# gunicorn service
deploy ALL=(ALL) NOPASSWD: /usr/sbin/service mlapi start, /usr/sbin/service mlapi stop, /usr/sbin/service mlapi restart, /usr/sbin/service mlapi status
I’m running the service using gunicorn - feel free to pick your own favourite app server.
Finally, you need save the anaconda environment.yml by running:
$ conda env export > environment.yml
Now you’re ready to setup the server:
$ fab -R prd setup
And deploy your code:
$ fab -R prd deploy
It should be running now, listening on the default port of 8000. You’ll
probably want to configure nginx as a proxy in front of gunicorn to secure the
service with authentication if exposing to the world.
If you have any issues, it’s useful to check the service is running:
$ service mlapi status
Also check the upstart logs at /var/log/upstart/mlapi.log.
And also check you can manually run the above gunicorn command to start your
server if it is failing to start for whatever reason.
There are comments.