Deploying a scikit-learn classifier to production

Share on:

Scikit-learn is a great python library for all sorts of machine learning algorithms, and really well documented for the model development side of things. But once you have a trained classifier and are ready to run it in production, how do you go about doing this?

There’s a few managed services that will do it for you, but for my situation these weren’t a good fit. We just wanted to deploy the model onto a modest sized Digital Ocean instance, as a REST API that can be externally queried.

Challenges

  • saving the model in a form that can be loaded onto a remote server
  • wrapping up the classifier in an API
  • installing a scikit-learn environment on a server
  • finally, deploying the code onto the remote server

Model persistence

The scikit docs have a good section on this topic:

http://scikit-learn.org/stable/modules/model_persistence.html

I’d recommend using joblib to serialize your model:

from sklearn.externals import joblib
from sklearn.pipeline import make_pipeline
from sklearn.feature_extraction.text import HashingVectorizer
from sklearn.svm import LinearSVC

# code to load training data into X_train, y_train, split train/test set

vec = HashingVectorizer()
svc = LinearSVC()
clf = make_pipeline(vec, svc)
svc.fit(X_train, y_train)

joblib.dump({'class1': clf}, 'models', compress=9)

I’m saving a single model under the key class1, but there’s scope to add several more classifiers as you go.

Also note, to keep the model size manageable even with a large number of features (as would happen with word vectors) the HashingVectorizer is used, because it is very memory efficient (constant with size of input) vectorizer. In validation the accuracy was very close to using a TfIdfVectorizer.

REST API

For an API, I’ve used flask to wrap up classifier predict() as a simple HTTP service:

from flask import jsonify, request, Flask
from sklearn.externals import joblib

models = joblib.load('models')
app = Flask(__name__)

@app.route('/', methods=['POST'])
def predict():
    text = request.form.get('text')
    results = {}
    for name, clf in models.iteritems():
        results[name] = clf.predict([text])[0]
    return jsonify(results)

if __name__ == '__main__':
    app.run()

This receives a POST / with JSON {"text": "..."}, and returns the first classification predicted from the model.

This is set out as a python a package (under mlapi/__init__.py), with a corresponding setup.py:

from setuptools import setup

__version__ = '0.1.0'

setup(
    name='mlapi',
    version=__version__,
    description='mlapi',
    author='Your Name',
    author_email='email@example.com',
    packages=['mlapi'],
    entry_points={},
)

Creating it as a python package means it can be built to a wheel (.whl) which makes it very easy to distribute for deployment, next…

Deploying

I use fabric to deploy - it’s nice and simple to setup.

For python installation on the server I use the Anaconda distribution of python miniconda. Miniconda removes all the difficulties of installing scikit-learn and the complex dependencies (numpy, scipy, etc.), taking the pain out of setting up the python environment to run your app.

It’ll run on any version of Linux - in this case I was using Ubuntu 14.04 LTS. The app service is started by Upstart, but equally Systemd can be used with some adaptions if that’s your flavour.

Create a fabfile.py to configure fabric:

from fabric.api import env, local, run, sudo
from fabric.operations import put
import anaconda

env.roledefs = {
    'prd': ['deploy@server1.example.com'],
}
APP_PATH = '/apps/mlapi'

def setup():
    # anaconda setup
    anaconda.install()
    run('mkdir -p %s/logs' % APP_PATH)
    put('deploy/mlapi.conf', '/etc/init/mlapi.conf', use_sudo=True)
    put('deploy/upstart_mlapi', '/etc/sudoers.d/mlapi', use_sudo=True)
    sudo('chown root:root /etc/init/mlapi.conf /etc/sudoers.d/mlapi')

def deploy():
    # ensure environment up to date
    put('environment.yml', APP_PATH)
    anaconda.create_env(APP_PATH+'/environment.yml')

    # install egg
    local('python setup.py bdist_wheel')
    wheel = 'mlapi-0.1.0-py2-none-any.whl'
    put('dist/'+wheel, APP_PATH)
    with anaconda.env('ml'):
        run('pip install -U %s/%s' % (APP_PATH, wheel))

    # deploy models
    put('models', APP_PATH)

    # restart gunicorn
    # stop, then start: ensure it succeeds first time
    run('sudo /usr/sbin/service mlapi stop; sudo /usr/sbin/service mlapi start')

anaconda.py is an extra module for fabric to manage installing the miniconda python distribution:

from fabric.api import cd, run
from fabric.contrib.files import exists
from fabric.context_managers import prefix

# defaults
CONDA_REPO = 'http://repo.continuum.io/miniconda/'
CONDA_VERS = 'Miniconda2-3.19.0-Linux-x86_64.sh'

def install(conda_repo=CONDA_REPO, conda_vers=CONDA_VERS, home='~'):
    anaconda = home+'/anaconda'
    if not exists(anaconda):
        run('mkdir -p %s/downloads' % home)
        with cd(home+'/downloads'):
            run('wget -nv -N %s%s' % (conda_repo, conda_vers))
            run('bash %s -b -p %s' % (conda_vers, anaconda))

def create_env(environment_yml, home='~'):
    anaconda_bin = '%s/anaconda/bin' % home
    with cd(anaconda_bin):
        if exists(anaconda_bin):
            run('./conda env update -f %s' % environment_yml)
        else:
            run('./conda env create -f %s' % environment_yml)

def env(name, home='~'):
    """Run with an anaconda environment"""
    return prefix('source %s/anaconda/bin/activate %s' % (home, name))

The two configuration files installed in setup() are the upstart configuration to /etc/init/mlapi.conf and a sudoers file /etc/sudoers.d/mlapi to allow a normal user to stop and restart the service without requiring root permissions.

mlapi.conf:

description "mlapi"

start on (filesystem)
stop on runlevel [016]

respawn
setuid deploy
setgid nogroup
chdir /apps/mlapi

exec /home/deploy/anaconda/envs/ml/bin/gunicorn mlapi:app --access-logfile /apps/mlapi/logs/access.log

upstart_mlapi:

# gunicorn service
deploy ALL=(ALL) NOPASSWD: /usr/sbin/service mlapi start, /usr/sbin/service mlapi stop, /usr/sbin/service mlapi restart, /usr/sbin/service mlapi status

I’m running the service using gunicorn - feel free to pick your own favourite app server.

Finally, you need save the anaconda environment.yml by running:

$ conda env export > environment.yml

Now you’re ready to setup the server:

$ fab -R prd setup

And deploy your code:

$ fab -R prd deploy

It should be running now, listening on the default port of 8000. You’ll probably want to configure nginx as a proxy in front of gunicorn to secure the service with authentication if exposing to the world.

If you have any issues, it’s useful to check the service is running:

$ service mlapi status

Also check the upstart logs at /var/log/upstart/mlapi.log.

And also check you can manually run the above gunicorn command to start your server if it is failing to start for whatever reason.