Deploying Python Applications and their Virtual Environments
Introduction
As noted in a past article, we leverage virtualenv and pip to isolate and manage some of our python applications. A natural next question is “How can a python virtual environment and related application be deployed to a production server?”. This article provides a conceptual overview of one way such deployments can be handled.
The Server Environment and Conventions
First, let’s discuss some assumptions about the server environment. In this article, a deployment server, development server(s), and production server(s) are all discussed. It can be assumed that all these servers are running the same operating system (in this case, RHEL 6). This provides a luxury which allows for transporting virtual environments from one host to another with no ill effects and no requirement to build new virtual environments for each host.
Additionally, there are some directory conventions used which help assure consistency from host to host. The virtual environment is located in a standard path such as /opt/companyname/. The code for each python application is then located in a directory inside the virtual environment root. This makes for a set of paths like so:
Example directories:
/opt/company/myapp/ # the virtual env root
/opt/company/myapp/myapp/ # the application root
/opt/company/myapp/myapp/lib/ # the application library
/opt/company/myapp/myapp/bin/appd.py # the application
The Build
The building of the python application is a two step process. First the virtual environment is created or updated. Next, the desired version of the application is exported from the repository. This work all takes place on the deployment server.
Steps to build the virtual env and application:
# Go to the standard app location
cd /opt/company/
# Create the virtual env if needed
virtualenv ./myapp
# Export the desired copy of the app inside the virtual env root
svn export $repouri /opt/company/myapp/myapp/
# Activate the virtualenv
cd /opt/company/myapp/ && source ./bin/activate
# Install the requirements
cd /opt/company/myapp/myapp/
pip install -r ./requirements.txt
Here’s an example script which would handle such a build:
The Deploy
Once the virtualenv and application are built, the deployment can be handled with some rsync and scripting work. This same model can be used to deploy to development servers or production servers, maintaining consistency across any environment. It can also be used to deploy your application to a new server. While a bit of a simplification, the deployment can be envisioned as a simple for-loop around rsync.
Example deployment loop:
for host in $devservers; do
rsync -avz --delete-after /opt/company/myapp $host:/opt/company/myapp
done
Here’s an example script which would handle such a build:
Closing
This describes one of many ways python applications and their virtual environments can be deployed to remote hosts. It is a fairly simple matter to assemble these techniques into shell scripts for semi-automated build and deployment. Such scripts can then be enhanced preferred conventions as well as the more intelligent handling of application restarts, rollbacks, configuration management, and other desired improvements particular to the application.
(Originally published on the Catalyst InsideOps blog)