Simplifying python development with virtual environments


Over the past year I’ve spent a considerable amount of time beefing up my development skills. One of the best ways I’ve found to improve my skills is by reading code from well respected developers, watching udemy and youtube videos, fixing bugs in code I’m not familiar with (you will find some doozies this way) and benchmarking code to see what the system and resource ramifications of a change are.

To allow myself to experiment in isolation I’ve relied heavily on Python virtual environments. Virtual environments are code sandboxes and everything (libraries, packages, etc.) in a given virtual environment is isolated from everything else on your system. You can hop between these environments quite easily and if you break something the breakage won’t leak out of the environment. To use virtual environments on a Ubuntu host you will first need to install a couple of packages with apt (packages are also available for CentOS):

$ sudo apt install virtualenvwrapper

Once the virtual environment packages are installed you will need to define the PROJECT_HOME (older versions use the WORKON_HOME environment varaible) environment variable to tell virtualenv where to store your environments:

$ mkdir /home/matty/python-virtualenv

$ export PROJECT_HOME=/home/matty/python-virtualenv

To get a better feel for how virtual environments work lets say you want to modify the foo module to see if a new algorithm would perform better than what is currently in place. To create a new environment to test this hypothesis you can run the mkproject helper with the name of the environment you want to create:

$ mkproject foo

This will create a new virtual environment and initialize it with an isolated python runtime environment. To see the environments on your system you can run lsvirtualenv:

$ lsvirtualenv

bar
===

foo
===

To sandbox yourself in the new enviroment you can run the workon helper with the name of the environment you want to work in:

$ workon foo

Once you are in a virtual environment you can install new packages with pip, pull down code with git or alter system libraries to your liking. All of these changes will be isolated to this environment. If you change a library or python program and something breaks that breakage will only exist in the virtual environment. This is incredibly useful for debugging modules from PyPi and evaluating changes without “polluting” a branch (less of an issue with a good CI suite) or your system libraries. To leave a virtual environment you can use the deactivate helper:

$ deactivate

Once you are finished vetting your change you can delete a virtual environment with the rmvirtualenv helper:

$ rmvirtualenv foo

The virtualenvwrapper package comes with several other useful helpers. showvirtualenv will show you details about a specific virtual environment, cpvirtualenv will duplicate an existing virtual environment and the various sitepackages helpers will allow you to work with the site-packages that are in your virtual environment. Great stuff!

This article was posted by Matty on 2017-07-19 11:42:00 -0400 -0400