Home Articles Categories Series
Pythonise Just now

Python virtual environments with the venv command

A quick introduction to creating and working with Python virtual environments with the built in venv command


Article Posted on by in Python
Julian Nash · 8 months ago in Python

Virtual environments are a way of creating a dedicated Python environment, allowing us to install packages, tinker and hack away whilst not interfering with our system Python!

They can be created, modified and deleted with ease from the command line using Pythons' built in venv script.

Whilst there are several third party tools for creating and managing virtual environments, the majority of people will be just fine using Python's built in venv, which we'll be covering in this article.

Why use a virtual environment

When working with Python, there's a high chance that you'll be using some third party packages/libraries which your software depends on, along with possibly working with several versions of Python across a range of projects on the same machine.

Creating virtual environments allows us to install packages, test, develop, hack and modify our software in a very controlled environment, ensuring everything works with a specific version of Python and specific versions of any third party packages & tools installed.

Virtual environments also have some neat features, like being able to generate a requirements.txt containing any packages installed in your environment which can also be installed with a simple command (We'll cover this later!)

Virtual environments can be easily created and deleted, making them disposable but powerful. They're also a great way to keep your projects isolated from your system installation of Python.

Difference between Windows, Linux & Mac

Virtual environments are similar on Windows, Linux & mac, with some minor differences which we'll cover throughout this guide. However, most of the commands are the same (with come minor differences to directory names and a couple of files)

Any commands or directories covered for Linux will be the same for Mac.

Tour of a virtual environment

Here's a tree view of a virtual environment created on Linux:

.
└── env
    ├── bin
    |   ├── activate
    |   ├── activate.csh
    |   ├── activate.fish
    |   ├── easy_install
    |   ├── easy_install-3.7
    |   ├── pip
    |   ├── pip3
    |   ├── pip3.7
    |   ├── python -> /home/julian/.pyenv/versions/3.7.2/bin/python
    |   └── python3 -> python
    ├── include
    ├── lib
    │   └── python3.7
    │       └── site-packages
    │           ├── pip
    │           │   ├── _internal
    │           │   │   ├── cli
    │           │   │   ├── commands
    │           │   │   ├── models
    │           │   │   ├── operations
    │           │   │   ├── req
    │           │   │   ├── utils
    │           │   │   └── vcs
    │           │   └── _vendor
    │           │       ├── cachecontrol
    │           │       │   └── caches
    │           │       ├── certifi
    │           │       ├── chardet
    │           │       │   └── cli
    │           │       ├── colorama
    │           │       ├── distlib
    │           │       │   └── _backport
    │           │       ├── html5lib
    │           │       │   ├── _trie
    │           │       │   ├── filters
    │           │       │   ├── treeadapters
    │           │       │   ├── treebuilders
    │           │       │   └── treewalkers
    │           │       ├── idna
    │           │       ├── lockfile
    │           │       ├── msgpack
    │           │       ├── packaging
    │           │       ├── pep517
    │           │       ├── pkg_resources
    │           │       ├── progress
    │           │       ├── pytoml
    │           │       ├── requests
    │           │       ├── urllib3
    │           │       │   ├── contrib
    │           │       │   │   └── _securetransport
    │           │       │   ├── packages
    │           │       │   │   ├── backports
    │           │       │   │   └── ssl_match_hostname
    │           │       │   └── util
    │           │       └── webencodings
    │           ├── pip-18.1.dist-info
    │           ├── pkg_resources
    │           │   ├── _vendor
    │           │   │   └── packaging
    │           │   └── extern
    │           ├── setuptools
    │           │   ├── _vendor
    │           │   │   └── packaging
    │           │   ├── command
    │           │   └── extern
    │           └── setuptools-40.6.2.dist-info
    └── lib64 -> lib

The 2 main directories to pay attention to are:

  • bin - Contains copies/symbolic links to the Python, pip & a few other binaries
  • lib/python3.7/site-packages - Contains any site packages installed in the environment

On windows, you'll find site-packages in Lib\site-packages

Any third party libraries installed with pip install <package> can be found in the site-packages directory (once the environment has been activated which we'll cover soon!)

Creating a virtual environment

To create a virtual environment on all platforms, we run the following (where env is the name of your environment):

python -m venv env

This will create a virtual environment in the current directory named env using the current version of your system Python.

You can also provide a path in place of the environment name:

python -m venv /path/to/your/env

Virtual environment options

To see the full range of options, run the following command:

python -m venv -h

You'll see a list of possible options for creating a new virtual environment:

usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear]
            [--upgrade] [--without-pip] [--prompt PROMPT]
            ENV_DIR [ENV_DIR ...]

Creates virtual Python environments in one or more target directories.

positional arguments:
  ENV_DIR               A directory to create the environment in.

optional arguments:
  -h, --help            show this help message and exit
  --system-site-packages
                        Give the virtual environment access to the system
                        site-packages dir.
  --symlinks            Try to use symlinks rather than copies, when symlinks
                        are not the default for the platform.
  --copies              Try to use copies rather than symlinks, even when
                        symlinks are the default for the platform.
  --clear               Delete the contents of the environment directory if it
                        already exists, before environment creation.
  --upgrade             Upgrade the environment directory to use this version
                        of Python, assuming Python has been upgraded in-place.
  --without-pip         Skips installing or upgrading pip in the virtual
                        environment (pip is bootstrapped by default)
  --prompt PROMPT       Provides an alternative prompt prefix for this
                        environment.

Once an environment has been created, you may wish to activate it, e.g. by
sourcing an activate script in its bin directory.

The output of python -m venv -h does a pretty good job of explaining the options for using the venv script so we won't bother going through them. The majority of cases should be just fine using the python -m venv env command and leaving the options as default.

Having said that, one option I do like to select is --prompt, providing a short but descriptive name of the project and environment.

The --prompt argument, followed by a name will be displayed just before your terminal propt when the environment is active, providing some visual feedback to let you know the environment you're working in.

On that note, let's activate the environment.

Activating a virtual environment

When an environment is activated, any python or pip commands will be directed to the binaries in your newly created environment! Prepending your newly created binaries in the bin or Scripts directory on your Python PATH. This is what allows us to install packages and run Python from the virtual environment.

Activation is slightly different between Windows & Linux/Mac (assuming your virtual environment is named env and you're in the same directory)

To activate on Windows:

Command prompt

env\Scripts\Activate.ps1

Powershell

env\Scripts\Activate.bat

To activate on Linux & Mac:

source env/bin/activate

Once your environment has been activated, you should see your prompt change and have the name of your environment prepended to it:

(env) julian@jnwt: /mnt/c/pythonise/tutorials/python/virtual_environments

If you passed the --prompt argument, followed by a name when you created the virtual environment, you would see the name you provided next to your prompt, for example:

python -m venv env --prompt FOOBAR
source env/bin/activate

Would show the following prompt when activated:

(FOOBAR) julian@jnwt: /mnt/c/pythonise/tutorials/python/virtual_environments

Now that the environment is activated, we can install packages with pip and any python commands will be run from the virtual environment.

Installing packages

With an active environment, any pip commands will be routed to the new environment.

For example, pip list will return a list of any installed site packages:

pip list
Package    Version
---------- -------
pip        18.1
setuptools 40.6.2

With an active environment, installing packages with pip is business as usual:

pip install <package>

Any packages installed can be found in the site-packages directory.

For example, we'll go ahead and install the bleach package, invoke the interactive Python interpreter and import it:

pip install bleach

Running, pip list shows us it's been installed:

Package      Version
------------ -------
bleach       3.1.0
pip          18.1
setuptools   40.6.2
six          1.12.0
webencodings 0.5.1

Running pip -V will return the version of pip along with a path to its location:

pip -V

Returns:

pip 18.1 from /mnt/c/pythonise/tutorials/python/virtual_environments/env/lib/python3.7/site-packages/pip (python 3.7)

Let's go ahead and invoke Python and import bleach:

python
Python 3.7.2 (default, Jan 23 2019, 16:31:34)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import bleach
>>>

Awesome! bleach was imported and no errors were thrown. All is good, our packages have been installed and Python is launching from the virtual environment.

Requirements

A requirements.txt file is a handly little text file that lists any packages installed in your Python environment.

With an activated environment, run the following command to create it:

pip freeze > requirements.txt

If we take a look at the requirements.txt file, we see:

bleach==3.1.0
six==1.12.0
webencodings==0.5.1

Obviously this will differ based on your installed packages! but you'll notice, we get the name of the package along with the current installed version number.

The pinning of the version number helps us control which versions of third party packages should be installed and hopefully work with our software!

This portable file now gives us a way to install packages, which we can do with the following command:

pip install -r requirements.txt

Assuming you have an activated virtual environment and you're currently in the same directory as the requirements.txt file, each package listed will be installed.

Deactivating a virtual environment

Again, another platform specific command to deactivate a virtual environment.

On Windows:

Command prompt

deactivate.ps1

Powershell

deactivate.bat

On Linux & Mac:

deactivate

ZSH & Fish terminals

Alternative terminal users including ZSH, Fish & CSH will find their respective activate binaries in the bin directory for activating their virtual environments.

Fish:

. env/bin/activate.fish

ZSH:

source env/bin/activate

CSH:

source env/bin/activate.csh

Wrapping up

Python virtual environments, in combination with pip freeze and requirements.txt are a simple yet powerful way to handle your dependencies and create robust, reliable and sharable software.

Consider combining virtual environments with pyenv, a simple Python version management tool for Linux & Mac (Tutorial)

Last modified · 19 Mar 2019
Did you find this article useful?
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License
Contents
Loading...