Virtual environments are a way of creating a dedicated Python environment, allowing us to install packages, tinker and hack away whilst not interfering with our system Python!
They can be created, modified and deleted with ease from the command line using Pythons' built in
Whilst there are several third party tools for creating and managing virtual environments, the majority of people will be just fine using Python's built in
venv, which we'll be covering in this article.
Why use a virtual environment
When working with Python, there's a high chance that you'll be using some third party packages/libraries which your software depends on, along with possibly working with several versions of Python across a range of projects on the same machine.
Creating virtual environments allows us to install packages, test, develop, hack and modify our software in a very controlled environment, ensuring everything works with a specific version of Python and specific versions of any third party packages & tools installed.
Virtual environments also have some neat features, like being able to generate a
requirements.txt containing any packages installed in your environment which can also be installed with a simple command (We'll cover this later!)
Virtual environments can be easily created and deleted, making them disposable but powerful. They're also a great way to keep your projects isolated from your system installation of Python.
Difference between Windows, Linux & Mac
Virtual environments are similar on Windows, Linux & mac, with some minor differences which we'll cover throughout this guide. However, most of the commands are the same (with come minor differences to directory names and a couple of files)
Any commands or directories covered for Linux will be the same for Mac.
Tour of a virtual environment
Here's a tree view of a virtual environment created on Linux:
. └── env ├── bin | ├── activate | ├── activate.csh | ├── activate.fish | ├── easy_install | ├── easy_install-3.7 | ├── pip | ├── pip3 | ├── pip3.7 | ├── python -> /home/julian/.pyenv/versions/3.7.2/bin/python | └── python3 -> python ├── include ├── lib │ └── python3.7 │ └── site-packages │ ├── pip │ │ ├── _internal │ │ │ ├── cli │ │ │ ├── commands │ │ │ ├── models │ │ │ ├── operations │ │ │ ├── req │ │ │ ├── utils │ │ │ └── vcs │ │ └── _vendor │ │ ├── cachecontrol │ │ │ └── caches │ │ ├── certifi │ │ ├── chardet │ │ │ └── cli │ │ ├── colorama │ │ ├── distlib │ │ │ └── _backport │ │ ├── html5lib │ │ │ ├── _trie │ │ │ ├── filters │ │ │ ├── treeadapters │ │ │ ├── treebuilders │ │ │ └── treewalkers │ │ ├── idna │ │ ├── lockfile │ │ ├── msgpack │ │ ├── packaging │ │ ├── pep517 │ │ ├── pkg_resources │ │ ├── progress │ │ ├── pytoml │ │ ├── requests │ │ ├── urllib3 │ │ │ ├── contrib │ │ │ │ └── _securetransport │ │ │ ├── packages │ │ │ │ ├── backports │ │ │ │ └── ssl_match_hostname │ │ │ └── util │ │ └── webencodings │ ├── pip-18.1.dist-info │ ├── pkg_resources │ │ ├── _vendor │ │ │ └── packaging │ │ └── extern │ ├── setuptools │ │ ├── _vendor │ │ │ └── packaging │ │ ├── command │ │ └── extern │ └── setuptools-40.6.2.dist-info └── lib64 -> lib
The 2 main directories to pay attention to are:
bin- Contains copies/symbolic links to the Python, pip & a few other binaries
lib/python3.7/site-packages- Contains any site packages installed in the environment
On windows, you'll find
Any third party libraries installed with
pip install <package> can be found in the
site-packages directory (once the environment has been activated which we'll cover soon!)
Creating a virtual environment
To create a virtual environment on all platforms, we run the following (where
env is the name of your environment):
python -m venv env
This will create a virtual environment in the current directory named
env using the current version of your system Python.
You can also provide a path in place of the environment name:
python -m venv /path/to/your/env
Virtual environment options
To see the full range of options, run the following command:
python -m venv -h
You'll see a list of possible options for creating a new virtual environment:
usage: venv [-h] [--system-site-packages] [--symlinks | --copies] [--clear] [--upgrade] [--without-pip] [--prompt PROMPT] ENV_DIR [ENV_DIR ...] Creates virtual Python environments in one or more target directories. positional arguments: ENV_DIR A directory to create the environment in. optional arguments: -h, --help show this help message and exit --system-site-packages Give the virtual environment access to the system site-packages dir. --symlinks Try to use symlinks rather than copies, when symlinks are not the default for the platform. --copies Try to use copies rather than symlinks, even when symlinks are the default for the platform. --clear Delete the contents of the environment directory if it already exists, before environment creation. --upgrade Upgrade the environment directory to use this version of Python, assuming Python has been upgraded in-place. --without-pip Skips installing or upgrading pip in the virtual environment (pip is bootstrapped by default) --prompt PROMPT Provides an alternative prompt prefix for this environment. Once an environment has been created, you may wish to activate it, e.g. by sourcing an activate script in its bin directory.
The output of
python -m venv -h does a pretty good job of explaining the options for using the
venv script so we won't bother going through them. The majority of cases should be just fine using the
python -m venv env command and leaving the options as default.
Having said that, one option I do like to select is
--prompt, providing a short but descriptive name of the project and environment.
--prompt argument, followed by a name will be displayed just before your terminal propt when the environment is active, providing some visual feedback to let you know the environment you're working in.
On that note, let's activate the environment.
Activating a virtual environment
When an environment is activated, any
pip commands will be directed to the binaries in your newly created environment! Prepending your newly created binaries in the
Scripts directory on your Python PATH. This is what allows us to install packages and run Python from the virtual environment.
Activation is slightly different between Windows & Linux/Mac (assuming your virtual environment is named
env and you're in the same directory)
To activate on Windows:
To activate on Linux & Mac:
Once your environment has been activated, you should see your prompt change and have the name of your environment prepended to it:
(env) julian@jnwt: /mnt/c/pythonise/tutorials/python/virtual_environments
If you passed the
--prompt argument, followed by a name when you created the virtual environment, you would see the name you provided next to your prompt, for example:
python -m venv env --prompt FOOBAR source env/bin/activate
Would show the following prompt when activated:
(FOOBAR) julian@jnwt: /mnt/c/pythonise/tutorials/python/virtual_environments
Now that the environment is activated, we can install packages with
pip and any
python commands will be run from the virtual environment.
With an active environment, any
pip commands will be routed to the new environment.
pip list will return a list of any installed site packages:
Package Version ---------- ------- pip 18.1 setuptools 40.6.2
With an active environment, installing packages with pip is business as usual:
pip install <package>
Any packages installed can be found in the
For example, we'll go ahead and install the
bleach package, invoke the interactive Python interpreter and import it:
pip install bleach
pip list shows us it's been installed:
Package Version ------------ ------- bleach 3.1.0 pip 18.1 setuptools 40.6.2 six 1.12.0 webencodings 0.5.1
pip -V will return the version of
pip along with a path to its location:
pip 18.1 from /mnt/c/pythonise/tutorials/python/virtual_environments/env/lib/python3.7/site-packages/pip (python 3.7)
Let's go ahead and invoke Python and import
Python 3.7.2 (default, Jan 23 2019, 16:31:34) [GCC 7.3.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import bleach >>>
bleach was imported and no errors were thrown. All is good, our packages have been installed and Python is launching from the virtual environment.
requirements.txt file is a handly little text file that lists any packages installed in your Python environment.
With an activated environment, run the following command to create it:
pip freeze > requirements.txt
If we take a look at the
requirements.txt file, we see:
bleach==3.1.0 six==1.12.0 webencodings==0.5.1
Obviously this will differ based on your installed packages! but you'll notice, we get the name of the package along with the current installed version number.
The pinning of the version number helps us control which versions of third party packages should be installed and hopefully work with our software!
This portable file now gives us a way to install packages, which we can do with the following command:
pip install -r requirements.txt
Assuming you have an activated virtual environment and you're currently in the same directory as the
requirements.txt file, each package listed will be installed.
Deactivating a virtual environment
Again, another platform specific command to deactivate a virtual environment.
On Linux & Mac:
ZSH & Fish terminals
Alternative terminal users including ZSH, Fish & CSH will find their respective
activate binaries in the
bin directory for activating their virtual environments.
Python virtual environments, in combination with
pip freeze and
requirements.txt are a simple yet powerful way to handle your dependencies and create robust, reliable and sharable software.
Consider combining virtual environments with
pyenv, a simple Python version management tool for Linux & Mac (Tutorial)