The Python Tutorial#Virtual Environments and Packages

12.1 Introduction

Python applications often use packages and modules that do not belong to standard libraries. Sometimes an application needs a specific version of the library, because the application may need a library with a specific bug to be fixed, or an application depends on an obsolete version of the interface in the library.

This means that a Python installation may not meet the needs of each application. If application A relies on version 1.0 of a particular module and application B relies on version 2.0, then requirements conflict, and installing either version of 1.0 or 2.0 will cause one of the applications to fail to run.

The solution to the above problem is to create virtual environment The virtual environment is a separate directory tree containing a specific Python version of the Python installation and some additional packages.

Different applications can use different virtual environments. To resolve previous requirements conflicts, application A can have a virtual environment with version 1.0 installed, while application B can have another virtual environment with version 2.0 installed. If application B needs to be upgraded to version 3.0 library, this will not affect the virtual environment of A.

12.2 Creating Virtual Environments

The module used to create and manage a virtual environment is called venv . venv usually installs the latest version of Python. If there are multiple versions of Python in the system, you can run Python 3 or any other version to select a specific Python version.

To create a virtual environment, you first need to decide to place the directory of the virtual environment, and then run the venv module with directory path in a scripted way:

python3 -m venv tutorial-env

If the tutorial-env directory does not exist, a tutorial-env directory is created, and subdirectories are created in the directory, including Python interpreters, standard libraries, and copies of various supporting files.

Once a virtual environment is created, it can be activated.

In Windows, run:

tutorial-env\Scripts\activate.bat

In Unix or MacOS, run:

source tutorial-env/bin/activate

(This script is written using the bash shell. If you use csh or fish shell, there are activae.csh and activate.fish scripts to choose from.

Activating the virtual environment changes the shell's prompts to show the virtual environment in use, and changes the environment so that running Python will get a specific version and Python installation. For example:

$ source ~/envs/tutorial-env/bin/activate
(tutorial-env) $ python
Python 3.5.1 (default, May  6 2016, 10:59:36)
  ...
>>> import sys
>>> sys.path
['', '/usr/local/lib/python35.zip', ...,
'~/envs/tutorial-env/lib/python3.5/site-packages']
>>>

12.3 Managing Packages with pip

Using a program called pip, you can install, update and remove packages. By default, PIP will be from Python Package Index https://pypi.python.org/pypi Installation package. You can browse Python Package Index using a browser or use pip's limited search features:

(tutorial-env) $ pip search astronomy
skyfield               - Elegant astronomy for Python
gary                   - Galactic astronomy and gravitational dynamics.
novas                  - The United States Naval Observatory NOVAS astronomy library
astroobs               - Provides astronomy ephemeris to plan telescope observations
PyAstronomy            - A collection of astronomy related tools for Python.
...

pip has a series of subcommands: "search", "install", "uninstall", "freeze" and so on (see Installing Python Modules Get the complete document of pip)

Specify the name of the package to install the latest version of the package:

(tutorial-env) $ pip install novas
Collecting novas
  Downloading novas-3.1.1.3.tar.gz (136kB)
Installing collected packages: novas
  Running setup.py install for novas
Successfully installed novas-3.1.1.3

You can also give the package name and follow == and version number to install the specified version of the package:

(tutorial-env) $ pip install requests==2.6.0
Collecting requests==2.6.0
  Using cached requests-2.6.0-py2.py3-none-any.whl
Installing collected packages: requests
Successfully installed requests-2.6.0

If you run this command again, pip will notice that the requested version is installed and nothing will be done. You can provide different version numbers or specified version packages, or run pip install --upgrade to the latest version:

(tutorial-env) $ pip install --upgrade requests
Collecting requests
Installing collected packages: requests
  Found existing installation: requests 2.6.0
    Uninstalling requests-2.6.0:
      Successfully uninstalled requests-2.6.0
Successfully installed requests-2.7.0

The command pip uninstall follows one or more package names or removes packages from the virtual environment.

pip show shows information about the specified package:

(tutorial-env) $ pip show requests
---
Metadata-Version: 2.0
Name: requests
Version: 2.7.0
Summary: Python HTTP for Humans.
Home-page: http://python-requests.org
Author: Kenneth Reitz
Author-email: me@kennethreitz.com
License: Apache 2.0
Location: /Users/akuchling/envs/tutorial-env/lib/python3.4/site-packages
Requires:

pip list shows all installed packages in the virtual environment:

(tutorial-env) $ pip list
novas (3.1.1.3)
numpy (1.9.2)
pip (7.0.3)
requests (2.7.0)
setuptools (16.0)

pip freeze produces a similar list of installed packages, but its output uses the format expected by pip install. The usual convention is to put the output in the requirements.txt file:

(tutorial-env) $ pip freeze > requirements.txt
(tutorial-env) $ cat requirements.txt
novas==3.1.1.3
numpy==1.9.2
requests==2.7.0

Next, requirements.txt can be submitted to version control and assembled as part of the application. Users can install all required packages using the install-r command:

(tutorial-env) $ pip install -r requirements.txt
Collecting novas==3.1.1.3 (from -r requirements.txt (line 1))
  ...
Collecting numpy==1.9.2 (from -r requirements.txt (line 2))
  ...
Collecting requests==2.7.0 (from -r requirements.txt (line 3))
  ...
Installing collected packages: novas, numpy, requests
  Running setup.py install for novas
Successfully installed novas-3.1.1.3 numpy-1.9.2 requests-2.7.0

pip has many other options. consult Installing Python Modules Get the complete pip document. If you develop a package and want it to be available on Python Package Index, consult Distributing Python Modules.