Working with Virtual Environments in Python for Beginners

Working with Virtual Environments in Python for Beginners - TutLinks
Working with Virtual Environments in Python for Beginners – TutLinks

Virtual Environments in Python provides an isolated and self contained instance of the version of Python and all other modules or packages that a specific project is dependent on. Virtual Environments appear in the form of directory with the name of the environment we choose and contain the version of Python, scripts, libraries and packages that are available with in the virtual environment.

Table of Contents

What you will learn

In this tutorial we will see a brief introduction to virtual environments. By the end of this tutorial you will understand the following:

  • Problems when virtual environments are not used
  • What is the need for Virtual Environments
  • Why it is suggested to work with virtual environments in Python
  • How to create, activate deactivate virtual environment in Linux and Windows
  • Install a package in activated virtual environment
  • Portability limitation of virtual environments
  • How to overcome the Portability limitation with pip freeze

Working with Virtual Environments

Suppose that you are working on two different Python Projects and one of the projects requires the very old absinthe version 0.11 of Flask and another requires version 1.1.1.

The problem occurs if you perform pip install flask directly on a PC level. Because the different projects that you work on or Python applications running on a PC require specific versions of Flask. Installing packages or Python modules at the system level may result in malfunctioning of other Python projects that have version specific requirements of their dependent modules. In order to get rid of this problem, Python has a concept of Virtual Environments.

The beauty of working with Python is it allows you to create and work in isolated environments for each of the Python project you work on a PC.

Virtual Environments in Python provides an isolated and self contained instance of the version of Python and all other modules or packages that a specific project is dependent on. Virtual Environments appear in the form of directory with the name of the environment we choose and contain the version of Python, scripts, libraries and packages that are available with in the virtual environment.

Working in activated virtual environments will scope the installation of Python packages to the active virtual environment. This facilitates working on different Python projects on a single PC that require different versions of the same package.

Create a Virtual Environment

The following is the standard command to set up a virtual environment.

python -m venv {name-of-virtualenv}
  • -m : is the module argument that takes the name of the module.
  • venv : is the name of module to create virtual environment. The standard module venv comes default with installation of Python.
  • {name-of-virtualenv} : is the name of the virtual environment of our choice.

Create a Virtual Environment named env or any name of your choice by typing the following command. Let’s name our virtual environment as env

Create a Python Virtual Environment in Windows OS

python -m venv env

Create a Python Virtual Environment in Unix based OS

python3 -m venv env

The above command tells the Python to use venv module that helps to create the virtual environment with the name env.

Virtual Environment Tree Directory Structure

The created virtual environment env will have the following folder structure

.
└───env
    ├───Include
    ├───Lib
    │   └───site-packages
    │       ├───pip
    │       │   ├───_internal
    │       │   │   ├───cli
    │       │   │   │   └───__pycache__
    │       │   │   ├───commands
    │       │   │   │   └───__pycache__
    │       │   │   ├───distributions
    │       │   │   │   └───__pycache__
    │       │   │   ├───models
    │       │   │   │   └───__pycache__
    │       │   │   ├───operations
    │       │   │   │   └───__pycache__
    │       │   │   ├───req
    │       │   │   │   └───__pycache__
    │       │   │   ├───utils
    │       │   │   │   └───__pycache__
    │       │   │   ├───vcs
    │       │   │   │   └───__pycache__
    │       │   │   └───__pycache__
    │       │   ├───_vendor
    │       │   │   ├───cachecontrol
    │       │   │   │   ├───caches
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───certifi
    │       │   │   │   └───__pycache__
    │       │   │   ├───chardet
    │       │   │   │   ├───cli
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───colorama
    │       │   │   │   └───__pycache__
    │       │   │   ├───distlib
    │       │   │   │   ├───_backport
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───html5lib
    │       │   │   │   ├───filters
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───treeadapters
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───treebuilders
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───treewalkers
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───_trie
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───idna
    │       │   │   │   └───__pycache__
    │       │   │   ├───lockfile
    │       │   │   │   └───__pycache__
    │       │   │   ├───msgpack
    │       │   │   │   └───__pycache__
    │       │   │   ├───packaging
    │       │   │   │   └───__pycache__
    │       │   │   ├───pep517
    │       │   │   │   └───__pycache__
    │       │   │   ├───pkg_resources
    │       │   │   │   └───__pycache__
    │       │   │   ├───progress
    │       │   │   │   └───__pycache__
    │       │   │   ├───pytoml
    │       │   │   │   └───__pycache__
    │       │   │   ├───requests
    │       │   │   │   └───__pycache__
    │       │   │   ├───urllib3
    │       │   │   │   ├───contrib
    │       │   │   │   │   ├───_securetransport      
    │       │   │   │   │   │   └───__pycache__       
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───packages
    │       │   │   │   │   ├───backports
    │       │   │   │   │   │   └───__pycache__       
    │       │   │   │   │   ├───rfc3986
    │       │   │   │   │   │   └───__pycache__       
    │       │   │   │   │   ├───ssl_match_hostname    
    │       │   │   │   │   │   └───__pycache__       
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───util
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───webencodings
    │       │   │   │   └───__pycache__
    │       │   │   └───__pycache__
    │       │   └───__pycache__
    │       ├───pip-19.2.3.dist-info
    │       ├───pkg_resources
    │       │   ├───extern
    │       │   │   └───__pycache__
    │       │   ├───_vendor
    │       │   │   ├───packaging
    │       │   │   │   └───__pycache__
    │       │   │   └───__pycache__
    │       │   └───__pycache__
    │       ├───setuptools
    │       │   ├───command
    │       │   │   └───__pycache__
    │       │   ├───extern
    │       │   │   └───__pycache__
    │       │   ├───_vendor
    │       │   │   ├───packaging
    │       │   │   │   └───__pycache__
    │       │   │   └───__pycache__
    │       │   └───__pycache__
    │       ├───setuptools-41.2.0.dist-info
    │       └───__pycache__
    └───Scripts

Activate the Virtual Environment

Once we have the virtual environment created, we need to activate that virtual environment.

Type in the following command based on your operating system to activate the virtual environment.

Activate Python virtual environment for Windows OS

Type the following command to activate Python virtual environment in Windows OS

env\Scripts\activate

Activate Python virtual environment for Unix based OS

Type the following command to activate Python virtual environment in Unix based OS

source ./env/bin/activate

After successful activation of the virtual environment you will see the name of the environment that got created will be prefixed to the command line terminal in rounded brackets. On Windows OS, this looks like

(env) c:\Users\navule\helloflask

If you have given any other name for the virtual environment, you should see the name in closed parentheses accordingly.

Upgrade pip

The first thing you should do after installing and activating a virtual environment is to upgrade pip. pip is a module installer in Python. That means, pip will help us to install Python modules so that we can consume it in our application.

pip in itself is a module that comes by default with Python installation. There are frequent upgrades to pip. Hence it is a best practice to upgrade the pip so that it will always get the dependencies from the trusted sources for our project.

Run the following command to upgrade pip.

python -m pip install --upgrade pip

Install Modules in activated virtual environment

From now on, you can install any project related modules based on the requirement. Lets say the application needs to have flask module, you perform the module installation by running the command in the virtual environment activated shell as follows:

pip install flask

Now if you look at the site-packages directory you will find the Flask package along with its dependent packages like jinja, werkzeug are also installed.

.
└───env
    ├───Include
    ├───Lib
    │   └───site-packages
    │       ├───click
    │       │   └───__pycache__
    │       ├───Click-7.0.dist-info
    │       ├───flask
    │       │   ├───json
    │       │   │   └───__pycache__
    │       │   └───__pycache__
    │       ├───Flask-1.1.1.dist-info
    │       ├───itsdangerous
    │       │   └───__pycache__
    │       ├───itsdangerous-1.1.0.dist-info
    │       ├───jinja2
    │       │   └───__pycache__
    │       ├───Jinja2-2.11.1.dist-info
    │       ├───markupsafe
    │       │   └───__pycache__
    │       ├───MarkupSafe-1.1.1.dist-info
    │       ├───pip
    │       │   ├───_internal
    │       │   │   ├───cli
    │       │   │   │   └───__pycache__
    │       │   │   ├───commands
    │       │   │   │   └───__pycache__
    │       │   │   ├───distributions
    │       │   │   │   └───__pycache__
    │       │   │   ├───index
    │       │   │   │   └───__pycache__
    │       │   │   ├───models
    │       │   │   │   └───__pycache__
    │       │   │   ├───network
    │       │   │   │   └───__pycache__
    │       │   │   ├───operations
    │       │   │   │   ├───build
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───install
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───req
    │       │   │   │   └───__pycache__
    │       │   │   ├───utils
    │       │   │   │   └───__pycache__
    │       │   │   ├───vcs
    │       │   │   │   └───__pycache__
    │       │   │   └───__pycache__
    │       │   ├───_vendor
    │       │   │   ├───cachecontrol
    │       │   │   │   ├───caches
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───certifi
    │       │   │   │   └───__pycache__
    │       │   │   ├───chardet
    │       │   │   │   ├───cli
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───colorama
    │       │   │   │   └───__pycache__
    │       │   │   ├───distlib
    │       │   │   │   ├───_backport
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───html5lib
    │       │   │   │   ├───filters
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───treeadapters
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───treebuilders
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───treewalkers
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───_trie
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───idna
    │       │   │   │   └───__pycache__
    │       │   │   ├───msgpack
    │       │   │   │   └───__pycache__
    │       │   │   ├───packaging
    │       │   │   │   └───__pycache__
    │       │   │   ├───pep517
    │       │   │   │   └───__pycache__
    │       │   │   ├───pkg_resources
    │       │   │   │   └───__pycache__
    │       │   │   ├───progress
    │       │   │   │   └───__pycache__
    │       │   │   ├───pytoml
    │       │   │   │   └───__pycache__
    │       │   │   ├───requests
    │       │   │   │   └───__pycache__
    │       │   │   ├───urllib3
    │       │   │   │   ├───contrib
    │       │   │   │   │   ├───_securetransport      
    │       │   │   │   │   │   └───__pycache__       
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───packages
    │       │   │   │   │   ├───backports
    │       │   │   │   │   │   └───__pycache__       
    │       │   │   │   │   ├───ssl_match_hostname    
    │       │   │   │   │   │   └───__pycache__       
    │       │   │   │   │   └───__pycache__
    │       │   │   │   ├───util
    │       │   │   │   │   └───__pycache__
    │       │   │   │   └───__pycache__
    │       │   │   ├───webencodings
    │       │   │   │   └───__pycache__
    │       │   │   └───__pycache__
    │       │   └───__pycache__
    │       ├───pip-20.0.2.dist-info
    │       ├───pkg_resources
    │       │   ├───extern
    │       │   │   └───__pycache__
    │       │   ├───_vendor
    │       │   │   ├───packaging
    │       │   │   │   └───__pycache__
    │       │   │   └───__pycache__
    │       │   └───__pycache__
    │       ├───setuptools
    │       │   ├───command
    │       │   │   └───__pycache__
    │       │   ├───extern
    │       │   │   └───__pycache__
    │       │   ├───_vendor
    │       │   │   ├───packaging
    │       │   │   │   └───__pycache__
    │       │   │   └───__pycache__
    │       │   └───__pycache__
    │       ├───setuptools-41.2.0.dist-info
    │       ├───werkzeug
    │       │   ├───debug
    │       │   │   ├───shared
    │       │   │   └───__pycache__
    │       │   ├───middleware
    │       │   │   └───__pycache__
    │       │   ├───wrappers
    │       │   │   └───__pycache__
    │       │   └───__pycache__
    │       ├───Werkzeug-1.0.0.dist-info
    │       └───__pycache__
    └───Scripts

Limitations with Virtual Environments venv

Virtual Environments are not Portable

Virtual environments are not portable. Virtual Environments will not work if you move them to a different directory within the same PC or across different PCs.

Though the virtual environments offer a great isolation to work with different versions of a module across projects by providing isolation between the projects, they completely take out the freedom of portability.

Hence it is not suggested to check-in the virtual environment directory to the source control. And the relative path of the virtual environment directory must be added to the .gitignore file if you are using git as source control for version tracking.

Freezing the Virtual Environment

To overcome the portability limitation, you can freeze the virtual environment to a text file that has all the module information.

The following command when run in an activated virtual environment will freeze all the modules with version information to a file name of our choice.

pip freeze > requirements.txt

The above command with the help of pip module freezes the requirements that the active virtual environment is dependent on to the file named requirements.txt. The requirements.txt file will be created by pip if it does not already exists and updates if it exists already.

If we look at contents of requirements.txt, it will show something similar to this.

# c:\Users\navule\helloflask\requirements.txt
Click==7.0
Flask==1.1.1
itsdangerous==1.1.0
Jinja2==2.11.1
MarkupSafe==1.1.1
Werkzeug==1.0.0

The file requirements.txt must be checked-in to the version control like git.

Restore the Python packages from requirements.txt

When the repository is ported to the other location or PCs, we can then perform the following activities to restore the packages that our app is dependent on.

  • Check out our project on a different PC
  • Create a new and fresh virtual environment
  • Activate the virtual environment
  • Restore/ Install the modules that the app is dependent on from requirements.txt

Run the following command to restore or install the modules from the requirements.txt

pip install -r requirements.txt

The above command tells the pip module of the activated virtual environment to recursively read and install all the modules mentioned in the requirements.txt

When to perform freeze

The freeze of modules is very important whenever we install a new package via pip. Running pip freeze > requirements.txt will automatically update the existing requirements file and thus when a new developer or a new deployment happens, the new or updated installer packages will be installed with pip install -r requirements.txt

Deactivating the Virtual Environment

Once you are done working with the application, you are suggested to deactivate the virtual environment. Run the following command to deactivate the virtual environment.

deactivate

Watch 📺 Demonstration of working with Virtual Environment on CentOS

Watch 📺 the real demo of creating Virtual Environment in CentOS.

Video

You can watch this video to understand how to work with virtual environments in Python. This video tutorial uses virtualenv to create virtual environment.

Tags: , , , , , ,

Navule Pavan Kumar Rao

I am a Full Stack Software Engineer with the Product Development experience in Banking, Finance, Corporate Tax and Automobile domains. I use SOLID Programming Principles and Design Patterns and Architect Software Solutions that scale using C#, .NET, Python, PHP and TDD. I am an expert in deployment of the Software Applications to Cloud Platforms such as Azure, GCP and non cloud On-Premise Infrastructures using shell scripts that become a part of CI/CD. I pursued Executive M.Tech in Data Science from IIT, Hyderabad (Indian Institute of Technology, Hyderabad) and hold B.Tech in Electonics and Communications Engineering from Vaagdevi Institute of Technology & Science.

This Post Has 7 Comments

Leave a Reply