Canopy Command Line Interface (CLI)

If, like most users, you do not share a machine and want the ability to use both the Canopy GUI and the command line, then you probably do not need to use the Canopy Command Line Interface (CLI).

The Canopy CLI is primarily intended for use in multi-user or command-line-only (non-GUI) configurations, by users comfortable working at the system command prompt. In particular, it is useful for administrators writing automation scripts for Canopy, such as for cluster deployments.

This section covers details of the CLI, installation of Canopy via the command line in a non-GUI environment, configuring Canopy for use in a multi-user environment, and setting up a plain Python environment similar to EPD or the Python.org Python distribution. In order to provide some background for these instructions, the next sub-section introduces virtual environments and describes how Canopy uses virtual environments.

Canopy Core and virtual environments

The Canopy installer installs what is referred to as “Canopy Core”. This is the core Python environment and a minimum set of packages needed to bootstrap Canopy itself. This environment should not be used as a Python environment because it is intended to be locked down to facilitate Canopy’s automatic update mechanism and reliable startup and fail-safe recovery mode. What’s the alternative?

Virtual environments, specifically ‘venv’ which we backported from Python 3.x, are a technology that enables the creation of multiple, lightweight, independent Python environments. Each virtual environment appears to be a self-contained Python installation, but loads the Python standard library and other common resources from a common base Python installation. This significantly reduces disk usage and sometimes load time. Optionally, virtual environments can also load packages from the base Python environment, again saving disk space when common, large packages are centrally installed.

Canopy uses virtual environments in two ways – to provide a protected Python environment for the Canopy GUI application to run in, and to provide one or more User Python environments which you can customize and run your own code in. Canopy Core is the base for each of these virtual environments, providing the core Python components and several common, large packages such as Qt and PySide.

Canopy Core can optionally be updated, such as to move to a new version of Python, and each of the virtual environments will be updated automatically as well. This eliminates the need to install a new Python environment and then re-install any third-party packages into that new environment just to update Python.

The Canopy GUI automatically creates two virtual environments named ‘System’ and ‘User’. System is where the Canopy GUI runs; no user code runs in this environment. Updates to this virtual environment are done via the Canopy update mechanisms. The User environment environment is where the IPython kernel and all user code run. This virtual environment is managed by Package Manager; any packages can be updated and installed without fear of disrupting the GUI. Similarly, updates to the Canopy GUI will not affect packages installed in the User environment and break your code. This two-environment structure is created automatically when the Canopy GUI is first started, or when the non-GUI setup option is used, described below.

Users wishing to use Canopy as a plain Python environment without the GUI can easily create one or more Python environments for this purpose. See Creating an EPD-like Python environment below for details.

Command line interface

The Canopy Command Line Interface (CLI) is available via the canopy_cli command, available both in the Canopy Core install, and in the Canopy User Python environment. The tables at the end of this sub-section list the default install paths for the Canopy CLI, on each operating system.

The canopy_cli command has the following sub-command structure:

canopy_cli [<sub-command>] [options] [positional arguments]

The following table shows the available sub-commands and their options and arguments.

Function Sub-command usage
Create a common install post-install-setup [-h] <various options>
Update Canopy update [-h] <venv-directory>
Create a Canopy venv setup [-h] [-v] [–set-default|–no-default] <directory>
Low-level venv creation venv [-h] [-s] [-u] <directory>
Register a Venv (Windows) register-venv <venv-directory>
Unregister a Venv (Windows) unregister-venv <venv-directory>

The CLI can also be run with no sub-command, as described below in these sections:

canopy_cli update

The canopy_cli update command downloads and applies any available updates to the Canopy Application. This is a command-line-only (no GUI popup) convenience, equivalent to “Canopy Application Updates” within the Canopy GUI.

It is possible that several updates could be available, which would need to be applied sequentially. This might occur if your current version of Canopy is a bit old, or for other technical reasons. In this case, one update would be downloaded and applied at each invocation of this update command.

A single update will modify either the Canopy Core environment, or the Canopy System virtual environment, but not both.

The Canopy Core environment contains Python itself, and a collection of packages. Updates to Canopy Core may affect both the Canopy GUI application and the Canopy User Python environment, both of which inherit from Core.

Updates to Core Python will always affect Canopy User Python environment. Updates to a Core package will only affect the User Python environment if that same package is not installed in User. Most such packages are installed in User, and therefore override (“shadow”) the Core package being updated. Note that if a Core package is updated, and not shadowed in User (for example MKL), then this could affect the operation of its dependent packages in User (for example numpy).

If the update command updates the Canopy System environment, then this only affects the Canopy GUI application itself, not the Canopy User Python environment.

In the case of an update to the Canopy Core environment, the update command’s optional <venv-directory> can point to a venv that was previously created by the user using either canopy_cli venv -s or venv -s. After the updated Core environment has been created, this will be updated so that it inherits from the new Core environment, rather than the old one.

canopy_cli setup

The canopy_cli setup command is used to construct a full, user-accessible Python virtual environment similar to the User Python environment that the Canopy GUI sets up (as described above). This Python virtual environment includes all of the Canopy packages that were bundled into your Canopy installer. This environment behaves equivalently to an EPD (Enthought Python Distribution) tree or an install of Python from Python.org. Please note that this Python virtual environment inherits from the Canopy Core Python environment and not from the User environment that one may expect. For more information about Canopy core and the environments, refer to this FAQ.

The -v option turns on verbose mode. <directory> specifies a directory in the file system where the Python environment will be created.

For example:

canopy_cli setup C:\Python27

This constructs a user-accessible Python environment in C:\Python27 on a Windows system, mirroring the behavior of a default EPD install. canopy_cli setup can be used multiple times on a single system to create multiple user-accessible Python virtual environments in different directories.

If --set-default is specified, the Canopy CLI automatically adds the appropriate paths to your environment to make this Python environment the default. On Windows this means adding to your PATH environment variable; on Mac OS and Linux the appropriate commands are appended to an existing .bashrc file.

canopy_cli venv

canopy_cli venv is a low-level command for simply creating a new virtual environment. This virtual environment is similar to what is produced by the canopy_cli setup command but with no packages installed.

By default, a venv created via the canopy_cli venv command will not load packages from its base Python environment. However, if the -s switch is specified (short for --system-site-packages), then the Python interpreter in the new venv will have access both to packages installed in the venv itself, and to packages in the base environment.

How is the base Python environment determined? The base Python environment is the same environment that was used when canopy_cli venv was run. For example:

C:\Program Files\Enthought\Canopy\App\Canopy_cli.exe venv C:\Users\Username\my_venv

This constructs an empty virtual environment in C:\Users\Username\my_venv and uses the Canopy Core install as the base. Since the -s switch was not provided, no packages will be inherited from Canopy Core.

Now consider the following commands:

C:\Program Files\Enthought\Canopy\App\Canopy_cli.exe setup C:\Users\Username\my_venv
C:\Users\Username\my_venv\Scripts\venv.exe -s C:\Users\Username\my_venv_test

In this scenario, the first command constructs a full Python environment in C:\Users\Username\my_venv, including at least all of the Canopy Express packages. The second command then creates a derivative environment in C:\Users\Username\my_venv_test and, because -s was specified, this environment inherits access to any packages in C:\Users\Username\my_venv that aren’t superseded by package installations in the C:\Users\Username\my_venv_test venv. This is a particularly useful technique when experimenting with package updates; you can quickly construct the test environment mirroring an existing one and install new package variations for testing code without disturbing an existing known-good environment. Note that in the second command we do not use Canopy_cli.exe. This is because in a venv constructed via the setup subcommand, there is no access to the Canopy_cli.exe however, there is a venv.exe command which serves the same purpose and has the same options as the venv subcommand.

The -u switch causes an existing virtual environment to be updated if any updates have been applied to the base Python environment. Typically this is used after the canopy_cli update command has applied patches to the base Python environment.

Each of the sub-commands supports the option -h or --help which provides a summary of the information on this page.

canopy_cli register / unregister

These commands are specific to users running on Microsoft Windows and support integration with Microsoft’s Python Tools for Visual Studio (PTVS) tool. Canopy provides a custom integration that allows PTVS to automatically detect the User Python environment by default. This integration doesn’t have a way to detect Canopy virtual environments created by the user, however, unless it gets a little help.

The register command allows a custom virtual environment to be registered such that it will be automatically detected by Canopy’s PTVS integration and will show up in the list of recognized Python environments in PTVS. The unregister command simply reverses this process and removes the environment from the list.

A typical scenario is to create a venv for a particular application or experiment and then register it to use PTVS to debug the code. For example:

canopy_cli.exe venv -s c:\projects\new_app1      # Constructs the venv as above
canopy_cli.exe register c:\projects\new_app1

After the register command completes the new venv will appear automatically in the PTVS “Python Environments” tab. This environment can be de-registered like this:

canopy_cli.exe unregister C:\projects\new_app1

Note that it can be unregistered even if it has already been deleted as long as the path matches the original location.

Canopy CLI’s default location in the Canopy Core

Platform Version Location
Windows XP 64 C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy\App\Canopy_cli.exe
Windows XP 32 C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy32\App\Canopy_cli.exe
Windows 7, 8 & Vista 64 C:\Users\<username>\AppData\Local\Enthought\Canopy\App\Canopy_cli.exe
Windows 7, 8 & Vista 32 C:\Users\<username>\AppData\Local\Enthought\Canopy32\App\Canopy_cli.exe
Mac OS X 64 /Applications/Canopy.app/Contents/MacOS/Canopy_cli
Mac OS X 32 /Applications/Canopy.app/Contents/MacOS/Canopy_cli
Linux 64 ~/Canopy/canopy_cli
Linux 32 ~/Canopy/canopy_cli

Canopy CLI’s default location in the Canopy User Python environment

Platform Version Location
Windows XP 64 C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy\User\Scripts\canopy_cli.exe
Windows XP 32 C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy32\User\Scripts\canopy_cli.exe
Windows 7, 8 & Vista 64 C:\Users\<username>\AppData\Local\Enthought\Canopy\User\Scripts\canopy_cli.exe
Windows 7, 8 & Vista 32 C:\Users\<username>\AppData\Local\Enthought\Canopy32\User\Scripts\canopy_cli.exe
Mac OS X 64 ~/Library/Enthought/Canopy_64bit/User/bin/canopy_cli
Mac OS X 32 ~/Library/Enthought/Canopy_32bit/User/bin/canopy_cli
Linux 64 ~/Enthought/Canopy_64bit/User/bin/canopy_cli
Linux 32 ~/Enthought/Canopy_32bit/User/bin/canopy_cli

Scenario: Creating an EPD-like Python environment (command-line only)

Update: As of Canopy version 1.6, this CLI option is deprecated. If you are an institutional Canopy subscriber and want to create a command-line-only Python installation, please contact Enthought technical support for a simpler solution.

All Canopy users have access to Python at the command line. In a standard Canopy installation, you may also access Python via the Canopy GUI application (IDE).

If you do not wish to use the Canopy GUI application, and would like to use Canopy’s Python only as a command-line Python distribution similar to EPD, this is easily done. To set up such an EPD-like distribution, you can execute the following command:

On Windows:

Canopy\App\Canopy_cli.exe --no-gui-setup setup C:\Python27

On Mac:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup setup ~/canopy

On Linux:

~/Canopy/canopy_cli --no-gui-setup setup ~/canopy

The target directory is determined by the user. It will be created for you wherever you specify (as long as you have sufficient permissions).

Please note that this Python virtual environment inherits from the Canopy Core Python environment and not from the User environment that one may expect. (Indeed, typically in this scenario, there is no User Python environment). For more information about Canopy core and the environments, refer to this FAQ.

If you want to make this Python environment the default on your system, specify the --set-default switch. Specifying this option will cause Canopy to add the appropriate bin directory (Scripts directory on Windows) to your PATH environment variable. On Mac OS and Linux systems, this is done by appending a line to your ~/.bash_profile file which activates the correct virtual environment. On Windows, this Python environment is also added to the system registry so third-party tools can correctly find it.

The -v switch can also be used to enable verbose output.

The --no-gui-setup switch shown in each of the above examples is optional, but is recommended to avoid possible GUI popups during the setup procedure.

This installed Canopy python distribution can now be used like any other python environment. The following are a few examples (but not limited to) of the usage:

  1. Install a package of your choice in this environment using easy_install or enpkg.
  2. Run python or ipython consoles.

Please note that this is a virtual environment which inherits from the Canopy core so the Canopy Core can not be uninstalled without breaking this environment.

To uninstall the Canopy python distribution that has been installed as above, just delete the directory into which the distribution was installed.

On Windows:

rmdir /s C:\Python27

asuming C:\Python27 is the installation directory

On Mac or Linux:

rm -rf ~/canopy

assuming ~/canopy is the installation directory

Scenario: Creating a standalone customizable virtual environment

Using the Canopy Core one can manage virtual environments that includes creating, upgrading and using the virtual environments.

On Windows:

Canopy\App\Canopy_cli.exe venv C:\Users\<username>\my_venv

assuming C:\Users\<username>\my_venv is the installation directory for the virtual environment.

This will be an empty virtual environment. In order to provide access to system site packages, do the following:

Canopy\App\Canopy_cli.exe venv -s C:\Users\<username>\my_venv

To upgrade this virtual environment, run the following command:

Canopy\App\Canopy_cli.exe venv -u C:\Users\<username>\my_venv

The environment will need to be upgraded if the core Canopy application was changed (re-installed or upgraded).

In order to use this virtual environment, you need to activate which can be done as follows:

C:\Users\<username>\my_venv\Scripts\activate.bat

To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command:

deactivate

On Mac OS X:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv

assuming ~/my_venv is the installation directory for the virtual environment

This will be an empty virtual environment. In order to provide access to system site packages, do the following:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv -s

To upgrade this virtual environment, run the following command:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv -u

In order to use this virtual environment, you need to activate it, which can be done as follows:

source ~/my_venv/bin/activate

To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command:

deactivate

On Linux:

~/Canopy/canopy_cli venv ~/my_venv

assuming ~/my_venv is the installation directory for the virtual environment

This will be an empty virtual environment. In order to provide access to system site packages, do the following:

~/Canopy/canopy_cli venv ~/my_venv -s

To upgrade this virtual environment, run the following command:

~/Canopy/canopy_cli venv ~/my_venv -u

In order to use this virtual environment, you need to activate which can be done as follows:

source ~/my_venv/bin/activate

To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command.

deactivate

Scenario: Setting up the User and System environments without a setup GUI

Normally, when Canopy is launched for the first time by clicking on its icon, it will open a GUI dialog window to prompt the user for the locations of the User and System environments. This will not work if you are remotely connected to the machine via a text-only interface such as ssh. In such a scenario, you can perform the environment setup without a GUI, by using the CLI --no-gui-setup option:

Platform Commmand
Windows Canopy\app\Canopy_cli.exe --no-gui-setup
Mac OS X /Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup
Linux ~/Canopy/canopy_cli --no-gui-setup

By default,the User and System environments will be created in the same default directories as if Canopy had been launched with a normal GUI dialog (Where are all of the Python packages in my User Python Environment?). Alternatively, the grandparent directory of the User and System directories can be explicitly specified from the command line via the --install-dir switch. For example:

On Windows:

Canopy\App\Canopy_cli.exe --no-gui-setup --install-dir C:\Users\<username>

On Mac:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup --install-dir /usr/local

On Linux:

~/Canopy/canopy_cli --no-gui-setup --install-dir /usr/local

On Mac OS and Linux, this example would create /usr/local/Canopy_64bit for Canopy 64-bit and /usr/local/Canopy_32bit for Canopy 32-bit install. On Windows, it would create C:\Users\<username>\Canopy for Canopy 64-bit and C:\Users\<username>\Canopy32 for Canopy 32-bit. Under this installation directory, you will find the User and System directories which are the Canopy User Python and Canopy System environments, respectively. Under the User/bin (User\Scripts on Windows) directory you will find two convenience scripts, canopy_cli (.exe on Windows) and canopy (.exe on Windows), which are described in detail here.

Note about file permissions on Mac and Linux

The setup commands shown above should generally be run as an admin user, but not as the root user, i.e. not using default sudo. You would need to ensure that the admin user has write access, and that all ordinary users have read and execute access, to the target directory (/usr/local/Canopy_64bit in this example, or whatever target directory you choose).

Scenario: Creating a Canopy common install

On multi-user systems, Canopy supports “common install” setup, where users share a common Python and set of packages, installed in a central location. This reduces disk usage because there is no need for packages to be duplicated across users. It also provides a consistent base of package versions for all users.

In Canopy 1.6, Canopy’s “open common install” (originally named just “common install”) has been deprecated. The new recommended method for installing Canopy for multiple users is the “managed common install”. For information about these options, please read the Knowledge Base article Canopy Common Install Options.