Canopy Command Line Interface (CLI)

(The CLI is new in Canopy 1.1. This section of the documentation will be further revised after Canopy 1.1 is released. You may read the latest revision online.)

The command line interface (CLI) is intended for users comfortable working at the command prompt on their system, users writing automation scripts around Canopy such as for cluster deployments, and users working in an environment where a graphical user interface (GUI) is not available.

This section covers details of the CLI, installation of Canopy via the command line in a non-GUI environment, configuring Canopy for use in a multi-user environment, and setting up a plain Python environment similar to EPD or the Python.org Python distribution. In order to provide some background for these instructions, an initial sub-section introduces virtual environments and describes how Canopy uses virtual environments.

Canopy Core and virtual environments

The Canopy installer installs what is referred to as “Canopy Core”. This is the core Python environment and a minimum set of packages needed to bootstrap Canopy itself. This environment should not be used as a Python environment because it is intended to be locked down to facilitate Canopy’s automatic update mechanism and reliable startup and fail-safe recovery mode. What’s the alternative?

Virtual environments, specifically ‘venv’ which we backported from Python 3.x, are a technology that enables the creation of multiple, lightweight, independent Python environments. Each virtual environment appears to be a self-contained Python installation, but loads the Python standard library and other common resources from a common base Python installation. This significantly reduces disk usage and sometimes load time. Optionally, virtual environments can also load packages from the base Python environment, again saving disk space when common, large packages are centrally installed.

Canopy uses virtual environments in two ways – to provide a protected Python environment for the Canopy GUI application to run in, and to provide one or more User Python environments which you can customize and run your own code in. Canopy Core is the base for each of these virtual environments, providing the core Python components and several common, large packages such as Qt and PySide.

Canopy Core can optionally be updated, such as to move to a new version of Python, and each of the virtual environments will be updated automatically as well. This eliminates the need to install a new Python environment and then re-install any third-party packages into that new environment just to update Python.

The Canopy GUI automatically creates two virtual environments named ‘System’ and ‘User’. System is where the Canopy GUI runs; no user code runs in this environment. Updates to this virtual environment are done via the Canopy update mechanisms. The User environment environment is where the IPython kernel and all user code run. This virtual environment is managed by Package Manager; any packages can be updated and installed without fear of disrupting the GUI. Similarly, updates to the Canopy GUI will not affect packages installed in the User environment and break your code. This two-environment structure is created automatically when the Canopy GUI is first started, or when the non-GUI setup option is used, described below.

Users wishing to use Canopy as a plain Python environment without the GUI can easily create one or more Python environments for this purpose. See Creating an EPD-like Python environment below for details.

Command line interface

The Canopy Command Line Interface (CLI) is available via the canopy_cli command, available both in the Canopy Core install, and in the Canopy User Python environment. The tables at the end of this sub-section list the default install paths for the Canopy CLI, on each operating system.

The canopy_cli command has the following sub-command structure:

canopy_cli [<sub-command>] [options] [positional arguments]

The following table shows the available sub-commands and their options and arguments.

Function Sub-command usage
Update Canopy update [-h] <venv-directory>
Create a Canopy venv setup [-h] [-v] [–default] <directory>
Low-level venv creation venv [-h] [-s] [-u] <directory>

The CLI can also be run with no sub-command, to do an initial no-GUI and/or system-wide installation, as described in the final sub-sections.

canopy_cli update

The canopy_cli update command applies any Canopy platform updates that may be pending. These updates patch the Canopy Core install with updates to the Python environment itself and any packages installed in the core or in the System environment. This command will not pop up a GUI and is a convenience provided to update Canopy. The optional <venv-directory> can point to a venv that has been created by the user – System or User are automatically updated when the Canopy GUI is started. This optional directory will be upgraded after the canopy update is applied.

canopy_cli setup

The canopy_cli setup command is used to construct a full, user-accessible Python environment similar to the User Python environment that the Canopy GUI sets up (as described above). This Python environment has all of the Canopy Express packages installed, plus the enpkg command which can be used to install additional packages from the Canopy software repository. This environment behaves equivalently to an EPD (Enthought Python Distribution) tree or an install of Python from Python.org. The -v option turns on verbose mode. <directory> specifies a directory in the file system where the Python environment will be created.

For example:

canopy_cli setup C:\Python27

This constructs a user-accessible Python environment in C:\Python27 on a Windows system, mirroring the behavior of a default EPD install. canopy_cli setup can be used multiple times on a single system to create multiple user-accessible Python virtual environments in different directories.

If --default is specified, the Canopy CLI automatically adds the appropriate paths to your environment to make this Python environment the default. On Windows this means adding to your PATH environment variable; on Mac OS and Linux the appropriate commands are appended to an existing .bashrc file.

canopy_cli venv

canopy_cli venv is a low-level command for simply creating a new virtual environment. This virtual environment is similar to what is produced by the canopy_cli setup command but with no packages installed.

By default, a venv created via the canopy_cli venv command will not load packages from its base Python environment. However, if the -s switch is specified (short for --system-site-packages), then the Python interpreter in the new venv will have access both to packages installed in the venv itself, and to packages in the base environment.

How is the base Python environment determined? The base Python environment is the same environment that was used when canopy_cli venv was run. For example:

C:\Program Files\Enthought\Canopy\App\Canopy_cli.exe venv C:\Users\Username\my_venv

This constructs an empty virtual environment in C:\Users\Username\my_venv and uses the Canopy Core install as the base. Since the -s switch was not provided, no packages will be inherited from Canopy Core.

Now consider the following commands:

C:\Program Files\Enthought\Canopy\App\Canopy_cli.exe setup C:\Users\Username\my_venv
C:\Users\Username\my_venv\Scripts\venv.exe -s C:\Users\Username\my_venv_test

In this scenario, the first command constructs a full Python environment in C:\Users\Username\my_venv, including at least all of the Canopy Express packages. The second command then creates a derivative environment in C:\Users\Username\my_venv_test and, because -s was specified, this environment inherits access to any packages in C:\Users\Username\my_venv that aren’t superseded by package installations in the C:\Users\Username\my_venv_test venv. This is a particularly useful technique when experimenting with package updates; you can quickly construct the test environment mirroring an existing one and install new package variations for testing code without disturbing an existing known-good environment. Note that in the second command we do not use Canopy_cli.exe. This is because in a venv constructed via the setup subcommand, there is no access to the Canopy_cli.exe however, there is a venv.exe command which serves the same purpose and has the same options as the venv subcommand.

The -u switch causes an existing virtual environment to be updated if any updates have been applied to the base Python environment. Typically this is used after the canopy_cli update command has applied patches to the base Python environment.

Each of the sub-commands supports the option -h or --help which provides a summary of the information on this page.

Canopy CLI’s default location in the Canopy Core

Platform Version Location
Windows XP 64 C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy\App\Canopy_cli.exe
Windows XP 32 C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy32\App\Canopy_cli.exe
Windows 7 & Vista 64 C:\Users\<username>\AppData\Local\Enthought\Canopy\App\Canopy_cli.exe
Windows 7 & Vista 32 C:\Users\<username>\AppData\Local\Enthought\Canopy32\App\Canopy_cli.exe
Mac OS X 64 /Applications/Canopy.app/Contents/MacOS/Canopy_cli
Mac OS X 32 /Applications/Canopy.app/Contents/MacOS/Canopy_cli
Linux 64 ~/Canopy/canopy_cli
Linux 32 ~/Canopy/canopy_cli

Canopy CLI’s default location in the Canopy User Python environment

Platform Version Location
Windows XP 64 C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy\User\Scripts\canopy_cli.exe
Windows XP 32 C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy32\User\Scripts\canopy_cli.exe
Windows 7 & Vista 64 C:\Users\<username>\AppData\Local\Enthought\Canopy\User\Scripts\canopy_cli.exe
Windows 7 & Vista 32 C:\Users\<username>\AppData\Local\Enthought\Canopy32\User\Scripts\canopy_cli.exe
Mac OS X 64 ~/Library/Enthought/Canopy_64bit/User/bin/canopy_cli
Mac OS X 32 ~/Library/Enthought/Canopy_32bit/User/bin/canopy_cli
Linux 64 ~/Enthought/Canopy_64bit/User/bin/canopy_cli
Linux 32 ~/Enthought/Canopy_32bit/User/bin/canopy_cli

Scenario: Creating an EPD-like Python environment

If you do not wish to use the Canopy development environment and would like to use Canopy’s Python as a pure Python distribution similar to EPD, this is easily supported by Canopy. To setup an EPD-like distribution, you can execute the following command:

On Windows:

Canopy\App\Canopy_cli.exe setup C:\Python27

On Mac:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli setup ~/canopy

On Linux:

~/Canopy/canopy_cli setup ~/canopy

Note that the trailing directory is determined by the user, so it will be created for you and it could as well be any other directory.

If you want to make this Python environment the default on your system, specify the --default switch. Specifying this option will cause Canopy to add the appropriate bin directory (Scripts directory on Windows) to your PATH environment variable. On Mac OS and Linux systems, this is done by appending a line to your ~/.bash_profile file which activates the correct virtual environment. On Windows, this Python environment is also added to the system registry so third-party tools can correctly find it.

The -v switch can also be used to enable verbose output.

This installed Canopy python distribution can now be used like any other python environment. The following are a few examples (but not limited to) of the usage:

  1. Install a package of your choice in this environment using easy_install or enpkg.
  2. Run python or ipython consoles.

Please note that this is a virtual environment which inherits from the Canopy core so the Canopy Core can not be uninstalled without breaking this environment.

To uninstall the Canopy python distribution that has been installed as above, just delete the directory into which the distribution was installed.

On Windows:

rmdir /s C:\Python27

asuming C:\Python27 is the installation directory

On Mac or Linux:

rm -rf ~/canopy

assuming ~/canopy is the installation directory

Scenario: Creating a standalone customizable virtual environment

Using the Canopy Core one can manage virtual environments that includes creating, upgrading and using the virtual environments.

On Windows:

Canopy\App\Canopy_cli.exe venv C:\Users\<username>\my_venv

assuming C:\Users\<username>\my_venv is the installation directory for the virtual environment.

This will be an empty virtual environment. In order to provide access to system site packages, do the following:

Canopy\App\Canopy_cli.exe venv -s C:\Users\<username>\my_venv

To upgrade this virtual environment, run the following command:

Canopy\App\Canopy_cli.exe venv -u C:\Users\<username>\my_venv

The environment will need to be upgraded if the core Canopy application was changed (re-installed or upgraded).

In order to use this virtual environment, you need to activate which can be done as follows:

C:\Users\<username>\my_venv\Scripts\activate.bat

To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command:

deactivate

On Mac OS X:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv

assuming ~/my_venv is the installation directory for the virtual environment

This will be an empty virtual environment. In order to provide access to system site packages, do the following:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv -s

To upgrade this virtual environment, run the following command:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv -u

In order to use this virtual environment, you need to activate it, which can be done as follows:

source ~/my_venv/bin/activate

To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command:

deactivate

On Linux:

~/Canopy/canopy_cli venv ~/my_venv

assuming ~/my_venv is the installation directory for the virtual environment

This will be an empty virtual environment. In order to provide access to system site packages, do the following:

~/Canopy/canopy_cli venv ~/my_venv -s

To upgrade this virtual environment, run the following command:

~/Canopy/canopy_cli venv ~/my_venv -u

In order to use this virtual environment, you need to activate which can be done as follows:

source ~/my_venv/bin/activate

To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command.

deactivate

Scenario: Setting up the User environment without a GUI

Normally, when Canopy is launched for the first time by clicking on its icon, it will prompt the user for information using a GUI. This will not work if you are remotely connected to the machine via a text-only interface such as ssh. In such a scenario, you can continue with the setup by disabling the GUI as shown:

Platform Commmand
Windows Canopy\app\Canopy_cli.exe --no-gui-setup
Mac OS X /Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup
Linux ~/Canopy/canopy_cli --no-gui-setup

This will prompt for the installation directory. The installation directory can also be specified from the command line via the --install-dir switch. For example:

On Windows:

Canopy\App\Canopy_cli.exe --no-gui-setup --install-dir C:\Users\username

On Mac:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup --install-dir /usr/local

On Linux:

~/Canopy/canopy_cli --no-gui-setup --install-dir /usr/local

Note that on Mac OS and Linux, this will create /usr/local/Canopy_64bit for Canopy 64-bit and /usr/local/Canopy_32bit for Canopy 32-bit install. On Windows, this would create C:\Users\<username>\Canopy for Canopy 64-bit and C:\Users\<username>\Canopy32 for Canopy 32-bit. Under this installation directory, you will find the User directory which is the User Python Environment. Under the User/bin (User\bin on Windows) directory you will find two convenience scripts, canopy_cli (.exe on Windows) and canopy (.exe on Windows), which are described in detail here.

Scenario: Creating a system-wide Canopy install

On a multi-user system, Canopy supports system-wide installation, where users share a common Python and set of packages, installed in a central location. This reduces disk usage because there is no need for packages to be duplicated across users. It also provides a consistent base of package versions for all users.

To create a central installation, a system administrator or other privileged user runs the Canopy command line tool with the --common-install switch. The installation directory can be any arbitrary location; this example installs to Z:\tools\Canopy on Windows and /usr/local on Linux and MacOS. The commands are:

On Windows:

Canopy\App\Canopy_cli.exe --common-install --install-dir=Z:\tools\Canopy

On Mac:

/Applications/Canopy.app/Contents/MacOS/Canopy_cli --common-install --install-dir=/usr/local

On Linux:

~/Canopy/canopy_cli --common-install --install-dir=/usr/local

(The --no-gui-setup switch, described in the previous sub-section, may additionally be used along with --common-install. The same installation is performed, but without using any GUI.)

This command will construct the subdirectory Canopy_64bit or Canopy_32bit (Canopy, Canopy32 for Windows) for 64-bit and 32-bit Canopy Python architectures, respectively. Under this subdirectory it will construct System and User virtual environments, in subdirectories named System and User, respectively. These are common central virtual environments that all users will access.

Important: If you have previously started Canopy from this account, you may have an existing locations.cfg file (see here for location by platform). This file must be deleted before running with the --common-install switch. If not, Canopy will warn you that the System and User locations have already been configured, and will exit without making any changes.

So that Canopy can find these common central virtual environments, when it is run by other users, the CLI when run with with --common-install creates a locations.cfg file in the root directory of Canopy Core. This is the same directory where canopy_cli is found on Windows and Linux systems; on Mac OS it is the directory (typically Canopy.app) containing Contents/MacOS/canopy_cli.

This central locations.cfg file looks like this:

{ 'common_system': u'/usr/local/Canopy_64bit/System',
  'common_user': u'/usr/local/Canopy_64bit/User'}

The format is that of a Python dictionary, with keys “common_system” and “common_user”. In this example, the paths reflect the path created on a 64-bit Linux install to /usr/local.

The initiating administrator’s own individual locations.cfg file will also be automatically set to point to these same central System and User directories. The initiator is the only user for whom that will automatically be true.

Subsequently, when any other user starts Canopy for the first time (from the Core installation), Canopy will recognize that the central locations.cfg already exists, and will set up this user as a regular user. Specifically, Canopy will create user-specific System and User virtual environments in that user’s account, and set the user’s locations.cfg to point to these user-specific directories.

These user virtual environments will inherit (not copy) from the common central System and User virtual environments just described. So regular users can read from but not write to these central virtual environments from which they inherit.

An administrator can also configure a set of default preferences to be shared with all users, via a central preferences.ini file located in the same directory as the central locations.cfg file. The file has the same format as an individual user’s preferences.ini file; each user’s individual preference setting will supersede the corresponding inherited default preference.

Updates in a system-wide installation

Since the initiating administrator’s System virtual environment is the central one, she can update the common installation of the Canopy GUI application using either the Canopy application’s update capabilities, or the canopy_cli update command described above. After any such update, all users will inherit the update the next time they start Canopy, unless they have done their own separate update to override it.

Likewise, if this administrator installs or updates any packages, whether using Package Manager or from the command line, this will affect the central User virtual environment, and therefore all users who inherit from it, the next time they run User Python or restart Canopy’s IPython kernel.

In contrast, if a regular user updates the application, or installs or updates any package, this update will be done in her own account only, and for herself only will override (shadow) the central installation of the application or the corresponding package (if any).

Caution: at present, the command line enpkg utility is unable to see the contents of parent virtual environments. In particular, if a regular user uses enpkg, it will not be aware of the inherited central User Python. If one does not realize this, one might unnecessarily install packages in one’s own account that are already installed in and accessible from the central User Python. Canopy’s Package Manager does not suffer from this shortcoming.