Canopy Command Line Interface (CLI)¶
If, like most users, you do not share a machine and want the ability to use both the Canopy GUI and the command line, then you probably do not need to use the Canopy Command Line Interface (CLI).
The Canopy CLI is primarily intended for use in multi-user or command-line-only (non-GUI) configurations, by users comfortable working at the system command prompt. In particular, it is useful for administrators writing automation scripts for Canopy, such as for cluster deployments.
This section covers details of the CLI, installation of Canopy via the command line in a non-GUI environment, configuring Canopy for use in a multi-user environment, and setting up a plain Python environment similar to EPD or the Python.org Python distribution. In order to provide some background for these instructions, the next sub-section introduces virtual environments and describes how Canopy uses virtual environments.
Canopy Core and virtual environments¶
The Canopy installer installs what is referred to as “Canopy Core”. This is the core Python environment and a minimum set of packages needed to bootstrap Canopy itself. This environment should not be used as a Python environment because it is intended to be locked down to facilitate Canopy’s automatic update mechanism and reliable startup and fail-safe recovery mode. What’s the alternative?
Virtual environments, specifically ‘venv’ which we backported from Python 3.x, are a technology that enables the creation of multiple, lightweight, independent Python environments. Each virtual environment appears to be a self-contained Python installation, but loads the Python standard library and other common resources from a common base Python installation. This significantly reduces disk usage and sometimes load time. Optionally, virtual environments can also load packages from the base Python environment, again saving disk space when common, large packages are centrally installed.
Canopy uses virtual environments in two ways – to provide a protected Python environment for the Canopy GUI application to run in, and to provide one or more User Python environments which you can customize and run your own code in. Canopy Core is the base for each of these virtual environments, providing the core Python components and several common, large packages such as Qt and PySide.
Canopy Core can optionally be updated, such as to move to a new version of Python, and each of the virtual environments will be updated automatically as well. This eliminates the need to install a new Python environment and then re-install any third-party packages into that new environment just to update Python.
The Canopy GUI automatically creates two virtual environments named ‘System’ and ‘User’. System is where the Canopy GUI runs; no user code runs in this environment. Updates to this virtual environment are done via the Canopy update mechanisms. The User environment environment is where the IPython kernel and all user code run. This virtual environment is managed by Package Manager; any packages can be updated and installed without fear of disrupting the GUI. Similarly, updates to the Canopy GUI will not affect packages installed in the User environment and break your code. This two-environment structure is created automatically when the Canopy GUI is first started, or when the non-GUI setup option is used, described below.
Users wishing to use Canopy as a plain Python environment without the GUI can easily create one or more Python environments for this purpose. See Creating an EPD-like Python environment below for details.
Command line interface¶
The Canopy Command Line Interface (CLI) is available via the canopy_cli command,
available both in the Canopy Core install, and in the Canopy User Python environment.
The tables at the end of this sub-section list the
default install paths for the Canopy CLI, on each operating system.
The canopy_cli command has the following sub-command structure:
canopy_cli [<sub-command>] [options] [positional arguments]
The following table shows the available sub-commands and their options and arguments.
| Function | Sub-command usage |
|---|---|
| Create a common install | post-install-setup [-h] <various options> |
| Update Canopy | update [-h] <venv-directory> |
| Create a Canopy venv | setup [-h] [-v] [–set-default|–no-default] <directory> |
| Low-level venv creation | venv [-h] [-s] [-u] <directory> |
| Register a Venv (Windows) | register-venv <venv-directory> |
| Unregister a Venv (Windows) | unregister-venv <venv-directory> |
The CLI can also be run with no sub-command, as described below in these sections:
canopy_cli update¶
The canopy_cli update command downloads and applies any available updates to
the Canopy Application. This is a command-line-only (no GUI popup) convenience,
equivalent to “Canopy Application Updates” within the Canopy GUI.
It is possible that several updates could be available, which would need to be applied sequentially. This might occur if your current version of Canopy is a bit old, or for other technical reasons. In this case, one update would be downloaded and applied at each invocation of this update command.
A single update will modify either the Canopy Core environment, or the Canopy System virtual environment, but not both.
The Canopy Core environment contains Python itself, and a collection of packages. Updates to Canopy Core may affect both the Canopy GUI application and the Canopy User Python environment, both of which inherit from Core.
Updates to Core Python will always affect Canopy User Python environment. Updates to a Core package will only affect the User Python environment if that same package is not installed in User. Most such packages are installed in User, and therefore override (“shadow”) the Core package being updated. Note that if a Core package is updated, and not shadowed in User (for example MKL), then this could affect the operation of its dependent packages in User (for example numpy).
If the update command updates the Canopy System environment, then this only affects the Canopy GUI application itself, not the Canopy User Python environment.
In the case of an update to the Canopy Core environment, the update command’s
optional <venv-directory> can point to a venv that was previously created by
the user using either canopy_cli venv -s or venv -s. After the updated Core
environment has been created, this will be updated so that it inherits from the
new Core environment, rather than the old one.
canopy_cli setup¶
The canopy_cli setup command is used to construct a full, user-accessible
Python virtual environment similar to the User Python environment that the
Canopy GUI sets up (as described above). This Python virtual environment
includes all of the Canopy packages that were bundled into your Canopy
installer. This environment behaves equivalently to an EPD (Enthought Python
Distribution) tree or an install of Python from Python.org. Please note that
this Python virtual environment inherits from the Canopy Core Python
environment and not from the User environment that one may expect. For more
information about Canopy core and the environments, refer to this
FAQ.
The -v option turns on verbose mode. <directory> specifies a directory in the file system where
the Python environment will be created.
For example:
canopy_cli setup C:\Python27
This constructs a user-accessible Python environment in C:\Python27 on a Windows system, mirroring the
behavior of a default EPD install. canopy_cli setup can be used multiple times on a single
system to create multiple user-accessible Python virtual environments in different directories.
If --set-default is specified, the Canopy CLI automatically adds the appropriate paths to your
environment to make this Python environment the default. On Windows this means adding
to your PATH environment variable; on Mac OS and Linux the appropriate commands are appended
to an existing .bashrc file.
canopy_cli venv¶
canopy_cli venv is a low-level command for simply creating a new virtual environment.
This virtual environment is similar to what is produced by the canopy_cli setup command but with no
packages installed.
By default, a venv created via the canopy_cli venv command will not load packages from its base
Python environment. However, if the -s switch is specified (short for --system-site-packages),
then the Python interpreter
in the new venv will have access both to packages installed in the venv itself, and to packages
in the base environment.
How is the base Python environment determined? The base Python environment is the same
environment that was used when canopy_cli venv was run. For example:
C:\Program Files\Enthought\Canopy\App\Canopy_cli.exe venv C:\Users\Username\my_venv
This constructs an empty virtual environment in C:\Users\Username\my_venv and uses the Canopy Core
install as the base. Since the -s switch was not provided, no packages will be inherited
from Canopy Core.
Now consider the following commands:
C:\Program Files\Enthought\Canopy\App\Canopy_cli.exe setup C:\Users\Username\my_venv
C:\Users\Username\my_venv\Scripts\venv.exe -s C:\Users\Username\my_venv_test
In this scenario, the first command constructs a full Python environment in C:\Users\Username\my_venv, including
at least all of the Canopy Express packages. The second command then creates a derivative
environment in C:\Users\Username\my_venv_test and, because -s was specified, this environment inherits access to
any packages in C:\Users\Username\my_venv that aren’t superseded by package installations in the
C:\Users\Username\my_venv_test venv. This is a particularly useful technique when experimenting with package
updates; you can quickly construct the test environment mirroring an existing one and install new
package variations for testing code without disturbing an existing known-good environment. Note
that in the second command we do not use Canopy_cli.exe. This is because in a venv
constructed via the setup subcommand, there is no access to the Canopy_cli.exe however,
there is a venv.exe command which serves the same purpose and has the same options as the venv
subcommand.
The -u switch causes an existing virtual environment to be updated if any updates
have been applied to the base Python environment. Typically this is used after the
canopy_cli update command has applied patches to the base Python environment.
Each of the sub-commands supports the option -h or --help which provides a summary of
the information on this page.
canopy_cli register / unregister¶
These commands are specific to users running on Microsoft Windows and support integration with Microsoft’s Python Tools for Visual Studio (PTVS) tool. Canopy provides a custom integration that allows PTVS to automatically detect the User Python environment by default. This integration doesn’t have a way to detect Canopy virtual environments created by the user, however, unless it gets a little help.
The register command allows a custom virtual environment to be registered such that it will be automatically detected by Canopy’s PTVS integration and will show up in the list of recognized Python environments in PTVS. The unregister command simply reverses this process and removes the environment from the list.
A typical scenario is to create a venv for a particular application or experiment and then register it to use PTVS to debug the code. For example:
canopy_cli.exe venv -s c:\projects\new_app1 # Constructs the venv as above
canopy_cli.exe register c:\projects\new_app1
After the register command completes the new venv will appear automatically in the PTVS “Python Environments” tab. This environment can be de-registered like this:
canopy_cli.exe unregister C:\projects\new_app1
Note that it can be unregistered even if it has already been deleted as long as the path matches the original location.
Canopy CLI’s default location in the Canopy Core¶
| Platform | Version | Location |
|---|---|---|
| Windows XP | 64 | C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy\App\Canopy_cli.exe |
| Windows XP | 32 | C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy32\App\Canopy_cli.exe |
| Windows 7, 8 & Vista | 64 | C:\Users\<username>\AppData\Local\Enthought\Canopy\App\Canopy_cli.exe |
| Windows 7, 8 & Vista | 32 | C:\Users\<username>\AppData\Local\Enthought\Canopy32\App\Canopy_cli.exe |
| Mac OS X | 64 | /Applications/Canopy.app/Contents/MacOS/Canopy_cli |
| Mac OS X | 32 | /Applications/Canopy.app/Contents/MacOS/Canopy_cli |
| Linux | 64 | ~/Canopy/canopy_cli |
| Linux | 32 | ~/Canopy/canopy_cli |
Canopy CLI’s default location in the Canopy User Python environment¶
| Platform | Version | Location |
|---|---|---|
| Windows XP | 64 | C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy\User\Scripts\canopy_cli.exe |
| Windows XP | 32 | C:\Documents and Settings\<username>\Local Settings\Application Data\Enthought\Canopy32\User\Scripts\canopy_cli.exe |
| Windows 7, 8 & Vista | 64 | C:\Users\<username>\AppData\Local\Enthought\Canopy\User\Scripts\canopy_cli.exe |
| Windows 7, 8 & Vista | 32 | C:\Users\<username>\AppData\Local\Enthought\Canopy32\User\Scripts\canopy_cli.exe |
| Mac OS X | 64 | ~/Library/Enthought/Canopy_64bit/User/bin/canopy_cli |
| Mac OS X | 32 | ~/Library/Enthought/Canopy_32bit/User/bin/canopy_cli |
| Linux | 64 | ~/Enthought/Canopy_64bit/User/bin/canopy_cli |
| Linux | 32 | ~/Enthought/Canopy_32bit/User/bin/canopy_cli |
Scenario: Creating an EPD-like Python environment (command-line only)¶
Update: As of Canopy version 1.6, this CLI option is strongly deprecated. If you are a Canopy subscriber and want to create a command-line-only Python installation, please use EPD 7.4 or greater. EPD installation instructions are included in this Knowledge Base article: “Installing Canopy to work with Canopy’s stand-alone Python distribution (EPD)”.
All Canopy users have access to Python at the command line. In a standard Canopy installation, you may also access Python via the Canopy GUI application (IDE).
If you do not wish to use the Canopy GUI application, and would like to use Canopy’s Python only as a command-line Python distribution similar to EPD, this is easily done. To set up such an EPD-like distribution, you can execute the following command:
On Windows:
Canopy\App\Canopy_cli.exe --no-gui-setup setup C:\Python27
On Mac:
/Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup setup ~/canopy
On Linux:
~/Canopy/canopy_cli --no-gui-setup setup ~/canopy
The target directory is determined by the user. It will be created for you wherever you specify (as long as you have sufficient permissions).
Please note that this Python virtual environment inherits from the Canopy Core Python environment and not from the User environment that one may expect. (Indeed, typically in this scenario, there is no User Python environment). For more information about Canopy core and the environments, refer to this FAQ.
If you want to make this Python environment the default on your system,
specify the --set-default switch. Specifying this option will cause Canopy to
add the appropriate bin directory (Scripts directory on Windows) to your PATH
environment variable. On Mac OS and Linux systems, this is done by appending a
line to your ~/.bash_profile file which activates the correct virtual
environment. On Windows, this Python environment is also added to the system
registry so third-party tools can correctly find it.
The -v switch can also be used to enable verbose output.
The --no-gui-setup switch shown in each of the above examples is optional,
but is recommended to avoid possible GUI popups during the setup procedure.
This installed Canopy python distribution can now be used like any other python environment. The following are a few examples (but not limited to) of the usage:
- Install a package of your choice in this environment using
easy_installorenpkg. - Run
pythonoripythonconsoles.
Please note that this is a virtual environment which inherits from the Canopy core so the Canopy Core can not be uninstalled without breaking this environment.
To uninstall the Canopy python distribution that has been installed as above, just delete the directory into which the distribution was installed.
On Windows:
rmdir /s C:\Python27
asuming C:\Python27 is the installation directory
On Mac or Linux:
rm -rf ~/canopy
assuming ~/canopy is the installation directory
Scenario: Creating a standalone customizable virtual environment¶
Using the Canopy Core one can manage virtual environments that includes creating, upgrading and using the virtual environments.
On Windows:
Canopy\App\Canopy_cli.exe venv C:\Users\<username>\my_venv
assuming C:\Users\<username>\my_venv is the installation directory for the
virtual environment.
This will be an empty virtual environment. In order to provide access to system site packages, do the following:
Canopy\App\Canopy_cli.exe venv -s C:\Users\<username>\my_venv
To upgrade this virtual environment, run the following command:
Canopy\App\Canopy_cli.exe venv -u C:\Users\<username>\my_venv
The environment will need to be upgraded if the core Canopy application was changed (re-installed or upgraded).
In order to use this virtual environment, you need to activate which can be done as follows:
C:\Users\<username>\my_venv\Scripts\activate.bat
To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command:
deactivate
On Mac OS X:
/Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv
assuming ~/my_venv is the installation directory for the
virtual environment
This will be an empty virtual environment. In order to provide access to system site packages, do the following:
/Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv -s
To upgrade this virtual environment, run the following command:
/Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv -u
In order to use this virtual environment, you need to activate it, which can be done as follows:
source ~/my_venv/bin/activate
To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command:
deactivate
On Linux:
~/Canopy/canopy_cli venv ~/my_venv
assuming ~/my_venv is the installation directory for the
virtual environment
This will be an empty virtual environment. In order to provide access to system site packages, do the following:
~/Canopy/canopy_cli venv ~/my_venv -s
To upgrade this virtual environment, run the following command:
~/Canopy/canopy_cli venv ~/my_venv -u
In order to use this virtual environment, you need to activate which can be done as follows:
source ~/my_venv/bin/activate
To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command.
deactivate
Scenario: Setting up the User and System environments without a setup GUI¶
Normally, when Canopy is launched for the first time by clicking on its icon, it will
open a GUI dialog window to prompt the user for the locations of the User and
System environments. This will not work if you are remotely connected to the machine
via a text-only interface such as ssh. In such a scenario, you can perform the
environment setup without a GUI, by using the CLI --no-gui-setup option:
| Platform | Commmand |
|---|---|
| Windows | Canopy\app\Canopy_cli.exe --no-gui-setup |
| Mac OS X | /Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup |
| Linux | ~/Canopy/canopy_cli --no-gui-setup |
By default,the User and System environments will be created in the same default
directories as if Canopy had been launched with a normal GUI dialog
(Where are all of the Python packages in my User Python Environment?).
Alternatively, the grandparent directory of the User and System directories can
be explicitly specified from the command line via the
--install-dir switch. For example:
On Windows:
Canopy\App\Canopy_cli.exe --no-gui-setup --install-dir C:\Users\<username>
On Mac:
/Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup --install-dir /usr/local
On Linux:
~/Canopy/canopy_cli --no-gui-setup --install-dir /usr/local
On Mac OS and Linux, this example would create /usr/local/Canopy_64bit for Canopy
64-bit and /usr/local/Canopy_32bit for Canopy 32-bit install. On Windows,
it would create C:\Users\<username>\Canopy for Canopy 64-bit and
C:\Users\<username>\Canopy32 for Canopy 32-bit. Under this installation
directory, you will find the User and System directories which are the
Canopy User Python and Canopy System environments, respectively.
Under the User/bin (User\Scripts on Windows) directory you
will find two convenience scripts, canopy_cli (.exe on Windows) and
canopy (.exe on Windows), which are described in detail
here.
Note about file permissions on Mac and Linux¶
The setup commands shown above should generally be run as an admin user, but
not as the root user, i.e. not using default sudo. You would need to ensure that the admin
user has write access, and that all ordinary users have read and execute access,
to the target directory (/usr/local/Canopy_64bit in this example, or whatever target
directory you choose).
Scenario: Creating a Canopy common install¶
On multi-user systems, Canopy supports “common install” setup, where users share a common Python and set of packages, installed in a central location. This reduces disk usage because there is no need for packages to be duplicated across users. It also provides a consistent base of package versions for all users.
Starting with Canopy 1.6, Canopy’s “open common install” (originally named just “common install”) is not recommended for most users. The new recommended method for installing Canopy for multiple users is the “managed common install”. For information about these options, please read the Knowledge Base article “Canopy Common Install Options”.