.. _canopy_cli: Canopy Command Line Interface (CLI) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ *If, like most users, you do not share a machine and want the ability to use both the Canopy GUI and the command line, then you probably do not need to use the Canopy Command Line Interface (CLI).* The Canopy CLI is primarily intended for use in multi-user or command-line-only (non-GUI) configurations, by users comfortable working at the system command prompt. In particular, it is useful for administrators writing automation scripts for Canopy, such as for cluster deployments. This section covers :ref:`details of the CLI`, :ref:`installation of Canopy via the command line in a non-GUI environment`, :ref:`configuring Canopy for use in a multi-user environment`, and :ref:`setting up a plain Python environment similar to EPD` or the Python.org Python distribution. In order to provide some background for these instructions, the next sub-section introduces virtual environments and describes how Canopy uses virtual environments. .. _core_canopy: Canopy Core and virtual environments ==================================== The Canopy installer installs what is referred to as "Canopy Core". This is the core Python environment and a minimum set of packages needed to bootstrap Canopy itself. This environment should *not* be used as a Python environment because it is intended to be locked down to facilitate Canopy's automatic update mechanism and reliable startup and fail-safe recovery mode. What's the alternative? Virtual environments, specifically 'venv' which we backported from Python 3.x, are a technology that enables the creation of multiple, lightweight, independent Python environments. Each virtual environment appears to be a self-contained Python installation, but loads the Python standard library and other common resources from a common base Python installation. This significantly reduces disk usage and sometimes load time. Optionally, virtual environments can also load packages from the base Python environment, again saving disk space when common, large packages are centrally installed. Canopy uses virtual environments in two ways -- to provide a protected Python environment for the Canopy GUI application to run in, and to provide one or more User Python environments which you can customize and run your own code in. Canopy Core is the base for each of these virtual environments, providing the core Python components and several common, large packages such as Qt and PySide. Canopy Core can optionally be updated, such as to move to a new version of Python, and each of the virtual environments will be updated automatically as well. This eliminates the need to install a new Python environment and then re-install any third-party packages into that new environment just to update Python. The Canopy GUI automatically creates two virtual environments named 'System' and 'User'. System is where the Canopy GUI runs; no user code runs in this environment. Updates to this virtual environment are done via the Canopy update mechanisms. The User environment environment is where the IPython kernel and all user code run. This virtual environment is managed by Package Manager; any packages can be updated and installed without fear of disrupting the GUI. Similarly, updates to the Canopy GUI will not affect packages installed in the User environment and break your code. This two-environment structure is created automatically when the Canopy GUI is first started, or when the non-GUI setup option is used, :ref:`described below`. Users wishing to use Canopy as a plain Python environment without the GUI can easily create one or more Python environments for this purpose. See :ref:`Creating an EPD-like Python environment` below for details. .. _command_line_interface: Command line interface ====================== The Canopy Command Line Interface (CLI) is available via the ``canopy_cli`` command, available both in the Canopy Core install, and in the Canopy User Python environment. The tables at the end of this sub-section list the default install paths for the Canopy CLI, on each operating system. The ``canopy_cli`` command has the following sub-command structure: :: canopy_cli [] [options] [positional arguments] The following table shows the available sub-commands and their options and arguments. =========================== =========================================================== Function Sub-command usage =========================== =========================================================== Create a common install post-install-setup [-h] Update Canopy update [-h] Create a Canopy venv setup [-h] [-v] [--set-default|--no-default] Low-level venv creation venv [-h] [-s] [-u] Register a Venv (Windows) register-venv Unregister a Venv (Windows) unregister-venv =========================== =========================================================== The CLI can also be run with no sub-command, as described below in these sections: | :ref:`cmd_line_setup` canopy_cli update ----------------- The ``canopy_cli update`` command downloads and applies any available updates to the Canopy Application. This is a command-line-only (no GUI popup) convenience, equivalent to "Canopy Application Updates" within the Canopy GUI. It is possible that several updates could be available, which would need to be applied sequentially. This might occur if your current version of Canopy is a bit old, or for other technical reasons. In this case, one update would be downloaded and applied at each invocation of this update command. A single update will modify either the Canopy Core environment, or the Canopy System virtual environment, but not both. The Canopy Core environment contains Python itself, and a collection of packages. Updates to Canopy Core may affect both the Canopy GUI application and the Canopy User Python environment, both of which inherit from Core. Updates to Core Python will always affect Canopy User Python environment. Updates to a Core package will only affect the User Python environment if that same package is not installed in User. Most such packages are installed in User, and therefore override ("shadow") the Core package being updated. Note that if a Core package is updated, and not shadowed in User (for example MKL), then this could affect the operation of its dependent packages in User (for example numpy). If the update command updates the Canopy System environment, then this only affects the Canopy GUI application itself, not the Canopy User Python environment. In the case of an update to the Canopy Core environment, the update command's optional can point to a venv that was previously created by the user using either ``canopy_cli venv -s`` or ``venv -s``. After the updated Core environment has been created, this will be updated so that it inherits from the new Core environment, rather than the old one. canopy_cli setup ----------------- The ``canopy_cli setup`` command is used to construct a full, user-accessible Python virtual environment similar to the User Python environment that the Canopy GUI sets up (as described above). This Python virtual environment includes all of the Canopy packages that were bundled into your Canopy installer. This environment behaves equivalently to an EPD (Enthought Python Distribution) tree or an install of Python from Python.org. Please note that this Python virtual environment inherits from the Canopy Core Python environment and not from the User environment that one may expect. For more information about Canopy core and the environments, refer to :ref:`this FAQ`. The ``-v`` option turns on verbose mode. ```` specifies a directory in the file system where the Python environment will be created. For example: :: canopy_cli setup C:\Python27 This constructs a user-accessible Python environment in C:\\Python27 on a Windows system, mirroring the behavior of a default EPD install. ``canopy_cli setup`` can be used multiple times on a single system to create multiple user-accessible Python virtual environments in different directories. If ``--set-default`` is specified, the Canopy CLI automatically adds the appropriate paths to your environment to make this Python environment the default. On Windows this means adding to your PATH environment variable; on Mac OS and Linux the appropriate commands are appended to an existing .bashrc file. canopy_cli venv ----------------- ``canopy_cli venv`` is a low-level command for simply creating a new virtual environment. This virtual environment is similar to what is produced by the ``canopy_cli setup`` command but with no packages installed. By default, a venv created via the ``canopy_cli venv`` command will not load packages from its base Python environment. However, if the ``-s`` switch is specified (short for ``--system-site-packages``), then the Python interpreter in the new venv will have access both to packages installed in the venv itself, and to packages in the base environment. How is the base Python environment determined? The base Python environment is the same environment that was used when ``canopy_cli venv`` was run. For example:: C:\Program Files\Enthought\Canopy\App\Canopy_cli.exe venv C:\Users\Username\my_venv This constructs an empty virtual environment in ``C:\Users\Username\my_venv`` and uses the Canopy Core install as the base. Since the ``-s`` switch was not provided, no packages will be inherited from Canopy Core. Now consider the following commands:: C:\Program Files\Enthought\Canopy\App\Canopy_cli.exe setup C:\Users\Username\my_venv C:\Users\Username\my_venv\Scripts\venv.exe -s C:\Users\Username\my_venv_test In this scenario, the first command constructs a full Python environment in ``C:\Users\Username\my_venv``, including at least all of the Canopy Express packages. The second command then creates a derivative environment in ``C:\Users\Username\my_venv_test`` and, because ``-s`` was specified, this environment inherits access to any packages in ``C:\Users\Username\my_venv`` that aren't superseded by package installations in the ``C:\Users\Username\my_venv_test`` venv. This is a particularly useful technique when experimenting with package updates; you can quickly construct the test environment mirroring an existing one and install new package variations for testing code without disturbing an existing known-good environment. Note that in the second command we do not use ``Canopy_cli.exe``. This is because in a venv constructed via the ``setup`` subcommand, there is no access to the ``Canopy_cli.exe`` however, there is a ``venv.exe`` command which serves the same purpose and has the same options as the ``venv`` subcommand. The ``-u`` switch causes an existing virtual environment to be updated if any updates have been applied to the base Python environment. Typically this is used after the ``canopy_cli update`` command has applied patches to the base Python environment. Each of the sub-commands supports the option ``-h`` or ``--help`` which provides a summary of the information on this page. .. _canopy_cli_register: canopy_cli register / unregister -------------------------------- These commands are specific to users running on Microsoft Windows and support integration with Microsoft's `Python Tools for Visual Studio `_ (PTVS) tool. Canopy provides a custom integration that allows PTVS to automatically detect the User Python environment by default. This integration doesn't have a way to detect Canopy virtual environments created by the user, however, unless it gets a little help. The register command allows a custom virtual environment to be registered such that it will be automatically detected by Canopy's PTVS integration and will show up in the list of recognized Python environments in PTVS. The unregister command simply reverses this process and removes the environment from the list. A typical scenario is to create a venv for a particular application or experiment and then register it to use PTVS to debug the code. For example: :: canopy_cli.exe venv -s c:\projects\new_app1 # Constructs the venv as above canopy_cli.exe register c:\projects\new_app1 After the register command completes the new venv will appear automatically in the PTVS "Python Environments" tab. This environment can be de-registered like this: :: canopy_cli.exe unregister C:\projects\new_app1 Note that it can be unregistered even if it has already been deleted as long as the path matches the original location. Canopy CLI's default location in the Canopy Core ------------------------------------------------ ===================== ======= =================================================================== Platform Version Location ===================== ======= =================================================================== Windows XP 64 ``C:\Documents and Settings\\Local Settings\Application Data\Enthought\Canopy\App\Canopy_cli.exe`` Windows XP 32 ``C:\Documents and Settings\\Local Settings\Application Data\Enthought\Canopy32\App\Canopy_cli.exe`` Windows 7, 8 & Vista 64 ``C:\Users\\AppData\Local\Enthought\Canopy\App\Canopy_cli.exe`` Windows 7, 8 & Vista 32 ``C:\Users\\AppData\Local\Enthought\Canopy32\App\Canopy_cli.exe`` Mac OS X 64 ``/Applications/Canopy.app/Contents/MacOS/Canopy_cli`` Mac OS X 32 ``/Applications/Canopy.app/Contents/MacOS/Canopy_cli`` Linux 64 ``~/Canopy/canopy_cli`` Linux 32 ``~/Canopy/canopy_cli`` ===================== ======= =================================================================== Canopy CLI's default location in the Canopy User Python environment -------------------------------------------------------------------- ===================== ======= =================================================================== Platform Version Location ===================== ======= =================================================================== Windows XP 64 ``C:\Documents and Settings\\Local Settings\Application Data\Enthought\Canopy\User\Scripts\canopy_cli.exe`` Windows XP 32 ``C:\Documents and Settings\\Local Settings\Application Data\Enthought\Canopy32\User\Scripts\canopy_cli.exe`` Windows 7, 8 & Vista 64 ``C:\Users\\AppData\Local\Enthought\Canopy\User\Scripts\canopy_cli.exe`` Windows 7, 8 & Vista 32 ``C:\Users\\AppData\Local\Enthought\Canopy32\User\Scripts\canopy_cli.exe`` Mac OS X 64 ``~/Library/Enthought/Canopy_64bit/User/bin/canopy_cli`` Mac OS X 32 ``~/Library/Enthought/Canopy_32bit/User/bin/canopy_cli`` Linux 64 ``~/Enthought/Canopy_64bit/User/bin/canopy_cli`` Linux 32 ``~/Enthought/Canopy_32bit/User/bin/canopy_cli`` ===================== ======= =================================================================== .. _create_epd_dist: Scenario: Creating an EPD-like Python environment (command-line only) ===================================================================== **Update: As of Canopy version 1.6, this CLI option is strongly deprecated. If you are a Canopy subscriber and want to create a command-line-only Python installation, please use EPD 7.4 or greater. EPD installation instructions are included in this Knowledge Base article:** `"Installing Canopy to work with Canopy's stand-alone Python distribution (EPD)" `_. *All* Canopy users have access to Python at the command line. In a standard Canopy installation, you may also access Python via the Canopy GUI application (IDE). If you do **not** wish to use the Canopy GUI application, and would like to use Canopy's Python **only** as a command-line Python distribution similar to EPD, this is easily done. To set up such an EPD-like distribution, you can execute the following command: On Windows: :: Canopy\App\Canopy_cli.exe --no-gui-setup setup C:\Python27 On Mac: :: /Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup setup ~/canopy On Linux: :: ~/Canopy/canopy_cli --no-gui-setup setup ~/canopy The target directory is determined by the user. It will be created for you wherever you specify (as long as you have sufficient permissions). Please note that this Python virtual environment inherits from the Canopy Core Python environment and not from the User environment that one may expect. (Indeed, typically in this scenario, there is no User Python environment). For more information about Canopy core and the environments, refer to :ref:`this FAQ`. If you want to make this Python environment the default on your system, specify the ``--set-default`` switch. Specifying this option will cause Canopy to add the appropriate bin directory (Scripts directory on Windows) to your PATH environment variable. On Mac OS and Linux systems, this is done by appending a line to your ``~/.bash_profile`` file which activates the correct virtual environment. On Windows, this Python environment is also added to the system registry so third-party tools can correctly find it. The ``-v`` switch can also be used to enable verbose output. The ``--no-gui-setup`` switch shown in each of the above examples is optional, but is recommended to avoid possible GUI popups during the setup procedure. This installed Canopy python distribution can now be used like any other python environment. The following are a few examples (but not limited to) of the usage: 1. Install a package of your choice in this environment using ``easy_install`` or ``enpkg``. 2. Run ``python`` or ``ipython`` consoles. Please note that this is a virtual environment which inherits from the Canopy core so the Canopy Core can not be uninstalled without breaking this environment. To uninstall the Canopy python distribution that has been installed as above, just delete the directory into which the distribution was installed. On Windows: :: rmdir /s C:\Python27 asuming ``C:\Python27`` is the installation directory On Mac or Linux: :: rm -rf ~/canopy assuming ``~/canopy`` is the installation directory .. _standalone_cust_venv: Scenario: Creating a standalone customizable virtual environment ================================================================ Using the :ref:`Canopy Core` one can manage virtual environments that includes creating, upgrading and using the virtual environments. On Windows: :: Canopy\App\Canopy_cli.exe venv C:\Users\\my_venv assuming ``C:\Users\\my_venv`` is the installation directory for the virtual environment. This will be an empty virtual environment. In order to provide access to system site packages, do the following:: Canopy\App\Canopy_cli.exe venv -s C:\Users\\my_venv To upgrade this virtual environment, run the following command:: Canopy\App\Canopy_cli.exe venv -u C:\Users\\my_venv The environment will need to be upgraded if the core Canopy application was changed (re-installed or upgraded). In order to use this virtual environment, you need to activate which can be done as follows:: C:\Users\\my_venv\Scripts\activate.bat To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command:: deactivate On Mac OS X:: /Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv assuming ``~/my_venv`` is the installation directory for the virtual environment This will be an empty virtual environment. In order to provide access to system site packages, do the following:: /Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv -s To upgrade this virtual environment, run the following command:: /Applications/Canopy.app/Contents/MacOS/Canopy_cli venv ~/my_venv -u In order to use this virtual environment, you need to activate it, which can be done as follows:: source ~/my_venv/bin/activate To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command:: deactivate On Linux:: ~/Canopy/canopy_cli venv ~/my_venv assuming ``~/my_venv`` is the installation directory for the virtual environment This will be an empty virtual environment. In order to provide access to system site packages, do the following: :: ~/Canopy/canopy_cli venv ~/my_venv -s To upgrade this virtual environment, run the following command: :: ~/Canopy/canopy_cli venv ~/my_venv -u In order to use this virtual environment, you need to activate which can be done as follows: :: source ~/my_venv/bin/activate To stop using this virtual environment which is activated, you can deactivate it by just running the deactive command. :: deactivate .. _cmd_line_setup: Scenario: Setting up the User and System environments without a setup GUI ========================================================================= Normally, when Canopy is launched for the first time by clicking on its icon, it will open a GUI dialog window to prompt the user for the locations of the User and System environments. This will not work if you are remotely connected to the machine via a text-only interface such as ssh. In such a scenario, you can perform the environment setup without a GUI, by using the CLI ``--no-gui-setup`` option: ============== ====================================================================== Platform Commmand ============== ====================================================================== Windows ``Canopy\app\Canopy_cli.exe --no-gui-setup`` Mac OS X ``/Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup`` Linux ``~/Canopy/canopy_cli --no-gui-setup`` ============== ====================================================================== By default,the User and System environments will be created in the same default directories as if Canopy had been launched with a normal GUI dialog (:ref:`venv_where_are_packages`). Alternatively, the grandparent directory of the User and System directories can be explicitly specified from the command line via the ``--install-dir`` switch. For example: On Windows: :: Canopy\App\Canopy_cli.exe --no-gui-setup --install-dir C:\Users\ On Mac: :: /Applications/Canopy.app/Contents/MacOS/Canopy_cli --no-gui-setup --install-dir /usr/local On Linux: :: ~/Canopy/canopy_cli --no-gui-setup --install-dir /usr/local On Mac OS and Linux, this example would create ``/usr/local/Canopy_64bit`` for Canopy 64-bit and ``/usr/local/Canopy_32bit`` for Canopy 32-bit install. On Windows, it would create ``C:\Users\\Canopy`` for Canopy 64-bit and ``C:\Users\\Canopy32`` for Canopy 32-bit. Under this installation directory, you will find the ``User`` and ``System`` directories which are the Canopy User Python and Canopy System environments, respectively. Under the ``User/bin`` (``User\Scripts`` on Windows) directory you will find two convenience scripts, ``canopy_cli`` (``.exe`` on Windows) and ``canopy`` (``.exe`` on Windows), which are described in detail :ref:`here`. Note about file permissions on Mac and Linux -------------------------------------------- The setup commands shown above should generally be run as an admin user, but not as the root user, i.e. not using default ``sudo``. You would need to ensure that the admin user has write access, and that all ordinary users have read and execute access, to the target directory (``/usr/local/Canopy_64bit`` in this example, or whatever target directory you choose). .. _sys_wide_canopy_install: Scenario: Creating a Canopy common install ========================================== On multi-user systems, Canopy supports "common install" setup, where users share a common Python and set of packages, installed in a central location. This reduces disk usage because there is no need for packages to be duplicated across users. It also provides a consistent base of package versions for all users. Starting with Canopy 1.6, Canopy's "open common install" (originally named just "common install") is not recommended for most users. The new recommended method for installing Canopy for multiple users is the "managed common install". For information about these options, please read the Knowledge Base article `"Canopy Common Install Options" `_.