[ Skip to the content ]

Institute of Formal and Applied Linguistics Wiki


[ Back to the navigation ]

Table of Contents

Python

In this guide, I expect you are using Python3 (and pip3), but (for old projects) you can use Python2 (and pip2) as well.

Virtual environments

You can use virtual environments, e.g. one for each project (or a group of projects), so the version requirements of different projects do not collide.

Create a new virtual environment with:

  python3 -m venv my-project-name

or (for a specific version of python):

  /opt/python/$PYTHON_VERSION/bin/python3 -m venv my-project-name

where my-project-name can be a relative or absolute path to a directory which will be created.
Caveat1: the path to the directory should be rather short, there is a limit on shebang paths so you can get strange errors like ipython3: bad interpreter: No such file or directory if your ipython3 is installed in the venv starts with #!/..a..very..long..path..
Caveat2: If my-project-name is a relative path, it will be concatenated with pwd -P, i.e. with your actual directory with all symlinks resolved. If you want symlinks in the path, you must provide absolute path as my-project-name. Note that e.g. /net/work/people is currently (2018-06) resolved by pwd -P to /lnet/.nfs/spec/work/people/, but this path does not work on machines with native Lustre support.)

You can use any Python version for the new environment, e.g. /opt/python/3.5.4/bin/python3 -m venv my-project-name-python3.5
Then activate it with

  source my-project-name/bin/activate

Now your prompt changes and you can install any modules (with pip3, but you can omit --user), e.g.

  pip3 install --upgrade pip tensorflow

If you run python, it will be the version used when creating venv. Check it with

  which python
  python -V

Once finished, exit the virtual environment with

  deactivate

If you decide you don't need the environment and any modules installed in this environment, simply delete the my-project-name directory.

Note that there are many alternative ways how to create virtual environments and install modules. The venv module is available only for Python 3.3 and newer. For older versions you can use the third-party virtualenv tool:

 pip3 install --user --upgrade pip
 pip3 install --user virtualenv
 virtualenv my-project-name
 source my-project-name/bin/activate
 

IPython in virtual envs.

When using virtuaenvs, ipython is (in some versions) confused and uses an incorrect python interpreter. The following alias fixes this issue:

alias ipython="python -c 'import IPython; IPython.terminal.ipapp.launch_new_instance()'"

Installing directly from Git

If you are developing (contributing to) a Python tool version in git and want to install the newest version, you can use pip3 git+https://github.com/…, provided the project has proper setup.py in the root directory.
You can also git clone the project manually and add the path to your $PYTHONPATH. This way whenever you git pull, you should make sure there are no new dependencies, e.g. with pip3 install –user -r requirements.txt if the project provides the requirements.txt file.
See e.g. Udapi or Neural Monkey for examples of both ways.


[ Back to the navigation ] [ Back to the content ]