Note

relion is distributed under a GPLv2 license, i.e. it is completely free, open-source software for both academia and industry.

Installation

The sections below explain how to download and install relion on your computer.

Note that relion depends on and uses several external programs and libraries.

C++ compiler:

RELION 4.0 requires a C++ compiler that fully supports the C++11 standard. For GCC, this means version 4.8.1 or later. Note that GCC 4.4, which comes with RedHat Enterprise Linux or CentOS 6.x, is too old.

To use Torch related features, you need C++14 support. For GCC, this means version 5.0 or later.

MPI:

Your system will need MPI runtime (most flavours will do). If you don’t have an MPI installation already on your system, we recommend installing OpenMPI.

CUDA:

If you have a modern GPU from nvidia with compute capability 3.5+, you can accelerate many jobs considerably. In order to compile relion with GPU-acceleration support, you’ll need to install cuda. Download it from NVIDIA website.

Note that CUDA toolkits support only a limited range of C compilers. Also note that a newer CUDA toolkit requires a newer GPU driver. Carefully read the release note and make sure you have a compatible set of GPU driver, C compiler and CUDA toolkit.

CTFFIND-4.1:

CTF estimation is not part of relion. Instead, relion provides a wrapper to Alexis Rohou and Niko Grigorieff’s ctffind 4 [RG15]. Alternatively, you may also use (the closed-source) gctf by Kai Zhang [Zha16], which may be downloaded from Kai’s website.

Ghostscript:

RELION uses Ghostscript to generate PDF files.

FLTK (only for GUI):

RELION uses FLTK as a GUI tool kit. This can be installed automatically (see below).

X Window system libraries (only for GUI):

RELION needs basic X11 libraries together with Xft for the GUI. Most Linux distributions have packages called libxft-dev or libXft-devel and libX11-devel. Note that you need developer packages if you build your own FLTK.

FFT libraries:

RELION needs an FFT library. The default is FFTW. This can be installed automatically (see below). Depending on your CPU, Intel MKL FFT or AMD optimised FFTW might run faster. See below how to use them.

libtiff:

RELION needs libtiff version >= 4.0. Most Linux distributions have packages called libtiff-dev or libtiff-devel. Note that you need a developer package.

libpng:

RELION needs libpng. Most Linux distributions have packages called libpng-dev or libpng-devel. Note that you need a developer package.

UCSF MotionCor2 (optional):

relion implements its own (CPU-only) implementation of the UCSF motioncor2 algorithm for whole-frame micrograph movie-alignment [ZPA+17]. If you want, you can still use the (GPU-accelerated) UCSF program. You can download it from David Agard’s page and follow his installation instructions. Note that using the UCSF program does not make full advantage of the opportunities provided in Bayesian polishing.

ResMap (optional):

Local-resolution estimation may be performed inside relion’s own postprocessing program. Alternatively, one can also use Alp Kucukelbir’s resmap [KST14]. Download it from Alp’s ResMap website and follow his installation instructions.

In practice, most of these dependencies can be installed by system’s package manager if you have the root priviledge.

In Debian or Ubuntu:

sudo apt install cmake git build-essential mpi-default-bin mpi-default-dev libfftw3-dev libtiff-dev libpng-dev ghostscript libxft-dev

In RHEL, Cent OS, Scientific Linux:

sudo yum install cmake git gcc gcc-c++ openmpi-devel fftw-devel libtiff-devel libpng-devel ghostscript libXft-devel libX11-devel

Download RELION

We store the public release versions of relion on GitHub, a site that provides code-development with version control and issue tracking through the use of git. We will not describe the use of git in general, as you will not need more than very basic features. Below we outline the few commands needed on a UNIX-system, please refer to general git descriptions and tutorials to suit your system. To get the code, you clone or download the repository. We recommend cloning, because it allows you very easily update the code when new versions are released. To do so, use the shell command-line:

git clone https://github.com/3dem/relion.git

This will create a local Git repository. All subsequent git-commands should be run inside this directory.

The master branch (default) contains the stable release of relion-4.0. By performing:

git checkout ver4.0

you can access the latest (developmental) updates for RELION 4.0x. The code there is not tested as throughfully as that in the master branch and not generally recommended.

The code will be intermittently updated to amend issues. To incorporate these changes, use the command-line:

git pull

inside you local repository (the source-code directory downloaded). If you have changed the code in some way, this will force you to commit a local merge. You are free to do so, but we will assume you have not changed the code. Refer to external instructions regarding git and merging so-called conflicts if you have changed the code an need to keep those changes.

Compilation

relion has an installation procedure which relies on cmake. You will need to have this program installed, but most UNIX-systems have this by default. You will need to make a build-directory in which the code will be compiled. This can be placed inside the repository:

cd relion
mkdir build
cd build

You then invoke cmake inside the build-directoy, but point to the source-directoy to configure the installation. This will not install relion, just configure the build:

cmake ..

The output will notify you of what was detected and what type of build will be installed. Because relion is rich in terms of the possible configurations, it is important to check this output. For instance:

  • The path to the MPI library.

  • GPU-capability will only be included if a CUDA SDK is detected. If not, the program will install, but without support for GPUs.

  • If FFTW is not detected, instructions are included to download and install it in a local directory known to the relion installation.

  • As above, regarding FLTK (required for GUI). If a GUI is not desired, this can be escaped as explained in the following section.

The MPI library must be the one you intend to use relion with. Compiling relion with one version of MPI and running the resulting binary with mpirun from another version can cause crash. Note that some software packages (e.g. CCPEM, crYOLO, EMAN2) come with their own MPI runtime. Sourcing/activating their environment might update PATH and LD_LIBRARY_PATH environmental variables and put their MPI runtime into the highest priority.

The MPI C++ compiler (mpicxx) and CUDA compiler (nvcc) internally calls a C++ compiler. This must match the compiler cmake picked up. Otherwise, the compilation might fail at the linking step.

Following the completion of cmake-configuration without errors, make is used to install the program:

make -j N

, where N is the number of processes to use during installation. Using a higher number simply means that it will compile faster.

Take note of any warnings or errors reported. relion will be installed in the build directory’s sub-directory called bin. To make the installation system-wide, see below.

Wherever you install relion, make sure your PATH environmental variable points to the directory containing relion binaries. Launching relion with a path like /path/to/relion is not the right way; this starts the right GUI, but the GUI might invoke other versions of relion in the PATH.

Configuration options

CMake allows configuration of many aspects of the installation, some of which are outlined here. Most options can be set by adding options to the cmake configuration. Under the below subheadings, some example replacement commands are given to substitute the original configuration command. It is also recommended to clean or purge your build-directory between builds, since CMake caches some of previous configurations:

cd build
rm -fr *

And of course, any of the below options can be combined.

Omitting the GUI:

cmake -DGUI=OFF .. (default is ON)

With this option, GUI programs (e.g. relion, relion_manualpick, relion_display) are not be built and FLTK becomes unnecessary.

Using single-precision on the CPU:

cmake -DDoublePrec_CPU=OFF .. (default is ON)

This will reduce (CPU but not GPU) memory consumption to about half. This is useful when memory hungry tasks such as motion correction and Polishing run out of memory. This is safe in most cases but please use the default double precision build if CtfRefine produces NaNs.

Using double-precision on the GPU:

cmake -DDoublePrec_GPU=ON .. (default is OFF)

This will slow down GPU-execution considerably, while this does NOT improve the resolution. Thus, this option is not recommended.

Compiling GPU-code for your architecture:

cmake -DCUDA_ARCH=52 .. (default is 35, meaning compute capability 3.5, which is the lowest supported by relion)

CUDA-capable devices have a so-called compute capability, which code can be compiled against for optimal performance. The compute capability of your card can be looked up at the table in NVIDIA website. WARNING: If you use a wrong number, compilation might succeed but the resulting binary can fail at the runtime.

Forcing build and use of local FFTW:

cmake -DFORCE_OWN_FFTW=ON ..

This will download, verify and install FFTW during the installation process.

Forcing build and use of AMD optimized FFTW:

cmake -DFORCE_OWN_FFTW=ON -DAMDFFTW=ON ..

This will download, verify and install AMD optimized version of FFTW during the installation process. This is recommended for AMD CPUs (e.g. Ryzen, Threadripper, EPYC).

Forcing build and use of Intel MKL FFT:

cmake -DMKLFFT=ON ..

This will use FFT library from Intel MKL. In contrast to the FFTW options above, this will not download MKL automatically. You have to install MKL and set relevants paths (usually by sourcing the mkl_vars.sh script).

Forcing build and use of local FLTK:

cmake -DFORCE_OWN_FLTK=ON ..

This will download, verify and install FLTK during the installation process. If any of these are not detected during configuration, this will happen automatically anyway, and you should not have to specify the below options manually.

Specify location of libtiff:

cmake -DTIFF_INCLUDE_DIR=/path/to/include -DTIFF_LIBRARY=/path/to/libtiff.so.5

This option is to use libtiff installed in non-standard location.

Specifying an installation location:

To allow relion a system-wide installation use:

cmake -DCMAKE_INSTALL_PREFIX=/path/to/install/dir/ ..
make -j N
make install

Warning

Do not specify the build directory itself as CMAKE_INSTALL_PREFIX. This does not work! If you are happy with binaries in the build directory, leave CMAKE_INSTALL_PREFIX as default and omit the make install step.

Enable accelerated CPU code path:

cmake -DALTCPU=ON

Note that this is mutually exclusive with GPU acceleration (-DCUDA=ON). Intel compilers are recommended for this option (see below).

Use Intel compilers:

Intel compilers often generate faster binaries for Intel CPUs, especially when combined with the accelerated CPU code path above. Intel compilers are available free of chage as part of Intel oneAPI HPC toolkit. To use Intel compilers, run below after sourcing Intel compilers’ initialization scripts:

cmake .. -DMKLFFT=ON \
-DCMAKE_C_COMPILER=icc -DCMAKE_CXX_COMPILER=icpc -DMPI_C_COMPILER=mpiicc -DMPI_CXX_COMPILER=mpiicpc \
-DCMAKE_C_FLAGS="-O3 -ip -g -xCOMMON-AVX512 -restrict " \
-DCMAKE_CXX_FLAGS="-O3 -ip -g -xCOMMON-AVX512 -restrict "

This generates binaries optimized with AVX512 instructions. If your CPU supports only up to AVX256, use -xCORE-AVX2 instead of -xCOMMON-AVX512.

If you don’t want to use Intel MPI, change mpiicc and mpiicpc accordingly. For example, to use OpenMPI with Intel compilers, specify mpicc and mpicxx after setting environmental variables OMPI_CC=icc and OMPI_CXX=icpc. See OpenMPI FAQ for details.

Set-up queue job submission

The GUI allows the user to submit jobs to a job queueing system with a single click. For this to work, a template job submission script needs to be provided for the queueing system at hand (e.g. TORQUE, PBS, SGE). In this script a set of strings (variables) in the template script is replaced by the values given in the GUI. The following table contains all defined variables:

String

Variable

Meaning

XXXcommandXXX

string

relion command + arguments

XXXqueueXXX

string

Name of the queue to submit job to

XXXmpinodesXXX

integer

The number of MPI processes to use

XXXthreadsXXX

integer

The number of threads to use on each MPI process

XXXcoresXXX

integer

The number of MPI processes times the number of threads

XXXdedicatedXXX

integer

The minimum number of cores on each node (use this to fill entire nodes)

XXXnodesXXX

integer

The total number of nodes to be requested

XXXextra1XXX

string

Installation-specific, see below

XXXextra2XXX

string

Installation-specific, see below

The XXXcommandXXX variable needs a special care. For non-MPI commands (e.g. relion_refine) not only the variable but the whole line is replaced. Thus, mpirun XXXcommandXXX will be mpirun relion_refine_mpi for an MPI job but relion_refine for a non-MPI job. Also note that some jobs consist of multiple lines of commands. See CCPEM threads (1 and 2) for typical pitfalls.

There are two environment variables that control the use of the entry of the ‘Minimum number of dedicated cores per node’ on the Running tabs of the GUI: RELION_MINIMUM_DEDICATED sets its default value (1 if not set); RELION_ALLOW_CHANGE_MINIMUM_DEDICATED sets whether the user will be able to change this entry. At LMB, we set the default to 24 and do not allow users to change it. In this way, we enforce that our hyper-threaded 12-core nodes get filled up entirely by individual relion jobs.

By default, the XXXextra1XXX, XXXextra2XXX, … variables are not used. They provide additional flexibility for queueing systems that require additional variables. They may be activated by first setting RELION_QSUB_EXTRA_COUNT to the number of fields you need (e.g. 2) and then setting the RELION_QSUB_EXTRA1, RELION_QSUB_EXTRA2, … environment variables, respectively. This will result in extra input fields in the GUI, with the label text being equal to the value of the environment variable. Likewise, their default values (upon starting the GUI) can be set through environment variables RELION_QSUB_EXTRA1_DEFAULT, RELION_QSUB_EXTRA2_DEFAULT, etc and their help messages can be set through environmental variables RELION_QSUB_EXTRA1_HELP, RELION_QSUB_EXTRA2_HELP and so on. But note that (unlike all other entries in the GUI) the extra values are not remembered from one run to the other.

The template job submission script may be saved in any location. By default, the one used at the LMB is present as gui/qsub.csh in the relion tar-ball. Upon installation this file is copied to the bin directory. It is convenient for the user if he does not have to select this file each time he opens the relion GUI in a new directory. Therefore, one may set the environment variable RELION_QSUB_TEMPLATE to point to the location of the script for the system at hand. This value will be pre-set as default in the GUI. (Note the user still has the liberty to define and use his own template!)

Note

If somehow the job queue submission cannot be set up, relion may still be run in parallel and on a job queueing system. The GUI comprises a Print command button that prints the entire relion command, including all arguments, to the screen. Pasting of this command to a job queue submission script, and manual submission of this script may then be used to submit the parallel job to a queueing system.

Edit the environment set-up

For relion, we source the following C-shell setup in our .cshrc file. You’ll need to change all the paths for your own system, and translate the script in case you use a bash shell (which uses export instead of setenv, etc).

#!/bin/csh -f

# Setup openMPI if not already done so
if ("" == "`echo $path | grep /public/EM/OpenMPI/openmpi/bin`") then
        set path=(/public/EM/OpenMPI/openmpi/bin $path)
endif
if ("1" == "$?LD_LIBRARY_PATH") then
        if ("$LD_LIBRARY_PATH" !~ */public/EM/OpenMPI/openmpi/lib*) then
                setenv LD_LIBRARY_PATH /public/EM/OpenMPI/openmpi/lib:$LD_LIBRARY_PATH
        endif
else
        setenv LD_LIBRARY_PATH /public/EM/OpenMPI/openmpi/lib
endif

# Setup |RELION| if not already done so
if ("" == "`echo $path | grep /public/EM/RELION/relion/bin`") then
   set path=(/public/EM/RELION/relion/bin $path)
endif
if ("1" == "$?LD_LIBRARY_PATH") then
        if ("$LD_LIBRARY_PATH" !~ */public/EM/RELION/relion/lib*) then
                setenv LD_LIBRARY_PATH /public/EM/RELION/relion/lib:$LD_LIBRARY_PATH
        endif
else
        setenv LD_LIBRARY_PATH /public/EM/RELION/relion/lib
endif

# CUDA for RELION
setenv PATH /public/EM/CUDA/Cuda7.0/bin:$PATH
setenv LD_LIBRARY_PATH /public/EM/CUDA/Cuda7.0/lib64:$LD_LIBRARY_PATH
setenv CUDA_HOME /public/EM/CUDA/Cuda7.0

# Where is qsub template script stored
setenv RELION_QSUB_TEMPLATE /public/EM/RELION/relion-prerelease/bin/qsub.csh

# Default PDF viewer
setenv RELION_PDFVIEWER_EXECUTABLE evince

# Default MOTIONCOR2 executable
setenv RELION_MOTIONCOR2_EXECUTABLE /public/EM/MOTIONCOR2/bin/MotionCor2_1.0.4

# Default CTFFIND-4.1+ executable
setenv RELION_CTFFIND_EXECUTABLE /public/EM/ctffind/ctffind.exe

# Default Gctf executable
setenv RELION_GCTF_EXECUTABLE /public/EM/Gctf/bin/Gctf

# Default ResMap executable
setenv RELION_RESMAP_EXECUTABLE /public/EM/ResMap/ResMap-1.1.4-linux64

# Default Topaz executable
setenv RELION_TOPAZ_EXECUTABLE /public/EM/RELION/topaz

# Enforce cluster jobs to occupy entire nodes with 24 hyperthreads
setenv RELION_MINIMUM_DEDICATED 24
# Do not allow the user to change the enforcement of entire nodes
setenv RELION_ALLOW_CHANGE_MINIMUM_DEDICATED 0

# Ask for confirmation if users try to submit local jobs with more than 12 MPI nodes
setenv RELION_WARNING_LOCAL_MPI 12

# Other useful variables
# RELION_MPI_RUN: The mpi runtime ('mpirun' by default)
# RELION_QSUB_NRMPI: The default for 'Number of MPI procs'
# RELION_MPI_MAX: The maximum number of MPI processes available from the GUI
# RELION_QSUB_NRTHREADS: The default for 'Number of threads'
# RELION_THREAD_MAX: The maximum number of threads per MPI process available from the GUI
# RELION_QUEUE_USE: The default for 'Submit to queue?'. "yes" or "no".
# RELION_QUEUE_NAME: The default for 'Queue Name"
# RELION_QSUB_COMMAND: The default for 'Queue submit command'
# RELION_MINIMUM_DEDICATED: The default for 'Minimum dedicated cores per node'
# RELION_ALLOW_CHANGE_MINIMUM_DEDICATED: Whether to allow a user to change the 'Minimum dedicated cores per node' field in the GUI
# RELION_SHELL: A shell used to launch CTFFIND/GCTF in CtfFind jobs ('csh' by default; only available from 3.1)
# RELION_SCRATCH_DIR: The default scratch directory in the GUI
# RELION_STACK_BUFFER: The buffer size used for MRC(S) file I/O, potentially useful on GPFS or Lustre file system. See https://github.com/3dem/relion/pull/783 for details.