=============================
HEASoft singularity container
=============================

.. contents:: :local:

--------------

HEASoft is a unified release of the FTOOLS and XANADU Software Packages.
This document describes how to use a singularity container with EASoft
on the HPC cluster at Fysikum.

HEASoft references:

- https://heasarc.gsfc.nasa.gov/lheasoft/
- https://heasarc.gsfc.nasa.gov/lheasoft/docker.html

An overview how to use singularity in a cluster environment:

- https://indico.fysik.su.se/event/7203/attachments/4610/5377/singularity-containers.pdf

--------------

Using existng container
-----------------------

The HEASoft singularity container is in 
``/cfs/home/sw/heasoft/heasoft-v6.29c.sif``.
A singularity shell can be started sourcing
``/cfs/home/sw/heasoft/init.sh``.
Here is an example usage:

::

    user@sol-login:~$ source /cfs/home/sw/heasoft/init.sh

    user@heasoft:~$ fversion
    31Aug2021_V6.29c

    user@heasoft:~$ fhelp heatools

    List of heatools tasks

       ftappend - Append a copy of an HDU from one file onto another file
       ...

    user@heasoft:~$ exit

    user@sol-login:~$

The ``init.sh`` script will load singularity module and then execute
a bash shell using the initial ``/home/heasoft/.bashrc``.
The contents of ``init.sh`` is:

.. code-block:: bash

    module load singularity

    LC_ALL=C singularity exec --hostname heasoft \
        /cfs/home/sw/heasoft/heasoft-v6.29c.sif \
        bash --rcfile /home/heasoft/.bashrc

--------------

Building singularity container from scratch
-------------------------------------------

One can build own version of the HEASoft singularity container 
using the following procedure. 

.. warning::

    The procedure must be executed with `root` privileges on a 
    local client computer. The required packages are `docker` and
    `singularity`

The first step is to build a docker container following the instructions
on https://heasarc.gsfc.nasa.gov/lheasoft/docker.html: ::

    mkdir heasoft
    cd heasoft
    wget https://heasarc.gsfc.nasa.gov/FTP/software/lheasoft/release/heasoft-6.29docker0.6.tar
    tar xvf heasoft-6.29docker0.6.tar
    make


Depending on the computer speed, the build process can take 3 or more hours.
To test the newly build HEASoft docker container, execute: ::

    docker run --rm -it heasoft:v6.29 bash
    fversion
    exit

Next, we export the docker image to tar.
First we list the images to find out the image name,
then we save the image as a tarball.  ::

    docker images
    docker save 78cafdfca446 -o heasoft-v6.29c-docker.tar

Finally, we convert the docker tarball to singularity sif-file
(which can be used on the cluster).  ::

    singularity build heasoft-v6.29c.sif \
        docker-archive://heasoft-v6.29c-docker.tar

The sif file can be now transferred to and used on the cluster.

----------

Customizing existing container
------------------------------

To customize an existing singularity container 
(e.g., to install additional software in it),
one first needs to unpack the container into a sandbox,
modify the sandbox and then build ``.sif`` file from the sandbox.
Note that the sandbox is ~8 GiB in size while the sif file is ~4 GiB.

.. warning::

    The procedure must be executed with `root` privileges on a 
    local client computer. The required package is `singularity`.

First, we build a sandbox from the sif file.  ::

    singularity build --sandbox heasoft-sandbox heasot-v6.29c.sif

Then we execute a shell in the sandbox and customize it.
The option ``-w`` indicates that the sandbox is writable 
(by default it is readonly). 
Note that the HEASoft container is based on Ubuntu. ::

    singularity shell -w heasoft-sandbox
    apt update
    apt install my-software
    exit

Finally, we build a sif file from the sandbox.  ::

    singularity build my-heasoft.sif heasoft-sandbox

The singularity container ``my-heasoft.sif`` can be now transferred to 
and used on the cluster.