.. _adding-custom-pkgs-to-cbcore-container:

Adding Custom Packages To cbcore Container
==========================================

You can add custom packages to the Cerebras cbcore container. This section describes how to add two specific packages: the ``spektral`` and ``networkx``, to your Cerebras cbcore container.

Adding ``spektral`` and ``networkx`` packages
---------------------------------------------

Requirements
~~~~~~~~~~~~

.. admonition:: Read this before you proceed further

    - Make sure that your system administrator has enabled the user control of binds for Singularity. See here for more: `Bind Paths and Mounts — Singularity container 3.0 documentation <https://sylabs.io/guides/3.0/user-guide/bind_paths_and_mounts.html#user-defined-bind-paths>`_ .

    - Know that you are responsible to maintain the directory of packages you are installing in the Cerebras container (the ``~/Cerebras-0.8.0-extra-python-packages`` directory in the example installation instructions below).

    - Ensure that you map the directory containing these packages to all the Slurm nodes in your cluster, for example via NFS.

Installation
~~~~~~~~~~~~

1. Create a directory shared between and accessible by all the hosts. For example, a directory with the name: ``~/Cerebras-0.8.0-extra-python-packages``.

2. Run the following ``singularity exec`` command with ``-B`` option to perform the following two tasks:

        - Access the ``~/Cerebras-0.8.0-extra-python-packages`` directory as ``$HOME/.local`` directory from within the Cerebras container, and
        - Install the ``spektral`` and ``networkx`` packages in ``$HOME/.local``. In reality these packages are installed in ``~/Cerebras-0.8.0-extra-python-packages`` directory but they are accessible within the Cerebras container as ``$HOME/.local`` directory.

        .. code-block:: bash

            singularity exec -B ~/Cerebras-0.8.0-extra-python-packages:$HOME/.local /<path>/cbcore-0.8.0-<identifier-string>.sif pip install --user spektral==1.0.7 networkx==2.6.2

Accessing the packages
~~~~~~~~~~~~~~~~~~~~~~

Run the following command to test that the modules in the installed packages are accessible from within the Cerebras container:

.. code-block:: bash

    singularity exec -B ~/Cerebras-0.8.0-extra-python-packages:$HOME/.local /<path>/cbcore-0.8.0-<identifier-string>.sif python -c 'import spektral; print(spektral.__file__)'

The output should resemble the following, if the ``$HOME`` value is ``/my/home/username``:

    ``/my/home/username/.local/lib/python3.7/site-packages/spektral/__init__.py``.