Alphafold

Overview

• DeepMind: AlphaFold: can accurately predict 3D models of protein structures and has the potential to accelerate research in every field of biology.

Documentation

• AlphaFold Protein Structure Database: Developed by DeepMind and EMBL-EBI

• AlphaFold Colab: “This Colab notebook allows you to easily predict the structure of a protein using a slightly simplified version of AlphaFold v2.1.0.“

• General Articles:

  • Deepmind: AlphaFold

  • Putting the power of AlphaFold into the world’s hands.

• ARCC are NOT domain experts on the science behind using Alphafold. We can best effort support on errors you come across, but not the use of the flags and databases used by Alphafold.

  • Please share any feedback you have, and we will develop this page for the wider community.

Using

Use the module name alphafold to discover versions available and to load the application.

Loading the particular alphafold module version will appropriate set the following environment variables: ALPHADB and ALPHABIN and set the associated singularity module version.

Running Alphafold

Alphafold has been built via a docker image, but has been converted to a Singularity image, so must be run using Singularity.

Flag Help

As versions of alphafold update, available options will change. On loading the alphafold module, a full list of flags can be found by running:

Data Files and Examples

Our test file T1050.fasta looks like this.

If you have alternative examples, please share.

TPU Warnings

TPUs are Google's specialized ASICs and are thus not available on our NVidia GPUs. The following form of warnings can be ignored:

CPU Mode:

Slurm parameters and alphafold flag:

The mem value will depend on your data, please share your findings/observations.

Notice that neither GPUs/TPUs are detecting, so only running in CPU mode. Since running in CPU mode, your output will contain very slow compile messages

GPU Mode:

With: use_gpu_relax=True

Slurm parameters and alphafold flag:

The P100 GPUs do not have TPU capabilities, so expect to see the Unable to initialize backend 'tpu' message. The job will finish with a Final timings message that lists timings for all the number of models details at the start - so in this example 5.

With: use_gpu_relax=False

At this stage, we haven’t noticed any observable difference with setting use_gpu_relax to True or False.

Looking at the flag help:

• The test that we are running might not be performing the “final relaxation step on the predicted models“ which is why we’re not seeing any significant difference.

• And/Or this might be because it uses the tensor core capabilities of the GPU, which we do not have on the P100s.

Current Recommendations:

• Our test file appears to require at least 64G, otherwise we have run into out of memory issues.

• If using a GPU, we have only had successes using the P100/A30s.

• The P100s do NOT have tensor cores.

  • Using 2 GPUs showed no speed increase over one.

• Although our V100s do have tensor cores, we have not had any successful tests. We believe this is due to the current NVidia drivers / cuda versions - we have ongoing testing for these.

Run Times: Observations

The timings below are all using the same dataset (see below) but each will have a different random seed so the runs are not deterministic (i.e. they have an element of randomness) so we can not expect the same resource allocation to run in the same time.

• Within a CPU only mode, 16 cores appears to run faster than 8 or 32.

• The cascade nodes are generally faster than the teton nodes - to be expected as they have a newer chipset.

• The teton-gpu P100s on are significantly faster. We only have 8 of these so expecting jobs to be queued.

These timings are only a small subset of possible resource dimensions that can be changed within a job submission, but provide some basic insight into what to consider. Also, consider that we have 180 teton nodes and 56 cascade nodes, so if you have time to run simulations (e.g. over the weekend) then you can submit a lot more jobs that have a better chance of not being queued than simply requesting the P100s and potentially having your jobs queued for hours/days.