DeepMind: AlphaFold: can accurately predict 3D models of protein structures and has the potential to accelerate research in every field of biology.
AlphaFold Protein Structure Database: Developed by DeepMind and EMBL-EBI
AlphaFold Colab: “This Colab notebook allows you to easily predict the structure of a protein using a slightly simplified version of AlphaFold v2.1.0.“
General Articles:
ARCC are NOT domain experts on the science behind using Alphafold. We can best effort support on errors you come across, but not the use of the flags and databases used by Alphafold.
Please share any feedback you have, and we will develop this page for the wider community.
Use the module name alphafold to discover versions available and to load the application.
Loading the particular alphafold module version will appropriate set the following environment variables: ALPHADB and ALPHABIN and set the associated singularity module version.
Alphafold has been built via a docker image, but has been converted to a Singularity image, so must be run using Singularity.
As versions of alphafold update, available options will change. On loading the alphafold module, a full list of flags can be found by running:
singularity run -B .:/etc $ALPHABIN/alphafold220.sif --help singularity run -B .:/etc $ALPHABIN/alphafold220.sif --helpfull |
Version | Data Tree | Example | ||
|---|---|---|---|---|
2.3.0 |
|
| ||
2.2.0 |
|
|
Our test file T1050.fasta looks like this.
>T1050 A7LXT1, Bacteroides Ovatus, 779 residues| MASQSYLFKHLEVSDGLSNNSVNTIYKDRDGFMWFGTTTGLNRYDGYTFKIYQHAENEPGSLPDNYITDIVEMPDGRFWINTARGYVLFDKERDYFITDVTGFMKNLESWGVPEQVFVDREGNTWLSVAGEGCYRYKEGGKRLFFSYTEHSLPEYGVTQMAECSDGILLIYNTGLLVCLDRATLAIKWQSDEIKKYIPGGKTIELSLFVDRDNCIWAYSLMGIWAYDCGTKSWRTDLTGIWSSRPDVIIHAVAQDIEGRIWVGKDYDGIDVLEKETGKVTSLVAHDDNGRSLPHNTIYDLYADRDGVMWVGTYKKGVSYYSESIFKFNMYEWGDITCIEQADEDRLWLGTNDHGILLWNRSTGKAEPFWRDAEGQLPNPVVSMLKSKDGKLWVGTFNGGLYCMNGSQVRSYKEGTGNALASNNVWALVEDDKGRIWIASLGGGLQCLEPLSGTFETYTSNNSALLENNVTSLCWVDDNTLFFGTASQGVGTMDMRTREIKKIQGQSDSMKLSNDAVNHVYKDSRGLVWIATREGLNVYDTRRHMFLDLFPVVEAKGNFIAAITEDQERNMWVSTSRKVIRVTVASDGKGSYLFDSRAYNSEDGLQNCDFNQRSIKTLHNGIIAIGGLYGVNIFAPDHIRYNKMLPNVMFTGLSLFDEAVKVGQSYGGRVLIEKELNDVENVEFDYKQNIFSVSFASDNYNLPEKTQYMYKLEGFNNDWLTLPVGVHNVTFTNLAPGKYVLRVKAINSDGYVGIKEATLGIVVNPPFKLAAALQHHHHHH |
If you have alternative examples, please share.
TPUs are Google's specialized ASICs and are thus not available on our NVidia GPUs. The following form of warnings can be ignored:
I0927 02:45:29.788146 47769932949376 tpu_client.py:54] Starting the local TPU driver. I0927 02:45:29.829137 47769932949376 xla_bridge.py:212] Unable to initialize backend 'tpu_driver': Not found: Unable to find driver in registry given worker: local:// |
Slurm parameters and alphafold flag:
#SBATCH --nodes=1 #SBATCH --ntasks-per-node=1 #SBATCH --cpus-per-task=8 #SBATCH --mem=64G --use_gpu_relax=False |
The mem value will depend on your data, please share your findings/observations.
Notice that neither GPUs/TPUs are detecting, so only running in CPU mode. Since running in CPU mode, your output will contain very slow compile messages
|
Slurm parameters and alphafold flag:
#SBATCH --nodes=1 #SBATCH --ntasks-per-node=1 #SBATCH --cpus-per-task=8 #SBATCH --mem=64G #SBATCH --gres=gpu:p100:1 #SBATCH --partiton=teton-gpu --use_gpu_relax=True |
The P100 GPUs do not have TPU capabilities, so expect to see the Unable to initialize backend 'tpu' message. The job will finish with a Final timings message that lists timings for all the number of models details at the start - so in this example 5.
|
#SBATCH --nodes=1 #SBATCH --ntasks-per-node=1 #SBATCH --cpus-per-task=8 #SBATCH --mem=64G #SBATCH --gres=gpu:p100:1 #SBATCH --partiton=teton-gpu --use_gpu_relax=False |
At this stage, we haven’t noticed any observable difference with setting use_gpu_relax to True or False.
Looking at the flag help:
--[no]run_relax: Whether to run the final relaxation step on the predicted models. Turning relax off might result in predictions with distracting stereochemical violations but might help
in case you are having issues with the relaxation stage.
(default: 'true')
...
--[no]use_gpu_relax: Whether to relax on GPU. Relax on GPU can be much faster than CPU, so it is recommended to enable if possible. GPUs must be available if this setting is enabled. |
The test that we are running might not be performing the “final relaxation step on the predicted models“ which is why we’re not seeing any significant difference.
And/Or this might be because it uses the tensor core capabilities of the GPU, which we do not have on the P100s.
Our test file appears to require at least 64G, otherwise we have run into out of memory issues.
If using a GPU, we have only had successes using the P100/A30s.
The P100s do NOT have tensor cores.
Using 2 GPUs showed no speed increase over one.
Although our V100s do have tensor cores, we have not had any successful tests. We believe this is due to the current NVidia drivers / cuda versions - we have ongoing testing for these.
The timings below are all using the same dataset (see below) but each will have a different random seed so the runs are not deterministic (i.e. they have an element of randomness) so we can not expect the same resource allocation to run in the same time.
Within a CPU only mode, 16 cores appears to run faster than 8 or 32.
The cascade nodes are generally faster than the teton nodes - to be expected as they have a newer chipset.
The teton-gpu P100s on are significantly faster. We only have 8 of these so expecting jobs to be queued.
These timings are only a small subset of possible resource dimensions that can be changed within a job submission, but provide some basic insight into what to consider. Also, consider that we have 180 teton nodes and 56 cascade nodes, so if you have time to run simulations (e.g. over the weekend) then you can submit a lot more jobs that have a better chance of not being queued than simply requesting the P100s and potentially having your jobs queued for hours/days.
Partition | cores:8 | cores:16 | cores:32 |
|---|---|---|---|
teton: cpu | 20:28:1 | 18:54:09 13:15:21 | 19:50:55 |
teton-cascade: cpu | 16:02:35 | 10:56:07 11:34.45 | 12:07:54 12:06:52 12:08:08 |
teton-gpu: 1 p100 | 02:38:10 02:49:23 03:04:28 02:43:51 | 02:39:39 02:53:48 02:53:40 02:58:28 | |
beartooth-gpu: 1 a30 | 02:39:51 |