Goal: Provide an exercise to work through that puts together the various concepts covered within this workshop.

1 Exercise: Put Things Together
2 Exercise: Extend with Pandas
3 Exercise: Suggested Answers: Create the Environment
4 Exercise: Suggested Answers: Run the Code
5 Exercise: Suggested Answers: Extend with Pandas

Exercise: Put Things Together

Exercise: Create a self contained Conda environment that would allow a user to run a python file that uses PyTorch.

The environment should be created within a location that can be shared with other users within a project.
Make sure the environment can utilize GPUs running with cuda 12.4.

Using the the following pytorch_gpu_test.py script, create a bash script to submit a job to the cluster that allows this to be run, using the created Conda environment, that utilizes one H100 NVidia GPU device on a single compute node, with 8 cores and 16G of memory.
Any output should be written into a slurms sub folder, with a filename of the form pytorch_<job-id>.out
Send email notification to yourself regards the status of the submission.

import torch
import math

print("PyTorch Version: " + str(torch.__version__))
print("Cuda Available: " + str(torch.cuda.is_available()))
print("Device Count: " + str(torch.cuda.device_count()))
print("Device Name: " + str(torch.cuda.get_device_name(0)))

dtype = torch.float
device = torch.device("cuda:0") # Uncomment this to run on GPU

# Create random input and output data
x = torch.linspace(-math.pi, math.pi, 2000, device=device, dtype=dtype)
y = torch.sin(x)

# Randomly initialize weights
a = torch.randn((), device=device, dtype=dtype)
b = torch.randn((), device=device, dtype=dtype)
c = torch.randn((), device=device, dtype=dtype)
d = torch.randn((), device=device, dtype=dtype)

learning_rate = 1e-6
for t in range(2000):
    # Forward pass: compute predicted y
    y_pred = a + b * x + c * x ** 2 + d * x ** 3

    # Compute and print loss
    loss = (y_pred - y).pow(2).sum().item()
    if t % 100 == 99:
        print(t, loss)

    # Backprop to compute gradients of a, b, c, d with respect to loss
    grad_y_pred = 2.0 * (y_pred - y)
    grad_a = grad_y_pred.sum()
    grad_b = (grad_y_pred * x).sum()
    grad_c = (grad_y_pred * x ** 2).sum()
    grad_d = (grad_y_pred * x ** 3).sum()

    # Update weights using gradient descent
    a -= learning_rate * grad_a
    b -= learning_rate * grad_b
    c -= learning_rate * grad_c
    d -= learning_rate * grad_d

print(f'Result: y = {a.item()} + {b.item()} x + {c.item()} x^2 + {d.item()} x^3')

Ignoring any cached packages, how much storage space does this Conda environment use?
Which version of Python is being used?

Exercise: Extend with Pandas

Exercise: Suggested Answers: Create the Environment

[~]$ cd /project/<project-name>/software/pytorch
[]$ module purge
[]$ module load miniconda3/24.3.0
[]$ conda create -p pytorch_env
...
# To activate this environment, use
#
#     $ conda activate /cluster/medbow/project/<project-name>/software/pytorch/pytorch_env
#
# To deactivate an active environment, use
#
#     $ conda deactivate

[]$ conda activate /cluster/medbow/project/<project-name>/software/pytorch/pytorch_env
(/cluster/medbow/project/<project-name>/software/pytorch/pytorch_env) []$ conda install pytorch torchvision torchaudio pytorch-cuda=12.4 -c pytorch -c nvidia
(/cluster/medbow/project/<project-name>/software/pytorch/pytorch_env) []$ conda deactivate
[]$

[pytorch]$ du -d 1 -h
6.3G    ./pytorch_env
6.3G

Exercise: Suggested Answers: Run the Code

Exercise: Suggested Answers: Extend with Pandas

Prev

Python, Conda and Pip: Suggested Best Practices

Workshop Home

Using Python, Conda and Pip on the Cluster

Next

Python, Conda and Pip: Summary

ARCC Wiki

Python, Conda and Pip: Exercise

Exercise: Put Things Together

Exercise: Extend with Pandas

Exercise: Suggested Answers: Create the Environment

Exercise: Suggested Answers: Run the Code

Exercise: Suggested Answers: Extend with Pandas