Page Comparison

Introduction: The workshop session will provide a quick tour covering high-level concepts, commands and processes for using Linux and HPC on our Beartooth cluster. It will cover enough to allow an attendee to access the cluster and to perform analysis associated with this workshop.

...

What is HPC

HPC stands for High Performance Computing and is one of UW ARCC’s core services. HPC is the practice of aggregating computing power in a way that delivers a much higher performance than one could get out of a typical desktop or workstation. HPC is commonly used to solve large problems, and has some common use cases:

...

• We typically have multiple users independently running jobs concurrently across compute nodes.

• Resources are shared, but do not interfere with any one else’s resources.

  • i.e. you have your own cores, your own block of memory.

• If someone else’s job fails it does NOT affect yours.

...

There are 2 types of HPC systems:

1. Homogeneous: All compute nodes in the system share the same architecture. CPU, memory, and storage are the same across the system. (Ex: NWSC’s Derecho)

2. Heterogeneous: The compute nodes in the system can vary architecturally with respect to CPU, memory, even storage, and whether they have GPUs or not. Usually, the nodes are grouped in partitions. Beartooth is a heterogeneous cluster and our partitions are described on the Beartooth Hardware Summary Table on our ARCC Wiki.

...

A reservation can be considered a temporary partition.

It is a set of compute nodes reserved for a period of time for a set of users/projects, who get priority use.

...

Important Dates:

1. After the 17th of June this reservation will stop and you will drop down to general usage if you have another Beartooth project.

2. The project itself will be removed after the 24th of June. You will not be able to use/access it. Anything you require please copy out of the project.

...

Note: While we use a webform to request Beartooth resources on Southpass, later training will show how resource configurations can be requested through command line via salloc or sbatch commands.

...

4. /apps (Specific to ARCC HPC) is like on Windows or on a Mac.

   1. Where applications are installed and where modules are loaded from. (More on that later).

5. /alcova (Specific to ARCC HPC).

   1. Additional research storage for research projects that may not require HPC but is accessible from beartooth.

   2. You won’t have access to it unless you were added to an alcova project by the PI.

...

Exercise: File Browsing in Southpass GUI

...

• The Beartooth Shell Access opens up a new browser tab that is running on a login node. Do not run any computation on these.
  [<username>@blog2 ~]$

• The SouthPass Interactive Desktop (terminal) is already running on a compute node.
  [<username>@t402 ~]$

...

Login Node Policy

...

1. Anything compute-intensive (tasks using significant computational/hardware resources - Ex: using 100% cluster CPU)

2. Long running tasks (over 10 min)

3. Any collection of a large # of tasks resulting in a similar hardware footprint to actions mentioned previously.  

4. Not sure?  Usesallocto be on the safe side. This will be covered later.
   Ex:salloc –-account=arccanetrain -–time 40:00

5. See more on ARCC’s Login Node Policy here

...

arcc-t10@blog2 ~]$ man pwd
NAME
       pwd - print name of current/working directory
SYNOPSIS
       pwd [OPTION]...
DESCRIPTION
       Print the full filename of the current working directory.
       -L, --logical
              use PWD from environment, even if it contains symlinks
       -P, --physical
              avoid all symlinks
       --help display this help and exit
       --version
              output version information and exit
       If no option is specified, -P is assumed.
       NOTE:  your  shell  may have its own version of pwd, which usually supersedes the version described here.  Please refer to your shell's documentation
       for details about the options it supports.

[arcc-t10@blog1 ~]$ cp --help
Usage: cp [OPTION]... [-T] SOURCE DEST
  or:  cp [OPTION]... SOURCE... DIRECTORY
  or:  cp [OPTION]... -t DIRECTORY SOURCE...
Copy SOURCE to DEST, or multiple SOURCE(s) to DIRECTORY.

...

Note: On Beartooth, vi maps to vim i.e. if you open vi, you're actually starting vim.

...

Demonstrating vi/vim text editor

...

Since the cluster has to cater for everyone we can not provide a simple desktop environment that provides everything.

Instead we provide modules that a user will load that configures their environment for their particular needs within a session.

...

• We’ve covered the following high-level concepts, commands and processes:

• What is HPC and what is a cluster - focusing on ARC’s Beartooth cluster.

• An introduction to Linux and its File System, and how to navigate around using an Interactive Desktop and/or using the command-line.

• Linux command-line commands to view, search, parse, sort text files.

• How to pipe the output of one command to the input of another, and how to redirect output to a file.

• Using vim as a command-line text editor and/or emacs as a GUI within an Interactive Desktop.

• Setting up your environment (using modules) to provide R/Python environments, and other software applications.

• Accessing compute nodes via a SouthPass Interactive Desktop, and requesting different resources (cores, memory, GPUs).

• Requesting interactive sessions (from a login node) using salloc.

• Setting up a workflow, within a script, that can then be submitted to the Slurm queue using sbatch, and how to monitor jobs.

...