Introduction: The workshop session will provide a quick tour covering high-level concepts, commands and processes for using Linux and HPC on our Beartooth cluster. It will cover enough to allow an attendee to access the cluster and to perform analysis associated with this workshop.
Goals:
...
Introduction: The workshop session will provide a quick tour covering high-level concepts, commands and processes for using Linux and HPC on our Beartooth cluster. It will cover enough to allow an attendee to access the cluster and to perform analysis associated with this workshop.
Goals:
Introduce ARCC and what types of services we provide including “what is HPC?”
Define “what is a cluster”, and how is it made of partitions and compute nodes.
How to access and start using ARCC’s Beartooth cluster - using our SouthPass service.
How to start an interactive desktop and open a terminal to use Linux commands within.
Introduce the basics of Linux, the command-line, and how its File System looks on Beartooth.
Introduce Linux commands to allow navigation and file/folder manipulation.
Introduce Linux commands to allow text files to be searched and manipulated.
Introduce using a command-line text-editor and an alternative GUI based application.
How to setup a Linux environment to use R(/Python) and start RStudio, by loading modules.
How to start interactive sessions to run on a compute node, to allow computation, requesting appropriate resources.
How to put elements together to construct a workflow that can be submitted as a job to the cluster, which can then be monitored.
...
Our training will help provide the foundation necessary for you to use Beartooth cluster, specifically to perform some of the exercises later in this workshop over the week.
Because of our limited time this morning, please submit any questions to the slack channel for this workshop and workshop instructors can address them as they are available.
More extensive and in-depth information and walkthroughs are available on our wiki and you under workshops/tutorials. You are welcome to dive into those in your own time. Content within them should provide you with a lot of the foundational concepts you would need to be familiar with to become a proficient HPC user.
...
Describe ARCC’s role at UW.
Provide resources for ARCC Researchers to seek help.
Introduce staff members, including those available throughout the workshop.
Introduce the concept of an HPC cluster, it’s architecture and when to use one.
Introduce the Beartooth HPC architecture, hardware, and partitions.
...
About ARCC and how to reach us
...
| |
|---|
...
What is a Compute Node?
...
We typically have multiple users independently running jobs concurrently across compute nodes.
Resources are shared, but to do interfere with any one else’s resources.
i.e. you have your own cores, your own block of memory.
If someone else’s job fails it does NOT affect yours.
...
Homogeneous vs Heterogeneous HPCs
...
Homogeneous: All compute nodes in the system share the same architecture. CPU, memory, and storage are the same across the system. (Ex: NWSC’s Derecho)
Heterogeneous: The compute nodes in the system can vary architecturally with respect to CPU, memory, even storage, and whether they have GPUs or not. Usually, the nodes are grouped in partitions. Beartooth is a heterogeneous cluster and our partitions are described on the described on the Beartooth Hardware Summary Table on our ARCC Wiki.
...
Beartooth Cluster: Heterogeneous: Partitions
...
Beartooth Hardware and Partitions
| Expand | ||||||||
|---|---|---|---|---|---|---|---|---|
| ||||||||
|
| Expand | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ||||||||||
|
See Beartooth Hardware Summary Table on
...
the ARCC Wiki.
Beartooth Hardware and Partitions
...
| title | Beartooth Partition Table |
|---|
...
| title | Beartooth GPU Table |
|---|
...
Reservation
A reservation can be considered a temporary partition.
It is a set of compute nodes reserved for a period of time for a set of users/projects, who get priority use.
For this workshop we will be using the following: biocompworkshop:
| Code Block |
|---|
ReservationName = biocompworkshop
StartTime = 06.09-09:00:00
EndTime = 06.17-17:00:00
Duration = 8-08:00:00
Nodes = mdgx01,t[402-421],tdgx01 NodeCnt=22 CoreCnt=720
Users = Groups=biocompworkshop |
Important Dates:
After the 17th of June this reservation will stop and you will drop down to general usage if you have another Beartooth project.
The project itself will be removed after the 24th of June. You will not be able to use/access it. Anything you require please copy out of the project.
...
02 Using Southpass to access the Beartooth HPC Cluster
...
| Expand | ||
|---|---|---|
| ||
|
...
What am I Using?
Remember:
The Beartooth Shell Access opens up a new browser tab that is running on a login node. Do not run any computation on these.
[<username>@blog2 ~]$The SouthPass Interactive Desktop (terminal) is already running on a compute node.
[<username>@t402 ~]$
...
Login Node Policy
As a courtesy to your colleagues, please do not run the following on any login nodes:
Anything compute-intensive (tasks using significant computational/hardware resources - Ex: using 100% cluster CPU)
Long running tasks (over 10 min)
Any collection of a large # of tasks resulting in a similar hardware footprint to actions mentioned previously.
Not sure? Use
sallocto be on the safe side. This will be covered later.
Ex:salloc –-account=arccanetrain -–time 40:00See more on ARCC’s Login Node Policy here
...
Vi/Vim Cheatsheet | Nano Cheatsheet |
|---|---|
 |  |
Note: On Beartooth, vi maps to vim.
...
Demonstrating vi/vim text editor
...
Vim Tutor is a walkthrough for new users to get used to Vim. Run |
|
...
*** Break ***
...
04 Using Linux to Search/Parse Text Files
...
Since the cluster has to cater for everyone we can not provide a simple desktop environment that provides everything.
Instead we provide modules that a user will load that configures their environment for their particular needs within a session.
...