This section of the workshop expands upon the previous section on Basic Globus transfers. It is still an introductory level of content, but may not apply to all use cases.

Discussed in this section are the transfer options in Globus, sharing data with others, and then briefly touch on the Globus CLI and flows functionality.

Globus Transfer Options

Beyond simply transferring data, Globus enables users optional features that enhance the functionality of data transfers. Here are a few that ARCC frequently uses:

Label - Because Globus keeps a record of each transfer along with the stats bout them, it can be hard to determine which transfer is which based solely on the transfer ID and date. Giving a transfer a human-readable label can help users keep track of each transfer.
Sync - to only transfer new or changed files. Selecting this option gives the ability to choose how files will be overwritten on the destination file system. The options for syncing can be where the modification time of a file is newer, the file size is different, the file on the source does not exist on the destination, or where the checksum is different. When using this option, ARCC recommends using the checksum option, to cover a vareity of scenarios
Preserve source file modification times - After transfer of a file, the modification time on the destination will reflect the modification time on the source instead of the time the transfer completed.

Transfer Option Example

In this example we expand the ‘Transfer and Timer Options’ window to provide a demonstration transfer that has a label, syncing, and preserving the file modification time. Notice that there are many more options that can be used as well including:

Encryption
Skipping on errors
Deleting files that don’t exist on source

Notification and Scheduling Options

At the bottom of the same Transfer and Timer Options window, there are options to disable notifications of transfers as well as scheduling transfers that may or may not repeat

Sharing with Globus

Sharing with guest collections is available on Globus endpoints managed by an active Subscription. You can create guest collections on Globus endpoints if you are a member of a Subscription Group.

By sharing with Globus, the person you are sharing with will need a Globus account and access to an Endpoint
Please contact ARCC if you are interested in sharing data with Globus

The Globus CLI

Globus does have a command line interface (CLI) that can be used within HPC workflows as well. ARCC does not have this installed by default, but a user can install it using the miniconda module on MedicineBow.

This still uses the same terminology as basic Globus transfers and users will have to be familiar with those key concepts before attempting to use the CLI

Globus Flows

Globus flows enables the automation of data transfers. ARCC is still investigating how this works, but generally speaking, you have to have access to each endpoint you are wanting to transfer to and from and then do three things to deploy a Flow: