Globus for Pathfinder/S3 Buckets

Note that this collection is still under construction and may go through changes in the future. If you have issues accessing the UWYO Globus Pathfinder collection, check our UW ARCC Announcements Page to see if anything has changed.

Access Setup:

Access to Pathfinder S3 Storage is managed by public/private key combinations. When you first set up access to your Pathfinder storage, you were likely provided a keyset from ARCC. This same keyset will be used to access your Pathfinder share through Globus. This access is all encompassing and provides full access permissions to the S3 bucket.

How to Access Pathfinder S3 Storage over Globus:

Select the Globus S3 Endpoint

Login with Globus at https://app.globus.org

Select “Endpoints” from the left navigation.

Enter “Pathfinder S3 Access” into the search textbox.

From the list of endpoints that appear, click on the “Pathfinder S3 Access” link.

You will be prompted to set up permission. Click continue:

Now choose the option on the right to “add credentials”.

Setup Credentials

Click on the “Credentials” tab of the “Pathfinder S3 Access” collection page.

Here is where you register your access key ID and secret key with Globus. This will be your public and private access key provided by ARCC to access your bucket. Please be aware that at this time, you may only register a single Pathfinder S3 bucket for access within Globus.

After you’ve entered them, click the “Continue” button, and you’ll be taken back to the full “Credentials” tab where you can see your saved access credentials.

At this point you are set up to access the S3 buckets with Globus. Click the “Overview” tab, and then the “Open in File Manager” button to see the S3 buckets and data that are available using your credentials.

Create Guest Collection from Pathfinder S3 Access Collection to Share with Others

Click on the “Collections” tab of the “Pathfinder S3 Access” endpoint, and then click on the “Add a Guest Collection”.

Enter in the path to the top level folder you want visible in your collection in the “Directory” field (or enter “/” to use all buckets available to those credentials in your collection). You can also click the “Browse” button to get a directory view and select the bucket or subfolder folder you want.

Enter a value for the “Display Name” field, and then click the “Create Collection” button.

Changing Globus Settings to Access Another Pathfinder S3 Bucket:

At this time, because you can only access a single S3 bucket from a single collection in Globus, you will need to swap access between buckets by going into Collections and selecting the collection details for the pathfinder collection:

Once in the collection details window for Pathfinder, click on the credentials tab at the top.

In the credentials window, assuming you have your keys saved to access this S3 bucket, hit delete to delete the Globus connection to this particular Pathfinder S3 Bucket.

Deleting this set of Access Credentials allows you to set up another one.

Follow the Access Setup instructions above to configure access for a different Pathfinder S3 Bucket over Globus.

Adding Additional Keys & Buckets

Now that you have your default s3 key setup, you can then add any additional keys and the corresponding buckets that you may have access to.

The first step is to find the Pathfinder mapped collection in the ‘Search Collections’ box and then clicking the three dots on the right side to begin to manage.

Once there, go to the ‘Credentials’ tab and click the ‘+' sign to the right of the section that says 'AWS Access Keys’

From here, you will get the option to add an additional access key and secret key combo. However, this time, you will need to add the names of any additional buckets that this key has access to individually.

To add more buckets, simply follow the same procedure of going to the collection, clicking the three dots to manage, going to the credentials page, and then adding them to the appropriate key.

Limitations:

Globus S3 does not support the following S3 features, currently:

Custom Metadata/Tagging: Custom Metadata and tags on existing objects are ignored when those objects are downloaded, and new objects will not have custom metadata or tags set.
Versions: When downloading a file from an S3 bucket, Globus will always access the latest version and no others.
ACLs: Access control lists on existing objects and buckets will influence what you can download through Globus but those ACLs are not currently copied into or out of S3. When uploading new objects, ACLs are not explicitly set.
Additional Checksum Algorithms: At this time, if an object in S3 needs to be verified, GLobus will re-download in order to compute the checksum.