Goal: Use various commands to view, inspect and search a text file.

Table of Contents

minLevel	1
maxLevel	1
outline	false
style	none
type	list
printable	true

...

Exercise: Setting Up

Info

Question:

There is a folder called intro_to_linux within the /project/

...

<project-name>/ folder.
How would you copy this folder into your home folder?

Note
Before moving on, please make sure you get this copied over correctly. All further exercises assume you will be working within this folder, within the home folder As you work through exercises take note of the `[]$ cd ~/intro_to_linux` - remember the tilda character `~` is a shortcut for the home folder.

...

Setting Up: Answer(s)

Answer: There are a number of ways…

# Move to your home folder and copy int this location. # The “.” marks the current working directory. [arccanetrain

Code Block

Info

Method 01: Move into your home folder and copy into this location.

The . marks the current working directory.

Code Block

[]$ cd 
[~]$ cp -r /project/arccanetrain<project-name>/intro_to_linux/ .
cp: cannot open 'intro_to_linux/workshop_me.txt' for reading: Permission denied
[~]$ ls
Desktop  Documents  Downloads  intro_to_linux
# Move into the

Info

Method 02: Move into the /project/

arccatrain

<project-name>/

folder

and

copy

from

there

, into

your

home.

# The “~” is short for your home folder.

Remember, the ~ is short for your home folder.

Code Block

[~]$ cd /project/arccanetrain<project-name>/
[arccanetrain<project-name>]$ cp -r intro_to_linux/ ~
cp: cannot open 'intro_to_linux/workshop_me.txt' for reading: Permission denied
[arccanetrain<project-name>]$ ls ~
Desktop  Documents  Downloads  intro_to_linux
# Why do we see the cp related permission denied?

Note

In both cases you do not have permissions to copy workshop_me.txt - this is intentional and a good remind to check the permissions/ownerships of files.

Question: What do you notice about the permissions of this file?

Code Block
# -rw------- 1 arcc<arcc-t05username> arccanetrain<project-name> 23 Oct 5 07:20 workshop_me.txt # What happens to this file? It does not get copied

Info

Only the owner of this file (<arcc-username>) has permission to read/write, and thus copy this file.

Any other user can not copy this file, and thus it will not be copied as part of everything else.

...

View the content of files

Command

Description

cat

Code Block
Usage: cat [OPTION]... [FILE]... Concatenate FILE(s) to standard output. -n, --number number all output lines

more

Code Block
more [options] <file>... A file perusal filter for CRT viewing. more is a filter for paging through text one screenful at a time.

head

Code Block

Usage: head [OPTION]... [FILE]...
Print the first 10 lines of each FILE to standard output.
 -n, --lines=[-]NUM       print the first NUM lines instead of the first 10;
                             with the leading '-', print all but the last
                             NUM lines of each file

...

View the content of files

Command

Description

tail

Code Block

Usage: tail [OPTION]... [FILE]...
Print the last 10 lines of each FILE to standard output.
 -f, --follow[={name|descriptor}]
                           output appended data as the file grows;
                             an absent option argument means 'descriptor’
 -n, --lines=[+]NUM       output the last NUM lines, instead of the last 10;
                             or use -n +NUM to output starting with line NUM

...

Exercises: View Files: Give it a go.

Info

Question: Within the intro_to_linux folder, how can we view the contents of the software.csv file?

How can we view the entire file?
How can we view the start of the file?
How can we view the end of the file?

...

Answers

Info

Within the intro_to_linux folder, how can we view the contents of the software.csv file?

1. How can we view the entire file?

Code Block

[]$ cd ~/intro_to_linux/
[intro_to_linux]$ cat software.csv
[intro_to_linux]$ cat -n software.csv

[intro_to_linux]$ more software.csv
# Press spacebar to scroll through.
# Press ‘q’ to quit at any time.
[intro_to_linux]$ more software.csv

Info
2. How can we view the start of the file?

Code Block
[intro_to_linux]$ head software.csv [intro_to_linux]$ head –n 5 software.csv

Info
3. How can we view the end of the file?

Code Block
[intro_to_linux]$ tail software.csv [intro_to_linux]$ tail –n 5 software.csv

...

Search for a string within a text file (grep)

Command

Description

grep

Code Block

Usage: grep [OPTION]... PATTERN [FILE]...
Search for PATTERN in each FILE.
Example: grep -i 'hello world' menu.h main.c
...
 -i, --ignore-case         ignore case distinctions
...
 -n, --line-number         print line number with output lines
...
 -r, --recursive           like --directories=recurse
...
# grep is case-sensitive

...

Examples: Search a file

Info
Remember, Linux in general, and grep are case-sensitive.

Info
Search for the capitalized `NVIDIA` text.

Code Block

[intro_to_linux]$ grep nvidia software.csv
[intro_to_linux]$
# Remember: grep is case-sensitive
[intro_to_linux]$ grep NVIDIA software.csv
libraries and toolkits,cuDNN,cudnn,beartooth,The NVIDIA CUDA Deep...
libraries and toolkits,TensorRT,,beartooth,"NVIDIA TensorRT, an...
# Nothing is returned.
[intro_to_linux]$ grep nvidia software.csv
[intro_to_linux]$
# Neither of the above picked up “Nvidia”.

Info

The above identified and returned two locations where NVIDIA was found.

But this would not pick up the text Nvidia.

Modify the command-line to ignore-case using the -i option.

Code Block

[intro_to_linux]$ grep -i NVidia software.csv
compiler,NVidia HPC SDK,nvhpc,"beartooth,teton"...
libraries and toolkits,cuDNN,cudnn,beartooth,The NVIDIA CUDA Deep...
libraries and toolkits,TensorRT,,beartooth,"NVIDIA TensorRT, an...
# Ignore the case of the word to search for.

Info
Modify the command-line to display the the line numbers within the file using the `-n` option.

Code Block

[intro_to_linux]$ grep -n -i NVidia software.csv
145:compiler,NVidia HPC SDK,nvhpc,"beartooth,teton"...
152:libraries and toolkits,cuDNN,cudnn,beartooth,The NVIDIA CUDA Deep...
166:libraries and toolkits,TensorRT,,beartooth,"NVIDIA TensorRT, an...

...

Examples: Search folders and files

Info
Use the wildcard `*` to search all files within the folder.

Code Block

[intro_to_linux]$ cd clusters/
[clusters]$ grep -i nvidia *
beartooth.html:    .../788758554/NVidia+HPC+SDK">NVidia HPC SDK</a></td>
teton.html:    .../788758554/NVidia+HPC+SDK">NVidia HPC SDK</a></td>
[clusters]$ cd ..

Info
Remember: case-sensitivity:

Code Block

[intro_to_linux]$ grep -i nvidia *
grep: clusters: Is a directory
software.csv:compiler,NVidia HPC SDK,nvhpc,"beartooth,teton"...
software.csv:libraries and toolkits,cuDNN,cudnn,beartooth,The NVIDIA CUDA Deep...
software.csv:libraries and toolkits,TensorRT,,beartooth,"NVIDIA TensorRT, an...

Info
Modify the command-line, using the `-r` option, to recursively search through folders within this folder.

Code Block

[intro_to_linux]$ grep -r -i nvidia *
clusters/teton.html:    .../788758554/NVidia+HPC+SDK">NVidia HPC SDK</a></td>
clusters/beartooth.html:    .../788758554/NVidia+HPC+SDK">NVidia HPC SDK</a></td>
software.csv:compiler,NVidia HPC SDK,nvhpc,"beartooth,teton"...
software.csv:libraries and toolkits,cuDNN,cudnn,beartooth,The NVIDIA CUDA Deep...
software.csv:libraries and toolkits,TensorRT,,beartooth,"NVIDIA TensorRT, an...

...

Exercises: Search Files

Info

We will be using the software.csv file:

Lets take a quick look at it using head to view the first few lines. How is this structured? What columns do we have?

Code Block

# The software.csv file takes the form:
[intro_to_linux]$ head software.csv
Type,Name,Module,Cluster,Description
application,Alphafold,alphafold,"beartooth,teton",AlphaFold...
application,Astral,astral,wildiris,ASTRAL is a tool...
application,Augustus,augustus,beartooth,AUGUSTUS is a program...
application,Avizo,avizo,loren-pre202308,Avizo is a general-purpose...
application,ANGSD,angsd,"beartooth,teton",ANGSD: is a software...
application,ANSYS,ansys,teton,"ANSYS is a general-purpose software...

Info

Questions:

Which named applications are related to the words

...

bayes?
Which files contain reference to IPA?

...

Answers

Info

Question 01: Which named applications are related to the words “bayes”?

We have a csv file, with comma separated columns, the second column represents the name of the application.

This assumes we want to find all instances of “bayes” regardless of case.

Code Block

[intro_to_linux]$ grep -i bayes software.csv
application,Bayescan,bayescan,beartooth,"BayeScan aims...
application,Beast1,beast1,wildiris,BEAST is a cross-platform program for Bayesian...
application,Beast2,beast2,beartooth,"BEAST 2 is a cross-platform program for Bayesian...
application,Freebayes,freebayes,beartooth,"freebayes is a Bayesian genetic...
application,Jags,jags,"beartooth,teton",Just Another Gibbs Sampler. It is a program for analysis of Bayesian hierarchical...
application,RevBayes,revbayes,wildiris,Bayesian phylogenetic...
application,ROHan,rohan,teton,"ROHan is a Bayesian framework...
application,SourceTracker2,sourcetracker2,"beartooth,teton","SourceTracker, a Bayesian approach...

Info

Which of the returned lines represent an application with bayes within it’s name column?

Code Block

[intro_to_linux]$ grep -i bayes software.csv
application,Bayescan,bayescan,beartooth,"BayeScan aims...
application,Freebayes,freebayes,beartooth,"freebayes is a Bayesian genetic...
application,RevBayes,revbayes,wildiris,Bayesian phylogenetic...

The other returned lines are “lines that contain the word bayes within it” - so not strictly what we are looking for.

Info

Question 02: Which files contain reference to IPA?

In this case we want to explicitly search for IPA (all upper-case) and recursively search through all folders.

Code Block
[intro_to_linux]$ grep -r IPA * clusters/beartooth.html: .../pages/1893597185/IPA">IPA</a></td> software.csv:application,IPA,ipa,beartooth,Improved Phased Assembler (IPA) is...

...

Workshop Home

Intro to Linux

Command-Line

CLI: View, Find and Search

Files

Next

Search for a File

Versions Compared

Old Version 5

New Version Current

Key

Exercise: Setting Up

Setting Up: Answer(s)

View the content of files

View the content of files

Exercises: View Files: Give it a go.

Answers

Search for a string within a text file (grep)

Examples: Search a file

Examples: Search folders and files

Exercises: Search Files

Answers

Page Comparison

Versions Compared

Old Version 5

New Version Current

Key

Exercise: Setting Up

Setting Up: Answer(s)

View the content of files

View the content of files

Exercises: View Files: Give it a go.

Answers

Search for a string within a text file (grep)

Examples: Search a file

Examples: Search folders and files

Exercises: Search Files

Answers