Goal: Use various commands to view, inspect and search a text file.
Setting Up
Question:
There is a folder called
intro_to_linuxwithin the/project/<project-name>/folder.How would you copy this folder into your home folder?
Before moving on, please make sure you get this copied over correctly.
All further exercises assume you will be working within this folder, within the home folder
As you work through exercises take note of the
[]$ cd ~/intro_to_linux- remember the tilda character~is a shortcut for the home folder.
Setting Up: Answer(s)
Answer: There are a number of ways…
Method 01: Move to your home folder and copy into this location.
The . marks the current working directory.
[]$ cd [~]$ cp -r /project/<project-name>/intro_to_linux/ . cp: cannot open 'intro_to_linux/workshop_me.txt' for reading: Permission denied [~]$ ls Desktop Documents Downloads intro_to_linux
Method 02: Move into the /project/<project-name>arccatrain/ folder and copy from there into your home.
Remember, the ~ is short for your home folder.
[~]$ cd /project/<project-name>/ [<project-name>]$ cp -r intro_to_linux/ ~ cp: cannot open 'intro_to_linux/workshop_me.txt' for reading: Permission denied [<project-name>]$ ls ~ Desktop Documents Downloads intro_to_linux
In both cases you do not have permissions to copy workshop_me.txt - this is intentional and a good remind to check the permissions/ownerships of files.
Question: What do you notice about the permissions of this file?
# -rw------- 1 <arcc-username> <project-name> 23 Oct 5 07:20 workshop_me.txt
Only the owner of this file (<arcc-username>) has permission to read/write, and thus copy this file.
Any other user can not copy this file, and thus it will not be copied as part of everything else.
View the content of files
Command | Description |
cat | Usage: cat [OPTION]... [FILE]... Concatenate FILE(s) to standard output. -n, --number number all output lines |
more | more [options] <file>... A file perusal filter for CRT viewing. more is a filter for paging through text one screenful at a time. |
head | Usage: head [OPTION]... [FILE]... Print the first 10 lines of each FILE to standard output. -n, --lines=[-]NUM print the first NUM lines instead of the first 10; with the leading '-', print all but the last NUM lines of each file |
View the content of files
Command | Description |
tail | Usage: tail [OPTION]... [FILE]...
Print the last 10 lines of each FILE to standard output.
-f, --follow[={name|descriptor}]
output appended data as the file grows;
an absent option argument means 'descriptor’
-n, --lines=[+]NUM output the last NUM lines, instead of the last 10;
or use -n +NUM to output starting with line NUM
|
Exercise
Question: Within the intro_to_linux folder, how can we view the contents of the software.csv file?
How can we view the entire file?
How can we view the start of the file?
How can we view the end of the file?
Answers
Within the intro_to_linux folder, how can we view the contents of the software.csv file?
1. How can we view the entire file?
[]$ cd ~/intro_to_linux/ [intro_to_linux]$ cat software.csv [intro_to_linux]$ cat -n software.csv [intro_to_linux]$ more software.csv # Press spacebar to scroll through. # Press ‘q’ to quit at any time.
2. How can we view the start of the file?
[intro_to_linux]$ head software.csv [intro_to_linux]$ head –n 5 software.csv
3. How can we view the end of the file?
[intro_to_linux]$ tail software.csv [intro_to_linux]$ tail –n 5 software.csv
Search for a string within a text file (grep)
Command | Description |
grep | Usage: grep [OPTION]... PATTERN [FILE]... Search for PATTERN in each FILE. Example: grep -i 'hello world' menu.h main.c ... -i, --ignore-case ignore case distinctions ... -n, --line-number print line number with output lines ... -r, --recursive like --directories=recurse ... # grep is case-sensitive |
Examples: Search a file
Remember, Linux in general, and grep are case-sensitive.
Search for the capitalized NVIDIA text.
[intro_to_linux]$ grep nvidia software.csv [intro_to_linux]$ # Remember: grep is case-sensitive [intro_to_linux]$ grep NVIDIA software.csv libraries and toolkits,cuDNN,cudnn,beartooth,The NVIDIA CUDA Deep... libraries and toolkits,TensorRT,,beartooth,"NVIDIA TensorRT, an...
The above identified and returned two locations where NVIDIA was found.
But this would not pick up the text Nvidia.
Modify the command-line to ignore-case using the -i option.
[intro_to_linux]$ grep -i NVidia software.csv compiler,NVidia HPC SDK,nvhpc,"beartooth,teton"... libraries and toolkits,cuDNN,cudnn,beartooth,The NVIDIA CUDA Deep... libraries and toolkits,TensorRT,,beartooth,"NVIDIA TensorRT, an...
Modify the command-line to display the the line numbers within the file using the -n option.
[intro_to_linux]$ grep -n -i NVidia software.csv 145:compiler,NVidia HPC SDK,nvhpc,"beartooth,teton"... 152:libraries and toolkits,cuDNN,cudnn,beartooth,The NVIDIA CUDA Deep... 166:libraries and toolkits,TensorRT,,beartooth,"NVIDIA TensorRT, an...
Examples: Search folders and files
Use the wildcard * to search all files within the folder.
[intro_to_linux]$ cd clusters/ [clusters]$ grep -i nvidia * beartooth.html: .../788758554/NVidia+HPC+SDK">NVidia HPC SDK</a></td> teton.html: .../788758554/NVidia+HPC+SDK">NVidia HPC SDK</a></td> [clusters]$ cd ..
Remember: case-sensitivity:
[intro_to_linux]$ grep -i nvidia * grep: clusters: Is a directory software.csv:compiler,NVidia HPC SDK,nvhpc,"beartooth,teton"... software.csv:libraries and toolkits,cuDNN,cudnn,beartooth,The NVIDIA CUDA Deep... software.csv:libraries and toolkits,TensorRT,,beartooth,"NVIDIA TensorRT, an...
Modify the command-line, using the -r option, to recursively search through folders within this folder.
[intro_to_linux]$ grep -r -i nvidia * clusters/teton.html: .../788758554/NVidia+HPC+SDK">NVidia HPC SDK</a></td> clusters/beartooth.html: .../788758554/NVidia+HPC+SDK">NVidia HPC SDK</a></td> software.csv:compiler,NVidia HPC SDK,nvhpc,"beartooth,teton"... software.csv:libraries and toolkits,cuDNN,cudnn,beartooth,The NVIDIA CUDA Deep... software.csv:libraries and toolkits,TensorRT,,beartooth,"NVIDIA TensorRT, an...
Exercises
We will be using the software.csv file:
Lets take a quick look at it using head to view the first few lines:
# The software.csv file takes the form: [intro_to_linux]$ head software.csv Type,Name,Module,Cluster,Description application,Alphafold,alphafold,"beartooth,teton",AlphaFold... application,Astral,astral,wildiris,ASTRAL is a tool... application,Augustus,augustus,beartooth,AUGUSTUS is a program... application,Avizo,avizo,loren-pre202308,Avizo is a general-purpose... application,ANGSD,angsd,"beartooth,teton",ANGSD: is a software... application,ANSYS,ansys,teton,"ANSYS is a general-purpose software...
Questions:
Which named applications are related to the words
bayes?Which files contain reference to
IPA?
Answers
Question 01: Which named applications are related to the words “bayes”?
This assumes we want to find all instances of “bayes” regardless of case.
[intro_to_linux]$ grep -i bayes software.csv application,Bayescan,bayescan,beartooth,"BayeScan aims... application,Beast1,beast1,wildiris,BEAST is a cross-platform program for Bayesian... application,Beast2,beast2,beartooth,"BEAST 2 is a cross-platform program for Bayesian... application,Freebayes,freebayes,beartooth,"freebayes is a Bayesian genetic... application,Jags,jags,"beartooth,teton",Just Another Gibbs Sampler. It is a program for analysis of Bayesian hierarchical... application,RevBayes,revbayes,wildiris,Bayesian phylogenetic... application,ROHan,rohan,teton,"ROHan is a Bayesian framework... application,SourceTracker2,sourcetracker2,"beartooth,teton","SourceTracker, a Bayesian approach...
Question 02: Which files contain reference to IPA?
In this case we want to explicitly search for IPA (all upper-case) and recursively search through all folders.
[intro_to_linux]$ grep -r IPA * clusters/beartooth.html: .../pages/1893597185/IPA">IPA</a></td> software.csv:application,IPA,ipa,beartooth,Improved Phased Assembler (IPA) is...
| Workshop Home | Next |