...
View the content of text-based files.
Search a file for a string.
Search for a file/folder by name.
Redirect output from commands and pipe commands together.
Be very exercise based to allow practice of commands and concepts.
...
| Table of Contents | ||
|---|---|---|
|
...
01 View/Search a File
01.01 Setting Up
Question:
...
| Code Block |
|---|
[intro_to_linux]$ grep -i bayes software.csv application,Bayescan,bayescan,beartooth,"BayeScan aims... application,Beast1,beast1,wildiris,BEAST is a cross-platform program for Bayesian... application,Beast2,beast2,beartooth,"BEAST 2 is a cross-platform program for Bayesian... application,Freebayes,freebayes,beartooth,"freebayes is a Bayesian genetic... application,Jags,jags,"beartooth,teton",Just Another Gibbs Sampler. It is a program for analysis of Bayesian hierarchical... application,RevBayes,revbayes,wildiris,Bayesian phylogenetic... application,ROHan,rohan,teton,"ROHan is a Bayesian framework... application,SourceTracker2,sourcetracker2,"beartooth,teton","SourceTracker, a Bayesian approach... [intro_to_linux]$ grep -r IPA * clusters/beartooth.html: .../pages/1893597185/IPA">IPA</a></td> software.csv:application,IPA,ipa,beartooth,Improved Phased Assembler (IPA) is... |
...
02 Search for a File
...
02.01a Searching for Files: find
...
| Code Block |
|---|
# dd.tx is actually a folder. # Notices the ’d’ in the long format list. [intro_to_linux]$ ls -l data total 4 drwxrwxr-x 6 arcc-t05 arcc-t05 2021 drwxrwxr-x 6 arcc-t05 arcc-t05 2022 drwxrwxr-x 5 arcc-t05 arcc-t05 2023 drwxrwxr-x 2 arcc-t05 arcc-t05 dd.tx [intro_to_linux]$ find . -type f -name "*.tx" ./data/2021/feb/february_01_2021.tx # We explicitly want lowercase. [intro_to_linux]$ find . -type f -iname "*.tx" ./data/2021/feb/february_01_2021.tx ./data/2022/20220723.TX |
...
03 Output Redirection and Pipes
...
06 03 Output Redirection and Pipes
Redirection of output: > vs >>
A redirect sends a channel of output to a file.
You can redirect a file as input to a command using
<and<<(not looked at).
Using pipe “|’
A pipe passes standard output as the standard input to another command
Examples of the form:
View a text file and pipe to grep.
Cat a list and sort by line.
Sort and then find unique items.
View folder contents and look for a specifically named name.
...
0603.01 Redirection of output: > vs >>
| Code Block |
|---|
# Writes out to the command line. [intro_to_linux]$ grep -i bayes software.csv # Redirects the output to a file called apps.txt [intro_to_linux]$ grep -i bayes software.csv > apps.txt [intro_to_linux]$ ls apps.txt clusters data software.csv [intro_to_linux]$ cat apps.txt # Overwrites any existing file called apps.txt [intro_to_linux]$ grep -i IPA software.csv > apps.txt [intro_to_linux]$ cat apps.txt [intro_to_linux]$ rm apps.txt # Overwrites existing apps.txt [intro_to_linux]$ grep -i bayes software.csv > apps.txt # Appends to the existing file. [intro_to_linux]$ grep -i IPA software.csv >> apps.txt |
...
0603.02 Example: Using pipe “|” from a file.
| Code Block |
|---|
[intro_to_linux]$ cat fruits.txt Gooseberry Apple Apricot Avocado Strawberry ... [intro_to_linux]$ cat fruits.txt | wc -l 97 |
...
0603.03 Example continued:
| Code Block |
|---|
# The order of items is the same as listed within the fruits.txt file. [intro_to_linux]$ cat fruits.txt | grep berry Gooseberry Strawberry Bilberry Blackberry Marionberry Blueberry Boysenberry Gooseberry Cloudberry Elderberry Goji berry Honeyberry Juniper berry Cranberry Cranberry Marionberry Gooseberry Mulberry Salmonberry Huckleberry Raspberry Salal berry |
...
0603.04 Example continued:
| Code Block |
|---|
# Notice the duplicates. [intro_to_linux]$ cat fruits.txt | grep berry | sort Bilberry Blackberry Blueberry Boysenberry Cloudberry Cranberry Cranberry Elderberry Goji berry Gooseberry Gooseberry Gooseberry Honeyberry Huckleberry Juniper berry Marionberry Marionberry Mulberry Raspberry Salal berry Salmonberry Strawberry |
...
0603.05 Example continued:
| Code Block |
|---|
# Duplicates have been removed leaving only the unique names. [intro_to_linux]$ cat fruits.txt | grep berry | sort | uniq Bilberry Blackberry Blueberry Boysenberry Cloudberry Cranberry Elderberry Goji berry Gooseberry Honeyberry Huckleberry Juniper berry Marionberry Mulberry Raspberry Salal berry Salmonberry Strawberry [intro_to_linux]$ cat fruits.txt | grep berry | sort | uniq | wc –l 18 |
...
0603.06 Example continued
| Code Block |
|---|
[intro_to_linux]$ cat fruits.txt | grep berry | sort | uniq > berries.txt [intro_to_linux]$ cat berries.txt Bilberry Blackberry Blueberry Boysenberry Cloudberry Cranberry Elderberry Goji berry Gooseberry Honeyberry Huckleberry Juniper berry Marionberry Mulberry Raspberry Salal berry Salmonberry Strawberry [intro_to_linux]$ cat berries.txt | wc -l 18 |
...
0603.07 Example: Pipe from ls command
| Code Block |
|---|
[intro_to_linux]$ ls -R [intro_to_linux]$ ls -R | grep "Feb" February ./data/2022/February: Feb ./data/2023/Feb: [intro_to_linux]$ ls -R | grep -i "Feb" feb ./data/2021/feb: february_01_2021.tx February ./data/2022/February: Feb ./data/2023/Feb: |
...
0603.08 Exercises
How does the
wccommand work? What are its options?How does the
sortcommand work? What are its options?How does the
uniqcommand work? What are its options?How many unique varieties of beans are there in the
vegetables.txtfile?
...
0603.09 Answers
4: How many unique varieties of beans ae there in the vegetables.txt file?
...
| Code Block |
|---|
[intro_to_linux]$ cat vegatables.txt | grep -i beans | sort | uniq -i | wc -l 12 |
...
04 More Intermediate Features, Next Steps, Suggestions and Summary
...
0704.01a More Intermediate Features
Environment Variables: Define the behavior of the environment: Try:
echo $HOMEecho $USERecho $SHELLecho $PATH
File searching/manipulation
sed: stream editor for filtering and transforming textgawk: pattern scanning and processing language
Ability to update file permission and ownership:
chmod/chownUser-case of sharing files/folders.
...
0704.01b More Intermediate Features
Aliases in
.bashrc.Create short-cuts of popular/frequently used commands.
Text editors: vi/vim/nano
vimtutortouch
Remote access with ssh.
...
0704.02 Next Steps, Suggestions
Next Steps on using Linux:
Practicing using Linux online.
Dual boot a Windows machine with Linux.
Run a container image.
UW Researcher? Create a project on the Beartooth cluster with your PI.
...
0704.03 Further Trainings: UWYO LinkedIn
...
Introduction to Linux
Learning Linux Command Line
Linux: Files and Permissions
Linux: Over and Installation
Learning Linux Shell Scripting
...
0704.04 Request an Account with ARCC
...
Wiki: https://arccwiki.atlassian.net/wiki/spaces/DOCUMENTAT/overview
Portal: https://arccwiki.atlassian.net/servicedesk/customer/portals
...
0704.05 Summary
In this workshop we have:
...