92.103
Using the Unix/MacOS Command Line

---

Objectives

Upon completion of this lesson, you will be able to:

• navigate the file system
• view directories and files
• change permissions
• customize the command prompt and environment

---

Overview

Welcome to the world of Linux! If you’re new to this operating system, one of the first things you’ll notice is the power of the command-line interface, commonly referred to as the shell. This tutorial will introduce you to the basics of the Linux shell, including navigating directories, manipulating files, and executing commands.

Most of the commands introduced in this tutorial also apply to MacOS ‘terminal’.

What is the Shell?

The shell is a command-line interface that allows users to interact directly with the operating system. While Linux also has graphical user interfaces (GUIs), the shell provides a more direct way to manage files, run programs, and configure the system.

Getting Started

1. Accessing the Terminal: To start using the shell, open the Terminal application. This is where you’ll enter all your commands.

2. Prompt: Once you open the terminal, you’ll see something like:

   [username@hostname ~]$

   This is called the shell prompt. It’s waiting for your command!

Basic Shell Commands

1. Navigating the Filesystem

• pwd (Print Working Directory): Shows the current directory you’re in.

  $ pwd

• ls (List): Displays the contents of the current directory.

  $ ls

• cd (Change Directory): Navigate to a different directory.

  $ cd /path/to/directory

  The special directory ~ is the user’s home directory. To navigate one directory up (the parent directory), use ..

2. File and Directory Manipulation

• mkdir (Make Directory): Create a new directory.

  $ mkdir new_directory_name

• rmdir (Remove Directory): Delete an empty directory.

  $ rmdir directory_name

• touch: Create a new empty file.

  $ touch new_file.txt

• rm (Remove): Delete a file.

  $ rm file_name.txt

3. Displaying File Contents

• cat (Concatenate and Display): Display the contents of a file.

  $ cat file_name.txt

• less: View the content of a file page by page (press q to exit).

  $ less file_name.txt

4. Searching and Filtering

• grep (Global Regular Expression Print): Search for specific text within files.

  $ grep "search_term" file_name.txt

Tips

1. Auto-completion: Press Tab to auto-complete file and directory names.
2. History: Press the Up Arrow key to scroll through previously entered commands.
3. Clearing the Screen: Type clear to clear the terminal screen.

Startup Profiles

.bashrc is a shell script that Bash runs whenever it is started interactively. It initializes an interactive shell session. This means that when you start a new terminal or a new shell session, the commands and configurations in .bashrc are executed to set up your environment.

.bashrc

1. Setting up PATH: You can modify the PATH environment variable in .bashrc to add new directories that contain executables.

   export PATH=$PATH:/path/to/directory

2. Aliases: You can define shortcuts for commands using aliases. For example:

   alias ll='ls -lah'

   Now, whenever you type ll in the terminal, it will execute ls -lah.

3. Functions: You can define shell functions for more complex operations.

4. Prompt Customization: You can customize how your shell prompt looks.

   PS1='\u@\h:\w\$ '

   This would make your prompt look something like username@hostname:current_directory$.

5. Environment Variables: You can set or modify environment variables.

   export JAVA_HOME=/path/to/java/home

6. Initialization: Run any commands you want to execute whenever a new shell session starts.

Building a .bashrc

1. Editing: You can open .bashrc in your favorite text editor (like nano or vim) and add or modify commands.

   nano ~/.bashrc

2. Reloading: After making changes to .bashrc, you need to reload it to see the effects in the current shell session.

   source ~/.bashrc

   Alternatively, you can use:

   . ~/.bashrc

3. Backup: It’s a good idea to make a backup of your .bashrc before making significant changes.

.bashrc vs .bash_profile

While .bashrc is executed for interactive non-login shells, .bash_profile (or .profile in some systems) is executed for login shells. When you log into a system (e.g., through SSH) or when you start a new shell session from a graphical login, .bash_profile is executed.

Typically, for users who have both files, you’ll see that .bash_profile sources .bashrc to ensure that the interactive configurations are also loaded during a login shell:

if [ -f ~/.bashrc ]; then
   source ~/.bashrc
fi

This setup ensures consistent behavior between login and non-login interactive shell sessions.

Searching and Filtering: grep

grep and fgrep are both commands used for searching text, but they interpret patterns differently. Here’s a breakdown of their differences:

1. Pattern Interpretation:

   • grep: Stands for “global regular expression print.” It searches for a pattern using regular expressions (regex) by default. So, if you provide a pattern with special regex characters, such as . (dot) or *, grep will interpret them as regex metacharacters. For instance, . will match any character, and * will match the preceding character zero or more times.

     echo "ab" | grep "a.b"

     In the above command, the pattern “a.b” will not match “ab” because the dot expects any character to be present between “a” and “b”.

   • fgrep: Stands for “fixed grep” or “fast grep.” It treats the pattern as a fixed string. This means it does not recognize regular expression metacharacters as special; they are treated as regular characters. Thus, fgrep is useful when you want to search for a string that might contain regex metacharacters and you don’t want them to be interpreted as such.

     echo "a.b" | fgrep "a.b"

     Here, the pattern “a.b” matches the string “a.b” because fgrep doesn’t interpret the dot as a regex character.

2. Speed:

   • Due to its literal interpretation of patterns, fgrep can be faster than grep when searching for fixed strings because it doesn’t need to process regex metacharacters.

3. Alternatives:

   • You can achieve fgrep behavior with grep using the -F option:

     echo "a.b" | grep -F "a.b"

4. Multiple Patterns:

   • One advantage of fgrep (or grep -F) is the ability to search for multiple fixed strings efficiently by placing each string on a new line in a pattern file.

     fgrep -f pattern_file.txt target_file.txt

     In the above command, each line in pattern_file.txt is treated as a fixed string, and fgrep searches for any of these lines in target_file.txt.

In short, use grep when you need the power and flexibility of regular expressions. Opt for fgrep (or grep -F) when you want to search for exact, fixed strings, especially when these strings might contain characters that have special meanings in regex.

Options for ‘grep’

grep is a powerful tool for searching text patterns within files. It stands for “global regular expression print.” Below, I’ll expand on how to use grep effectively with various options and examples.

Basic Usage

Search for a pattern within a file:

grep "pattern" file.txt

Common Options

1. -i: Ignore case. It makes the search case-insensitive.

grep -i "pattern" file.txt

2. -v: Invert the match. Returns lines that do not match the pattern.

grep -v "pattern" file.txt

3. -r or -R: Recursively search for the pattern in a directory.

grep -r "pattern" /path/to/directory/

4. -l: List only the filenames containing the pattern (instead of the actual matched lines).

grep -l "pattern" /path/to/directory/*

5. -n: Show line numbers along with the matched lines.

grep -n "pattern" file.txt

6. -c: Count the number of lines that match the pattern.

grep -c "pattern" file.txt

7. -w: Match whole words only (the pattern is searched as a whole word).

grep -w "pattern" file.txt

8. -o: Show only the matched part of the line.

grep -o "pattern" file.txt

9. -A [num]: Show [num] lines after the match.
10. -B [num]: Show [num] lines before the match.
11. -C [num]: Show [num] lines before and after the match. This is useful for context.

grep -A 3 -B 2 "pattern" file.txt

12. -E: Enables extended regular expressions (like +, ?, |, {}, and () without backslashes). Often available as egrep.

grep -E "pat(tern1|tern2)" file.txt

Practical Examples for ‘grep’

1. Search for a pattern in multiple files:

grep "pattern" file1.txt file2.txt

2. Search for lines not containing a pattern:

grep -v "pattern" file.txt

3. Search for a pattern only in .txt files recursively:

grep -r --include="*.txt" "pattern" /path/to/directory/

4. Search for either ‘pattern1’ or ‘pattern2’:

grep "pattern1\|pattern2" file.txt

Or using extended regular expressions:

grep -E "pattern1|pattern2" file.txt

5. Count the number of files that have a particular pattern:

grep -rl "pattern" /path/to/directory/ | wc -l

6. Highlighting the matched pattern: By default, some grep installations highlight the matched text. If yours doesn’t, you can use the --color option.

grep --color "pattern" file.txt

7. Using regular expressions to match multiple instances: To find lines that contain two consecutive vowels:

grep -E "[aeiou]{2}" file.txt

In summary, grep is a versatile and powerful tool that can be used to search through text data quickly. Mastering grep can considerably speed up text-processing tasks and help in debugging, analyzing logs, or finding specific content in large datasets.

Regular Expressions

Absolutely! Regular expressions (often abbreviated as “regex” or “regexp”) are sequences of characters that define a search pattern. They can be used for string matching and replacement. Let’s delve into the details:

Basic Regular Expressions

1. Literals:
   • The simplest form of pattern matching supported by regular expressions is the match of a string literal. regex hello
2. Dot (.):
   • The . character matches any single character except a newline. regex h.llo
3. Character Classes ([]):
   • A set of characters enclosed in square brackets ([]) makes up a character class. This matches any single character in the set. regex [aeiou] # matches any single vowel
4. Negate Character Classes ([^ ]):
   • By placing a ^ symbol inside the square brackets, only characters not in the character class will be matched. regex [^aeiou] # matches any single non-vowel character
5. Predefined Character Classes:
   • \d: Matches any digit. Equivalent to [0-9].
   • \D: Matches any non-digit. Equivalent to [^0-9].
   • \w: Matches any word character (alphanumeric and underscore). Equivalent to [a-zA-Z0-9_].
   • \W: Matches any non-word character.
   • \s: Matches any whitespace character (spaces, tabs, line breaks).
   • \S: Matches any non-whitespace character.
6. Quantifiers:

   • *: Matches 0 or more of the preceding element. regex ab*c # matches "ac", "abc", "abbc", etc.

   • +: Matches 1 or more of the preceding element. regex ab+c # matches "abc", "abbc", but not "ac"

   • ?: Matches 0 or 1 of the preceding element. regex ab?c # matches "ac" or "abc"

   • {n}: Matches exactly n occurrences of the preceding element.

   • {n,}: Matches n or more occurrences.

   • {n,m}: Matches between n and m occurrences. regex a{2,4} # matches "aa", "aaa", or "aaaa"

7. Anchors:
   • ^: Matches the start of a line.
   • $: Matches the end of a line. regex ^Hello # matches "Hello" at the beginning of a line world$ # matches "world" at the end of a line
8. Alternation (|):
   • Acts like an OR operator. regex cat|dog # matches "cat" or "dog"
9. Grouping (()):
   • Elements within parentheses are treated as a single unit.
   • Also captures the matched text for use with backreferences. regex (abc)+ # matches "abc", "abcabc", etc.
10. Backreferences:
    • Refers to the text matched by a previous capture group.
    • \1 refers to the text of the first capture group, \2 for the second, and so on. regex (a)b\1 # matches "aba"
11. Escape Sequences:
    • To match characters that have special meanings in regex, you escape them with a backslash \. regex \. # matches a literal dot \\ # matches a backslash \[ # matches a square bracket

Advanced Concepts

1. Lookaheads and Lookbehinds:

   • Lookaheads allow you to match a group ahead of your main pattern without including it in the result. regex q(?=u) # matches the "q" in "queue" but not the "u"

   • Lookbehinds are similar, but they look behind. regex (?<=a)b # matches the "b" after an "a"

2. Non-capturing Groups:
   • Use (?:...) to create a non-capturing group. regex (?:abc)+ # matches "abc", "abcabc", etc., but does not capture the match
3. Modifiers:
   • Some regex engines allow modifiers to change how the regex is interpreted. regex (?i)abc # matches "abc", "ABC", "AbC", etc. (case-insensitive)

Regular expressions are powerful but can be complex. Mastering them requires practice. Test your regular expressions using tools like regex101 to ensure they match what you intend and to understand their behavior better.

Summary

Navigating a system from the command line is a key skill that all programmers and software developers should possess. This tutorial covered just the basics of the Linux shell, but there’s a vast world of commands and functions to explore. As you become more familiar with the shell, you’ll find it’s an incredibly powerful tool that can simplify many tasks and offer fine-grained control over your Linux system.

Keep practicing, explore further, and soon, you’ll master the Linux shell!

---

---

References

None.

Errata

Let us know.