Toggle Menu

The Shell


  • The shell is a program that reads commands and interprets them.
  • Shell commands interact with the file system.

Shell Commands

cat: Prints out the content of a file or combine multiple files to standard output
  • cat file1.txt prints out file1.txt
  • cat file1.txt file2.txt combines both files
  • cat/cat - takes in keyboard input and outputs it (ctrl-D ends)
  • cat file1.txt - file2.txt takes in file1.txt, then keyboard input (end with ctrl-D), then file2.txt
  • man cat provides info about cat command
Ctrl-C vs. Ctrl-D
  • Ctrl-C: kills a process and stops what it is doing by sending an interrupt signal
  • Ctrl-D: EOF signal
head: Displays first few (10) lines of a file
tail: Displays last few (10) lines of a file
-n[# of lines]/-[# of lines]: specify number of lines

ls: lists the contents of the current directory
ls -a: lists all contents including hidden files (files that start with '.')
ls -l: lists the contents of a directory in a long listing format (ls -l [filename] displays info for a file in long listing format)

pwd: shows the current working directory

cd: change directory
  • cd/cd ~ home directory
  • cd [directory] specified directory
  • cd .. parent directory
  • cd / root directory
  • cd - previous directory
  • Absolute directories begin with /
  • Relative directories begin with
    • ./ (current)
    • ../ (parent)
    • ~/ (home)
exit: terminates the shell session
echo: write arguments, separated by spaces and terminated with a newline
rm: remove
rm -f: force remove
less: loads a text file one page at a time rather than all at once
wc: displays the number of words, bytes, and lines in a file
wc -w: word count
wc -l: line count
sort: sort lines of text file (alphabetical)
  • Lines starting with a number will appear before lines starting with a letter.
  • Sorted in increasing alphabetical order based on first character
  • Lines starting with a lowercase letter will appear before lines starting with the same letter in uppercase.
  • sort -r: sorts in reverse order

Standard File Streams

There are 3 standard file streams:
  1. stdin with descriptor value of 0 (e.g. keyboard)
  2. stdout with descriptor value of 1 (e.g. terminal)
  3. stderr with descriptor value of 2 (e.g. log file)
Input Redirection: stdin is often supplied by directing an input to come from a file or previous output command using Pipe.
  • doCommand < input_file
  • e.g. cat < file1.txt redirects keyboard input from file1.txt. Note that this is different from cat file1.txt which opens the file and prints out content
Output Redirection: stdout is often redirected to a file, stderr is usually printed to terminal but sometimes redirected to a log file
  • doCommand > output_file for stdout
  • doCommand 2> error_file for stderr
  • doCommand > output_file 2>&1 redirects the error messages to the same location where output is (or in bash: doCommand >% output_file)
  • doCommand >> output_file appends to start of file
  • e.g. cat > output_file.txt saves content of keyboard input to file
  • e.g. cat file1.txt file2.txt combines file1 and file2 to new file

Wildcards

  • used for pattern matching and searching for files names with specific characters
  • ? replaces a single character or digit
  • * replaces a string of characters of digits
  • e.g. cat file?.txt shows the content of all files that start with the word 'file' then a single character or digit of type .txt
  • e.g. ls *file*.txt list all files that has the word 'file' anywhere in its name

Pipes

  • can pipe the output of certain commands into the input of another command (i.e. chains commands)
  • commandA | commandB | commandC
  • head -n11 file3.txt | tail -n2 displays lines 10 and 11 of file3.txt

Users and Groups

  • Each user is assigned a userid
  • who: lists the currently logged-in users to the system
    whoami: lists current user

    File Permissions

    Files have three types of permissions:
    1. read (r)
    2. write (w)
    3. execute (x)
    There are 4 main types of users:
    1. user (u)
    2. group (g)
    3. others (o)
    4. all (a)
    File permissions are ordered by user, group, others. For example,
    drwxr-xr-x means:
    • d: directory
    • rwx: user has rwx access
    • r-x: group has r and x access
    • r-x: others have r and x access
    chmod: change permissions of a file
    • + adds permission
    • - removes permission
    • = sets permission
    • e.g. chmod u-r file removes read permission from user for file
    • e.g. chmod o+x file adds execute permission to others
    • e.g. chmod a=rwx file gives everybody full control
    which: search where an application resides (e.g. which ls searches where the ls command resides)
    whereis: search broader range of directories and locate the source and man packages of the application
    tree: displays tree view of file system (e.g. tree cs246 displays tree view of cs246 folder)
    tac: start from last line of a file and displays its contents backwards (e.g. tac file1.txt file2.txt file3.txt displays the contents of each file backwards, in order)
    touch: update the file timestamp to match the current time
    • can be used to update the access, change and modify time
    • if file doesnt exist, it creates an empty file
    • touch -t sets the time
    • Create a file called tomato and set the time stamp of tomato to June 6, 2006, 6:07
      touch tomato
      touch -t 0606060606 tomato
      touch tomato
    stat: displays the detailed status of a particular file or file system
    mkdir: creates new directory
    rmdir: removes empty directory
    mv: move or/and rename a file (e.g. mv alice.txt cs246/alicewonderland.txt moves alice.txt into cs246 and rename to alicewonderland.txt)
    sudo: runs a program by the use of security privilege of another user often the root or super user
    locate: search through a previously created database of files and directory to find a match of your entry
  • Move alice.txt into cs246 and rename to alicewonderland.txt. Update the directory to reflect changes and locate the new location of alicewonderland
  • mv alice.txt cs246/alicewonderland.txt
    updatedb
    locate alicewonderland

    awk: extract and then print specific content of the file to generate reports
  • e.g. output the whole line
  • awk '{ print $0 }' fileName.txt
    cal: displays the calendar
    find: searches for file
    • find [dir] -name file_name
    • e.g. find / -name ubunto > ubuntofiles.txt 2> logfile.txt less ubuntofiles.txt less logfile.txt finds file named ubunto in the root dir and save results into a file, save errors into log file and display the first page of the two new files
    • -delete removes all files named file_name that are in the specified directory
    • -exec [command] executes command on specified files. All following arguments are args to command until a ;. {} is the variable for each file
    • e.g. find / -name "*.txt" -exec rm {} \; searches all txt files and deletes them.
    • -size [size] searches for all files of a specific size.
      • b: 512-byte blocks (default if no suffix is used)
      • c: bytes
      • w: two-byte words
      • k: Kilobytes (units of 1024 bytes)
      • M: Megabytes (units of 1048576 bytes)
      • G: Gigabytes (units of 1073741824 bytes)
      • +: bigger than specified size
      • -: smaller than specified size
      • e.g. find / -size +100M: searches for all files bigger than 100MB
    Difference between find and locate:
    1. locate is faster than find
    2. find has more functionality (can select files more than just name)
    3. find doesn't necessarily search the entire filesystem, points to a subdirectory
    4. find can perform operations to files it finds (exec, delete, size)
    5. locate's output can be outdated (needs to be updated by updatedb)
    uniq: removes repeated, adjacent lines in a fil
    e.g.
    • sort file3.txt file4.txt | uniq sorts file3 and file4 and removes duplicates
    • sort file3.txt file4.txt | uniq | wc -l counts number of lines in unique combination of file3 and file4
    grep: searches for a pattern in a file name and prints all matching lines
    • grep [pattern] directory
    • -v print line NOT matched by the pattern
    • -i case-insensitive
    • -egrep/grep -E uses extended regular expressions, such as +, ?, |, (, )
    • e.g. egrep "(T|t)he" 1001nights.txt | wc -l counts the number of lines where the word 'the' or 'The' appeared in 1001night.txt
    shutdown: halts the machine
    • -h schedules shutdown at current time
    • e.g. shutdown -h 10:00 schedules shut down at 10am
    • -r restarts machine

    Regular Expressions (RegEx)

    • . matches any single character
    • | matches either
    • $ matches end of string
    • ^ matches beginning of string
    • * matches preceding item 0 or more times
    • + matches preceding item 1 or more times
    • ? matches preceding item 0 or 1 times
    • {n} matches preceding item exactly n occurences
    • [...] matches any character in set
    • e.g. (W|w)onderland matches Wonderland or wonderland
    • [C|c][S|s][0-9]{3} matches any CS course
    You can test your RegEx at https://regex101.com/ or https://regex101.com/