The Shell
- The shell is a program that reads commands and interprets them.
- Shell commands interact with the file system.
Shell Commands
cat: Prints out the content of a file or combine multiple files to standard output
cat file1.txt
prints out file1.txt
cat file1.txt file2.txt
combines both files
cat
/cat -
takes in keyboard input and outputs it (ctrl-D ends)
cat file1.txt - file2.txt
takes in file1.txt, then keyboard input (end with ctrl-D), then file2.txt
man cat
provides info about cat command
Ctrl-C vs. Ctrl-D
- Ctrl-C: kills a process and stops what it is doing by sending an interrupt signal
- Ctrl-D: EOF signal
head: Displays first few (10) lines of a file
tail: Displays last few (10) lines of a file
-n[# of lines]
/
-[# of lines]
: specify number of lines
ls: lists the contents of the current directory
ls -a: lists all contents including hidden files (files that start with '.')
ls -l: lists the contents of a directory in a long listing format (
ls -l [filename]
displays info for a file in long listing format)
pwd: shows the current working directory
cd: change directory
cd
/cd ~
home directory
cd [directory]
specified directory
cd ..
parent directory
cd /
root directory
cd -
previous directory
- Absolute directories begin with
/
- Relative directories begin with
./
(current)
../
(parent)
~/
(home)
exit: terminates the shell session
echo: write arguments, separated by spaces and terminated with a newline
rm: remove
rm -f: force remove
less: loads a text file one page at a time rather than all at once
wc: displays the number of words, bytes, and lines in a file
wc -w: word count
wc -l: line count
sort: sort lines of text file (alphabetical)
- Lines starting with a number will appear before lines starting with a letter.
- Sorted in increasing alphabetical order based on first character
- Lines starting with a lowercase letter will appear before lines starting with the same letter in uppercase.
sort -r
: sorts in reverse order
Standard File Streams
There are 3 standard file streams:
stdin
with descriptor value of 0 (e.g. keyboard)
stdout
with descriptor value of 1 (e.g. terminal)
stderr
with descriptor value of 2 (e.g. log file)
Input Redirection:
stdin
is often supplied by directing an input to come from a file or previous output command using Pipe.
doCommand < input_file
- e.g.
cat < file1.txt
redirects keyboard input from file1.txt. Note that this is different from cat file1.txt
which opens the file and prints out content
Output Redirection:
stdout
is often redirected to a file,
stderr
is usually printed to terminal but sometimes redirected to a log file
doCommand > output_file
for stdout
doCommand 2> error_file
for stderr
doCommand > output_file 2>&1
redirects the error messages to the same location where output is (or in bash: doCommand >% output_file
)
doCommand >> output_file
appends to start of file
- e.g.
cat > output_file.txt
saves content of keyboard input to file
- e.g.
cat file1.txt file2.txt
combines file1 and file2 to new file
Wildcards
- used for pattern matching and searching for files names with specific characters
?
replaces a single character or digit
*
replaces a string of characters of digits
- e.g.
cat file?.txt
shows the content of all files that start with the word 'file' then a single character or digit of type .txt
- e.g.
ls *file*.txt
list all files that has the word 'file' anywhere in its name
Pipes
- can pipe the output of certain commands into the input of another command (i.e. chains commands)
commandA | commandB | commandC
head -n11 file3.txt | tail -n2
displays lines 10 and 11 of file3.txt
Users and Groups
Each user is assigned a userid
who: lists the currently logged-in users to the system
whoami: lists current user
File Permissions
Files have three types of permissions:
- read (r)
- write (w)
- execute (x)
There are 4 main types of users:
- user (u)
- group (g)
- others (o)
- all (a)
File permissions are ordered by user, group, others. For example,
drwxr-xr-x
means:
d
: directory
rwx
: user has rwx access
r-x
: group has r and x access
r-x
: others have r and x access
chmod: change permissions of a file
+
adds permission
-
removes permission
=
sets permission
- e.g.
chmod u-r file
removes read permission from user for file
- e.g.
chmod o+x file
adds execute permission to others
- e.g.
chmod a=rwx file
gives everybody full control
which: search where an application resides (e.g.
which ls
searches where the ls command resides)
whereis: search broader range of directories and locate the source and man packages of the application
tree: displays tree view of file system (e.g.
tree cs246
displays tree view of cs246 folder)
tac: start from last line of a file and displays its contents backwards (e.g.
tac file1.txt file2.txt file3.txt
displays the contents of each file backwards, in order)
touch: update the file timestamp to match the current time
stat: displays the detailed status of a particular file or file system
mkdir: creates new directory
rmdir: removes empty directory
mv: move or/and rename a file (e.g.
mv alice.txt cs246/alicewonderland.txt
moves alice.txt into cs246 and rename to alicewonderland.txt)
sudo: runs a program by the use of security privilege of another user often the root or super user
locate: search through a previously created database of files and directory to find a match of your entry
Move alice.txt into cs246 and rename to alicewonderland.txt. Update the directory to reflect changes and locate the new location of alicewonderland
mv alice.txt cs246/alicewonderland.txt
updatedb
locate alicewonderland
awk: extract and then print specific content of the file to generate reports
e.g. output the whole line
awk '{ print $0 }' fileName.txt
cal: displays the calendar
find: searches for file
find [dir] -name file_name
- e.g.
find / -name ubunto > ubuntofiles.txt 2> logfile.txt less ubuntofiles.txt less logfile.txt
finds file named ubunto in the root dir and save results into a file, save errors into log file and display the first page of the two new files
-delete
removes all files named file_name that are in the specified directory
-exec [command]
executes command on specified files. All following arguments are args to command until a ;. {} is the variable for each file
- e.g.
find / -name "*.txt" -exec rm {} \;
searches all txt files and deletes them.
-size [size]
searches for all files of a specific size.
b
: 512-byte blocks (default if no suffix is used)
c
: bytes
w
: two-byte words
k
: Kilobytes (units of 1024 bytes)
M
: Megabytes (units of 1048576 bytes)
G
: Gigabytes (units of 1073741824 bytes)
+
: bigger than specified size
-
: smaller than specified size
- e.g.
find / -size +100M
: searches for all files bigger than 100MB
Difference between find and locate:
- locate is faster than find
- find has more functionality (can select files more than just name)
- find doesn't necessarily search the entire filesystem, points to a subdirectory
- find can perform operations to files it finds (exec, delete, size)
- locate's output can be outdated (needs to be updated by updatedb)
uniq: removes repeated, adjacent lines in a fil
e.g.
sort file3.txt file4.txt | uniq
sorts file3 and file4 and removes duplicates
sort file3.txt file4.txt | uniq | wc -l
counts number of lines in unique combination of file3 and file4
grep: searches for a pattern in a file name and prints all matching lines
grep [pattern] directory
-v
print line NOT matched by the pattern
-i
case-insensitive
-egrep
/grep -E
uses extended regular expressions, such as +, ?, |, (, )
- e.g.
egrep "(T|t)he" 1001nights.txt | wc -l
counts the number of lines where the word 'the' or 'The' appeared in 1001night.txt
shutdown: halts the machine
-h
schedules shutdown at current time
- e.g.
shutdown -h 10:00
schedules shut down at 10am
-r
restarts machine
Regular Expressions (RegEx)
.
matches any single character
|
matches either
$
matches end of string
^
matches beginning of string
*
matches preceding item 0 or more times
+
matches preceding item 1 or more times
?
matches preceding item 0 or 1 times
{n}
matches preceding item exactly n occurences
[...]
matches any character in set
- e.g.
(W|w)onderland
matches Wonderland or wonderland
[C|c][S|s][0-9]{3}
matches any CS course