Text Editors

stdout (Standard Out)

I/O (input/output streams)

$ echo Hello World > Peanuts.txt

Writes Hello World to a new file called Peanuts.txt

Processes use I/O streams to receive input and return output.

echo takes the input of the keyboard and returns it as output to the screen. > Is an I/O redirection operator that changes where the standard output goes. if the file already exist the contents will be overwritten.

$ echo Hello World » Peanuts.txt

> is used to add the output to the end of the file instead of overwriting it. This will also create Peanuts.txt if it doesn’t already exist.

stdin (Standard In) <

cat < peanuts.txt > banana.txt

inputs cat to peanuts.txt and outputs to Banana.txt

stderr (Standard Error)

$ ls /fake/directory > peanuts.txt

trying to ls a directory that doesn’t exist

Output: ls: cannot access /fake/directory: No such file or directory

This does not output to peanut.txt because you cannot output a stderror the same way as stdout.

Use a file descriptor to do this. This is a non-negative number that is used to access a file or stream.

The file descriptor for stdin, stdout, and stderr is 0,1, and 2 respectively.

Redirect stderr to a file

$ ls /fake/directory 2> peanuts.txt

To see both stderr and stdout in a file

$ ls /fake/directory > peanuts.txt 2>&1

There is a shorter way to redirect both stdout and stderr to a file

$ ls /fake/directory &> peanuts.txt

To get rid of stderr messages completely

$ ls fake/directory 2> /dev/null

redirects output to /dev/null

pipe and tee

$ ls -la /etc | less

| allows us to take the stdout of a command and make that the stdin to another process.

$ ls | tee peanuts.txt

tee writes the output to two different streams.

env (Environment)

Environment variables store info to variables in your specific user environment.

$ echo $HOME

Shows what is stored in the HOME variable

$ echo $USER

Shows what is stored in the USER variable

$ env

Shows all environment info used by shell and other processes

You can access a variable by placing $ in front of it like the above examples

The system will check the paths in the PATH variable for the command you are trying to run. If it is not in any of those paths then it will not find the command. If you downloaded a tool and place the commands in a path not listed then it will not find the tool when you run the command. You can include other paths in this list.

cut

$ cut -c 5 file.txt

outputs the 5th character in each line of file.txt space also counts as a character.

$ cut -f 2 file.txt

-f flag sorts by fields. Fields are separated by tabs. So this would output the field after the first tab which is the second field.

$ cut -f 1 -d “;” file.txt

Changes tab delimiter to a ; delimiter

paste

$ paste -s file2.txt

combines file2.txt into one line using tabs to separate. -s makes everything go on one line.

$ paste -d ’ ’ -s file2.txt

changes the delimiter to a space

head

by default, the head command will show you the first 10 lines in a file

$ head /var/log/syslog

shows the first 10 lines in /var/log/syslog

$ head -n 15 /var/log/syslog

-n flag is for number of lines, this would output 15 lines. You can output as many lines as you want.

tail

Shows last 10 lines of file by default

$ tail /var/log/syslog

$ tail -n 15 /var/log/syslog

Shows 15 lines instead of 10

$ tail -f /var/log/syslog

-f will follow the file as it grows

Expand and Unexpand

$ expand sample.txt

converts any tabs in the file to a series of spaces.

$ expand file.txt > spacefile.txt

put that output into a new file

$ unexpand -a result.txt

converts spaces back to tabs

Join and Split

$ join file1.txt file2.txt

Joined via matching list fields

-1 refers to file3.txt -2 refers to list2.txt and specifies the field from each file that you want.

$ split somefile

splits into different files by default it will split them once they reach a 1000 line limit. The files are names x** by default

sort

useful for sorting lines

$ sort file.txt

Sorts file in alphabetical order

$ sort -r file.txt

-r flag reverses the order of the sort

$ sort -n file.txt

sorts by numerical value

tr (Translate)

allows you to translate a set of characters into another set of characters

$ tr a-z A-Z

hello

HELLO

uniq (Unique)

Useful tool for parsing text.

$ uniq reading.txt

Outputs contents of reading.txt without duplicates.

$ uniq -c reading.txt

Outputs a count of each word

$ unique -u reading.txt

Outputs only unique words

$ unique -d reading.txt

Only outputs duplicate values

This will not detect duplicate lines if they are not adjacent

To overcome this limitation:

$ sort reading.txt | uniq

This will output duplicated because the sort puts all of the duplicates together first.

wc and nl

The wc (word count) command shows the total count of words in a file

$ wc /etc/passwd

Output: 96 265 5925 /etc/passwd

displays number of lines, number of words, and number of bytes respectively

To just see the count of a certain field, use the -l, -w, or -c respectively

$ wc -l /etc/psswd

$ nl file.txt

shows number of lines in the file

grep

Allows you to search files for characters that match a certain pattern. Like if a string is in a certain file or file in a certain directory.

$ grep fox sample.txt

looks for fox

$ grep -i somepattern somefile

looks for patterns that are case insensitive

$ env | grep -i User

Grabs User from env without needing CAPS for USER variable

$ ls /somedir | grep ‘.txt$’

returns all files ending with .txt in somedi