11 months

The operating system Linux

Classification of UNIX/Linux
UNIX is a Multi-User/Multi-Tasking operating system and exists in many different versions:Solaris, AIX, XENIX, HP-UX, SINIX, Linux.
Operating System (OS): Sum of all programs which are required to operate a computer and which control and monitor the application programs.

*UNIX has been originally written in the programming language C, and is therefore a classical platform for C-programs. UNIX contains well suited environments for program development (C, C++,Java, Fortran...)
*UNIX is mainly used for scientific-technical applications on mainframes and workstations but has become, because of Linux, also popular for classical PC-applications throughout the last years.
*UNIX is perfectly suited for application in networks. Larger systems and networks require an administrator.
*UNIX offers various alternatives for the solution of most tasks. The multitude of commands (more than in any other OS) are brief and flexible.

Linux is available (also via the internet) in different distributions (S.u.S.E., Fedora, Debian etc.). Meanwhile, there is a variety of direct-start (live) systems, which can be started, without installation, directly from CD or other bootable storage devices (Knop pix, Ubuntu, . . . ).The source code of Linux is free.

Since UNIX is a multi-user operating system, it can deal with several users simultaneously. Each user needs a user account. Each user has a personal environment (home directory, shell), which can be accessed only by her-/himself (and by the system administrator and those people who know the password – legitimate or by hacking). Inside the system, the user is identified by his user ID (UID) and his group identity (group ID, GID).

There are two user types:
• ‘normal’ users with restricted rights and the
• system administrator (root) with all privileges. The latter is responsible for the installation, configuration and maintenance of the system as well
as the user administration.

Working at external terminals
To login to a distant host, one has to provide the corresponding IP address, either numerical or as the complete host name name.domain. In local networks (CIP-Pool), the brief hostname (without domain) is sufficient. To establish the connection and to encrypt the transmitted data, one should use exclusively the so-called “secure” commands. Avoid ftp and use sftp instead. With ftp, even the password is not encoded!

ssh (-4) -X -l username hostname
ssh (-4) -X [email protected]
Enables logging in to an arbitrary host which can be located via an IP address (if one knows the user account and the password). Logoff with exit, Ctrl-D or logout.

command scp
To copy files from one host to another, the command scp (“secure copy”) is used, see also cp.

File systems
The following file-system objects can be found
• ‘normal’ (text-) files
• executable files (binary files or shell scripts)
• directories
• device files
• pipes
• symbolic or hard links

Listing files and directories
To find out what is in your home directory type: ls
The ls command lists the contents of your current working directory.
To list all files in your home directory including those whose names begin with a dot, type: ls -a

Making Directories
In addition to ls there are other commands for working with files that can be used together with file name patterns.
command mkdir, rmdir
mkdir directory
rmdir directory
mkdir creates an empty directory, rmdir deletes an empty directory.

Changing to a different directory (cd)
To change to the directory you have just made, type: cd unixstuff

In Linux . means the current directory, so typing: cd .
 (..) means the parent of the current directory, so typing: cd ..

Pathnames (pwd)
Pathnames enable you to work out where you are in relation to the whole file-system. 

~ (your home directory)
Home directories can also be referred to by the tilde ~ character. It can be used to specify paths starting at your home directory. 

Shell Shortcuts for bash
Ctrl-A (jump to start of line)
Ctrl-E (jump to end of line)
Ctrl-K (delete (kill) everything from the cursor onwards
Ctrl-W (delete the previous word only)
Ctrl-Y (paste whatever was just deleted)
Ctrl-C (kill/exit a running process)
Ctrl-L (clear the screen)
Ctrl-R (search for previously executed commands)
Tab (auto-complete command or file/directory name)
↑ / ↓ (scroll back / forwards through previously entered commands)

ls   list files and directories
ls -a  list all files and directories
mkdir   make a directory
cd   directory change to named directory
cd   change to home-directory
cd ~   change to home-directory
cd ..  change to parent directory
pwd   display the path of the current directory

Copying Files and Directories (cp)
cp file1 file2
cp file1 [file2 ...] directory
cp -r dir1 dir2
cp -r dir1 [dir2 ...] directory
copies files or directories. The original file/directory remains unmodified.
-r directories are copied recursively with all subdirectories.

 Moving files and Directories (mv)
The move command has a variety of similar but subtly different uses. It can be used to move a file to a different location (i.e. a different directory). It can also be used to move multiple files to a different directory. 
Rename or move files or directories. Similar to cp, but the original is ‘destroyed’. First command from above renames files, other commands move files/directories. 

 Removing Files (rm) and Directories (rmdir)
Delete files and/or directories. After deleting, the deleted files cannot be recovered! Use rm only with greatest caution. E.g., the command rm -r * deletes
recursively (in most cases without further inquiry) the complete file tree below the current directory.

-i delete only after confirmation
-r directories will be recursively deleted (with all subdirectories)
-f force: suppress all safety inquiries

File permissions/Access rights
The UNIX file system distinguishes between three different access rights or file mode bits. 
r read: permits the reading of file contents, or, for directories, the listing of their content.
w write: permits the modification of files. To create or delete files, the parent directory(ies) need write access as well.
x execute: permits the execution of binary files (commands, programs) and of shell scripts from the command line. For directories, the x bit is required to change into this directory and to access the files/directories inside.

Access rights are individually defined for
u the owner of the object
g the group to which the object belongs
o all other users
a all users

clear (clear screen)
This will clear all text and leave you with the prompt at the top of the window.

cat (concatenate)
The command cat can be used to display the contents of a file on the screen.

The command less writes the contents of a file onto the screen a page at a time.

The head command writes the first ten lines of a file to the screen. 

The tail command writes the last ten lines of a file to the screen.

Simple searching using less
Using less, you can search through a text file for a keyword (pattern).
For example, to search through science.txt for the word 'science', type 
less science.txt
then, still in less (i.e. don't press q to quit), type a forward slash (/) followed by the word to search for, e.g.

grep is one of many standard UNIX utilities. It searches files for specified words or patterns.
grep science science.txt

To ignore upper/lower case distinctions, use the -i option, i.e. type
grep -i science science.txt
Some of the other options of grep are:
-v display those lines that do NOT match 
-n precede each matching line with the line number 
-c print only the total count of matched lines 
grep -ivc science science.txt

wc (word count)
A handy little utility is the wc command, short for word count. 

cp file1 file2    copy file1 and call it file2
mv file1 file2    move or rename file1 to file2
rm file           remove a file
rmdir directory   remove a directory
cat file         Display or concatenate a file
less file        display a file a page at a time
head file        display the first few lines of a file
tail file        display the last few lines of a file
grep 'keyword' file     search a file for keywords
wc file             count number of lines/words/characters in file

 Redirecting Standard Output 

We use the > symbol to redirect the output of a command. Many of the commands we have seen so far write their output to the terminal (for instance cat, ls, grep, tail, head and wc all write to STDOUT). However, we can redirect the output of any of these commands to a file instead (the file can have any name you chose. If the file does not exist it will be created, if it does exist it will be replaced.

Appending data to an existing file
If we want to add/append data to an existing file, rather than overwrite it, we need to use the >> operator.

Redirecting Standard Error
Standard error and standard output are very similar. Both are generally written to the terminal and it is not always obvious what is STDOUT and what is STDERR. 

Redirecting Standard Input
Similarly, we can use the < operator to redirect STDIN. For example, the sort command reads input from STDIN (the keyboard) and produces an alphabetically or numerically sort list.


Furthermore, many UNIX commands act as so-called filters: They read from the standard input and write to the standard output. Thus, they can be combined via
so-called pipes such that the output of one command acts as the input of another. A re-direction to a file with ‘>’ or ‘>>’ can be present only at the end of such a chain.
To see who is on the system with you, type
One method to get a sorted list of names is to type,
who > names.txt
sort < names.txt

The symbol for a pipe is the vertical bar |
who | sort
will give the same result as above, but quicker and cleaner. To find out how many 
users are logged on, type
who | wc -l

command > file            redirect standard output to a file
command 2> file           redirect standard error to a file
command >> file           append standard output to a file
command < file            redirect standard input from a file
command1 | command2       pipe the output of command1 to the input of command2
cat file1 file2 > file0      concatenate file1 and file2 to file0
sort               sort data
who                 list users currently logged in

The characters * and ?
The character * is called a wildcard, and will match against none or more character(s) in a file (or directory) name.
ls list*
This will list all files in the current directory starting with list....Try typing
ls *list
This will list all files in the current directory ending with ....list

The character ? will match exactly one character. So ls ?ouse will match files like 
house and mouse, but not grouse. Try typing
ls ?list
ls list?
If you need to match a limit number of patterns you can use {pattern1,pattern2,etc}
ls list{1,2}
This can be used with most commands:
mkdir newdir{1,2,3,4,5}
The previous command would create 5 new directories.

Getting Help
On-line Manuals

There are online manuals which give information about most commands. The manual pages tell you which options a particular command can take, and how each option modifies the behavior of the command. Type man command to read the manual page for a particular command.

For example, to find out more about the wc (word count) command, type
man wc
whatis wc

When you are not sure of the exact name of a command,
man –k keyword
will give you the commands with keyword in their manual page header. For example, 
try typing
man –k list 

* match any number of characters
? match one character
man command read the online manual page for a command
whatis command brief description of a command
apropos keyword match commands with the keyword in their man pages

Access rights on files
 r (or -), indicates read permission (or otherwise), that is, the presence or absence of permission to read and copy the file 
 w (or -), indicates write permission (or otherwise), that is, the permission (or otherwise) to change a file 
 x (or -), indicates execution permission (or otherwise), that is, the permission to execute a file, where appropriate 

Access rights on directories
 r allows users to list files in the directory; 
 w means that users may delete files from the directory or move files into it; 
 x means the right to access files in the directory. This implies that you may read files in the directory provided you have read permission on the individual files. 

-rwxrwxrwx a file that everyone can read, write and execute and delete.
-rw------- a file that only the owner can read and write - no one else can read or write and no one has execution rights.

Changing access rights (chmod)
The options of chmod are as follows
Symbol Meaning
u user
g group
o other
a all
r read
w write (and delete)
x execute (and access directory)
+ add permission
- take away permission

Processes and Jobs
A process is an executing program identified by a unique PID (process identifier). To see information about your processes, with their associated PID and status, type

Running background processes
To background a process, type an & at the end of the command line.
sleep 10 &
[1] 6259
The & runs the job in the background and returns the prompt straight away, allowing you to run other programs while waiting for that one to finish.
The first line in the above example is typed in by the user; the next line, indicating job number and PID, is returned by the machine. The user is be notified of a job number (numbered from 1) enclosed in square brackets, together with a PID and is notified when a background process is finished. Backgrounding is useful for jobs which will take a long time to complete.

Listing suspended and background processes
When a process is running, backgrounded or suspended, it will be entered onto a list along with a job number.
An example of a job list could be
[1] Suspended sleep 100
[2] Running firefox
[3] Running nedit
To restart (foreground) a suspended process, type
fg %jobnumber

Killing a process
To kill a job running in the foreground, type ^c (control-c). For example, run
sleep 100
To kill a suspended or background process, type
kill %jobnumber
For example, run
sleep 100 &
If it is job number 4, type
kill %4

ps (process status)
Alternatively, processes can be killed by finding their process numbers (PIDs) and 
using kill PID_number
sleep 100 &
 20077 pts/5 S 0:05 sleep 100
 21563 pts/5 T 0:00 netscape
 21873 pts/5 S 0:25 nedit
To kill off the process sleep 100, type
kill 20077
and then type ps again to see if it has been removed from the list. If a process refuses to be killed, uses the -9 option, i.e. type
kill -9 20077

ls -lag                     list access rights for all files
chmod [options] file        change access rights for named file
command &                   run command in background
^C                          kill the job running in the foreground
^Z                          suspend the job running in the foreground
bg                          background the suspended job
jobs                        list current jobs
fg %1                       foreground job number 1
kill %1                     kill job number 1
ps                          list current processes
kill 26152                  kill process number 26152

To check your current quota and how much of it you have used, type

The df command reports on the space left on the file system. For example, to find out how much space is left on the fileserver, type
df .

du (disk usage)
The du command outputs the number of kilobytes used by each subdirectory.
du *
du –s *
du –sh *

The homeusage command will do the same as running du –sh * in your home directory but will output in ascending order of size to make it easy to see where you are using space. You do not need to be in your home directory to run this command.

This command compresses a file.
To uncompress the file, use the gunzip command.

file classifies the named files according to the type of data they contain, for example ascii (text), pictures, compressed data, etc..
file filename

The shell keeps an ordered list of all the commands that you have entered. Each command is given a number according to the order it was entered.
You can use the exclamation character (!) to recall commands easily.

!! # recall last command
!-3 # recall third most recent command
!5 # recall 5th command in list
!grep # recall last command starting with grep
You can increase the size of the history buffer by typing

find is a powerful but rather complicated command for finding files. By default it searches recursively from the directory specified.
The following example will look for any object called file1, and will start searching from the current working directory:
find . –name file1
To find all objects beginning with file, a wildcard can be used, but it must be quoted:
find . –name “file*”

locate is a very quick way of finding files on a large system. It performs a similar role to the find command but works in a very different way. find looks through the file system until it finds your files (which can be slow but is almost always correct);locate on the other hand searches a database in which the locations of files are maintained. This is far quicker but doesn't reflect very recent changes to the file system, because the database is usually only updated once a day.
locate filename
locate -i filename
locate -r filename

wget is a web client (not a browser). It can be used download files from web and ftp sites:
wget URL

Environment Variables
The value of this is the current operating system you are using. When using variables it’s necessary to refer to them with a $ sign at the start so that the shell knows that we are referring to a variable. Type
echo $OSTYPE

More examples of environment variables are
 $USER - Your login name
 $HOME - Pathname of your home directory
 $HOSTNAME - Name of the computer you are using
 $PATH - Directories the shell searches to find commands
 $SHELL – The shell you are using (should be bash!)
Environment variables are displayed using the env command. To show all values of these variables, type
env | less

The which command shows you the full path to a command (provided that the file is in 
the path)
which command
which wget
If there are multiple programs with the same name, you can use:
which -a command
to list them all. However you should realize that if there are multiple programs with the same name in your path only the one listed first will be executed.

More important commands
a2ps           converts ASCII text to PostScript. Often required to print text under Linux.

diff file1 file2          compares two files. If they are identical, no output.

touch       file sets the current time stamp for a file. Can be used to create an empty file.

finger account    displays additional information for the user of a certain account.

gv      datei.ps displays PostScript files and files of related formats

acroread file.pdf   displays pdf files and allows for simple manipulations

gimp file   starts the image manipulation program gimp (similar to photoshop).Allows to view, manipulate and print image files (e.g., *.jpg, *.tif, *.png).

ps2pdf file.ps    converts ps-files to pdf-files. The file file.pdf will be automatically created.

gzip file. Compresses file via Lempel-Ziv algorithm.The file file.gz is created and the file file deleted. Typical compression factor ∼3.

tar “tape archive”. Nowadays mainly used to create one single file from a file tree, which then, e.g.,can be sent by email. Reverse process also with
tar. tar -cvf direc.tar direc creates (c) file (f) direc.tar from directory direc. Verbose progress is displayed (v). 
tar -xvf direc.tarre-creates original file tree under original name (./direc).
tar -zcvf direc.tgz direc
tar -zxvf direc.tgz additional compression/dekompression via gzip

grep searches for text within given files.

make and the Makefile
The make command allows programmers to manage large programs or groups of programs. It aids in developing large programs by keeping track of which portions of the entire program have been changed, compiling only those parts of the program which have changed since the last compile.
The make program gets its set of compile rules from a text file called Makefile which resides in the same directory as the source files. It contains information on how to compile the software, e.g. the optimization level, whether to include debugging info in the executable. 

 Building the package
Now you can go ahead and build the package by running the make command.
After a minute or two (depending on the speed of the computer), the executables will be created. You can check to see everything compiled successfully by typing
make check
If everything is okay, you can now install the package.
make install

It is important to realize that while the
make check
make install
sequence is extremely common, it is not a standard and there is no absolute guarantee that software will install this way. However, the make command is a standard Linux command which reads in a user-written file (the makefile) that describes how to build and install the software. 

Process administration
A process is a running program or script and consists of
• the program/script itself and
• the corresponding environment, which consists of all required additional information necessary to ensure a correct program flow.

Characteristics of a process are (among others)
• a unique process ID (PID),
• PID of the parent process (PPID),
• User and group number of the owner and
• priority of the process

Normally, when a process has been started from a shell, the shell cannot be used for other input until the end of the process. But processes and programs can also be run in the background. To enable this feature, the command line which calls the process/program must end with an ampersand, '&'.

Automation of commands using shell scripts
Basic shell script
Create and edit a file called test.sh:
echo “Hello $USER”

The line at the beginning is called a shebang, sha-bang, hashbang, pound-bang or hash-pling and tells the computer that the script should be run in a bash shell. The script knows that everything beginning with a dollar sign ($) is a variable and will use the value of the environment variable $USER.
Once you have finished editing the file, close the editor and make the file executable with ‘chmod’.
chmod +x test.sh

Using variables
echo “Hello $USER”
NUMLOGINS=$(who | grep $USER | wc -l)
echo “You have $NUMLOGINS login sessions”

Looping with ‘for’
for user in gg78 ljg2 root 
 NUMLOGINS=$(who | grep $user | wc –l)
 echo “$user has $NUMLOGINS sessions”

Numbers can be used instead of variables in the loop so that the script can keep track of which number loop it is on:
for i in 1 2 3 4 5 
 echo “loop $i”
If the script needs to loop a lot of times, this notation can be used:
for i in {1..20}
This will loop 20 times with the value of $i incrementing by 1 each time. If incrementing by a value other than 1 is required then use the ‘seq’ command, eg:
for i in $(seq 0 0.5 4)
This will give values for $i from 0 to 4, incrementing by 0.5.

Conditional statements
“if / then / else” statements allow conditional branching within a script. This script tests the value of a variable and then decides which branch to take (and therefore what to output), depending on that value.
NUMLOGINS=$(who | grep $USER | wc –l)
if [ $NUMLOGINS -gt 1 ]
 echo "$USER is logged in with at least 2 sessions"
 echo "$USER has less than 2 sessions"
The ‘if … then … else’ statement must be finished with ‘fi’. The ‘else’ option is optional – the script will carry on past ‘fi’ if you do not use it.
‘-gt’ is short for ‘greater than’. Other tests include:
 -eq is equal to
 -ne is not equal to
 -ge is greater than or equal to
 -lt is less than 
 -le is less than or equal to