View source

== Introduction == 
In Linux philosophy every thing is a file. all hardware for linux nothing but files for example:
Keyboard, Screen, Printer, Mouse, Hard Disk, Fingerprint-scanner, .... 

One of smart guy now will ask so how come linux differentiate between all this files ?

The answer is with file descriptors, each pieces of this hardware have got unique file descriptors associated with it
linux provides each process that is run on it access to three important files. They are standard input, standard output and standard error. And each of these special files (standard input, output and error) have got the file descriptors 0, 1 and 2 respectively.

 debian:/dev# ls -ali /dev/std*
 2658 lrwxrwxrwx 1 root root 15 2009-06-24 09:44 /dev/stderr -> /proc/self/fd/2
 2656 lrwxrwxrwx 1 root root 15 2009-06-24 09:44 /dev/stdin -> /proc/self/fd/0
 2657 lrwxrwxrwx 1 root root 15 2009-06-24 09:44 /dev/stdout -> /proc/self/fd/1

But what is standard input and standard output?

the purpose we invite the computer for is to give the machine some information and let the machine do the computing in this information "<b>input</b>" and sure we need the result "<B>output</B>". in typical operation your keyboard is your standard input (stdin), and the screen is the standard output (stdout). if any thing wrong happen screen also will be standard error (stderr). Take care we are in linux world where everything is configurable, we can change all this later on when we need too

== Output Redirection ==

let say we need to have file include numbers from 1 to million, you have two choice
*Like all respectable cave man open file type them from your stander input "keyboard" into file -- meet you in next century --
*Like 21st century man do it simply by computer
let's think with this 21st century man how we can do it,there is already command in linux can print sequence of numbers this command called seq all we have to give to this command where to start and where to end, we still have small problem seq will print this numbers to it's (stout) which is screen we force seq to print them into file so....
what we need called redirecting output which can be achieved by using the ">" (greater-than symbol) this symbol can tell linux to change the stnout to other destination in our case the filename.


 debian:/tmp# seq 1 5
 1
 2
 3
 4
 5
 debian:/tmp# seq 1 5 > filename
 debian:/tmp# cat filename 
 1
 2
 3
 4
 5
 debian:/tmp#

the notable thing in ">" behavior is that if file does not exists file name will be created - or overwritten if it already exists, so take care.
it is important to note that when creating files, the output redirection operators are interpreted by the shell before the commands are executed. This means that any output files created through redirection are opened first. For this reason, you cannot modify a file in place, like this:
 debian:/tmp# cat filename  > filename 
 cat: filename: input file is output file
 debian:/tmp# grep 1 filename > filename 
 debian:/tmp# cat filename 
 debian:/tmp# 
If file1 contains something of importance, this command would be a disaster because an empty file1 would overwrite the original. The grep command would
be last to execute, resulting in a complete data loss from the original file1 file because the file that replaced it was empty.

Now let's make the task more hard for computer we need this numbers to be order randomly,
we have them already in filename all we need is use command sort with -R option to make computer do this job  

 debian:/tmp# sort -R filename 
 5
 3
 4
 1
 2
=== pipe ===
sure you think now do we have to do this in two step create the numbers file then sort them, there is short way which can be done using "|" (pipe) operator which sends the standard output of one command to another command as standard input.
so here is how our command will be
 debian:/tmp# seq 1 5 | sort -R  > filename
 debian:/tmp# cat filename 
 1
 4
 2
 5
 3

== Input Redirection ==
if you grasp the idea behind how i/o redirection work then you can guess what this command mean and do
 debian:/tmp# cat
 <B>http://www.msamir.net</B>
 http://www.msamir.net
 ^C
 debian:/tmp# cat < filename 
 this is test file you think i say something wrong  dsfafsadfdsa
in short cat command take any thing from stdin and give it back in stdout and in our example we make his stdin filename so it give it's content back in stdout

== Error Redirection ==
let's digging more deeper in redirection try to guess what is going on in this example
 redhat:/tmp/example# ls
 redhat:/tmp/example# ls msamir.net
 ls: cannot access msamir.net: No such file or directory
 redhat:/tmp/example# ls msamir.net > error.txt
 ls: cannot access msamir.net: No such file or directory
 redhat:/tmp/example# cat error.txt 
 redhat:/tmp/example# ls msamir.net 2> error.txt
 redhat:/tmp/example# cat error.txt 
 ls: cannot access msamir.net: No such file or directory
 redhat:/tmp/example# touch msamir.net
 redhat:/tmp/example# ls msamir.net 2> error.txt
 msamir.net
 redhat:/tmp/example# cat error.txt 
 redhat:/tmp/example# ls msamir.net 1> error.txt
 redhat:/tmp/example# cat error.txt 
 msamir.net
 redhat:/tmp/example#
I am in empty directory as you see ls command list no file in first command. when i try ls none existing file "msamir.net" it give me error messages  back, till now nothing special happen. when i try to redirect this error messages to my error.txt file surprisingly find it's not working as some of you might have been expected !!!!

think in it well i never say ">" will redirect every thing i say it will redirect only stdout but linux designer elite of the the hacker know that nothing is perfect and all the time error can happen so thy make for error place to go to allow poor user/programmer to trace error as separate output, by default linux kernel understand ">" as redirection to stdout if we need to tell the kernel something else we will have to give the kernel the file descriptors before the ">" symbol and in our case we need to redirect stderr which have file descriptor 2 so this is what i do in 5th command and also this is why 6th command show that error.txt file now include the error message.

i will leave the reset as exercise for you

as we must understand by now stdout and stderr by default both go to screen let's try change both to go to other files "yes screen was a file too"
 msamir@RHCE:~$ find / -name msamir.net > result 2>error
 ^C
 msamir@RHCE:~$ cat error 
 find: `/root/Mail': Permission denied
 find: `/root/.config': Permission denied
 find: `/root/.ssh': Permission denied
 msamir@RHCE:~$ cat result 
 /tmp/example/msamir.net
 msamir@RHCE:~$
input/output can be redirected to the same file:
 msamir@RHCE:~$ find / -name msamir.net &>result
 ^C
 msamir@RHCE:~$ cat result 
 find: `/root/.gconf': Permission denied
 find: `/root/Mail': Permission denied
 find: `/root/.config': Permission denied
 find: `/root/.ssh': Permission denied
 find: `/BACKUP/lost+found': Permission denied
 /tmp/example/msamir.net
 find: `/etc/ssl/private': Permission denied 
== All Redirection Cases ==
 redirect stdout to outputfile.                       # command > outputfile
                                                      # command 1> outputfile
 Append stdout to file.                               # command >> file
                                                      # command 1>> file
 Append stderr to errfile.                            # command 2>> errfile
 Append both stdout and stderr to file.               # command >> file 2>&1
 Send stderr to errfile.                              # command 2> errfile
 Send both stdout and stderr to allfile.              # command > allfile 2>&1
 Send stdout to outputfile and stderr to errfile.     # command > outputfile 2> errfile
 Receive stdin from infile.                           # command < infile
 Pipe stdout from command1 to command2.               # command1 | command2
 Pipe stdout and stderr from command1 to command2.    # command1 2>&1 | command2

== tee ==
In some situation you maybe want to run a program and send its output to a file while at the same time viewing the output on the screen, this is tee utility job and it's purpose

<blockquote>
<pre>
SYNOPSIS
       tee [OPTION]... [FILE]...
DESCRIPTION
       Copy standard input to each FILE, and also to standard output.
       -a, --append
              append to the given FILEs, do not overwrite
        If a FILE is -, copy again to standard output.
</pre>
</blockquote>
=== tee examples ===
<pre>
msamir@centos:~$ find / -name msamir.net 2>&1 | tee result
find: `/root/.gconf': Permission denied
.
.
/tmp/example/msamir.net
.
.
^C
msamir@centos:~$ cat result 
find: `/root/.gconf': Permission denied
.
.
/tmp/example/msamir.net
.
.
msamir@centos:~$ 
</pre>
== xargs ==
<blockquote>xargs is a command on Unix and most Unix-like operating systems. It is useful when one wants to pass a large number of arguments to a command. Until Linux kernel 2.6.23, arbitrarily long lists of parameters could not be passed to a command [1], so xargs will break the list of arguments into sublists small enough to be acceptable.</blockquote>
xargs - build and execute command lines from standard input
<pre>
user1@suse:~$ find /etc/ld.so.conf.d/ -type f | xargs -n 1 du -sh 
4.0K	/etc/ld.so.conf.d/libc.conf
4.0K	/etc/ld.so.conf.d/i486-linux-gnu.conf
</pre>

{|border="0" width="100%" style="background-color: #FFFFE6; border:1px Solid #C5C58B;"
|-
|width="30%"|'''«'''
|align="center" rowspan="2"|<big>[[Red_Hat_Linux_Essentials|RedHat Linux Essentials]]<br>''Standard I/O and Pipes''</big>
|width="30%" align="right" | '''»'''
|-
|[[Unit 6 - Using the bash Shell|Using the bash Shell]]
|align="right" | [[Unit 8 - Text Processing Tools|Text Processing Tools]]
|-
|}