Difference: TWikiBarTalk002 (1 vs. 12)

Revision 1225 Dec 2017 - Main.PauloSantana

Line: 1 to 1
 

Pub Talk Part II


Line: 471 to 471
  Waiter, another round for me and my pal, please!'
Added:
>
>
next

-- PauloSantana - 25 Dec 2017

 
META TOPICMOVED by="JarbasJunior" date="1157129235" from="TWikiBar.TWikiBarPapo013" to="TWikiBar.TWikiBarTalk002"

Revision 1118 Mar 2014 - JulioNeves

Line: 1 to 1
 

Pub Talk Part II



Added:
>
>
This is a very new translation from Portuguese to English. Please contribute to the development of this site indicating errors and and/or suggesting corrections and materials for this person.
 'Waiter, get me a pint and don't worry about my lad over here, he's finally getting to meet a real operating system and he's got a lot to learn!'

'So my friend, could you get anything of what I've said so far?'

Revision 1007 Jan 2007 - JulioNeves

Line: 1 to 1
 

Pub Talk Part II


Line: 19 to 19
 

The great grep

Changed:
<
<
'Waiter, this time I'll try a caipirinha - the Brazilian National Drink - (See how to prepare it)!'
>
>
'Waiter, this time I'll try a caipirinha - the Brazilian National Drink wink - (See how to prepare it)!'
  'So, I told you that grep matches regular expressions to the lines of an "input". But what are those "inputs"? Well, there are different ways of defining those inputs. Let's see!'

Revision 901 Sep 2006 - JarbasJunior

Line: 1 to 1
 

Pub Talk Part II


Line: 468 to 468
  Waiter, another round for me and my pal, please!'
Added:
>
>
META TOPICMOVED by="JarbasJunior" date="1157129235" from="TWikiBar.TWikiBarPapo013" to="TWikiBar.TWikiBarTalk002"

Revision 802 Aug 2006 - JulioNeves

Line: 1 to 1
 

Pub Talk Part II


Line: 173 to 173
 The software I'm intended to develop is called musinc, and it will include registers on my music file. I will inform the content of each album as a parameter whenever I run the software, this way:

Changed:
<
<
$ musinc "álbum^interprete~musica:interprete~musica:..."
>
>
$ musinc "album^musician~music:musician~music:..."
 

That way, the software musinc will get data from each album as if it were a variable. The only difference between a received parameter and a variable is that the first one gets numerical names (I know it sounds strange... what I meant was that they get one character names), such as $1, $2, $3, ..., $9. Let's make a test:

Line: 299 to 299
 Since it is a is very functional script, I'll simply attach the received parameter at the end of the file songs. Let's include 3 albums and see if it works (in order to simplify, I'll suppose each album contains just 2 songs):

Changed:
<
<
$ musinc "album 3^Artista5~Musica5:Artista6~Musica5" $ musinc "album 1^Artista1~Musica1:Artista2~Musica2" $ musinc "album 2^Artista3~Musica3:Artista4~Musica4"
>
>
$ musinc "album 3^Musician5~Music5:Musician6~Music5" $ musinc "album 1^Musician1~Music1:Musician2~Music2" $ musinc "album 2^Musician3~Music3:Musician4~Music4"
 

Listing the content of songs.

$ cat musics
Changed:
<
<
album 3^Artista5~Musica5:Artista6~Musica6 album 1^Artista1~Musica1:Artista2~Musica2 album 2^Artista3~Musica3:Artista4~Musica4
>
>
album 3^Musician5~Music5:Musician6~Music6 album 1^Musician1~Music1:Musician2~Music2 album 2^Musician3~Music3:Musician4~Music4
 

It is not as functional as it was supposed to be... it could be a lot better. The albums are out of order, complicating the research. Let's change the script and test it again:

Line: 327 to 327
 Including another one

Changed:
<
<
$ musinc "album 4^Artista7~Musica7:Artista8~Musica8"
>
>
$ musinc "album 4^Musician7~Music7:Musician8~Music8"
 

Now let's see what happens to the song file:

$ cat musics
Changed:
<
<
album 1^Artista1~Musica1:Artista2~Musica2 album 2^Artista3~Musica3:Artista4~Musica4 album 3^Artista5~Musica5:Artista6~Musica5 album 4^Artista7~Musica7:Artista8~Musica8
>
>
album 1^Musician1~Music1:Musician2~Music2 album 2^Musician3~Music3:Musician4~Music4 album 3^Musician5~Music5:Musician6~Music5 album 4^Musician7~Music7:Musician8~Music8
 

I simply inserted a line that classifies the file musics, pointing the output to the same file (that's how the option -o works), after attaching each album.

Line: 361 to 361
 
$ muslist "album 2" grep: can't open 2
Changed:
<
<
musicas: album 1^Artista1~Musica1:Artista2~Musica2 musicas: album 2^Artista3~Musica3:Artista4~Musica4 musicas: album 3^Artista5~Musica5:Artista6~Musica6 musicas: album 4^Artista7~Musica7:Artista8~Musica8
>
>
musicas: album 1^Musician1~Music1:Musician2~Music2 musicas: album 2^Musician3~Music3:Musician4~Music4 musicas: album 3^Musician5~Music5:Musician6~Music6 musicas: album 4^Musician7~Music7:Musician8~Music8
 

'What a mess!! Where is the mistake? I put the parameter between inverted commas so that shell would not split it into two...'

Line: 407 to 407
 
$ muslist "album 2"
Changed:
<
<
album2^Artista3~Musica3:Artista4~Musica4
>
>
album2^Musician3~Music3:Musician4~Music4
 

Pay attention too to the fact that grep locates the chain of characters in any position of the register, so, this way we can search for album, song, singer or even for pieces of information. As soon as we get started with conditional commands, we'll get a new version of muslist that asks us in which of the fields the research will be performed.'

Line: 421 to 421
 #!/bin/bash # Consulta CDs (versao 3) #
Changed:
<
<
grep -i "$*" musicas
>
>
grep -i "$*" musics
 $ muslist album 2
Changed:
<
<
album 2^Artista3~Musica3:Artista4~Musica4
>
>
album 2^Musician3~Music3:Musician4~Music4
 

The option $* stands for all parameters, and in that program it will be substituted by the chain album 2 (according to the previous example), and it will do what you wanted it to.

Line: 437 to 437
 Before developing it, I'd like to introduce you to a very useful option of the grep family. Meet the option -v. This option lists every input register, but the ones found by the command. Let's see the example:

Changed:
<
<
$ grep -v "album 2" musicas album 1^Artista1~Musica1:Artista2~Musica2 album 3^Artista5~Musica5:Artista6~Musica6 album 4^Artista7~Musica7:Artista8~Musica8
>
>
$ grep -v "album 2" musics album 1^Musician1~Music1:Musician2~Music2 album 3^Musician5~Music5:Musician6~Music6 album 4^Musician7~Music7:Musician8~Music8
 

As I've mentioned, that grep from the example lists all the registers but the ones that refer to album 2, and that happens because it fits into the parameters of the command. Now we are ready to develop the script that will remove the lost CD from your CD Library. It looks like this:

Line: 451 to 451
 # Delete CDs from Library (version 1) # grep -v "$1" musics > /tmp/mus$$
Changed:
<
<
mv -f /tmp/mus$$ musicas
>
>
mv -f /tmp/mus$$ musics
 

The first line sends the file musics to /tmp/mus$$, but extracting the registers that conform to the grep='s research. Afterwards, it moves (or renames, if you prefer this word) =/tmp/mus$$ to musics.

Revision 701 Aug 2006 - JulioNeves

Line: 1 to 1
 

Pub Talk Part II


Line: 359 to 359
 Let's run it looking for album 2. As we have previously seen, when informing the sequence of characters album 2, it is necessary to prevent Shell from interpreting it (otherwise it would read two parameters). Let's try the following:

Changed:
<
<
$ muslist "álbum 2"
>
>
$ muslist "album 2"
 grep: can't open 2 musicas: album 1^Artista1~Musica1:Artista2~Musica2 musicas: album 2^Artista3~Musica3:Artista4~Musica4

Revision 616 Dec 2005 - Main.CarlinhosCecconi

Line: 1 to 1
 

Pub Talk Part II


Line: 125 to 125
 Let's take another example. You know what are the first four positions of the output of a ls -s command for an ordinary file (not a directory, nor a link, nor anything...) should be:

Added:
>
>
 
  - -   s (suid)  
 Position    1st     2nd     3rd   4th
  Possible values   - r w x
Line: 219 to 220
 Since we are talking about parameters, let me give you some hints:

Added:
>
>
 
Meaning of the main variables
$0   Name of the program
  Variable     Meaning

Revision 514 Dec 2005 - AurelioAHeckert

Line: 1 to 1
 

Pub Talk Part II


Line: 25 to 25
  Searching a file:
Changed:
<
<
%TERMINALon%
>
>
 $ grep mary /etc/passwd
Changed:
<
<
%TERMINALoff%
>
>
  Searching more than one file:
Changed:
<
<
%TERMINALon%
>
>
 $ grep grep *.sh
Changed:
<
<
%TERMINALoff%
>
>
  Searching the output of a command
Changed:
<
<
%TERMINALon%
>
>
 $ who | grep pelegrino
Changed:
<
<
%TERMINALoff%
>
>
  Considering the 1st example - which is the simplest one - I searched the occurrences of the word mary in any position of the file /etc/passwd. If I wanted to search it as a login name - or, in other words, just at the begining of the registers of that file - I should execute:
Changed:
<
<
%TERMINALon%
>
>
 $ grep '^rafael' /etc/passwd
Changed:
<
<
%TERMINALoff%
>
>
  'Hold on, hold on... what's that caret (circumflex ^) and those apostrophes for?'
Line: 71 to 71
 
  • egrep ('e' standing for extended)
    Is a very powerful tool that uses regular expressions. It is often seen as the slowest brother of the grep family, hence it is more likely to use it when it is necessary to elaborate a regular expression that grep does not accept;
  • fgrep ('f' standing for fast, or file)
    As its own name points out, is the fast brother of the family. It is fast running (it is about 30% faster than grep and 50% faster than egrep), but it is does not allow the use of regular expressions
Changed:
<
<
%ATTENTIONon%
>
>
Pinguim com placa de atenção (em inglês)
 The considerations above on speed are valid to the Unix grep family. grep is faster running on Linux, because the other two (fgrep and egrep) are shell scripts that execute grep.

And I must say: I don't like that solution.

Changed:
<
<
%ATTENTIONoff%
>
>
  'Now that you know the differences among the tree, tell me: What do you think about the examples I gave before the explanation?'
Line: 89 to 89
  There are two options in that case:
Changed:
<
<
%TERMINALon%
>
>
 $ egrep (Linux | linux) arquivo.txt
Changed:
<
<
%TERMINALoff%
>
>
  or
Changed:
<
<
%TERMINALon%
>
>
 $ grep [Ll]inux arquivo.txt
Changed:
<
<
%TERMINALoff%
>
>
  In the first case, the complex regular expression (Linux | linux) uses the parentheses to group up the options and the pipe (|) as a logical "or", which means that you are searching Linux or linux.
Line: 105 to 105
  Another example. If you want to list the subdirectories of a directory, you should run:
Changed:
<
<
%TERMINALon% $ ls -l | grep '^d'%OUTon%
>
>
$ ls -l | grep '^d'
 drwxr-xr-x 3 root root 4096 Dec 18 2000 doc drwxr-xr-x 11 root root 4096 Jul 13 18:58 freeciv drwxr-xr-x 3 root root 4096 Oct 17 2000 gimp
Line: 117 to 117
 drwxrwxr-x 3 root root 4096 Jan 17 2000 pixmaps drwxr-xr-x 3 root root 4096 Jul 2 20:30 scribus drwxrwxr-x 3 root root 4096 Jan 17 2000 sounds
Changed:
<
<
drwxr-xr-x 3 root root 4096 Dec 18 2000 xine%OUToff% %TERMINALoff%
>
>
drwxr-xr-x 3 root root 4096 Dec 18 2000 xine
  As you can see above, the circumflex (^) limits the search to the first position of the long output of the ls command. The apostrophes tell the shell not to 'understand' the circumflex (^).
Line: 133 to 133
  Thus, in order to find out what are the executable files in a directory, you should:
Changed:
<
<
%TERMINALon% $ ls -la | egrep '^-..(x|s)'%OUTon%
>
>
$ ls -la | egrep '^-..(x|s)'
 -rwxr-xr-x 1 root root 2875 Jun 18 19:38 rc -rwxr-xr-x 1 root root 857 Aug 9 22:03 rc.local
Changed:
<
<
-rwxr-xr-x 1 root root 18453 Jul 6 17:28 rc.sysinit%OUToff% %TERMINALoff%
>
>
-rwxr-xr-x 1 root root 18453 Jul 6 17:28 rc.sysinit
  Once again the caret (^) limits the search to the begining of each line, hence, the listed occurrences are the ones that start with a -, followed by anything (the full stop - a dot - in a regular expression denotes any character), once again followed by any character, followed by an x or a s.

The same result would be found with the command:

Changed:
<
<
%TERMINALon%
>
>
 $ ls -la | grep '^-..[xs]'
Changed:
<
<
%TERMINALoff%
>
>
  and the search would be faster.
Line: 171 to 171
  The software I'm intended to develop is called musinc, and it will include registers on my music file. I will inform the content of each album as a parameter whenever I run the software, this way:
Changed:
<
<
%TERMINALon%
>
>
 $ musinc "álbum^interprete~musica:interprete~musica:..."
Changed:
<
<
%TERMINALoff%
>
>
  That way, the software musinc will get data from each album as if it were a variable. The only difference between a received parameter and a variable is that the first one gets numerical names (I know it sounds strange... what I meant was that they get one character names), such as $1, $2, $3, ..., $9. Let's make a test:
Changed:
<
<
%TERMINALon% $ cat teste%OUTon%
>
>
$ cat teste
 #!/bin/bash # Program to test how to inform the parameters echo "1o. parm -> $1" echo "2o. parm -> $2"
Changed:
<
<
echo "3o. parm -> $3"%OUToff% %TERMINALoff%
>
>
echo "3o. parm -> $3"
  Let's run it now:
Changed:
<
<
%TERMINALon% $ teste informing parameters to test%OUTon% bash: teste: cannot execute%OUToff% %TERMINALoff%
>
>
$ teste informing parameters to test bash: teste: cannot execute
  OOPS, there is a detail I've forgotten: we have to make the file executable before running it:
Changed:
<
<
%TERMINALon%
>
>
 $ chmod 755 teste
Changed:
<
<
$ teste informing parameters to test%OUTon%
>
>
$ teste informing parameters to test
 1o. parm -> informing 2o. parm -> parameters
Changed:
<
<
3o. parm -> to%OUToff% %TERMINALoff%
>
>
3o. parm -> to
  Interestingly, the last word test was not considered by our program. That is because the program just considered the three first parameters. Let's execute it another way:
Changed:
<
<
%TERMINALon% $ teste "informing parameters" to test%OUTon%
>
>
$ teste "informing parameters" to test
 1o. parm -> informing parameters 2o. parm -> to
Changed:
<
<
3o. parm -> test%OUToff% %TERMINALoff%
>
>
3o. parm -> test
  With inverted commas Shell did not consider the blank space between the two first words, making it consider them as a single parameter.
Line: 229 to 229
 
  • Examples
Making changes on the program teste, in order to use the variables we have just seen. Let's do it this way:
Changed:
<
<
%TERMINALon% $ cat teste%OUTon%
>
>
$ cat teste
 #!/bin/bash # Program to test how to inform the parameters (2nd Version) echo The program $0 received $# parameters echo "1o. parm -> $1" echo "2o. parm -> $2" echo "3o. parm -> $3"
Changed:
<
<
echo Todos de uma só \"tacada\": $*%OUToff% %TERMINALoff%
>
>
echo Todos de uma só \"tacada\": $*
  Note that preceding the inverted commas I inserted a inverted slash, in order to tell Shell not to interpret them. Let's run the program.
Changed:
<
<
%TERMINALon% $ teste informing parameters to test%OUTon%
>
>
$ teste informing parameters to test
 The program teste received 4 parameters 1o. parm -> informing 2o. parm -> parameters 3o. parm -> to
Changed:
<
<
Todos de uma "tacada": informing parameters to test%OUToff% %TERMINALoff%
>
>
Todos de uma "tacada": informing parameters to test
  As I've said before, the parameters are numbered from 1 to 9, but that does not mean that it is not possible to use more than 9 parameters. Let's test it:

  • Example:
Changed:
<
<
%TERMINALon% $ cat teste%OUTon%
>
>
$ cat teste
 #!/bin/bash # Program to test how to inform the parameters (3rd Version) echo The program $0 received $# parameters
Line: 264 to 264
 shift echo "2nd parm -> $1" shift 2
Changed:
<
<
echo "4th Parm -> $4"%OUToff% %TERMINALoff%
>
>
echo "4th Parm -> $4"
  Let's run it now:
Changed:
<
<
%TERMINALon% $ teste informing parameters to test%OUTon%
>
>
$ teste informing parameters to test
 The program teste received 4 parameters 11th parm -> informing1 2nd parm -> parameters
Changed:
<
<
4th parm -> test%OUToff% %TERMINALoff%
>
>
4th parm -> test
  There are two remarkable points about this script:
Line: 286 to 286
 
  • Examples:
Changed:
<
<
%TERMINALon% $ cat musinc%OUTon%
>
>
$ cat musinc
 #!/bin/bash # Cadastra CDs (Version 1) #
Changed:
<
<
echo $1 >> musics%OUToff% %TERMINALoff%
>
>
echo $1 >> musics
  Since it is a is very functional script, I'll simply attach the received parameter at the end of the file songs. Let's include 3 albums and see if it works (in order to simplify, I'll suppose each album contains just 2 songs):
Changed:
<
<
%TERMINALon%
>
>
 $ musinc "album 3^Artista5~Musica5:Artista6~Musica5" $ musinc "album 1^Artista1~Musica1:Artista2~Musica2" $ musinc "album 2^Artista3~Musica3:Artista4~Musica4"
Changed:
<
<
%TERMINALoff%
>
>
  Listing the content of songs.
Changed:
<
<
%TERMINALon% $ cat musics%OUTon%
>
>
$ cat musics
 album 3^Artista5~Musica5:Artista6~Musica6 album 1^Artista1~Musica1:Artista2~Musica2
Changed:
<
<
album 2^Artista3~Musica3:Artista4~Musica4%OUToff% %TERMINALoff%
>
>
album 2^Artista3~Musica3:Artista4~Musica4
  It is not as functional as it was supposed to be... it could be a lot better. The albums are out of order, complicating the research. Let's change the script and test it again:
Changed:
<
<
%TERMINALon% $ cat musinc%OUTon%
>
>
$ cat musinc
 #!/bin/bash # Cadastra CDs (versao 2) # echo $1 >> musics
Changed:
<
<
sort musics -o musics%OUToff% %TERMINALoff%
>
>
sort musics -o musics
  Including another one
Changed:
<
<
%TERMINALon%
>
>
 $ musinc "album 4^Artista7~Musica7:Artista8~Musica8"
Changed:
<
<
%TERMINALoff%
>
>
  Now let's see what happens to the song file:
Changed:
<
<
%TERMINALon% $ cat musics%OUTon%
>
>
$ cat musics
 album 1^Artista1~Musica1:Artista2~Musica2 album 2^Artista3~Musica3:Artista4~Musica4 album 3^Artista5~Musica5:Artista6~Musica5
Changed:
<
<
album 4^Artista7~Musica7:Artista8~Musica8%OUToff% %TERMINALoff%
>
>
album 4^Artista7~Musica7:Artista8~Musica8
  I simply inserted a line that classifies the file musics, pointing the output to the same file (that's how the option -o works), after attaching each album.
Line: 346 to 346
  Listing with the cat command is totally out, let's make a program called muslist that lists the album whose name is given as parameter:
Changed:
<
<
%TERMINALon% $ cat muslist%OUTon%
>
>
$ cat muslist
 #!/bin/bash # Search for CDs (version 1) #
Changed:
<
<
grep $1 musicas%OUToff% %TERMINALoff%
>
>
grep $1 musicas
  Let's run it looking for album 2. As we have previously seen, when informing the sequence of characters album 2, it is necessary to prevent Shell from interpreting it (otherwise it would read two parameters). Let's try the following:
Changed:
<
<
%TERMINALon% $ muslist "álbum 2"%OUTon%
>
>
$ muslist "álbum 2"
 grep: can't open 2 musicas: album 1^Artista1~Musica1:Artista2~Musica2 musicas: album 2^Artista3~Musica3:Artista4~Musica4 musicas: album 3^Artista5~Musica5:Artista6~Musica6
Changed:
<
<
musicas: album 4^Artista7~Musica7:Artista8~Musica8%OUToff% %TERMINALoff%
>
>
musicas: album 4^Artista7~Musica7:Artista8~Musica8
  'What a mess!! Where is the mistake? I put the parameter between inverted commas so that shell would not split it into two...'
Line: 387 to 387
  grep has understood that it was supposed to look for the chain of characters album on the files 2 and musics. But, since there is no arquivo 2, an error has occurred. Moreover, since the word album was found in every register of musicas, all registers were listed.
Changed:
<
<
%TIPon%
>
>
Pinguim com placa de dica (em inglês)
 Use inverted commas whenever there is a blank space or a <TAB> in the chain of characters that grep will run. That helps the words after the blank space or <TAB> from being interpreted as file names.
Changed:
<
<
%TIPoff%
>
>
  On the other side, it is better not to consider the case of the letters in the research. The following program would solve two problems at the same time:
Changed:
<
<
%TERMINALon% $ cat muslist%OUTon%
>
>
$ cat muslist
 #!/bin/bash # Search for CDs (version 2) #
Changed:
<
<
grep -i "$1" musics%OUToff% %TERMINALoff%
>
>
grep -i "$1" musics
  In that case, the option -i tells grep not to consider the case of the letters. Another point is the parameter $1 that was inserted between inverted commas so that grep would understand the chain of characters as a single argument.
Changed:
<
<
%TERMINALon% $ muslist "album 2"%OUTon% album2^Artista3~Musica3:Artista4~Musica4%OUToff% %TERMINALoff%
>
>
$ muslist "album 2" album2^Artista3~Musica3:Artista4~Musica4
  Pay attention too to the fact that grep locates the chain of characters in any position of the register, so, this way we can search for album, song, singer or even for pieces of information. As soon as we get started with conditional commands, we'll get a new version of muslist that asks us in which of the fields the research will be performed.'
Line: 414 to 414
  'You are right! Let me show you another way, then:
Changed:
<
<
%TERMINALon% $ cat muslist%OUTon%
>
>
$ cat muslist
 #!/bin/bash # Consulta CDs (versao 3) # grep -i "$*" musicas $ muslist album 2
Changed:
<
<
album 2^Artista3~Musica3:Artista4~Musica4%OUToff% %TERMINALoff%
>
>
album 2^Artista3~Musica3:Artista4~Musica4
  The option $* stands for all parameters, and in that program it will be substituted by the chain album 2 (according to the previous example), and it will do what you wanted it to.
Line: 434 to 434
  Before developing it, I'd like to introduce you to a very useful option of the grep family. Meet the option -v. This option lists every input register, but the ones found by the command. Let's see the example:
Changed:
<
<
%TERMINALon% $ grep -v "album 2" musicas%OUTon%
>
>
$ grep -v "album 2" musicas
 album 1^Artista1~Musica1:Artista2~Musica2 album 3^Artista5~Musica5:Artista6~Musica6
Changed:
<
<
album 4^Artista7~Musica7:Artista8~Musica8%OUToff% %TERMINALoff%
>
>
album 4^Artista7~Musica7:Artista8~Musica8
  As I've mentioned, that grep from the example lists all the registers but the ones that refer to album 2, and that happens because it fits into the parameters of the command. Now we are ready to develop the script that will remove the lost CD from your CD Library. It looks like this:
Changed:
<
<
%TERMINALon% $ cat musexc%OUTon%
>
>
$ cat musexc
 #!/bin/bash # Delete CDs from Library (version 1) # grep -v "$1" musics > /tmp/mus$$
Changed:
<
<
mv -f /tmp/mus$$ musicas%OUToff% %TERMINALoff%
>
>
mv -f /tmp/mus$$ musicas
  The first line sends the file musics to /tmp/mus$$, but extracting the registers that conform to the grep='s research. Afterwards, it moves (or renames, if you prefer this word) =/tmp/mus$$ to musics.

Revision 401 Dec 2005 - JulioNeves

Line: 1 to 1
 

Pub Talk Part II


Line: 294 to 294
 echo $1 >> musics%OUToff% %TERMINALoff%
Deleted:
<
<
 Since it is a is very functional script, I'll simply attach the received parameter at the end of the file songs. Let's include 3 albums and see if it works (in order to simplify, I'll suppose each album contains just 2 songs):
Changed:
<
<
>
>
%TERMINALon% $ musinc "album 3^Artista5~Musica5:Artista6~Musica5" $ musinc "album 1^Artista1~Musica1:Artista2~Musica2" $ musinc "album 2^Artista3~Musica3:Artista4~Musica4" %TERMINALoff%
  Listing the content of songs.
Changed:
<
<
>
>
%TERMINALon% $ cat musics%OUTon% album 3^Artista5~Musica5:Artista6~Musica6 album 1^Artista1~Musica1:Artista2~Musica2 album 2^Artista3~Musica3:Artista4~Musica4%OUToff% %TERMINALoff%
  It is not as functional as it was supposed to be... it could be a lot better. The albums are out of order, complicating the research. Let's change the script and test it again:
Changed:
<
<
>
>
%TERMINALon% $ cat musinc%OUTon% #!/bin/bash # Cadastra CDs (versao 2) # echo $1 >> musics sort musics -o musics%OUToff% %TERMINALoff%
  Including another one
Changed:
<
<
>
>
%TERMINALon% $ musinc "album 4^Artista7~Musica7:Artista8~Musica8" %TERMINALoff%
  Now let's see what happens to the song file:
Added:
>
>
%TERMINALon% $ cat musics%OUTon% album 1^Artista1~Musica1:Artista2~Musica2 album 2^Artista3~Musica3:Artista4~Musica4 album 3^Artista5~Musica5:Artista6~Musica5 album 4^Artista7~Musica7:Artista8~Musica8%OUToff% %TERMINALoff%
 
Changed:
<
<

I simply inserted a line that classifies the file musicas, pointing the output to the same file (that's how the option -o works), after attaching each album.

>
>
I simply inserted a line that classifies the file musics, pointing the output to the same file (that's how the option -o works), after attaching each album.
  WOW! Now it is nice and almost functional. But attention and don't panic! That is not the final version. The next version of the program will be a lot better and more friendly! We'll develop it as soon as we learn how to get data from the screen and how to format the input.
Changed:
<
<
Examples Listing with the cat command is totally out, let's make a program called muslist that lists the album whose name is given as parameter:

Let's run it looking for album 2. As we have previously seen, when informing the sequence of characters 'album 2', it is necessary to prevent shell from interpreting it (otherwise it would read two parameters). Let's try the following:

>
>
  • Examples
 
Added:
>
>
Listing with the cat command is totally out, let's make a program called muslist that lists the album whose name is given as parameter:
 
Added:
>
>
%TERMINALon% $ cat muslist%OUTon% #!/bin/bash # Search for CDs (version 1) # grep $1 musicas%OUToff% %TERMINALoff%
 
Added:
>
>
Let's run it looking for album 2. As we have previously seen, when informing the sequence of characters album 2, it is necessary to prevent Shell from interpreting it (otherwise it would read two parameters). Let's try the following:
 
Changed:
<
<
'
>
>
%TERMINALon% $ muslist "álbum 2"%OUTon% grep: can't open 2 musicas: album 1^Artista1~Musica1:Artista2~Musica2 musicas: album 2^Artista3~Musica3:Artista4~Musica4 musicas: album 3^Artista5~Musica5:Artista6~Musica6 musicas: album 4^Artista7~Musica7:Artista8~Musica8%OUToff% %TERMINALoff%
  'What a mess!! Where is the mistake? I put the parameter between inverted commas so that shell would not split it into two...'
Changed:
<
<
'Yeap, but pay attention to how grep is running:

Even putting album 2 between inverted commas, when shell sees $1 it splits it into two arguments. So, the final content of the line that grep has executed is:

>
>
'Yeap, but pay attention to how grep is running:
 
Added:
>
>
    grep $1 musics
 
Added:
>
>
Even putting album 2 between inverted commas, when Shell sees $1 it splits it into two arguments. So, the final content of the line that grep has executed is:
 
Added:
>
>
    grep album 2 musics
 
Changed:
<
<
As the grep syntax is:
>
>
As the grep syntax is:
 
Changed:
<
<
grep  [arq1, arq2, ..., arqn]
>
>
    grep  [arq1, arq2, ..., arqn]
 
Changed:
<
<
Grep has understood that it was supposed to look for the chain of characters 'album' on the files arquivos 2 and musicas. But, since there is no arquivo 2, an error has occurred. Moreover, since the word album was found in every register of musicas, all registers were listed.
>
>
grep has understood that it was supposed to look for the chain of characters album on the files 2 and musics. But, since there is no arquivo 2, an error has occurred. Moreover, since the word album was found in every register of musicas, all registers were listed.
 
Changed:
<
<
HINT Use inverted commas whenever there is a blank space or a TAB in the chain of characters that grep will run. That helps the words after the blank space or TAB from being interpreted as file names.
>
>
%TIPon% Use inverted commas whenever there is a blank space or a <TAB> in the chain of characters that grep will run. That helps the words after the blank space or <TAB> from being interpreted as file names. %TIPoff%
  On the other side, it is better not to consider the case of the letters in the research. The following program would solve two problems at the same time:
Added:
>
>
%TERMINALon% $ cat muslist%OUTon% #!/bin/bash # Search for CDs (version 2) # grep -i "$1" musics%OUToff% %TERMINALoff%
 
Added:
>
>
In that case, the option -i tells grep not to consider the case of the letters. Another point is the parameter $1 that was inserted between inverted commas so that grep would understand the chain of characters as a single argument.
 
Added:
>
>
%TERMINALon% $ muslist "album 2"%OUTon% album2^Artista3~Musica3:Artista4~Musica4%OUToff% %TERMINALoff%
 
Changed:
<
<
In that case, the option -i tells grep not to consider the case of the letters. Another point is the option "$1" that was inserted between inverted commas so that grep would understand the chain of characters as a single argument.

Pay attention too to the fact that grep locates the chain of characters in any position of the register, so, this way we can search for album, song, singer or even for pieces of information. As soon as we get started with conditional commands, we'll get a new version of muslist that asks us in which of the fields the research will be performed.'

>
>
Pay attention too to the fact that grep locates the chain of characters in any position of the register, so, this way we can search for album, song, singer or even for pieces of information. As soon as we get started with conditional commands, we'll get a new version of muslist that asks us in which of the fields the research will be performed.'
  'Hold on pal! That putting between inverted commas thing is not really a friendly way of doing that...'

'You are right! Let me show you another way, then:

Added:
>
>
%TERMINALon% $ cat muslist%OUTon% #!/bin/bash # Consulta CDs (versao 3) # grep -i "$*" musicas $ muslist album 2 album 2^Artista3~Musica3:Artista4~Musica4%OUToff% %TERMINALoff%
 
Added:
>
>
The option $* stands for all parameters, and in that program it will be substituted by the chain album 2 (according to the previous example), and it will do what you wanted it to.
 
Changed:
<
<

The option "$*" stands for all parameters, and in that program it will be substituted by the chain album 2 (according to the previous example), and it will do what you wanted it to.

You should have realized by now that the problem about shell is not how to do something, but what is the best way of doing it (as you've seen, the range of options is huge!).'

>
>
You should have realized by now that the problem about Shell is not if if does or not something, but what is the best way of doing it (as you've seen, the range of options is huge!).'
  'But what if I have to exclude a CD? Once I forgot a CD of mine under the sun and when I looked at it again... it was lost. What if that happened again?'
Changed:
<
<
'Well, let's make another script called musexc, in order to solve that kind of problem.

Before developing it, I'd like to introduce you to a very useful option of the grep family. Meet the option -v. This option lists every input register, but the ones found by the command. Let's see the example:

As I've mentioned, that grep from the example lists all the registers but the ones that refer to album 2, and that happens because it fits into the parameters of the command. Now we are ready to develop the script that will remove the lost CD from your CD Library. It looks like this:

>
>
'Well, let's make another script called musexc, in order to solve that kind of problem.'
 
Added:
>
>
Before developing it, I'd like to introduce you to a very useful option of the grep family. Meet the option -v. This option lists every input register, but the ones found by the command. Let's see the example:
 
Added:
>
>
%TERMINALon% $ grep -v "album 2" musicas%OUTon% album 1^Artista1~Musica1:Artista2~Musica2 album 3^Artista5~Musica5:Artista6~Musica6 album 4^Artista7~Musica7:Artista8~Musica8%OUToff% %TERMINALoff%
 
Added:
>
>
As I've mentioned, that grep from the example lists all the registers but the ones that refer to album 2, and that happens because it fits into the parameters of the command. Now we are ready to develop the script that will remove the lost CD from your CD Library. It looks like this:
 
Added:
>
>
%TERMINALon% $ cat musexc%OUTon% #!/bin/bash # Delete CDs from Library (version 1) # grep -v "$1" musics > /tmp/mus$$ mv -f /tmp/mus$$ musicas%OUToff% %TERMINALoff%
 
Changed:
<
<
The first line sends the file musicas to /tmp/mus$$, but extracting the registers that conform to the grep's research. Afterwards, it moves (or renames, if you prefer this word) /tmp/mus$$ to musicas.
>
>
The first line sends the file musics to /tmp/mus$$, but extracting the registers that conform to the grep='s research. Afterwards, it moves (or renames, if you prefer this word) =/tmp/mus$$ to musics.
 
Changed:
<
<
I used the file /tmp/mus$$ as a work copy, because, as I've mentioned previously, the $$ contains the PID (Process Identification), because of that, when others edit the file musicas, a different work copy will be made, and that avoids running over other's files.'
>
>
I used the file /tmp/mus$$ as a work copy, because, as I've mentioned previously, the $$ contains the PID (Process IDentification), because of that, when others edit the file musics, a different work copy will be made, and that avoids running over other's files.
  'And that's it?'
Line: 420 to 466
  Waiter, another round for me and my pal, please!'
Deleted:
<
<
%CONSTRUCAOon% Sorry, but in a couple of days the whole text will be "TWiked" %CONSTRUCAOoff%

Revision 330 Nov 2005 - JulioNeves

Line: 1 to 1
 

Pub Talk Part II


Line: 150 to 150
  and the search would be faster.
Changed:
<
<
Building a CD Library
>
>

Building a CD Library

 'Let me use a nice and didactic example: the process of building a CD Library. Keep in mind that it is as possible to develop software to organize audio CDs, as it is to data CDs (including those you get when you buy magazines, those you burn for yourself, etc.).'

'Hold on a sec. Where am I taking the CD data from?'

Line: 158 to 159
 'Firstly I'll show you how your software can obtain data from those who are using it, afterwards I'll show you how to get data from the screen or from a file.'
Changed:
<
<
Informing the Parameters
>
>

Informing the Parameters

  'In our case, the layout of a music file will be:'
Added:
>
>
 name of the album^artist~name of the song:..:singer of the song
Added:
>
>
 
Changed:
<
<
As you can see above, a circumflex (^) separates the name of the album from the rest of the register (which contains information on each song and on its singer). The artist and the name of the song are separated by a tilde (~), and a colon (:) separates name of the song and name of the singer.

The software I'm intended to develop is called musinc, and it will include registers on my music file. I will inform the content of each album as a parameter whenever I run the software, this way:

That way, the software musinc will get data from each album as if it were a variable. The only difference between a received parameter and a variable is that the first one gets numerical names (I know it sounds strange... what I meant was that they get one character names), such as $1, $2, $3, ..., $9. Let's make a test:

Examples

>
>
As you can see above, a circumflex (^) separates the name of the album from the rest of the register (which contains information on each song and on its singer). The artist and the name of the song are separated by a tilde (~), and a colon (:) separates name of the song and name of the singer.
 
Added:
>
>
The software I'm intended to develop is called musinc, and it will include registers on my music file. I will inform the content of each album as a parameter whenever I run the software, this way:
 
Added:
>
>
%TERMINALon% $ musinc "álbum^interprete~musica:interprete~musica:..." %TERMINALoff%
 
Added:
>
>
That way, the software musinc will get data from each album as if it were a variable. The only difference between a received parameter and a variable is that the first one gets numerical names (I know it sounds strange... what I meant was that they get one character names), such as $1, $2, $3, ..., $9. Let's make a test:
 
Added:
>
>
%TERMINALon% $ cat teste%OUTon% #!/bin/bash # Program to test how to inform the parameters echo "1o. parm -> $1" echo "2o. parm -> $2" echo "3o. parm -> $3"%OUToff% %TERMINALoff%
  Let's run it now:
Changed:
<
<
>
>
%TERMINALon% $ teste informing parameters to test%OUTon% bash: teste: cannot execute%OUToff% %TERMINALoff%
  OOPS, there is a detail I've forgotten: we have to make the file executable before running it:
Added:
>
>
%TERMINALon% $ chmod 755 teste $ teste informing parameters to test%OUTon% 1o. parm -> informing 2o. parm -> parameters 3o. parm -> to%OUToff% %TERMINALoff%
 
Added:
>
>
Interestingly, the last word test was not considered by our program. That is because the program just considered the three first parameters. Let's execute it another way:
 
Added:
>
>
%TERMINALon% $ teste "informing parameters" to test%OUTon% 1o. parm -> informing parameters 2o. parm -> to 3o. parm -> test%OUToff% %TERMINALoff%
 
Added:
>
>
With inverted commas Shell did not consider the blank space between the two first words, making it consider them as a single parameter.
 
Changed:
<
<
Interestingly, the last word 'test' was not considered by our program. That is because the program just considered the three first parameters. Let's execute it another way:

The double inverted commas tell our program not to consider the blank space between the two first words, making it consider them as a single parameter.'

Parametric Hints 'Since we are talking about parameters, let me give you some hints:

Meaning of the main variables variable Meaning $0 Name of the program $# Amount of informed parameters $* Set of all parameters (similar to $@)

Examples Making changes on the program test, in order to use the variables we have just seen. Let's do it this way:

Note that preceding the inverted commas I inserted a inverted slash, in order to tell shell not to interpret them. Let's run the program.

>
>

Parametric Hints

 
Added:
>
>
Since we are talking about parameters, let me give you some hints:
 
Added:
>
>
Meaning of the main variables
$*   Set of all parameters (similar to $@)  
  Variable     Meaning
$0   Name of the program
$#   Amount of informed parameters
 
Added:
>
>
  • Examples
Making changes on the program teste, in order to use the variables we have just seen. Let's do it this way:
 
Changed:
<
<
As I've said before, the parameters are numbered from 1 to 9, but that does not mean that it is not possible to use more than 9 parameters. Let's test it
>
>
%TERMINALon% $ cat teste%OUTon% #!/bin/bash # Program to test how to inform the parameters (2nd Version) echo The program $0 received $# parameters echo "1o. parm -> $1" echo "2o. parm -> $2" echo "3o. parm -> $3" echo Todos de uma só \"tacada\": $*%OUToff% %TERMINALoff%
 
Changed:
<
<
Example:
>
>
Note that preceding the inverted commas I inserted a inverted slash, in order to tell Shell not to interpret them. Let's run the program.
 
Added:
>
>
%TERMINALon% $ teste informing parameters to test%OUTon% The program teste received 4 parameters 1o. parm -> informing 2o. parm -> parameters 3o. parm -> to Todos de uma "tacada": informing parameters to test%OUToff% %TERMINALoff%
 
Added:
>
>
As I've said before, the parameters are numbered from 1 to 9, but that does not mean that it is not possible to use more than 9 parameters. Let's test it:
 
Added:
>
>
  • Example:
 
Added:
>
>
%TERMINALon% $ cat teste%OUTon% #!/bin/bash # Program to test how to inform the parameters (3rd Version) echo The program $0 received $# parameters echo "11th parm -> $11" shift echo "2nd parm -> $1" shift 2 echo "4th Parm -> $4"%OUToff% %TERMINALoff%
  Let's run it now:
Changed:
<
<
>
>
%TERMINALon% $ teste informing parameters to test%OUTon% The program teste received 4 parameters 11th parm -> informing1 2nd parm -> parameters 4th parm -> test%OUToff% %TERMINALoff%
  There are two remarkable points about this script:
Changed:
<
<
1- In order to show that the parameters range from $1 to $9, I wrote an echo $11 and what happened? It was interpreted as a $1 followed by the character 1, and the result was just1; 2- The command shift, whose syntax is shift n (in which n is a variable that can assume any numerical value - although its default is 1), does not consider the first n parameters, making the first parameter the one numbered n+1.
>
>
  1. In order to show that the parameters range from $1 to $9, I wrote an echo $11 and what happened? It was interpreted as a $1 followed by the character 1, and the result was informing1;
  2. The command shift, whose syntax is shift n (in which n is a variable that can assume any numerical value - although its default is 1), does not consider the first n parameters, making the first parameter the one numbered n+1.
 
Changed:
<
<
Well, now that you know a little bit more about informing parameters, let's return to our CD Library and create our script for including CDs on bank called music. It is a very simple script (as simple as everything else in shell) and I'll list you so that you can see:

Examples:

>
>
Well, now that you know a little bit more about informing parameters, let's return to our CD Library and create our script for including CDs on bank called musics. It is a very simple script (as simple as everything else in Shell) and I'll list you so that you can see:
 
Added:
>
>
  • Examples:
 
Added:
>
>
%TERMINALon% $ cat musinc%OUTon% #!/bin/bash # Cadastra CDs (Version 1) # echo $1 >> musics%OUToff% %TERMINALoff%
 

Revision 229 Nov 2005 - JulioNeves

Line: 1 to 1
 

Pub Talk Part II



Changed:
<
<
     - Waiter, get me a pint and don't worry about my lad over here, he's finally getting to meet a real operating system and he's got a lot to learn!
>
>
'Waiter, get me a pint and don't worry about my lad over here, he's finally getting to meet a real operating system and he's got a lot to learn!'
 
Changed:
<
<
     - So my friend, could you get anything of what I've said so far?
>
>
'So my friend, could you get anything of what I've said so far?'
 
Changed:
<
<
     - Well, I can get what you mean, but I actually don't see what's the point of it.
>
>
'Well, I can get what you mean, but I actually don't see what's the point of it.'
 
Changed:
<
<
     - Take it easy pal! We're just begining... What I've said so far is a taste of what lies ahead. As soon as we start developing structured programs, you'll see how useful those tools can be. After learning that, you'll see how easy it is to reach the top shellves. Now, tell me: how do you like the grep family?
>
>
'Take it easy pal! We're just begining... What I've said so far is a taste of what lies ahead. As soon as we start developing structured programs, you'll see how useful those tools can be. After learning that, you'll see how easy it is to reach the top shellves. Now, tell me: how do you like the grep family?'
 
Changed:
<
<
     - Pardon me! I don't know any grep family'
>
>
'Pardon me! I don't know any grep family'
 
Changed:
<
<
     - Sure, sure... grep is an an acronym for "global regular expression print" - although there is a legend that tells that the name grep comes from ed (a text editor that is vim's grampa), in which the search command was g/_regular expression_/p, or g/_re_/p.
>
>
'Sure, sure... grep is an an acronym for "global regular expression print" - although there is a legend that tells that the name grep comes from ed (a text editor that is vim's grampa), in which the search command was g/_regular expression_/p, or g/_re_/p.'
 
Changed:
<
<
     - Well, this grep command takes regular expressions and matches them to the lines of an "input". By the way, there is this guy - Aurélio Marinho Jargas - who maintains a webpage that can give you all the hints, clues and even tutorials you want about regular expressions (regexp). If you feel like learning to program in Shell, Perl, Python, etc. you're better to see what he's got!
>
>
'Well, this grep command takes regular expressions and matches them to the lines of an "input". By the way, there is this guy - Aurélio Marinho Jargas - who maintains a webpage that can give you all the hints, clues and even tutorials you want about regular expressions (regexp). If you feel like learning to program in Shell, Perl, Python, etc. you're better to see what he's got!'
 

The great grep

Changed:
<
<
     - Waiter, this time I'll try a caipirinha - the Brazilian National Drink - (See how to prepare it)!
>
>
'Waiter, this time I'll try a caipirinha - the Brazilian National Drink - (See how to prepare it)!'
 
Changed:
<
<
     - So, I told you that grep matches regular expressions to the lines of an "input". But what are those "inputs"? Well, there are different ways of defining those inputs. Let's see!'
>
>
'So, I told you that grep matches regular expressions to the lines of an "input". But what are those "inputs"? Well, there are different ways of defining those inputs. Let's see!'
  Searching a file:
Line: 47 to 47
 $ grep '^rafael' /etc/passwd %TERMINALoff%
Changed:
<
<
     - Hold on, hold on... what's that caret (circumflex ^) and those apostrophes for?
>
>
'Hold on, hold on... what's that caret (circumflex ^) and those apostrophes for?'
 
Changed:
<
<
     - The caret (^), as you'd know if you had read the other articles on regular expressions I told you about, constrains the matches to the begining of the lines and the apostrophes (') tell grep not to understand that circumflex, in order to be searched for.
>
>
'The caret (^), as you'd know if you had read the other articles on regular expressions I told you about, constrains the matches to the begining of the lines and the apostrophes (') tell grep not to understand that circumflex, in order to be searched for.'
  The 2nd example will list all the lines of all the files with the extension .sh that have the world grep. Since I use this extension to my Shell scripts, what I've done is to look for a good grep example in all my scripts.
Line: 77 to 77
 And I must say: I don't like that solution. %ATTENTIONoff%
Changed:
<
<
     - Now that you know the differences among the tree, tell me: What do you think about the examples I gave before the explanation?
>
>
'Now that you know the differences among the tree, tell me: What do you think about the examples I gave before the explanation?'
 
Changed:
<
<
     - I thought fgrep would solve your problem a lot faster than grep.
>
>
'I thought fgrep would solve your problem a lot faster than grep.'
 
Changed:
<
<
     - Perfect!! I see you got what I said! Let's see some other examples to make their differences even clearer.
>
>
'Perfect!! I see you got what I said! Let's see some other examples to make their differences even clearer.'
 
  • Examples
Line: 150 to 150
  and the search would be faster.
Added:
>
>
Building a CD Library 'Let me use a nice and didactic example: the process of building a CD Library. Keep in mind that it is as possible to develop software to organize audio CDs, as it is to data CDs (including those you get when you buy magazines, those you burn for yourself, etc.).'

'Hold on a sec. Where am I taking the CD data from?'

'Firstly I'll show you how your software can obtain data from those who are using it, afterwards I'll show you how to get data from the screen or from a file.'

Informing the Parameters

'In our case, the layout of a music file will be:'

name of the album^artist~name of the song:..:singer of the song

As you can see above, a circumflex (^) separates the name of the album from the rest of the register (which contains information on each song and on its singer). The artist and the name of the song are separated by a tilde (~), and a colon (:) separates name of the song and name of the singer.

The software I'm intended to develop is called musinc, and it will include registers on my music file. I will inform the content of each album as a parameter whenever I run the software, this way:

That way, the software musinc will get data from each album as if it were a variable. The only difference between a received parameter and a variable is that the first one gets numerical names (I know it sounds strange... what I meant was that they get one character names), such as $1, $2, $3, ..., $9. Let's make a test:

Examples

Let's run it now:

OOPS, there is a detail I've forgotten: we have to make the file executable before running it:

Interestingly, the last word 'test' was not considered by our program. That is because the program just considered the three first parameters. Let's execute it another way:

The double inverted commas tell our program not to consider the blank space between the two first words, making it consider them as a single parameter.'

Parametric Hints 'Since we are talking about parameters, let me give you some hints:

Meaning of the main variables variable Meaning $0 Name of the program $# Amount of informed parameters $* Set of all parameters (similar to $@)

Examples Making changes on the program test, in order to use the variables we have just seen. Let's do it this way:

Note that preceding the inverted commas I inserted a inverted slash, in order to tell shell not to interpret them. Let's run the program.

As I've said before, the parameters are numbered from 1 to 9, but that does not mean that it is not possible to use more than 9 parameters. Let's test it

Example:

Let's run it now:

There are two remarkable points about this script:

1- In order to show that the parameters range from $1 to $9, I wrote an echo $11 and what happened? It was interpreted as a $1 followed by the character 1, and the result was just1; 2- The command shift, whose syntax is shift n (in which n is a variable that can assume any numerical value - although its default is 1), does not consider the first n parameters, making the first parameter the one numbered n+1.

Well, now that you know a little bit more about informing parameters, let's return to our CD Library and create our script for including CDs on bank called music. It is a very simple script (as simple as everything else in shell) and I'll list you so that you can see:

Examples:

Since it is a is very functional script, I'll simply attach the received parameter at the end of the file songs. Let's include 3 albums and see if it works (in order to simplify, I'll suppose each album contains just 2 songs):

Listing the content of songs.

It is not as functional as it was supposed to be... it could be a lot better. The albums are out of order, complicating the research. Let's change the script and test it again:

Including another one

Now let's see what happens to the song file:

I simply inserted a line that classifies the file musicas, pointing the output to the same file (that's how the option -o works), after attaching each album.

WOW! Now it is nice and almost functional. But attention and don't panic! That is not the final version. The next version of the program will be a lot better and more friendly! We'll develop it as soon as we learn how to get data from the screen and how to format the input.

Examples Listing with the cat command is totally out, let's make a program called muslist that lists the album whose name is given as parameter:

Let's run it looking for album 2. As we have previously seen, when informing the sequence of characters 'album 2', it is necessary to prevent shell from interpreting it (otherwise it would read two parameters). Let's try the following:

'

'What a mess!! Where is the mistake? I put the parameter between inverted commas so that shell would not split it into two...'

'Yeap, but pay attention to how grep is running:

Even putting album 2 between inverted commas, when shell sees $1 it splits it into two arguments. So, the final content of the line that grep has executed is:

As the grep syntax is:

grep  [arq1, arq2, ..., arqn]

Grep has understood that it was supposed to look for the chain of characters 'album' on the files arquivos 2 and musicas. But, since there is no arquivo 2, an error has occurred. Moreover, since the word album was found in every register of musicas, all registers were listed.

HINT Use inverted commas whenever there is a blank space or a TAB in the chain of characters that grep will run. That helps the words after the blank space or TAB from being interpreted as file names.

On the other side, it is better not to consider the case of the letters in the research. The following program would solve two problems at the same time:

In that case, the option -i tells grep not to consider the case of the letters. Another point is the option "$1" that was inserted between inverted commas so that grep would understand the chain of characters as a single argument.

Pay attention too to the fact that grep locates the chain of characters in any position of the register, so, this way we can search for album, song, singer or even for pieces of information. As soon as we get started with conditional commands, we'll get a new version of muslist that asks us in which of the fields the research will be performed.'

'Hold on pal! That putting between inverted commas thing is not really a friendly way of doing that...'

'You are right! Let me show you another way, then:

The option "$*" stands for all parameters, and in that program it will be substituted by the chain album 2 (according to the previous example), and it will do what you wanted it to.

You should have realized by now that the problem about shell is not how to do something, but what is the best way of doing it (as you've seen, the range of options is huge!).'

'But what if I have to exclude a CD? Once I forgot a CD of mine under the sun and when I looked at it again... it was lost. What if that happened again?'

'Well, let's make another script called musexc, in order to solve that kind of problem.

Before developing it, I'd like to introduce you to a very useful option of the grep family. Meet the option -v. This option lists every input register, but the ones found by the command. Let's see the example:

As I've mentioned, that grep from the example lists all the registers but the ones that refer to album 2, and that happens because it fits into the parameters of the command. Now we are ready to develop the script that will remove the lost CD from your CD Library. It looks like this:

The first line sends the file musicas to /tmp/mus$$, but extracting the registers that conform to the grep's research. Afterwards, it moves (or renames, if you prefer this word) /tmp/mus$$ to musicas.

I used the file /tmp/mus$$ as a work copy, because, as I've mentioned previously, the $$ contains the PID (Process Identification), because of that, when others edit the file musicas, a different work copy will be made, and that avoids running over other's files.'

'And that's it?'

'Yeah, man! Well, those programs we've made are quite basic, because we still lack knowledge about some tools. But, while I have another pint, you can practice using the examples, and I promise you will develop a nice control system for your CDs.

Next time we meet, I'll show you how conditional commands work and we'll improve those scripts.'

'That's it for now... but before:

Waiter, another round for me and my pal, please!'

 %CONSTRUCAOon% Sorry, but in a couple of days the whole text will be "TWiked" %CONSTRUCAOoff%

Revision 117 Nov 2005 - JulioNeves

Line: 1 to 1
Added:
>
>

Pub Talk Part II



     - Waiter, get me a pint and don't worry about my lad over here, he's finally getting to meet a real operating system and he's got a lot to learn!

     - So my friend, could you get anything of what I've said so far?

     - Well, I can get what you mean, but I actually don't see what's the point of it.

     - Take it easy pal! We're just begining... What I've said so far is a taste of what lies ahead. As soon as we start developing structured programs, you'll see how useful those tools can be. After learning that, you'll see how easy it is to reach the top shellves. Now, tell me: how do you like the grep family?

     - Pardon me! I don't know any grep family'

     - Sure, sure... grep is an an acronym for "global regular expression print" - although there is a legend that tells that the name grep comes from ed (a text editor that is vim's grampa), in which the search command was g/_regular expression_/p, or g/_re_/p.

     - Well, this grep command takes regular expressions and matches them to the lines of an "input". By the way, there is this guy - Aurélio Marinho Jargas - who maintains a webpage that can give you all the hints, clues and even tutorials you want about regular expressions (regexp). If you feel like learning to program in Shell, Perl, Python, etc. you're better to see what he's got!

The great grep

     - Waiter, this time I'll try a caipirinha - the Brazilian National Drink - (See how to prepare it)!

     - So, I told you that grep matches regular expressions to the lines of an "input". But what are those "inputs"? Well, there are different ways of defining those inputs. Let's see!'

Searching a file:

%TERMINALon% $ grep mary /etc/passwd %TERMINALoff%

Searching more than one file:

%TERMINALon% $ grep grep *.sh %TERMINALoff%

Searching the output of a command

%TERMINALon% $ who | grep pelegrino %TERMINALoff%

Considering the 1st example - which is the simplest one - I searched the occurrences of the word mary in any position of the file /etc/passwd. If I wanted to search it as a login name - or, in other words, just at the begining of the registers of that file - I should execute:

%TERMINALon% $ grep '^rafael' /etc/passwd %TERMINALoff%

     - Hold on, hold on... what's that caret (circumflex ^) and those apostrophes for?

     - The caret (^), as you'd know if you had read the other articles on regular expressions I told you about, constrains the matches to the begining of the lines and the apostrophes (') tell grep not to understand that circumflex, in order to be searched for.

The 2nd example will list all the lines of all the files with the extension .sh that have the world grep. Since I use this extension to my Shell scripts, what I've done is to look for a good grep example in all my scripts.

And look!! grep accepts as input the output of another command, as long as it is indicated by a pipe symbol (|) - this is very common in shell and it accelerates enourmously the execution of commands, since it takes the output of a command and reads it as if it were a file.

So, looking at the 3rd example, the command who lists the users who are logged in the same machine as you are (remember: Linux is a multi user system) and the command grep verifies whether the user pelegrino is working or not.

The grep family

You know, the command grep is widely known, because it is frequently used, but what most people don't know is that there are three commands in the grep family. They are:

  • grep
  • egrep
  • fgrep

Their main features are:

  • grep
    Can (or cannot) use simple regular expressions, but when it is not the case of using them, it is better to execute fgrep (it is faster);
  • egrep ('e' standing for extended)
    Is a very powerful tool that uses regular expressions. It is often seen as the slowest brother of the grep family, hence it is more likely to use it when it is necessary to elaborate a regular expression that grep does not accept;
  • fgrep ('f' standing for fast, or file)
    As its own name points out, is the fast brother of the family. It is fast running (it is about 30% faster than grep and 50% faster than egrep), but it is does not allow the use of regular expressions

%ATTENTIONon% The considerations above on speed are valid to the Unix grep family. grep is faster running on Linux, because the other two (fgrep and egrep) are shell scripts that execute grep.

And I must say: I don't like that solution. %ATTENTIONoff%

     - Now that you know the differences among the tree, tell me: What do you think about the examples I gave before the explanation?

     - I thought fgrep would solve your problem a lot faster than grep.

     - Perfect!! I see you got what I said! Let's see some other examples to make their differences even clearer.

  • Examples

I know that there is a text talking about Linux, but I'm not quite sure on whether the word Linux is written with a capital L or with a small one, what should I do?

There are two options in that case:

%TERMINALon% $ egrep (Linux | linux) arquivo.txt %TERMINALoff%

or

%TERMINALon% $ grep [Ll]inux arquivo.txt %TERMINALoff%

In the first case, the complex regular expression (Linux | linux) uses the parentheses to group up the options and the pipe (|) as a logical "or", which means that you are searching Linux or linux.

In the second case, on the other hand, the regular expression [Ll]inux means that you are searching a word that starst with L or l followed by inux. Since this expression is simpler, grep itself can solve it, so I think it is a more recomendable one (remember: egrep is slower).

Another example. If you want to list the subdirectories of a directory, you should run:

%TERMINALon% $ ls -l | grep '^d'%OUTon% drwxr-xr-x 3 root root 4096 Dec 18 2000 doc drwxr-xr-x 11 root root 4096 Jul 13 18:58 freeciv drwxr-xr-x 3 root root 4096 Oct 17 2000 gimp drwxr-xr-x 3 root root 4096 Aug 8 2000 gnome drwxr-xr-x 2 root root 4096 Aug 8 2000 idl drwxrwxr-x 14 root root 4096 Jul 13 18:58 locale drwxrwxr-x 12 root root 4096 Jan 14 2000 lyx drwxrwxr-x 3 root root 4096 Jan 17 2000 pixmaps drwxr-xr-x 3 root root 4096 Jul 2 20:30 scribus drwxrwxr-x 3 root root 4096 Jan 17 2000 sounds drwxr-xr-x 3 root root 4096 Dec 18 2000 xine%OUToff% %TERMINALoff%

As you can see above, the circumflex (^) limits the search to the first position of the long output of the ls command. The apostrophes tell the shell not to 'understand' the circumflex (^).

Let's take another example. You know what are the first four positions of the output of a ls -s command for an ordinary file (not a directory, nor a link, nor anything...) should be:

      -
 Position    1st     2nd     3rd   4th
  Possible values   - r w x
  - -   s (suid)  

Thus, in order to find out what are the executable files in a directory, you should:

%TERMINALon% $ ls -la | egrep '^-..(x|s)'%OUTon% -rwxr-xr-x 1 root root 2875 Jun 18 19:38 rc -rwxr-xr-x 1 root root 857 Aug 9 22:03 rc.local -rwxr-xr-x 1 root root 18453 Jul 6 17:28 rc.sysinit%OUToff% %TERMINALoff%

Once again the caret (^) limits the search to the begining of each line, hence, the listed occurrences are the ones that start with a -, followed by anything (the full stop - a dot - in a regular expression denotes any character), once again followed by any character, followed by an x or a s.

The same result would be found with the command:

%TERMINALon% $ ls -la | grep '^-..[xs]' %TERMINALoff%

and the search would be faster.

%CONSTRUCAOon% Sorry, but in a couple of days the whole text will be "TWiked" %CONSTRUCAOoff%

 
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Wiki-SL? Send feedback