• About different Linux and Unix shells. UNIX command shell

    =====================================================

    LABORATORY WORK No. 1 (completion time - 3 hours)

    Topic: "Introduction toshell

    1. Introduction to shell

    Shell is a command interpreter (command processor) that acts as an interface between the user and the Unix kernel. Solaris offers three main command processors:

      The default Bourne shell is /sbin/sh

      C Command Processor -- /bin/csh

      Korn shell-- ./bin/ksh

    In addition to those listed, Solaris provides additional shells:

      J shell -- /sbin/jsh

      Restricted Korn l -- / usr/ bin/rsh

      T-shell -- /usr/bin/tcsh

      GNU Bourne Again -- /usr/bin/bash

      Z-shell – usr/bin/zsh

    The choice of command processor is a matter of personal preference.

    2. Run commands

    Any command in Unix consists of the name of the program to be launched (command), options (keys) and arguments passed to the program. The command line is delimited by spaces or tabs.

    Example

    $ /bin/ping –t 10 192.168.2.13

    argument

    shell invitation

    If, when starting the program, the path to executable file, then the OS sequentially scans the directories specified in the PATH variable. If there is a program with the same name in the directory being viewed, then it is launched for execution. The current working directory is not searched for programs unless it is specified in the PATH variable.

    ! For security reasons, it is recommended that the administrator not add the user's current directory and home (personal) directories to the PATH variable.

    Examples

    Run a program specifying the full path

    $ / usr/ local/ bin/ my_ program

    Running a program from a higher (parent) directory

    $ ../ local/ bin/ my_ program

    Running a program from the current directory

    $ ./ my_ program

    3. Special characters in shell

    Some shell characters have special meaning.

    3.1. File name generation templates

    ? (question mark) matches any one character except the first dot;

    (square brackets) define a group of symbols (one symbol from the group is selected);

    - (minus sign) defines the range of valid characters;

    ! (exclamation mark) rejects the following group of characters;

    * (asterisk symbol) matches any number of characters except the first dot.

    3.2. I/O redirection symbols

    < - input redirection;

    >, >> - output redirection;

    2>, 2>> - redirection of error messages;

    | - conveyor.

    3.3. Substitution characters

    $name_variableq - variable substitution;

    $(command) or `team` - command substitution;

    ~ - tilde substitution.

    4. Search

    For quick search files and directories, use the whereis command; for deeper ones, use the find command (search for files that meet the specified requirements;).

    5. Text editor vi

    The text editor vi is a universal editor present in any Unix. In addition to editing text files, the vi editor can be used to edit the command line.

    The vi editor has 3 operating modes (see figure).

    Logging into vi editor

    Exit vi

    The “~” (tilde) sign in the first position marks empty (non-existent) lines of the file.

    The main mode is command mode, in which commands are entered by pressing a certain sequence of keys (they are not displayed on the screen in any way).

    Moving through text in command mode is done using arrows and keys (left), (down), (up), (to the right).

    Some vi commands are listed in the table.

    Deleting the current line

    Copying the current line to the buffer

    Cursor movement

    Entering characters marked by the cursor into the buffer

    Insert new line from below

    Inserting a new line at the top

    Adding after the cursor

    Append after current line

    Insert before cursor

    Insert before current line

    Replacing a character with a character entered from the keyboard

    Pasting text from the clipboard

    . (dot)

    Repeat the last command executed

    Cancel last command

    Undo all changes to the current line

    Deleting a character above the cursor

    In input mode, all characters typed on the keyboard are inserted into the text.

    In mode last line the command you enter is displayed on the last line of the screen.

    6. Help

    Unix has an electronic directory containing a description of the purpose of commands, their syntax, examples of use, etc. The directory is called by the man command:

    7. Basic commands shell

    who– displaying information about active users;

    echo– output messages to the terminal;

    banner– output messages to the terminal in capital letters;

    man– calling the online help system;

    date – output of the current date;

    write– transfer of messages to another user’s terminal;

    mesg– allowing/prohibiting the display of messages from other users;

    mail– sending/receiving mail;

    news– familiarization with system news;

    pwd– displaying the absolute route name of the current working directory;

    CD– changing the working directory;

    ls– displaying information about the contents of the directory;

    mkdir– creating a catalog;

    rmdir– deleting a directory;

    touch– updating the file timestamp;

    cp– copying files;

    cat– combining and displaying the contents of files;

    more– page-by-page viewing of file contents.

    mv – moving or renaming a file;

    rm– file deletion;

    alias– creating a pseudonym;

    tr– character conversion;

    exit– completion of the current shell-A;

    tee– interception of pipeline results;

    cut – selecting fields from a row;

    grep– search by template;

    pr– output the file to standard output in a specified format;

    sort– sorting;

    head– output of the first lines of the file;

    tail– output of the last lines of the file;

    wc– counting the number of characters, words and lines;

    read, echo - reading and outputting variable values;

    test- evaluating the meaning of an expression;

    expr, let- calculation of arithmetic expressions;

    8. Shell-programs

    Shell allows you to save a sequence of commands in a file and then execute it. To run a file with a shell program for execution, you need to add the execute right to the access rights:

    $ chmod +x program_file_name

    Branching in shell programs is organized using the following operators:

    if-then-else

    Loop operators:

    EXERCISES

    1. Which of the following commands will result in a welcome message being displayed on the screen? Which ones are not? Why?

    $echo hello

    $ echo hello

    $ echo HELLO, WORLD

    $ banner hello

    $ BANNER HELLO, WORLD

    2. Print a message over multiple lines using commands echo And banner.

    3. Print the date in two lines: on the first line day, month, year, on the second line current time, providing the output with a comment.

    4. Using the command write, send a message to the console. Send messages to multiple terminals simultaneously.

    5. Using the command mesg, determine whether messages are allowed on your terminal. Ban messages. What will the system's reaction be if someone tries to send you a message?

    6. Determine your name HOME- catalogue.

    7. Browse the directory subtree starting from the /export/home directory using the commands cd, ls And pwd.

    8.. Create in your HOME-directory subdirectories of the form:

    fruit flowers

    apple pear grapes rose violet dandelion

    dark green

    9. Being in your own HOME-directory, create the following subdirectories using one command line:

    A/B/C/D

    10. Being in your own HOME-directory, delete all subdirectories of the directory A.

    11. While in HOME-directory, create a file macintosh in the existing catalog apple and several files in directories dark And green. Login to the directory flowers. While in the directory flowers, copy all subdirectories fruits together with the files contained in them into a specially created directory basket.

    12. While in the directory flowers, delete the directory fruits.

    13. Read the file .profile using commands cat And more.

    14. Create in HOME-directory text file myfile from multiple lines using the command cat. Create a text file MyFile, by writing the same lines into it. How many files did you get? Why?

    15. View the contents of the file created in task 2.2 myfile. Copy the file myfile to file mycopy. View the contents of both files.

    16. Move the file mycopy to the catalog flowers.

    17. While in HOME-catalog, create a link mylink per file mycopy located in the directory flowers. View the link file.

    18. Add a line to the file mylink. Which of the files mylink, mycopy, myfile changed? Why?

    19. Using what command can you determine the number of links? Determine the number of links for files mylink, mycopy, myfile.

    Delete the file mycopy. What happened to the link file? Determine the number of links for files mylink, myfile.

    20. Create an alias dir, which prints the contents of the current directory in an extended format.

    21. Log out and log in again. Does alias work? dir? What should be done to ensure that the nickname is not lost between sessions?

    22. Create an alias point, which prints a list of files in the working directory whose names begin with a dot.

    23. Using the command touch, create files in a new directory with names such that at the same time:

    Template a* 5 files matched;

    Template * a corresponded to 4 files;

    Template ??.? matched 3 files;

    Template * aa* corresponded to 2 files;

    Template??? matched 1 file.

    24. Which command should you enter to do the following:

    a) display the names of all files starting with a dot;

    b) display the names of all files ending with “.txt”;

    c) display the names of all files containing the word “my”;

    25. Replace all lowercase letters in one given file with uppercase ones, and in another, all uppercase letters with lowercase ones. Eliminate any duplicate spaces from the file.

    26. Create a file named * . Delete only this file. Be careful when using file name generation characters!

    27. Create a file that has a space in its name. How to delete such a file?

    28. Using the command cat double the contents of a file by appending the original contents to the end of the same file. Go to the directory flowers. Add to file spisok directory content list flowers. View the contents of the file

    29. Try to read using the command cat non-existent file. What is the system's response? Do the same thing by redirecting error messages to a file myerror. What do you see on the screen? View the file myerror.

    30. Create a pipeline to list only the names and permissions of files that are in at the moment are in your working directory.

    31. Change the built pipeline so that the list is saved in a file spisok yours HOME-directory, and only the number of files in the list was displayed on the screen.

    32. Display the contents of a file /etc/passwd, sorted by username field.

    33. Create an alias loggedon, which will display an alphabetically ordered list of names of users working on the system.

    33. Display the current date in capital letters using commands date And banner.

    34. Assign to a variable I.A.M. Your registration name. Launch another one shell. Do you see this variable? What do you need to do to see it in the generated shell? Change the value of a variable I.A.M. in the generated shell. Come out of the begotten shell. Look at the value of this variable in the source shell. Explain the result.

    35. Write shell-program info, which will ask the user to enter name, address, day, month and year of birth and display this information in the reverse order entered

    36. Write shell- a program that will display a prompt for entering a number and store the entered number in a variable Y, and print a message “Y is greater than 7”, if value Y more than 7, and “Y isnot greater than 7” otherwise.

    37. Write a shell program that displays the following statistics:

    a) your name;

    b) the number of arguments with which it is launched;

    c) prints each of its arguments and the length of the argument in characters;

    38. Write shell- a program that determines the number of arguments in command line and produces an error message if the number of arguments is not equal to three, or the arguments themselves if their number is equal to three.

    39. Write shell- a program that will prompt you to enter a user ID, check the ID against those used in the system, and display the full name HOME-directory or, in the case of an invalid identifier, display an error message.

    40. Write a shell program hello, which provides the following response to command line arguments:

    Argument “-d”- the program will execute the command date;

    Argument “-l”- the program will display the contents of the current directory;

      If there are no arguments or invalid arguments on the command line, the program will print help about its options.

    41. Write a program words, which will prompt the user to enter one word at a time until he enters the word end. Remember all the words you entered. After entering a word end display all entered words on the screen.

    42. Change the system prompt so that it contains the full path name of the current working directory .

    43. Write a program virus, which creates an executable copy of itself with a different name and then deletes itself.

    44. Write a program virus2 , which searches the current directory for command language programs and adds to their text a command to display the word Infected!”.

    45. Write a program virus3 , which adds its own code to the command language programs it finds to infect other programs.

    46. ​​Write a program virus4 , by modifying your virus so that when infected, the infecting code is removed from the infecting program.

    47. Write a program antivirus, which would find all shell programs infected with your virus.

    48. Modify the developed program so that it not only finds infected programs in the directory specified to it, but also “cures” them by saving the infected version in a new file with the ending added to its name . vir, and removing the execution attribute from such a file.

    49. Write a program virus5 , which would:

    a) would infect shell programs in the current directory and its subdirectories with a virus only if the user entered the command ls;

    b) would behave like a team ls, without revealing anything about your work on the terminal.

    50. Write a virus6 program based on the programs virus2 -virus5 , the infecting code of which could not be detected by the program you developed antivirus, and infection would occur for any value of the variable PATH.

    51. Write a program supervirus, the launch of which infects your files with the virus developed in the previous task, and every time you log into the system, an attempt is made to infect the files of your friends. Program supervirus erases itself after the first launch.

    52. Write a program superantivirus, which detects and completely “cures” (if the appropriate option is specified) all files in the specified directory and its subdirectories from all developed viruses.

    Command shell in UNIX is the command line interface in Unix-like operating systems, that is, it executes commands that the user issues or that are read from files. Such files with shell commands are called shell scripts (scripts, programs). These scripts are not compiled, but rather interpreted by the shell. This means that the shell reads the script from start to finish, line by line, looking for the commands specified there and executing them; In contrast to this approach, the compiler converts the entire program into a form suitable for machine execution - the file with this code can then be used in a shell script. Feature shell language - many operations that are built-in in traditional programming languages ​​are performed by calling external programs.

    One of the purposes of the shell is to provide a user environment that can be customized using configuration files.

    Types of shells

    UNIX provides several different shells to choose from:

    sh, or Bourne shell ( Bourne Shell) one of the first shells to be used in UNIX-like environments. This is a basic shell, small and with a small set of capabilities. It is the de facto standard shell, and is present on every UNIX system. On Linux, /bin/sh can be a symbolic link to bash. This is done to ensure compatibility with UNIX applications. bash, or the new Bourne shell ( Bourne Again Shell) standard Linux shell. Most Linux users have the default shell installed bash. In a sense bash- this is an add-on to sh, a set of add-ons and additional modules. Thus, the new Bourne shell is compatible with the regular Bourne shell: commands running in sh, will work in bash, but not necessarily the other way around. csh, or C-oriented shell ( C Shell) the syntax of this shell is similar to the C programming language. Typically, this shell is used by programmers. tcsh, or an improved C-oriented wrapper ( Turbo C Shell) superstructure over csh. ksh, or Korn shell ( Korn Shell) Was written in the early 1980s and received the best features of the Bourne shell and csh at that time. It is an add-on for the Bourne shell and guarantees the execution of scripts written for the Bourne shell. The POSIX standard is a limited-capability Korn shell.

    The /etc/shells file contains information about the shells present on the system:

    Mia: ~> cat /etc/shells /Bin/bash /Bin/sh /Bin/tcsh /Bin/csh

    The user shell is installed in the /etc/passwd file. For example, user string mia which has a shell installed bash :

    Mia: L2NOfqdlPrHwE: 504:504: Mia Maya:/home/mia:/bin/bash

    Switching between shells

    The user can launch another shell at any time. For example, if by default it works under the controls bash, but it’s more convenient for him to perform some actions under the shell csh, it launches this shell

    Mia:~>csh

    To return to control bash you need to type exit.

      wish (UNIX shell)- This term has other meanings, see Wish. wish (Windowing Shell) is a simple scriptable or interactive UNIX command shell for the X Window System and Mac OS X. It provides users with control over components... ... Wikipedia

      Command shell- Operating system shell (from English. shell) an operating system (OS) command interpreter that provides an interface for user interaction with system functions. In general, there are shells with two types of interface for ... Wikipedia

      UNIX shell

      Unix shell- UNIX command shell (English Unix shell, often just “shell” or “sh”) command interpreter, used in operating systems of the POSIX family of compatible shells, dating back to the Bourne shell, which appeared in Unix Version 7. Contents 1 ... ... Wikipedia

      Unix Shell- UNIX command shell (English Unix shell, often just “shell” or “sh”) a command interpreter used in operating systems of the POSIX family of compatible shells, dating back to the Bourne shell, which appeared in Unix Version 7. Contents 1 ... ... Wikipedia

      Command line- The Command Line Interpreter request is redirected here. A separate article is needed on the topic “Command Line Interpreter”. Appearance shells (eng. Co ... Wikipedia

      UNIX System V- Many descendants of System V... Wikipedia

      Unix System V- Many descendants of System V AT T UNIX System V is one of the versions of the operating system in 1989. 4 major versions were released, Releases 1, 2, 3 and 4. System V Release 4 (SVR4) was the most successful and popular and served as the source for ... ... Wikipedia

      UNIX - Family tree UNIX systems UNIX (read Unix) is a family of portable, multitasking and multi-user ... Wikipedia

      Unix- Family tree of UNIX systems UNIX (read Unix) is a group of portable, multitasking and multi-user operating systems. First UNIX system was developed in 1969 in the Bell Labs division of AT T. Since then it has been created ... Wikipedia

    Books

    • Shell scripts. Linux, OS X and Unix. Guide, Taylor Dave. Shell scripts help system administrators and programmers to automate routine tasks ever since the first computers appeared. Since the release of the first one... Buy for 1217 UAH (Ukraine only)
    • Shell scripts. Linux, OS X and Unix, Taylor Dave, Perry Brandon. Shell scripts have helped system administrators and programmers automate routine tasks since the dawn of computers. Since the release of the first...
    • Tutorial

    Why and for whom is the article?

    Initially, this was a reminder for students who are starting to work with Unix-like systems. In other words, the article is intended for those who have no previous experience working with the Unix command line, but for one reason or another want or need to learn how to interact effectively with it.

    There will be no retelling of mana (documentation), and the article does not in any way cancel or replace reading them. Instead, I will talk about the main things (commands, techniques and principles) that you need to understand from the very beginning of working in the unix shell in order for the work to be effective and enjoyable.

    The article concerns full-fledged unix-like environments, with a fully functional shell (preferably zsh or bash) and a fairly wide range of standard programs.

    What is shell

    Shell (shell, aka “command line”, aka CLI, aka “console”, aka “terminal”, aka “black window with white letters”) is a text interface for communicating with operating system(well, strictly speaking, this is program, which provides such an interface, but now this difference is insignificant).

    In general, work through a shell looks like this: the user (i.e. you) enters a command from the keyboard, presses Enter, the system executes the command, writes the result of the execution to the screen, and again waits for the next command to be entered.

    Typical shell type:

    The shell is the primary way to interact with all Unix-like server systems.

    Where are command line systems found?

    Where a unix shell might be waiting for you, popular options:
    • MacOS (bash);
    • remote access to the server for work or for a personal web project;
    • home file server with remote access;
    • Ubuntu, PC-BSD on laptop/desktop - unix-like systems today are easy to install and use.

    What problems are reasonable to solve with a shell?

    Natural tasks for which the shell is suitable, useful and indispensable:
    • interactive work in the terminal:
      • performing compilation, running jobs via make;
      • comparison of text files;
      • fast ad-hoc data analysis (number of unique IPs in the log, distribution of records by hours/minutes, etc.);
      • one-time mass actions (kill many processes; if you work with a version control system, reverse or resolve a bunch of files);
      • diagnostics of what is happening in the system (semaphores, locks, processes, descriptors, disk space, etc.);
    • scripting:
      • installation scripts, for which you cannot rely on the presence of other interpreters - this is not for beginners;
      • functions for customizing the interactive shell (affecting the invitation, changing the directory, setting environment variables) are also not exactly for beginners;
      • one-time scripts such as mass file recoding;
      • makefiles.

    Absolutely first steps

    Let's get started: sign in and sign out

    Make sure you know exactly how to start the shell and how to exit it.

    If you are working on a machine with Ubuntu installed, you need to launch the Terminal program. When finished, you can simply close the window.

    On MacOS - also launch Terminal.

    To access a remote server, use ssh (if you have MacOS, Ubuntu or another unix-like system locally) or putty (if you have Windows).

    Who am I, where am I?

    Run the following commands:
    • hostname - displays the name of the machine (server) you are currently on;
    • whoami - displays your login (your name in the system);
    • tree -d / |less - pseudo-graphic representation of the directory tree on the machine; exit from scrolling - q ;
    • pwd - displays the directory you are currently in; on the command line you cannot be “just like that”, you must be in some directory (=current directory, working directory). The current working directory is probably displayed in your prompt.
    • ls - list of files in the current directory; ls /home - list of files in the specified directory;

    Command history (history)

    An important property of a full-fledged command line is the command history.

    Run several commands: hostname, ls, pwd, whoami. Now press the up key. The previous command appears in the input line. You can use the up and down keys to move forward and backward through the history. When you get to hostname, press Enter - the command will be executed again.

    Commands from history can not only be executed repeatedly, but also edited. Scroll through the history to the ls command, add the -l switch to it (it turns out ls -l , there is a space before the minus, but not after). Press Enter - the modified command will be executed.

    Scrolling through history, editing and re-executing commands are the most common actions when working on the command line, so get used to it.

    Copy-paste

    The command line is very text-centric: commands are text, input data for most standard programs is text, and the output is most often text.

    The great thing about text is that it can be copied and pasted, and this is true on the command line as well.

    Try the command date +"%y-%m-%d, %A"
    Did you enter it entirely by hand or copied it from the article? Make sure you can copy it, paste it into a terminal and execute it.

    Once you've learned how to use man, make sure you can copy and run example commands from the help. To check, look up the EXAMPLES section in the date program help, copy and run the first example given (just in case: the dollar sign is not part of the command , this is a symbolic image of an input prompt).

    How exactly to copy text from the terminal and paste it into the terminal depends on your system and its settings, so give universal instructions, unfortunately, it won't work. On Ubuntu, try this: copy - just select with the mouse, paste - the middle mouse button. If it doesn’t work, or if you have a different system, look on the Internet or ask more experienced friends.

    Keys and options

    As you've explored the history of commands, you've already encountered that the ls command has at least two options. If you call it just like that, it outputs a simple list:

    Akira@latitude-e7240: ~/shell-survival-quide> ls Makefile shell-first-steps.md shell-first-steps.pdf shell-survival-quide.md shell-survival-quide.pdf
    If you add the -l switch, detailed information is displayed for each file:

    Akira@latitude-e7240: ~/shell-survival-quide> ls -l total 332 -rw-rw-r-- 1 akira akira 198 Feb 13 11:48 Makefile -rw-rw-r-- 1 akira akira 15107 Feb 14 22:26 shell-first-steps.md -rw-rw-r-- 1 akira akira 146226 Feb 13 11:49 shell-first-steps.pdf -rw-rw-r-- 1 akira akira 16626 Feb 13 11 :45 shell-survival-quide.md -rw-rw-r-- 1 akira akira 146203 Feb 13 11:35 shell-survival-quide.pdf
    This is a very typical situation: if you add special modifiers (keys, options, parameters) to a command call, the behavior of the command changes. Compare: tree / and tree -d / , hostname and hostname -f .

    In addition, commands can take file names, directory names, or simply text strings as parameters. Try:

    Ls -ld /home ls -l /home grep root /etc/passwd

    man

    man - Help with the commands and programs available on your machine, as well as system calls and the C standard library.

    Try: man grep, man atoi, man chdir, man man.

    Scrolling forward and backward is done with the “up”, “down”, “PageUp”, “PageDown” buttons, exiting the help view is done with the q button. Search for specific text in a help article: press / (forward slash), enter text to search, press Enter. Move to next occurrences - key n.

    All help articles are divided into categories. The most important:

    It is necessary to indicate from which category the certificate should be shown in cases of coincidence of names. For example, man 3 printf describes a function from the C standard library, and man 1 printf describes a console program with the same name.

    You can view a list of all help articles available on your machine using the man -k command. (the dot is also part of the komada).

    less

    When you need to view very long text in a small terminal window (the contents of a file, a long man, etc.), special “pager” programs are used (from the word page, that is, page flippers). The most popular scroller is less, and it is what provides you with the scrolling when you read man pages.

    Try and compare the behavior:

    Cat /etc/bash.bashrc cat /etc/bash.bashrc |less

    You can transfer the file to the pager directly in the parameters:

    Less /etc/bash.bashrc

    Scrolling up and down - buttons "up", "down", "PageUp", "PageDown", exit - button q. Search for specific text: press / (forward slash), enter the text to search, press Enter. Move to next occurrences - key n. (Do you recognize the instructions about man ? No wonder, less is also used to display help.)

    Rights

    Any file or directory is associated with a set of “rights”: the right to read the file, the right to write to the file, the right to execute the file. All users are divided into three categories: file owner, file owner group, and all other users.

    You can view file permissions using ls -l . For example:

    > ls -l Makefile -rw-r--r-- 1 akira students 198 Feb 13 11:48 Makefile
    This output means that the owner (akira) can read and write the file, the group (students) can only read, and all other users can also only read.

    If you receive a permission denied message while working, this means that you do not have sufficient permissions for the object you wanted to work with.

    Read more in man chmod.

    STDIN, STDOUT, conveyors (pipes)

    There are 3 standard data streams associated with each executing program: input data stream STDIN, output data stream STDOUT, error output stream STDERR.

    Run the wc program, enter the text Good day today, press Enter, enter the text good day, press Enter, press Ctrl+d. The wc program will show statistics on the number of letters, words and lines in your text and end:

    > wc good day today good day 2 5 24
    IN in this case you supplied a two-line text to the program's STDIN, but received three numbers in STDOUT.

    Now run the command head -n3 /etc/passwd , it should look something like this:

    > head -n3 /etc/passwd root:x:0:0:root:/root:/bin/bash daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin bin:x: 2:2:bin:/bin:/usr/sbin/nologin
    In this case, the head program did not read anything from STDIN, but wrote three lines to STDOUT.

    You can imagine it this way: the program is a pipe into which STDIN flows and STDOUT flows out.

    The most important property of the Unix command line is that “pipe” programs can be connected to each other: the output (STDOUT) of one program can be passed as input data (STDIN) to another program.

    Such a construction of connected programs is called a pipe in English, or a conveyor or pipe in Russian.

    Combining programs into a pipeline is done with the symbol | (vertical bar)

    Run the command head -n3 /etc/passwd |wc , it will look something like this:

    > head -n3 /etc/passwd |wc 3 3 117
    Here's what happened: the head program output three lines of text in STDOUT, which immediately went to the input of the wc program, which in turn counted the number of characters, words and lines in the resulting text.

    You can combine as many programs as you like into a pipeline. For example, you can add another wc program to the previous pipeline, which will count how many words and letters were in the output of the first wc:

    > head -n3 /etc/passwd |wc |wc 1 3 24

    Creating pipelines (pipes) is a very common task when working on the command line. For an example of how this is done in practice, read the section “Creating a one-line pipeline.”

    I/O redirection

    The output (STDOUT) of a program can not only be transferred to another program via a pipeline, but also simply written to a file. This redirection is done using > (greater than sign):

    Date > /tmp/today.txt
    As a result of executing this command, the file /tmp/today.txt will appear on the disk. View its contents using cat /tmp/today.txt

    If a file with the same name already existed, its old contents will be destroyed. If the file did not exist, it will be created. The directory in which the file is created must exist before the command is executed.

    If you don't want to overwrite a file, but rather add output to the end of it, use >> :

    Date >> /tmp/today.txt
    Check what is now written in the file.

    In addition, you can pass any file to the program instead of STDIN. Try:

    Wc

    What to do when something is unclear

    If you encounter system behavior that you don’t understand, or want to achieve a certain result, but don’t know exactly how, I advise you to proceed in the following order (by the way, this applies not only to shells):
    • As clearly as possible, formulate the question or task - there is nothing more difficult than solving “something I don’t know what”;
    • remember if you have already encountered the same or similar problem - in this case, it is worth trying the solution that worked last time;
    • read the appropriate man pages (if you understand which man pages are suitable in your case) - perhaps you will find suitable examples of using commands, the necessary options or links to other commands;
    • think: is it possible to change the task a little? - perhaps, by slightly changing the conditions, you will get a problem that you already know how to solve;
    • ask your clearly formulated question in a search engine - perhaps the answer can be found on Stack Overflow or other sites;
    If none of the above helps, seek advice from a teacher, an experienced colleague or friend. And don’t be afraid to ask “stupid” questions - it’s not a shame not to know, it’s a shame not to ask.

    If you solve a difficult problem (on your own, with the help of the Internet or other people), write down your solution in case the same problem arises again for you or your friends. You can record it in a simple text file, in Evernote, or publish it on social networks.

    Working methods

    Copy-and-paste- from man pages, from articles on StackOverflow, etc. The command line consists of text, take advantage of this: copy and use example commands, write down successful findings as a keepsake, publish them on Twitter and blogs.

    Pull the previous command from the history, add another command to the pipeline, run, repeat.Cm. See also the section “Creating a one-liner pipeline.”

    Basic commands

    • change to another directory: cd ;
    • viewing the contents of files: cat, less, head, tail;
    • file manipulation: cp, mv, rm;
    • viewing directory contents: ls , ls -l , ls -lS ;
    • directory structure: tree , tree -d (directory can be passed as a parameter);
    • search for files: find . -name ... ;

    Analytics

    • wc, wc -l;
    • sort -k - sort by the specified field;
    • sort -n - numeric sorting;
    • diff - file comparison;
    • grep , grep -v , grep -w , grep "\ " , grep -E - search for text;
    • uniq , uniq -c - string uniqueization;
    • awk - in the awk "(print $1)" option, to leave only the first field from each line, $1 can be changed to $2, $3, etc.;

    System diagnostics

    • ps axuww - information about processes (running programs) running on the machine;
    • top - interactive viewing of the most resource-intensive processes;
    • df - used and free disk space;
    • du - total size of files in the directory (recursively with subdirectories);
    • strace , ktrace - what system calls the process makes;
    • lsof - what files the process uses;
    • netstat -na, netstat -nap - which ports and sockets are open in the system.

    You may not have some programs; they need to be installed additionally. In addition, some options of these programs are available only to privileged users (root).

    Bulk and semi-automatic execution

    Skip this section at first; you will need these commands and constructs when you get to simple shell scripting.
    • test - checking conditions;
    • while read - loop line by line STDIN ;
    • xargs - substitution of strings from STDIN into parameters of the specified program;
    • seq - generation of sequences of natural numbers;
    • () - combine the output of several commands;
    • ; - do one thing after another;
    • && - execute if the first command completes successfully;
    • || - execute if the first command fails;
    • tee - duplicate the program output to STDOUT and to a file on disk.

    Miscellaneous

    • date - current date;
    • curl - downloads a document from the specified url and writes the result to STDOUT;
    • touch - update file modification date;
    • kill - send a signal to the process;
    • true - does nothing, returns true, useful for organizing eternal loops;
    • sudo - execute the command as root "a.

    Creating a one-liner pipeline

    Let's look at an example of a real task: we need to kill all task-6-server processes running as the current user.

    Step 1.
    Understand which program produces approximately the necessary data, even if not in its pure form. For our task, it is worth getting a list of all processes in the system: ps axuww. Launch.

    Step 2.
    Look at the data received with your eyes, come up with a filter that will throw out some of the unnecessary data. This is often grep or grep -v . Use the “Up” key to pull out the previous command from the history, assign an invented filter to it, and run it.

    Ps axuww |grep `whoami`
    - only processes of the current user.

    Step 3.
    Repeat step 2 until you get the clean data you need.

    "
    - all processes with the required name (plus, perhaps, extra ones like vim task-6-server.c, etc.),

    Ps axuww |grep `whoami` | grep "\ " | grep -v vim ps axuww |grep `whoami` | grep "\ " | grep -v vim |grep -v less
    - only processes with the required name

    Ps axuww |grep `whoami` | grep "\ " | grep -v vim |grep -v less |awk "(print $2)"

    Pids of the required processes, step 3 completed

    Step 4.
    Apply a suitable final handler. Using the “Up” key, we pull out the previous command from the history and add processing that will complete the solution to the problem:

    • |wc -l to count the number of processes;
    • >pids to write pids to a file;
    • |xargs kill -9 kill processes.

    Training tasks

    Want to practice new skills? Try the following tasks:
    • get a list of all files and directories in your home directory;
    • get a list of all man articles from category 2 (system calls);
    • count how many times the word grep appears in the grep program man page;
    • count how many processes are currently running as root;
    • find which command appears in the maximum number of help categories (man);
    • count how many times the word var appears on the page ya.ru.
    Hint: you will need find , grep -o , awk "(print $1)" , regular expressions in grep , curl -s .

    What to study next?

    If you start to like the command line, don't stop, keep improving your skills.

    Here are some programs that will definitely come in handy if you live on the command line:

    • find with complex options
    • apropos
    • locate
    • telnet
    • netcat
    • tcpdump
    • rsync
    • screen
    • zgrep, zless
    • visudo
    • crontab -e
    • sendmail
    In addition, over time it is worth mastering a scripting language, such as perl or python, or even both.

    Who needs this?

    Is it even worth learning the command line and shell scripting today? Definitely worth it. I’ll give just a few examples of Facebook’s requirements for candidates who want to get a job at FB.

    Annotation: The concept of a command shell. Overview of command shells. Bash command shell. Features of operation (command history, operator "!", actions by pressing a key ). Multitasking in the console. Assignments. Job management. Midnight commander environment variables

    Progress of the lesson

    1. In the world of Linux and Unix, working on a computer is inextricably linked with the concept command shell– a program that allows the user to interact with the system by entering and executing commands. Nevertheless, command shell is a regular program. You can prove this by setting the default shell in the file passwd another program for the user. But in order for the system to know it as a shell, you need to add the absolute filename to /etc/shells.

    Linux comes with several command shells, their composition may vary depending on the distribution, but you can always find:

    Bourne Shell (sh) - the oldest and most common command shell for Unix systems. There is not a single Unix system where it is not used.

    Bourne Again Shell (bash) – an extended Bourne Shell. It has a lot of pleasant advantages, which is why it has become so popular recently. It is the default shell for almost all Linux distributions.

    Also popular shells are:

    csh– shell, the command system of which is close to the C programming language

    tcsh is a shell whose command system is close to the Tcl programming language.

    zsh is a shell that probably has the widest capabilities. It is an extension of sh (bourne shell).

    Since Linux uses bash by default, we'll talk about it.

    2. Command shell bash was originally a freely distributed version of the Bourne Shell. Subsequently, when its capabilities grew, it began to be considered an independent product. The main features of bash are:

    Table 1.1.
    Opportunity Comment
    1 Editing a line Ability to edit the entered command instead of rewriting it again
    2 Organization of channels Opportunity I/O redirection, organizing channels between performed tasks
    3 Ease of use Using command aliases, command history, autocompletion
    4 Job management Ability to create and manage background jobs
    5 Flexibility of customization Using per-user login script files, environment variables

    We'll look at I/O redirection and pipes later in Lesson 8. Therefore, we will omit this point. But let’s talk in more detail about the features of working in bash and the advantages it provides.

    Bash automatically writes all commands typed by the user to a file ~/.bash_history. To manage this file, use the history command. history is builtin command bash. That is, there is no executable file corresponding to this command. Herself command shell performs all actions. Entered without parameters, it simply lists all the commands stored in this file and is identical to the cat ~/.bash_history command.

    Command history exists to simplify the set of commonly used commands. The history of commands can be sorted through the list using the keys <вверх>And<вниз> .

    Another way is to type ! and start command and press . The last command from the history, the first letters of which match those typed, will be executed. For example:

    $ !/usr /usr/bin/perl ./ptest.pl OK $ !xfonts bash: !xfonts: event not found $

    But how can we speed up input if the history does not yet contain the command we need? In this case, the key will help us . After typing the first few letters of the command (or file path), press And Bash will automatically complete your command (or path element). In the event that several files are suitable or no file is suitable, the system will produce a sound signal. If the button press again, then when several files come up, the system will display a list, and when there are none, it will repeat the sound signal

    3. From the first lesson, you should have remembered that Linux is a multitasking environment. However, until now you have not yet been able to take advantage of its multitasking. At first glance, it seems that the console does not allow you to use the multitasking capabilities of the system, and only in the graphical environment can you run two or more programs simultaneously. But that's not true! The console is also multitasking.

    Firstly, you can open several consoles by opening a program in each of them. Switching between consoles will be done using the keys Ctrl+ , Where x– console number.

    And even in one console, using job management commands, you can take full advantage of the multitasking system.

    – a key combination that sends an undetectable signal to a process sigstop. Allows you to stop the execution of a process to transfer control to the command line.

    command & – the & symbol after the command allows you to run it in the background.

    jobs – displays a list of current shell jobs.

    bg<#j>– puts job #j into the background. Before this, the task must be stopped using the key combination . If at the moment the interpreter has only one task, then the number need not be specified.

    fg<#j>– puts job #j into foreground execution mode. The job must be stopped by a key combination or be in the background. If at the moment the interpreter has only one task, then the number need not be specified.

    $ man bash ^Z + Stopped man bash $ vim ^Z vim + Stopped vim $ bg 1 + man bash & $ jobs + Stopped man bash + Stopped vim $ fg 2 + vim $ fg + man bash $

    4. Environment Variables– system information indicating your preferences, such as text editor default, search paths for executable files, etc., as well as user, system, and shell credentials such as username, version of Linux, etc., used by the command interpreter and other programs.

    Frequently used variables by the user are:

    PATH – the variable contains the paths in which the system should look for executable files if the full or relative path to them.

    PWD – variable contains the full name of the current directory.

    HOME – variable contains full path user's home directory.

    HOSTNAME – variable contains the computer name.

    LOGNAME – contains username

    SHELL – contains the name of the command shell running in the current session.

    USER - contains username, whose session is open now.

    The list of variables installed on the system can be seen using the export command, entered without parameters.

    The bash command interpreter also has its own variables. For local variables to become system variables, they must be exported using the same export command. For example:

    $ export declare -x HOME="/home/gserg" declare -x HOSTNAME="WebMedia" declare -x LANG="ru_RU.KOI8-R" declare -x LOGNAME="gserg" declare -x PATH="/bin: /usr/bin:/usr/local/bin:/home/gserg/bin" declare -x PWD="/home/gserg" declare -x SHELL="/bin/bash" declare -x TERM="Eterm" declare -x USER="gserg" $ EDITOR=/bin/vim $ export EDITOR $ export declare -x EDITOR="/bin/vim" declare -x HOME="/home/gserg" declare -x HOSTNAME="WebMedia" declare -x LANG="ru_RU.KOI8-R" declare -x LOGNAME="gserg" declare -x PATH="/bin:/usr/bin:/usr/local/bin:/home/gserg/bin:" declare - x PWD="/home/gserg" declare -x SHELL="/bin/bash" declare -x TERM="Eterm" declare -x USER="gserg" $

    Team unset deletes a system variable. For example:

    $ unset EDITOR $ export declare -x HOME="/home/gserg" declare -x HOSTNAME="WebMedia" declare -x LANG="ru_RU.KOI8-R" declare -x LOGNAME="gserg" declare -x PATH=" /bin:/usr/bin:/usr/local/bin:/home/gserg/bin:" declare -x PWD="/home/gserg" declare -x SHELL="/bin/bash" declare -x TERM= "Eterm" declare -x USER="gserg" $

    5. In today's lesson we will introduce you to another, slightly non-standard, command shell Midnight commander. This is not command shell in the usual sense. This is a text file manager - analogue Norton Commander or Far. Midnight commander launched with the mc command. We'll talk about its capabilities.

    Screen Midnight commander divided into two parts. Almost the entire screen space is occupied by two panels with a list of directories and files. By default, the second line from the bottom of the screen is the command line, where you can run regular shell commands, and the bottom line shows tooltips for function keys ( F1-F10). The top line of symbols contains a menu with which you can perform many functions. To use the menu, you can click the mouse on the desired item or press a key F9 and using the keys cursor control select the desired item.

    Panels Midnight commander provide viewing of two directories simultaneously. One of the panels is active (in the sense that the user can perform certain actions with the files and directories located in it). In the active panel, the name of one of the files or directories is highlighted, and the panel title in the top line is highlighted in color. The title name is the same as the name of the directory currently displayed. Almost all operations are performed in the active panel. Some operations such as transfer or copying files use a passive panel as a place where copying, transferring, etc. is done.

    Now let's talk about the basic keyboard shortcuts that will help you work with Midnight commander.

    To change the active panel, use the keys or +

    to mark a file, click or +

    click for help

    will call the file viewer

    by using you edit the file

    will allow you to copy the .

    move or rename a file

    create directory

    Key will allow you to delete a file and/or directory

    , as already mentioned, opens access to the menu.

    - will allow you to get out of Midnight commander.

    will move the pointer to the beginning of the list of files,

    - on the contrary - to the end of the list.

    And will change the position of the indicator to the page up and down respectively.

    Key <*> on an additional keyboard will allow you to invert the selection of files (it does not affect directories)

    Key <+> on the additional keyboard will allow you to mark files by mask, and <-> unmark files by mask.

    + - update the contents of the directory (by rereading from disk or from the network)

    + - swap the right and left panels.

    + - remove/return panels.

    Although keyboard shortcuts are the optimal tool for getting the job done with Midnight commander as fast and convenient as possible, it is quite difficult for beginners to learn them all at once. To fill this gap and add other capabilities that lack keyboard shortcuts, Midnight commander has a menu (callable by F9).

    The menu consists of the following items: "Left Pane", "File", "Command", "Settings", "Right Pane".

    "Left/Right Panel"- these menu items are exactly the same. The only difference between them is that the actions performed will be addressed to the left or right panel.

    "List Format"- opens a dialog box in which you can select the view in which the list of files/directories will be displayed. There are standard, short and extended formats to choose from. However, in this window the user can determine the type of panel that suits him by selecting the “User Defined” switch.

    "Quick view"- puts the panel into automatic viewing mode for files selected on the adjacent panel. Focus automatically switches to the opposite panel.

    "Information"- switches the panel to view information about the file highlighted in the adjacent panel, such as position, permissions and owner, file system and the device on which it is located, the number of hard links associated with this file, as well as information about the device on which the file is located.

    "Tree"- translates Midnight commander into a mode similar to the operating mode of Explorer from OS Windows. In the panel to which the command is applied "Tree", a directory tree is built, which can be navigated using arrows cursor control, keys PageUp, PageDown, Home, End. The adjacent panel displays the contents of the directory highlighted in the tree.

    "Sort Order"- opens a dialog box in which you can select an attribute by which files and directories will be sorted in a list such as name, extension, edit time, access time, attribute change time, size, node (on which the file is located). You can also leave the files unsorted, sort them case sensitive, or sort them in reverse order.

    "Filter"- allows you to select file names that will be displayed in the panel using a regular expression entered in the dialog box.

    "FTP connection"- using this command you can establish a connection with a remote (or even local) computer using the protocol ftp. If only the remote server address is entered, then Midnight commander will try to establish an anonymous connection. The full line with which the node is specified is:

    ftp:user_name:password@server_address:port/directory_on_server

    After the connection is established, work with remote file system works similarly to working with a local FS.

    "Shell connection"- allows you to open a network connection using the protocol FISH (FIle transfer over SHell– file transfer via shell). FISH uses protocols RSH (Remote SHell– removed shell) or SSH (Secure SHell– protected shell, analogue RSH, but with support for encryption of transmitted data). The complete string with which the user can specify the remote node is:

    sh:user_name@server_address:options/directory_on_server

    Parameter username, options and directory_on_server not required. If username not specified, then Midnight commander will attempt to log on to the remote system with the username used on the local computer.

    "Revise"- analogue of a key combination + - causes the list of files and directories in the current panel to be updated by rereading them from disk or over the network.

    "File"- menu section, the items of which provide basic functions for processing files and directories, such as:

    "User Menu"- allows you to call a menu that is set by the user himself. Also called by the key .

    "View file"- analogue of a function performed by pressing . Allows you to view the highlighted file (or enter the directory). Supports many formats such as text formats, archive, Winword DOC, Linux executables, etc.

    "Viewing file..."- the same as the previous paragraph, but does not act on the highlighted file, but on the one whose name and path will be entered in the dialog box.

    "View Team"- allows you to execute a command and view it standard output in file viewing mode.

    "Editing"- opens the file for editing. Simple built-in text editor has a sufficient set of built-in functions for editing configuration files, program source codes, etc., and built-in automatic syntax highlighting makes editing more convenient and edited texts more readable.

    "Copy"- copies a file from the active panel to the passive one. Analogous to a function called by . By default, the file (or group of files) highlighted in the active panel is considered to be copied, and the destination is the directory opened in the passive panel. This can be changed by adjusting the field values ​​in the dialog that opens after calling this command.

    "Access rights"- allows you to change access rights to a file (or group of files) in the dialog box.

    "Symbolic link"- creates a symbolic link. By default, the file highlighted in the active panel is taken as the addressee, and the created link will have the same name and be located in the directory opened in the passive panel. The user can change this in the dialog box that opens.

    "Owner/Group"- changes the owner and/or group to which the file/directory belongs.

    "Rights (extended)"- allows you to simultaneously change access rights to a file and its owner and/or group. Permissions are represented as three rwx sequences for owner, group, and all users.

    "Renaming"- allows you to rename/move a file. Analogous to a function called by . By default, the file (or group of files) highlighted in the active panel is considered to be moved/renamed, and the destination is the directory opened in the passive panel. This can be changed by adjusting the field values ​​in the dialog that opens after calling this command.

    "Creating a directory"- creates a directory. Analogous to a function called by . By default, the directory is created in the directory opened in the active panel. This can be changed if you specify the full path to the directory to be created in the dialog that opens.

    "Delete"- deletes a file/group of files/directory. Analogous to a function called by .

    "Change directory"- changes the current directory. Similar to the cd command in the Bash shell. The required directory is entered in the dialog box.

    "Mark group"- Marks a group of files by mask in the directory opened in the active panel. Analogous to a function called using <+> on an additional keyboard.