Configuring and compiling the Linux kernel. Anatomy of the Linux kernel

Imagine you have a Linux kernel image for a phone on Android based, but you do not have the corresponding sources or kernel header files. Imagine that the kernel has support for loading modules (thankfully), and you want to build a module for that kernel. There are several good reasons, why can’t you just build a new kernel from source and just leave it at that (for example, the assembled kernel does not have support for some important device, like LCD or touchscreen). With the Linux kernel's constantly changing ABI and lack of source and header files, you might think you've reached a dead end.

As a statement of fact, if you build a kernel module using different header files than those used to build the kernel image you have, the module will fail to load, with errors depending on how header files were different from what was required. He may complain about bad signatures, bad versions, and other things.

Kernel configuration

If the kernel image you have is kind enough to provide a /proc/config.gz file, you can start with that, otherwise, you can try to start with the default configuration, but in this case you need to be extremely careful ( While I won't go into detail about using the default configuration as I have been fortunate enough not to have to resort to it, there will be some detail below as to why the correct configuration is so important).

Assuming that you have arm-eabi-gcc available in one of the paths in your PATH environment variable, and that a terminal is open in the kernel source folder, you can begin configuring the kernel and installing header files and scripts:

$ mkdir build $ gunzip config.gz > build/.config # or whatever to prepare .config $ make silentoldconfig prepare headers_install scripts ARCH=arm CROSS_COMPILE=arm-eabi- O=build KERNELRELEASE=`adb shell uname - r`
The silentoldconfig build will most likely ask if you want to enable certain options. You can choose the defaults, but this may very well not work.

You can use something else in KERNELRELEASE, but it must match exactly the version of the kernel from which you plan to load the module.

Writing a simple module

#include /* Needed by all modules */ #include /* Needed for KERN_INFO */ #include /* Needed for the macros */ static int __init hello_start(void) ( printk(KERN_INFO "Hello world\n"); return 0; ) static void __exit hello_end(void) ( printk(KERN_INFO "Goodbye world\n"); ) module_init(hello_start); module_exit(hello_end);
Place the following text in a Makefile in the same directory:

Obj-m = hello.o
Assembly of the module is quite simple, but at this stage the resulting module will not be able to load.

Module assembly

MODULE_INFO(vermagic, VERMAGIC_STRING);
The VERMAGIC_STRING value is determined by the UTS_RELEASE macro, which is located in the include/generated/utsrelease.h file generated by the kernel build system. By default, this value is determined by the kernel version and the git repository status. This is what KERNELRELEASE sets when configuring the kernel. If VERMAGIC_STRING does not match the kernel version, loading a module will result in a message like this in dmesg:

Hello: version magic "3.0.21-perf-ge728813-00399-gd5fa0c9" should be "3.0.8-perf"
Next, we also have here a definition of the module structure:

Struct module __this_module __attribute__((section(".gnu.linkonce.this_module"))) = ( .name = KBUILD_MODNAME, .init = init_module, #ifdef CONFIG_MODULE_UNLOAD .exit = cleanup_module, #endif .arch = MODULE_ARCH_INIT, );
On its own, this definition looks harmless, but the struct module structure defined in include/linux/module.h has an unpleasant surprise:

Struct module ( (...) #ifdef CONFIG_UNUSED_SYMBOLS (...) #endif (...) /* Startup function. */ int (*init)(void); (...) #ifdef CONFIG_GENERIC_BUG (.. .) #endif #ifdef CONFIG_KALLSYMS (...) #endif (...) (... plenty more ifdefs ...) #ifdef CONFIG_MODULE_UNLOAD (...) /* Destruction function */ void (*exit) (void); (...) #endif (...) )
This means that in order for the init pointer to end up in the right place, CONFIG_UNUSED_SYMBOLS must be defined according to what our kernel image is using. What about the exit pointer, these are CONFIG_GENERIC_BUG , CONFIG_KALLSYMS , CONFIG_SMP , CONFIG_TRACEPOINTS , CONFIG_JUMP_LABEL , CONFIG_TRACING , CONFIG_EVENT_TRACING , CONFIG_FTRACE_MCOUNT_RECORD and CONFIG_MODULE_UNLOAD .

Are you starting to understand why we're usually supposed to use exactly the same header files that our kernel was built with?

Static const struct modversion_info ____versions __used __attribute__((section("__versions"))) = ( ( 0xsomehex, "module_layout" ), ( 0xsomehex, "__aeabi_unwind_cpp_pr0" ), ( 0xsomehex, "printk" ), );
These definitions come from the Module.symvers file, which is generated according to the header files.

Each such entry represents the symbol required by the module and what signature the symbol must have. The first character, module_layout , depends on what the struct module looks like, that is, it depends on which configuration options mentioned earlier are enabled. The second, __aeabi_unwind_cpp_pr0, is an ARM ABI specific function, and the last one is for our printk function calls.

The signature of each symbol may differ depending on the kernel code for the function and the compiler used to build the kernel. This means that if you build a kernel from source, as well as modules for that kernel, and then rebuild the kernel after modifying, for example, the printk function, even in a compatible way, the modules originally built will not load with the new kernel.

So, if we build a kernel with sources and configurations close enough to those that were used to build the kernel image we have, there is a chance that we will not get the same signatures as in our kernel image, and it will crash would when loading the module:

Hello: disagrees about version of symbol symbol_name
Which means we need the correct Module.symvers file for the kernel image, which we don't have.

Studying the kernel

The kernel uses next function to search in your symbol table (in kernel/module.c):

Bool each_symbol_section(bool (*fn)(const struct symsearch *arr, struct module *owner, void *data), void *data) ( struct module *mod; static const struct symsearch arr = ( ( __start___ksymtab, __stop___ksymtab, __start___kcrctab, NOT_GPL_ONLY , false ), ( __start___ksymtab_gpl, __stop___ksymtab_gpl, __start___kcrctab_gpl, GPL_ONLY, false ), ( __start___ksymtab_gpl_future, __stop___ksymtab_gpl_future, __start___kcrctab_gpl_future, WILL_BE_GPL_ONLY, false ), #ifdef CONFIG_UNUSE D_SYMBOLS ( __start___ksymtab_unused, __stop___ksymtab_unused, __start___kcrctab_unused, NOT_GPL_ONLY, true ), ( __start___ksymtab_unused_gpl, __stop___ksymtab_unused_gpl, __start___kcrctab_unused_gpl, GPL_ONLY, true ), #endif ); if (each_symbol_in_section(arr, ARRAY_SIZE(arr), NULL, fn, data)) return true (...)
The structure used in this function is defined in include/linux/module.h:

Struct symsearch ( const struct kernel_symbol *start, *stop; const unsigned long *crcs; enum ( NOT_GPL_ONLY, GPL_ONLY, WILL_BE_GPL_ONLY, ) license; bool unused; );
Note: this code kernel has not changed significantly over the past four years (apparently since the release of kernel 3.0 in question - approx.).

What we have above in the each_symbol_section function are three (or five when CONFIG_UNUSED_SYMBOLS is enabled) fields, each containing the beginning of the symbol table, its end and two flags.

This data is static and persistent, which means that it will appear in the kernel binary as is. By scanning the kernel for three consecutive sequences of three pointers in the kernel's address space followed by integer values ​​from the definitions in each_symbol_section , we can determine the location of the symbol and signature tables, and recreate the Module.symvers file from the kernel binary.

Unfortunately, most kernels today are compressed (zImage), so a simple search by compressed image impossible. The compressed kernel is actually a small binary followed by a compressed stream. You can scan a zImage file to find the compressed stream and extract the decompressed image from it.

Tags: Add tags

Loading any modern operating system is a complex multi-step process. The boot process may vary slightly among different Linux distributions, but general scheme is approximately the same and consists of the following stages:

Execute BIOS code. Initialization of equipment. Choice bootable media. Reading the bootloader into RAM and transferring control to it. The boot loader usually occupies one sector on the disk and is limited in size to 384 bytes (512 bytes - disk sector, minus 128 bytes - partition table). Depending on type boot device The boot sector can be read from different places:

• When booting from a floppy disk or hard drive, the bootloader is read from the first sector of the physical media;
• When booting from a CD/DVD - from the first sector of the boot disk image located in the CD data structure;
• When booting over a network, from the first sector of the boot disk image downloaded from the server via the tftp protocol.

The screen at this stage displays information about BIOS version, the process of checking the RAM found hard drives. The bootloader code is too small to include information printing functionality, but it can output short messages about errors.

Reading the main bootloader (GRUB, LiLo, NTLDR) into memory and executing its code. Since the bootloader is very small, as a rule, sectors from which the main bootloader code must be read are hardcoded into its code. On an HDD, this may be the space between the MBR and the first partition on the disk (track zero). On a floppy disk and when using a disk image when booting from a CD and over a network, the main bootloader can be located immediately after the primary bootloader and occupy the entire volume of the image.

Loading the kernel (vmlinuz) and auxiliary disk image (initrd). The main bootloader is intelligent enough to find the configuration file, the kernel image file, and the auxiliary disk image file in the file system. If necessary, the kernel image is unpacked into RAM, and a memory area is formed containing parameters passed from the bootloader to the kernel, including the address of the auxiliary disk image.

The auxiliary disk is necessary for modern Linux systems due to the modularity of the kernel and contains the drivers (ATA, NFS, RAID, etc.) necessary to gain access to the main file system.

At this stage, a process with pid=1 is created, in which the init script is executed, located in the root directory of the auxiliary disk. Options passed to the kernel are actually passed to init as command line arguments.

The script contains the download necessary drivers in the form of kernel modules, creating temporary device files in the /dev directory to access these modules, scanning disk partitions to detect and initialize RAIDs and logical volumes. After the logical drives are initialized, an attempt is made to mount the root file system specified by the root= parameter. In the case of diskless network boot, an attempt is made to mount the root directory via NFS.

Messages appear on the screen about loading drivers and searching for virtual volumes of the LVM subsystem. The stage is completed by remounting the root directory onto the main file system and loading the main program /sbin/init (or its equivalent) into the process with pid=1.

In classic UNIX and older versions of Linux (up to about 2012), the init program reads the /etc/inittab configuration file, initializes text consoles and, as a rule, starts the necessary services using a set of scripts located in the /etc/init.d directories and /etc/rc*.d . On modern Linux distributions, the /sbin/init file contains more modern program start services. The most popular of these programs are upstart and systemd, which can significantly reduce the time of this boot phase.

At this stage, lines are displayed on the screen indicating that services have started and information about the success this process( or ).

GRUB boot loader

Boot from the installation disk into Rescue mode. To do this, at the time of boot, you must enter linux rescue at the boot prompt:

If everything goes well, the root directory of the main system will be mounted in /mnt/sysimage, the boot directory in /mnt/sysimage/boot. Additionally, the current /proc, /sys, and /dev directories will be mounted into the corresponding /mnt/sysimage subdirectories. If this does not happen, then you will have to do these operations manually.

When all directories are mounted, you can change the root directory

#if it turns out that you forgot to mount something, you can exit using ^D chroot /mnt/sysimage

and rebuild the initrd

#copy the old file cp -p /boot/initramfs-$(uname -r).img /boot/initramfs-$(uname -r).img.bak #create a new dracut -f #if the kernel version on the main system is different from version on the installation disk, specify it explicitly dracut -f /boot/initramfs-2.6.32-358.el6.x86_64.img 2.6.32-358.el6.x86_64

#copy the old file cp -p /boot/initrd-$(uname -r).img /boot/initrd-$(uname -r).img.bak #create a new one mkinitrd -f -v /boot/initrd-$( uname -r).img $(uname -r) #if the kernel version in the main system differs from the version on the installation disk, specify it explicitly mkinitrd -f -v /boot/initrd-2.6.18-371.el5.img 2.6. 18-371.el5

Cd/sync telinit 6

Full example with i2o_block driver (Adaptec 2010S SCSI adapter), which does not load automatically. The example runs on CentOS 5 because the standard CentOS 6 kernel does not support this driver.

After booting from the CD into Rescue mode, a message appears that the Linux partitions were not found and you need to mount them yourself.

#Load the driver insmod i2o_block #Check that everything worked lsmod .... dmesg ... #Create device files based on the information in dmesg mkdir /dev/i2o mknod /dev/i2o/hda b 80 0 mknod /dev/i2o/ hda1 b 80 1 mknod /dev/i2o/hda2 b 80 2 #Activate VolumeGroup lvm vgchange -a y #Mount volumes mkdir /mnt/sysimage mount /dev/mapper/VolGroup00-LogVol00 /mnt/sysimage mount /dev/i2o/hda1 / mnt/sysimage/boot #Mount special directories mount --bind /proc /mnt/sysimage/proc mount --bind /dev /mnt/sysimage/dev mount --bind /sys /mnt/sysimage/sys

Further according to the instructions, only when creating a disk image you need to specify the mkinitrd program additional option--preload=i2o_block and disable readahead services, as they cause the i2o_block driver to hang:

Chkconfig early-readahead off chkconfig later-readahead off

Last time we talked about what happens when Linux boots: first the bootloader starts, it loads the kernel and deploys a temporary disk in RAM, the kernel starts the init process, init finds the real root disk, performs such a tricky revolution - instead of a temporary virtual disk on a real disk is mounted in the same place in the root directory, from this real disk the init process loads another init that is on this real disk. After all these operations, UNIX returns to its normal operating state.

In this lecture I will tell you what does classical program init in combination with System V style rc.d scripts. System V is the classic version of UNIX on which commercial UNIXes are built.

As the name suggests, rc.d is a directory. There is a UNIX tradition - if the entire configuration of something fits into one file, and it is called config, then when it is divided into separate files that are connected to the main one, they create a directory with the same name and add config.d to the name.d. The letter d means that this is a directory and the auxiliary parts of the configuration file are located there. The init program configuration file format has two traditions: the System V variant, in which each configuration detail is kept in a separate file in the rc.d directory, and the BSD tradition, in which there is a single /etc/rc file containing many scripts and variables. which are responsible for the behavior of the system.

In any case, when the system starts, we create a process with PID=1, in which a program called init is launched. As you saw last time, if the init program is killed, the kernel panics and stops all work.

The classic System V init reads the /etc/inittab file and carries out a number of instructions that are written in this file. Inittab is a text file, each line of which is, in fact, one command or some kind of rule of behavior. Inittab looks like this:

id:3:initdefault:

si::sysinit:/etc/rc.d/rc.sysinit

l3:3:wait:/etc/rc.d/rc 3

ca::ctrlaltdel:/sbin/shutdown -t3 -r now

There is a label at the beginning of the line. I don’t really understand what the big meaning of this label is. We can assume that this is simple text and that’s all. The second item is either the so-called load level or an empty value. The load level is either a single number from 0 to 6, or a list of numbers separated by commas. Next comes some action. The actions are as follows: wait, respawn, sysinit, ctrlaltdel. There are other actions, but these are the most used. Finally, at the end of the line there is a certain command written with the name of the executable file and the arguments that must be passed to this command.

The sysinit action is executed once at system startup.

The ctrlaltdel action isn't really an action at all - it's a handler for the control alt del shortcut. The click itself is intercepted by the system kernel, and information about this is sent to the init process, which must execute a specific command. For example, the shutdown command can be issued, which will shut down the computer. In principle, you can register any other program here, for example, echo, which, after pressing control alt del, will issue a message to all terminals of the system. fireplace console like this

The wait action means that you need to run the command, wait until it finishes and only then continue processing the next lines. I don’t know if such actions can be launched in parallel. Most likely not.

The respawn action means that you need to run the program and, without waiting for it to complete, go to further actions. If this program subsequently ends, it must be restarted.

So, there is a single execution with waiting for results and multiple execution in asynchronous mode - they started, waited until it finished, launched the words.

Load levels are a convention that allows you to control which services are loaded. The closest analogue in Windows is loading into safe mode, when only a limited number of drivers are loaded and a minimum number of services start, loading with debugging, when each action is additionally logged, and a regular full loading.

Linux traditionally has 6 boot options. This division is quite arbitrary.

0 and 6 are off. 0 is a complete power off, and 6 is a reboot mode.

4 is completely skipped on Linux

There are four loading levels remaining:

1 - single-user mode. If you pass it to the bootloader keyword single, then we will find ourselves in single-user mode, where only one process is running and this is the system administrator shell. This mode is used to restore the system.

3 - normal multi-user text mode, when all services are running, the network is working, all drivers are working.

2 - also text mode, but without connecting network drives. The fact is that traditional network file nfs system, which is used in UNIX, is extremely resistant to network damage. If we turn off the file server or cut the network cable, the nfs network file system will make numerous attempts to recover and these attempts are so long that I have never been able to wait until the error message finally appears. Perhaps this will happen in an hour, or maybe in 6 hours. All this time, the nfs driver will hold the computer, preventing you from doing anything. Therefore, if our network crashed or the file server in the settings says that at startup it is necessary to mount external drives, then the attempt will boot into full mode will cause everything to freeze for you. For this case, the second loading option is provided - everything is the same as in the third, only network drives don't connect. Myself network adapter works, the IP address is assigned, the Internet is available.

5 - the same as 3, but with the launch of the x window - graphical interface.

mode 2 includes 1 + multiplayer mode. 3 enables 2+ mounting of network file systems. Finally, 5 includes 3 + launch of the graphics subsystem. Whether this will be implemented consistently or not is a distribution issue. Generally speaking, administrators can independently configure the inittab file so that these modes are launched sequentially, or they can make everything completely independent - by switching to the next mode, we remove everything that was done in the previous step and configure everything from scratch.

Let's look at the lines of a real file. They are very simple.

l3:3:wait:/etc/rc.d/rc 3

Some program is launched, which must perform all the necessary actions that are expected at the third level. Probably, at the third level you need to configure network interfaces, launch the terminal driver, and start some services. Only after all this is completed will we be able to work in the system. Since we need to wait for the launch to complete, we select the wait action.

The launcher is called rc and is launched with the level number as a parameter. The init program itself is quite simple. She can read her file line by line with simple syntax and start new processes by launching some auxiliary programs. All load level logic is hidden in the rc script. By running rc with parameter 3 we will go to the third level, with parameter 5 - to the fifth.

The rc program is also very simple. This is a script that executes all files in directories corresponding to the load level, for example /etc/rc3.d/. These directories contain executable files that take one parameter - either start or stop. If the file is launched with the start parameter, then it starts the service, if with the stop parameter, it stops it. For example, network start will configure network interfaces, and network stop will put interfaces in a disabled state. In addition to network interfaces, there are scripts for connecting/disconnecting network file systems, starting/stopping services, etc.

Names of files in directories built according to certain rules. They begin with either the letter K or the letter S, followed by a number and the name of the service.

The rc script looks through the contents of the rc3 directory and selects from there all files that begin with the letter K (kill). The files are ordered in ascending order and executed with the stop parameter. Then the same actions are performed with files starting with the letter S (start), which are launched with the start parameter. That, in general, is the entire procedure for moving to a certain level.

We can assume that the /etc/rc0.d/ directory contains only files starting with the letter K, since everything needs to be stopped when shutting down, and the /etc/rc1.d/ directory will contain one file on the letter S to launch the administrator console.

For ease of programming, there is a separate directory /etc/init.d/, which contains the same files only without the letters and numbers at the beginning of the name. In fact, the files in the level directories are just symbolic links to the main files. So /etc/rc3.d/S10apache is a link to the file /etc/init.d/apache. Letters and numbers in the names of links are needed so that the rc script calls them in the right order and with the right arguments.

In systems that are built on this principle, in order to start or stop any service in the /etc/init.d/ directory, you need to find the file that corresponds to it and start it with the start or stop parameter. What's not to like about starting services this way - by explicitly calling scripts. The point is that in command line Linux autocompletion works great. With its help you can very quickly enter the path to the startup file.

To hide a specific implementation from the user, two auxiliary programs were written on top of the system of scripts and symbolic links.

The chkconfig program allows you to manipulate symbolic links to the corresponding scripts. To see what starts and what stops at each level, you can use the ls command and list the scripts in the corresponding directory, but it’s easier to use the chkconfig –list command. The chkconfig program runs through all the rc directories and produces a list of what starts and what stops at each level. If we want a certain service to automatically start when the system starts, we run chkconfig<имя службы>on and the script creates a link to run in the desired directory and with correct name. Run chkconfig<имя службы>off causes the start link to be removed and the stop link to be created. Thus, the chkconfig program allows you to manage the list of services that start at system startup.

Another program - service is used to manually start and stop services. Service is a wrapper that allows you not to access the script directly, but to specify the name of the service and say whether we want to start or stop it. The bash I use doesn't have autocompletion for the service command, so it's easier for me to type the path to the scripts.

In startup scripts, the start and stop arguments must be processed. In addition, you can come up with some of your own arguments that will do something useful.

Most scripts implement a status option, which shows whether the service is running or not. When we execute start, the script, after successfully starting the service, receives its PID and writes it to specific file. The stop command deletes the file. Typically such files are created in the /var/run/ directory. The status command checks whether such a file exists. If it is not there, it reports that the service is not running. If the file exists, it extracts the process ID from it and checks the current list of processes. If this identifier is present, everything is running, if the program breaks down for some reason, then the status indicates that an attempt was made to start this service - the file exists, but the service itself is not running.

The restart option sequentially executes two commands inside the script - first stop, and then start. This is a completely optional command - just convenient. Finally, there are services that allow you to re-read some configuration files on the fly. For them, a reload command is added, the task of which is to send a signal to the service that the configuration has changed. A separate case, the save and load commands for saving the firewall configuration.

If the system administrator, instead of stopping or starting individual services, wants to transfer the entire system to a certain level, then this can be achieved in one of two ways. You can call the /sbin/init program directly. If you call it with a certain number as a parameter, it will execute all the instructions from the inittab file for which the corresponding level was specified. If you run, for example, /sbin/init 1, then init will find in its configuration file all the lines that contain level 1 and will execute them. On some systems, the shutdown command is implemented as /sbin/init 0, since level 0 corresponds to stopping the system. Recently, to switch between levels, a special program called telinit, which is a link to init. Its task is to send a signal to the init process that the administrator wants to move to a certain level. telinit q tells init to reread the inittab file. In older systems, this was achieved by sending a SIGHUP signal to the process with PID=1 (kill –HUP 1).

A few more lines in inittab, this is the launch of terminals

1:2345:respawn:/sbin/mingetty tty1

In order to provide interactive access to the system, you may have a number of lines of this kind in the inittabe. 2345 are the levels at which the command must be run, respawn means that the program must be restarted if it terminates. getty is a terminal management program. Traditionally, a terminal in UNIX is called a teletype because the first terminals were electric typewriters. Accordingly, tty is an abbreviation for teletypewriter. Mingetty is a program that can work with virtual terminals on personal computer. It can configure the terminal driver, and as parameters it receives the name of the terminal device that needs to be configured. In the /dev/ directory there is a device file tty1, which corresponds to the first virtual terminal. If we had a modem and we wanted to initialize it at boot time, we could call getty with the ttyS0 parameter, which corresponds to the COM1 port. When initializing the modem, it would be possible to set additional parameters: connection speed 19200 baud, 7 or 8 bits per byte, parity, number of stop bits.

S0:2345:respawn:/sbin/getty ttyS0 19200 8 n 1

Last time I drew a chain in which a process makes a copy of itself by calling fork, the child copy by calling exec loads another program into its memory, and after completion reports this to the parent process.

Text user sessions are organized in the following chains: first, init makes a copy of itself and runs the mingetty program in it. Mingetty initializes the terminal and keyboard, and then runs the login program in the same process. Login displays prompts for entering a name and password and, if everything went well, it assigns user privileges to itself and in the same process, overwriting itself, launches a user interpreter, for example, bash. When the user types the exit command, the interpreter ends the life path of this process. When a process terminates, init receives a signal about it. Init looks at what it is supposed to do, sees the respawn action, runs the mingetty program again, which reinitializes the terminal and repeats everything. Thus, each session is inside one process. As soon as we left the session, our process ended and a program immediately launched that would clean up the terminal after us and restore all settings to default.

There is another special keyword in the inittab file, initdefault - the default level. If init received the single parameter through the kernel, then we will boot to level 1. If nothing was passed through the bootloader, then the default value is used. If, after installing the graphical shell, it turns out that our computer is rather weak for graphics, then we can set the default level to 3, and after the next reboot we find ourselves at the third level - that is, in text mode. We installed the system without graphical mode, then additionally installed all the packages for x window, changed the default level to 5, and after the next reboot we went straight into graphical mode.

In this script system, sometimes you want to do something of your own, for example, at startup, delete all files in the /tmp/ directory. For this there is separate file called /etc/rc.local, which runs after all the others. This is just a script without parameters in which you can write anything you want. For example, on one of my routers, at the time the system starts, routing tables are written in this file. I was too lazy to look for where the corresponding standard scripts from the distribution are located and it turned out to be easier to register the commands in rc.local.

The Linux kernel contains over 13 million lines of code and is one of the largest open source projects in the world. So what is the Linux kernel and what is it used for?

The core is the most low level software that interacts with computer hardware. It is responsible for the interaction of all applications running in user space down to the physical hardware. Also allows processes known as services to receive information from each other using the IPC system.

Types and versions of the kernel

You already know what the Linux kernel is, but what types of kernels are there? Eat various ways and architectural considerations when building kernels from scratch. Most kernels can be one of three types: monolithic kernel, microkernel, and hybrid. The Linux kernel is a monolithic kernel, while the Windows and OS X kernels are hybrid. Let's take an overview of these three types of kernels.

Microkernel

Microkernels implement an approach in which they only manage what they are supposed to: CPU, memory, and IPC. Almost everything else on the computer is treated as an accessory and handled in user mode. Microkernels have the advantage of portability; they can be used on other hardware, and even another operating system, as long as the OS attempts to access the hardware in a compatible manner.

Microkernels are also very small in size and are more secure since most processes run in user mode with minimal privileges.

Pros

• Portability
• Small size
• Low memory consumption
• Safety

Cons

• Hardware accessible via drivers
• Hardware is slower because drivers run in user mode
• Processes must wait their turn to receive information
• Processes cannot access other processes without waiting

Monolithic core

Monolithic kernels are the opposite of microkernels because they cover not only the processor, memory and IPC, but also include things like device drivers, file system management, I/O system. Monolithic kernels give better access to the hardware and enable better multitasking because if a program needs to get information from memory or another process, it doesn't have to wait in a queue. But this can cause some problems, because many things are performed in superuser mode. And this can harm the system if done incorrectly.

Pros:

• More direct access to hardware
• Easier exchange of data between processes
• Processes respond faster

Cons:

• Large size
• Takes up a lot of RAM
• Less secure

Hybrid core

Hybrid kernels can choose what to work with in user mode and what in kernel space. Often device and file system drivers are in user space, while IPC and system calls are in kernel space. This solution takes the best of both previous ones, but requires more work from equipment manufacturers. Because all responsibility for the drivers now lies with them.

Pros

• Ability to choose what will work in kernel and user space
• Smaller in size than a monolithic core
• More flexible

Cons

• May work slower
• Device drivers are released by manufacturers

Where are the kernel files stored?

Where is the Linux kernel located? The kernel files of Ubuntu or any other Linux distribution are located in the /boot folder and are called the vmlinuz version. The name vmlinuz comes from the Unix era. In the sixties, kernels were usually called simply Unix; in the 90s, Linux kernels were also called Linux.

When virtual memory was developed to facilitate multitasking, the letters vm appeared in front of the file name to indicate that the kernel supported this technology. For some time the kernel was called vmlinux, but then the image no longer fit into memory bootstrap, and was compressed. After this, the last letter x was changed to z to indicate that zlib compression was used. This particular compression is not always used; sometimes you can find LZMA or BZIP2, so some kernels are simply called zImage.

The version number consists of three digits, the version number of the Linux kernel, your version number and patches or fixes.

The /boot package contains not only the Linux kernel, but also files such as initrd.img and system.map. Initrd is used as a small virtual disk that fetches and executes the actual kernel file. The System.map file is used to manage memory while the kernel is not yet loaded, and configuration files can specify which kernel modules are included in the kernel image when built.

Linux kernel architecture

Since the Linux kernel has a monolithic structure, it is larger and much more complex than other types of kernels. This design feature attracted much controversy in the early days of Linux and still carries some of the design flaws inherent in monolithic kernels.

But to get around these shortcomings, the Linux kernel developers did one thing - kernel modules that can be loaded at runtime. This means you can add and remove kernel components on the fly. Everything can go beyond adding hardware functionality, you can run server processes, enable virtualization, and completely replace the kernel without a reboot.

Imagine being able to install a Windows update package without having to constantly reboot.

Kernel modules

What if Windows already had all the drivers you needed by default, and you could only enable the ones you needed? This is exactly the principle that Linux kernel modules implement. Kernel modules, also known as loadable modules (LKM), are essential for keeping the kernel running with all the hardware without using up all the RAM.

The module expands functionality base kernel for devices, file systems, system calls. Loadable modules have a .ko extension and are usually stored in the /lib/modules/ directory. Thanks to its modular nature, you can customize the kernel very easily by installing and loading modules. Automatic download or module unloading can be configured in configuration files or upload and download on the fly using special commands.

Third-party, proprietary, closed-source modules are available on some distributions such as Ubuntu, but they are not shipped by default and must be installed manually. For example, NVIDIA video driver developers do not provide source code, but instead they compiled their own modules in .ko format. Although these modules appear to be free, they are not free. That's why they are not included in many distributions by default. The developers believe that there is no need to pollute the kernel with proprietary software.

Now you are closer to answering the question what is the Linux kernel. The core is not magic. It is very necessary for the operation of any computer. The Linux kernel is different from OS X and Windows because it includes all the drivers and does a lot of things supported out of the box. Now you know a little more about how your software works and what files are used to do it.

The most basic component of the operating room Linux systems there is a core. It is the kernel that acts as an intermediate link between user programs and computer hardware. In all binary distributions, we do not need to worry about assembling and configuring the kernel; the distribution developers have already done everything for us. But if we want to build our distribution ourselves or install the latest version of the kernel, we will have to build the kernel manually.

The first option used to be relevant for those who wanted to receive maximum performance from its equipment, but now, given the rapid increase in computer power, the increase in performance when assembling the kernel is completely unnoticeable. Nowadays, building the kernel may be necessary for users of non-binary distributions such as Gentoo, those who want to make some changes to the kernel, get the latest version of the kernel, and, of course, those who want to fully understand the operation of their system.

In this tutorial we will look at how to build the Linux kernel. The first part will tell you how to configure the kernel in automatic mode. So to speak, for those who do not want to understand how it works, who only need to get the finished product as an output - the assembled kernel. In the second part we will look at the main stages manual settings kernels, this process is complex and slow, but I will try to give the basics so that you can figure it out yourself.

The very first thing you need to do is download the kernel sources. It is best to take the sources from the website of your distribution, if they are there, or from the official kernel website: kernel.org. We'll look at downloading sources from kernel.org.

Before downloading the sources, we need to decide on the version of the kernel that we will build. There are two main versions of releases - stable (stable) and release candidates (rc), there are, of course, also stable ones with a long period of support (longterm), but it is important now to understand the first two. Stable ones, as a rule, are not the newest, but they are already well-tested kernels with a minimum number of bugs. Test ones, on the contrary, are the newest, but may contain various errors.

So, when we have decided on the version, go to kernel.org and download the necessary sources in tar.xz format:

This article will use the latest at the moment unstable version 4.4.rc7.

You can also get the Linux kernel sources using the git utility. First, let's create a folder for the sources:

mkdir kernel_sources

To download the latest version, type:

git clone https://github.com/torvalds/linux

Unpacking kernel sources

Now we have saved sources. Go to the source folder:

cd linux_sources

Or if you downloaded the Linux kernel using a browser, then first create this folder and copy the archive into it:

mkdir linux_sources

cp ~/Downloads/linux* ~/linux_sources

Unpack the archive using the tar utility:

And go to the folder with the unpacked kernel, I have this:

cd linux-4.4-rc7/

Automatic configuration of Linux kernel build

Before we start building the Linux kernel, we will have to configure it. As I said, first we’ll look at the automatic option for setting up the kernel build. Your system already has a assembled, configured by the distribution manufacturer, and a fully working kernel. If you don’t want to deal with the intricacies of the kernel configuration, you can simply extract the ready-made settings of the old kernel and generate settings for the new one based on them. We will only have to specify the values ​​for the new parameters. Considering that there have been no major changes and no major changes are planned in the latest versions, you can respond to all these parameters as the configuration script suggests.

The parameters of the kernel used are stored in an archive at /proc/config.gz. Let's unpack the config and place it in our folder using the zcat utility:

During the process, you will need to answer several questions. These are new options that have changed or been added to the new kernel and support for new hardware, in most cases you can choose the default option. Usually there are three options: y - enable, n - do not include, m - enable as a module. The recommended option is written in capital letters; to select it, simply press Enter.

It will take you about 10 minutes to do everything. Once the process is complete, the kernel is ready to be built. Next we'll look at manually configuring the kernel, but you can skip straight to building the Linux kernel.

Manual Linux Kernel Tuning

Manual configuration is a complex and time-consuming process, but it allows you to understand how your system works, what functions are used and create a kernel with minimal the right set functions to suit your needs. We will consider only the main steps that need to be completed for the kernel to be assembled and working. You will have to figure out everything else yourself based on the kernel documentation. Fortunately, the configuration utility has extensive documentation for each parameter that will help you understand what other settings you need to enable.

Let's begin. To launch the Linux kernel settings menu, type:

This will open a utility with the ncurses interface:

As you can see, some required options are already included to make the setup process easier for you. Let's start with the most basic settings. To enable the parameter press y, to enable it by module - m, to move use the arrow and Enter keys, you can return to the top level with the Exit button Open the item General Setup.

Here we set the following parameters:

Local Version- local version of the kernel, will increase by one with each build, so that new kernels do not replace old ones during installation, set the value to 1.

Automatically append version information to the version string- add version to the name of the kernel file.

Kernel Compression Mode- kernel image compression mode, the most effective lzma.

Default Hostname- computer name displayed at the input prompt

POSIX Message Queues- support for POSTIX queues

Support for paging of anonymous memory - enable swap support

Control Group support- support for a mechanism for distributing resources between groups of processes

Kernel.config support And Enable access to .config through /proc/config.gz- enable the ability to extract the kernel configuration via /proc/config.gz

That's it, go back up a level and turn it on Enable loadable module support, This function allows loading external modules, then open its menu and enable:

Again we go back and open Processor type and features:

Processor family (Opteron/Athlon64/Hammer/K8)- select your processor type.

Let's go back again and go to the section File systems, check all the necessary boxes here.

Be sure to turn it on The Extended 3 (ext3) filesystem And The Extended 4 (ext4) filesystem- to support standard ext3 and ext4 file systems

We return and go to Kernel hacking.

Here we include Magic SysRq key- support for SysRq magic functions, not an essential thing, but sometimes useful.

There is one more point left, the most difficult, because you will have to go through it yourself. Device Drivers- you need to go through the sections and enable drivers for your equipment. By equipment I mean non-standard hard drives, mice, USB devices, webcams, Bluetooth, WIFI adapters, printers, etc.

You can see what equipment is connected to your system with the command:

Once you've completed all the steps, the kernel is ready to build, but you'll likely have a lot to figure out.

To exit press the button a couple of times Exit.

Building the Linux kernel

After all preparations are completed, the Linux kernel can be built. To start the build process, run:

make && make modules

Now you can go have a coffee or take a walk, because the assembly process is long and will take about half an hour.

Installing a new kernel

When the kernel and modules are assembled, the new kernel can be installed. You can manually copy the kernel file to the bootloader folder:

cp arch/x86_64/boot/bzImage /boot/vmlinuz

Or you can simply execute the installation script, immediately installing the modules at the same time:

sudo make install && sudo make modules_install

After installation, do not forget to update the Grub bootloader configuration:

grub-mkconfig -o /boot/grub/grub.cfg

And reboot the computer to see the new kernel in action:

Conclusions

That's it. In this article, we looked in detail at how to build the Linux kernel from source. This will be useful to everyone who wants to better understand their system, and to those who want to get the most new version kernels in your system. If you have any questions, ask in the comments!