How to restore a file system in fsck. Checking the FS and recovering deleted files in Linux

Due to various problems or an unexpected computer shutdown, the file system may become damaged. During a normal shutdown, all file systems are mounted read-only and all unsaved data is written to disk.

But if the power is turned off unexpectedly, some data will be lost, and important data may be lost, resulting in damage to the device itself. file system. In this article we will look at how to recover the fsck file system for several popular file systems, and also talk about how ext4 recovery works.

As you know, the file system contains all the information about all the files stored on the computer. This is the file data itself and the metadata that controls the location and attributes of files in the file system. As I already said, the data is not immediately written to hard drive, and for some time are in RAM and in case of an unexpected shutdown, due to a certain set of circumstances, the file system may be damaged.

Modern file systems are divided into two types - journaled and non-journaled. Journaled file systems log all actions that are about to be performed, and after execution they erase these records. This allows you to very quickly understand whether the file system has been damaged. But it doesn't help much with recovery. To recover a Linux file system, you need to check each block of the file system and find bad sectors.

The fsck utility is used for these purposes. In fact, this is a shell for other utilities that are focused on working only with one or another file system, for example, there is one utility for fat, and a completely different one for ext4.

On most systems, the fsck check runs automatically for the root partition, but this does not apply to other partitions, and also will not work if you disable the check.

fsck basics

In this article we will look at manual work with fsck. You may need a LiveCD to run the utility from if the root partition is damaged. If not, the system will be able to boot into recovery mode and you will use the utility from there. You can also run fsck on an already booted system. Only the work requires superuser rights, so do it via sudo.

Now let's look at the syntax of the utility itself:

$fsck [options] [filesystem_options][disk_partition]

Basic options specify how the fsck shell utility behaves. A disk partition is a partition device file in the /dev directory, for example /dev/sda1 or /dev/sda2. The file system options are specific to each individual scan utility.

Now let's look at the most useful options fsck:

• -l- don't run another instance of fsck for this hard drive until the current one finishes running. For SSD, the parameter is ignored;
• -t- set the types of file systems that need to be scanned. It is not necessary to specify the device; you can check several partitions with one command by simply specifying desired type file system. This could be the file system itself, for example ext4, or its options in the opts=ro format. The utility scans all file systems mounted in fstab. If you also specify a section, then a check of the specified type will be applied to it, without autodetection;
• -A- check all file systems from /etc/fstab. This is where the file system scanning parameters specified in /etc/fstab are applied, including priority. First of all, the root is checked. Typically used at system startup;
• -C- show the progress of the file system check;
• -M- do not check if the file system is mounted;
• -N- do nothing, show that the check was completed successfully;
• -R- do not check the root file system;
• -T- do not show information about the utility;
• -V- the most detailed output.

These were global utility options. Now let’s look at the options for working with the file system, there are fewer of them, but they will be more interesting:

• -a- during the check, correct all detected errors, without any questions. This option is obsolete and is not recommended;
• -n- perform only a file system check, do not fix anything;
• -r- ask before correcting each error, used by default for ext file systems;
• -y- answers all questions about error correction in the affirmative, we can say that this is the equivalent of a.
• -c- find and blacklist everything broken blocks on your hard drive. Available only for ext3 and ext4;
• -f- forced check of the file system, even if according to the log it is clean;
• -b- set the superblock address if the main one was damaged;
• -p- another modern analogue of the -a option, it checks and corrects automatically. Basically, you can use one of three options for this purpose: p, a, y.

Now we have sorted everything out and you are ready to perform file recovery linux systems. Let's get down to business.

How to restore a file system in fsck

Let's say you've already booted into a LiveCD system or recovery mode. Well, in a word, we are ready to restore ext4 or any other damaged FS. The utility is already installed by default in all distributions, so there is no need to install anything.

File system recovery

If your file system is on a partition with address /dev/sda1 run:

sudo fsck -y /dev/sda1

It is not necessary to specify the y option, but if you do not do this, the utility will simply flood you with questions that need to be answered yes.

Restoring a damaged superblock

Usually this team copes with all damage with a bang. But if you did something serious and damaged the superblock, then fsck may not help. The superblock is the beginning of the file system. Without it, nothing will work.

But don’t rush to say goodbye to your data, everything can still be restored. Using this command, we look at where the backup superblocks were written:

sudo mkfs -t ext4 -n /dev/sda1

This command actually creates a new file system. Instead of ext4, substitute the file system into which the partition was formatted; the block size must also match, otherwise nothing will work. With the -n option, no changes are made to the disk, but only information is displayed, including about superblocks.

Now we have six backup addresses superblocks and we can try to recover the file system using each of them, for example:

sudo fsck -b 98304 /dev/sda1

After doing this, you will most likely be able to restore your file system. But let's look at a couple more examples.

Checking a clean file system

Let's check the file system, even if it is clean:

sudo fsck -fy /dev/sda1

Bad sectors

Or else we can find bad sectors and don’t write anything else in them:

sudo fsck -c /dev/sda1

File system installation

You can specify which file system should be scanned on the partition, for example:

sudo fsck -t ext4 /dev/sdb1

Checking all file systems

Using the -A flag you can check all file systems attached to the computer:

But such a command will only work in recovery mode; if the root partition and other partitions are already mounted, it will give an error. But you can exclude the root partition from the check by adding R:

sudo fsck -AR -y

Or exclude all mounted file systems:

You can also check not all file systems, but only ext4, to do this, use the following combination of options:

sudo fsck -A -t ext4 -y

Or you can also filter by mount options in /etc/fstab, for example let's check for filesystems that are mounted read-only:

sudo fsck -A -t opts=ro

Checking mounted file systems

I said before that it is impossible. But if there is no other way out, then it is possible, although it is not recommended. To do this, you must first remount the file system to read-only mode. For example:

sudo mount -o remount,ro /dev/sdb1

And now check the fsck file system in forced mode:

sudo fsck -fy /dev/sdb1

View information

If you do not want to correct anything, but just view the information, use the -n option.

Sometimes, for various reasons (as a result of a failure, incorrect shutdown), file systems accumulate errors. The errors themselves are “mismatched” data structures. Naturally, if such a situation arises, it is necessary to put the damaged thing in order as soon as possible. The utility copes with this task perfectly. fsck. It is indeed very effective and system administrators very often use it first of all to restore or repair file systems.

How does fsck work?

Utility fsck (F ile S system Consistency Che ck) initially deeply checked all data structures in a row, i.e. the entire file system. To find errors, she used methods heuristic analysis to speed up and optimize the process of finding errors. However, even in this case, for large file systems, this procedure could take many hours.

Later, a scheme for assessing the state of the file system was implemented, which is based on the sign of a “clean bit of the file system.” If a failure occurred and the file system (FS) was incorrectly dismantled, then this bit was set in the FS superblock. By default, in Linux systems, at one of the stages of system boot, file systems are checked, which are registered in the files /etc/fstab, /etc/vfstab, as well as in /etc/filesystems. Thus, by analyzing the “clean bit” of the FS during system boot, the utility determines whether it is worth checking.

Journaled file systems currently allow the utility to work only with those data structures that really need to be repaired or restored. If necessary, fsck can restore the entire FS thanks to the same FS logs.

Some features of using fsck in Linux

For Linux systems, quite often (especially when using the ext FS), the FS check can be organized in such a way that it will be carried out after a certain number of uninstalls, even if the FS is completely functional. This is especially true for desktop computers, which can be turned off/on every day, rebooted due to the nature of their operation and application, as well as due to free access to them for connection external devices. In such cases, checking the FS (although it is a useful and favorable procedure) turns out to be too frequent and therefore pointless.

By default, in Linux, the FS is checked after 20 dismantlings. In order to change the number of dismantlings after which a FS check is needed, you need to use the command tune2fs:

$ sudo tune2fs -c 50 /dev/sda1 tune2fs 1.44.1 (24-Mar-2018) Setting maximal mount count to 50

fsck syntax and basic options

The team fsck the following syntax:

Fsck [parameter] -- [FS parameters] [<файловая система> . . .]

Main parameters:

In addition to the basic options for fsck, there are also specific ones that depend on the task being performed and/or the FS. You can read about this in more detail in the corresponding pages using the command man fsck. The table of contents of the main manual for the utility (under "SEE ALSO") contains links to other pages, such as fstab(5), mkfs(8), fsck.ext2(8), fsck.ext3(8), etc. Information These links can be viewed by running the man command with the appropriate parameters, for example man fsck.ext3.

The following table lists additional (special) options, as well as the most commonly used options, allowing you to use the command with maximum flexibility and efficiency:

Examples of using fsck

For the most typical situation, typical for cases when you need to restore (or rather “repair”) the FS, for example on the /dev/sdb2 device, you should use the command:

$ sudo fsck -y /dev/sdb2

Here the -y option is necessary, because without it you will have to give confirmation too often. The following command will allow you to perform a forced check of the FS, even if it is working:

$ sudo fsck -fy /dev/sdb2

One of the most useful is the option that allows you to mark bad sectors and this option is used most often. Typically, such situations (with damaged sectors) occur after failures caused by an abnormal power outage:

$ sudo fsck -c /dev/sdb2

Work with file systems must be carried out when they are unmounted from partitions. However, if a situation arises when you still need to check on mounted FS, then before using the fsck command with the appropriate option, you must first remount the desired FS in read-only mode:

$ sudo mount remount,ro /dev/sdb2 $ sudo fsck -fy /dev/sdb2

To specify which FS to use for a partition:

$ sudo fsck -t ext4 -y /dev/sdb2

If fsck fails to correct/repair the FS (which happens very rarely), then this may be due to a damaged FS superblock. It can also be restored because superblocks are backed up. But first you need to find out at what addresses these copies were written, and then try to restore the superblock from one of them backup copies:

$ sudo fdisk -l $ sudo mkfs -t ext4 -n /dev/xvdb1 $ sudo fsck -b 163840 /dev/xvdb1

The -l command is mentioned in this example to illustrate the fact that you first need to know which device to work with, since it displays a list (in this conclusion omitted) available partitions. The mkfs command is intended for creating a file system, but with the -n option it can be used to obtain information about the file system, including the location of superblocks. You should ensure that the -t switch for mkfs specifies the file system corresponding to the actual state, in in this case ext4.

Conclusion

In this article we looked at the operation and use of the utility fsck. As can be seen from the article, using the utility doesn't provide much complexity. And its capabilities for checking and restoring file systems in Linux are quite large, so knowledge of this utility system administrator simply necessary.

If you find an error, please highlight a piece of text and click Ctrl+Enter.

I also had to face this problem. My one friend who has installed Ubuntu like an old one ASUS laptop, and who simply does not want to turn on his brain at least sometimes, came to me with such a problem. The new Ubuntu 12.10 is installed on his laptop and very often the system simply does not want to boot, throwing it into a black screen or freezing on a purple background. But in lately This message started popping up, something like “The operating system could not boot. Select for further actions the desired key..." And then there is a description of what you need to click. I don’t remember exactly which keys the system suggests pressing, but the meaning is that to automatically correct errors, press such and such a key, for manual debugging another, and to ignore this message you are asked to press a third button. Automatic error correction did not lead to anything and the loading of the operating system never reached its logical conclusion. So I decided to try the famous team fsck.

First you need to boot from either bootable flash drive with Ubuntu (Lubuntu, Xubuntu, Kubuntu, etc.), or with Ubuntu disk Live CD. Now we need to find out which Ubuntu partition we need to scan to fix the file system. Launch Terminal (Ctrl-Alt-T) and execute the command:

sudo fdisk -l

This command will show us all the disks and flash drives that are mounted to the system. I'll give an example with mine personal computer, and not with a friend’s laptop. Here's what I got:

ubuntu@ubuntu:~$ sudo fdisk -l

Disk /dev/sda: 640.1 GB, 640135028736 bytes
255 heads, 63 sectors/track, 77825 cylinders, total 1250263728 sectors



Disk identifier: 0x0009d6f7

/dev/sda1 * 2048 61442047 30720000 83 Linux
/dev/sda2 61442048 73730031 6143992 82 Linux swap / Solaris
/dev/sda3 73730048 1250263039 588266496 83 Linux

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xb9ff6f01

Device Boot Start End Blocks Id System
/dev/sdb1 * 16065 100197404 50090670 83 Linux
/dev/sdb2 105322201 976771071 435724435+ 5 Extended
/dev/sdb3 100197405 105322139 2562367+ 82 Linux swap / Solaris
/dev/sdb5 105322203 832110591 363394194+ 7 HPFS/NTFS/exFAT
/dev/sdb6 832112640 860755218 14321289+ 83 Linux
/dev/sdb7 860758016 862613503 927744 82 Linux swap / Solaris
/dev/sdb8 862615552 976771071 57077760 83 Linux

Partition table entries are not in disk order

Disk /dev/sdc: 8115 MB, 8115978240 bytes
250 heads, 62 sectors/track, 1022 cylinders, total 15851520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xc3072e18

Device Boot Start End Blocks Id System
/dev/sdc1 * 32 15847625 7923797 b W95 FAT32

As you can see from the command output sudo fdisk -l, I have 2 hard drives(sda)640 GB and (sdb)500 GB, as well as a flash drive (sdc)8GB, from which I actually booted. I know that my base system with Ubuntu 12.04 is located on the sda ​​disk, and the partition with the operating system is called sda1.

Now that we know the partition that needs to be scanned, we can actually start checking it. In the Terminal:

sudo fsck -y -f -c /dev/sda1

If you see an error, you most likely need to unmount this partition:

sudo umount /dev/sda1

Command keys and parameters fsck:

y- always answer yes to all questions (there is an alternative: key p - starts checking in fully automatic mode);

f- forced file system check (even if the file system is marked as fully functional)

c- looks for bad blocks, and then marks them accordingly

/dev/sda1- device or partition that needs to be checked. Although the team may have a different appearance. For example:

sudo fsck -p /dev/sda1

In this case, only the -p switch has been added. You just read about all the fsck command keys and add exactly the keys you need. To find out about all the program's capabilities, enter in the Terminal:

man fsck

This is what the Terminal produced after checking:

ubuntu@ubuntu:~$ sudo fsck -y -f -c /dev/sda1
fsck from util-linux 2.20.1
e2fsck 1.42.5 (29-Jul-2012)
Checking for bad blocks (read-only test): 0.00% done, 0:00 elapsed. (0/0/0 errdone
/dev/sda1: Updating bad block inode.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information

Linux is one of the most reliable operating systems you'll ever see, but that doesn't mean the hardware running Linux is as reliable. Hard drives may work with errors and, as a result, you will get errors on your file systems. No matter how reliable your operating system if you accidentally deleted necessary files or catalogs. However, do not despair if something similar happens to you. Linux has everything you need to help you recover lost files as a result of deletion or failure of disks and file systems. What tools are we talking about? First of all, we will look at the utilities e2fsck, scalpel And lsof. In today's post, we will see how using such a set of tools you can correct FS errors and recover deleted files.

Checking FS ext2/ext3/ext4 using e2fsck

Utility e2fsck is a descendant of the famous UNIX utility fsck, designed to check file systems. With the help e2fsck you can check for errors and perform repair work on file systems ext2/ext3/ext4.

One of the most important points The thing about working with e2fsck is that it can only be used to work on an unmounted file system, otherwise you can get yourself even more headaches, which is what the utility itself warns about when you try to launch it to work on a mounted file system. If the FS being checked is not root, then you can shut down all users and switch to single-user mode ( init 1), unmount the FS and work with it.

However, the author still recommends using one and, after booting from it, perform all the work. Using this method, you will have unmounted file files at your complete disposal without the need to perform any additional actions.

If for some reason you choose the first option, then after you switch to single-user mode:

# init 1

unmount the file system needed for work:

# umount /dev/sdb1

and after successful unmounting, run e2fsck:

# e2fsck -y /dev/sdb1

Option "-y" informs the utility e2fsck that we agree in advance with all her questions and are leaving to drink coffee, in the hope that she will do everything on her own. Depending on the size of the file system, verification and recovery may take some time. After completing the check, you can always run the test again to make sure that no new errors have arisen in the FS, which may be caused by hardware problems with the drive.

After all checks and repair work are completed, you can mount the verified file system and return back to multi-user mode. Or you can simply reboot the system.

Recovering deleted files using /proc and lsof

Now let's look at the recovery process deleted files. In general, the reason you can recover a deleted file is the fact that "file" is just a reference to the file's inode ( inode). It is in inode information about the physical location of the file is stored. When you delete a file, you are actually just deleting the link to inode, while the descriptor itself will exist for some time: until the process that previously opened this file releases the corresponding descriptor for writing. Thus, there is some time, albeit short, during which it is possible to restore the contents of a deleted file. The key to this process is the file system, which contains, among other things, information about all processes running on the system and the files they open. Each process running in the system has a corresponding PID directory in /proc. Knowing the PID of the process that is still holding the deleted file open, we can always restore its contents from the /proc// directory of the process that opened it. Let's get on simple example Let's see how it's done.

First let's create some file:

$ echo "Very important data" > ~/myfile.txt

Now we have a file myfile.txt with important data, located in the home directory. Let's try to remove it and then restore it as follows. First we will open the file for viewing with the command less, after which we will pause its work, thus leaving the file we need open. So, step by step.

Open the file with the command less to view

$ less ~/myfile.txt

Once the file is open and you can see its contents, click Ctrl+z to pause execution less.

Delete the file:

$ rm ~/myfile.txt

Make sure the file no longer exists

$ ls -l ~/myfile.txt

Since the work previously launched by us less is not completed yet, the file remains open for it and is not actually deleted. Let's restore it.

First you need to find out the PID of the process that opened the file and the file descriptor number. This can be done using the program lsof:

$lsof | grep myfile.txt less 2675 ashep 4r REG 8.1 37 294478 /home/ashep/myfile.txt (deleted)

In the second output field lsof contains PID - 2675, and the fourth descriptor number is 4. Now you can start recovery:

$ cp /proc/2675/fd/4 ~/recovered.txt

Check if the content is in the file we need:

$ cat ~/recovered.txt Very important data

As you can see, everything went well and we were able to recover the deleted file.

Recovering deleted files using Scalpel

Once the process that opened the file exits, recovering the file becomes more difficult because the inode is freed and all communication between the data in the disk blocks and the file system is lost. As long as the data is not physically overwritten on the disk, it is possible to restore it using the utility Scalpel. This tool traverses the contents of the disk block by block and analyzes its contents, trying to find signs that files exist there. To search Scalpel uses patterns from sequences of bytes inherent in certain file types. For example, PNG files contain the sequence of bytes in the header \x50\x4e\x47.

Scalpel you will find in the repositories of most modern distributions. After installing the utility, the first thing you need to do is decide what files the program will look for when running. Search pattern definitions are in the file /etc/scalpel/scalpel.conf. By default, the contents of the file are completely commented out and before you start working, you need to uncomment the necessary templates and/or add your own. The template description format is quite simple:

Extension case_sensitive size header

• extension defines the file extension that Scalpel will be added upon restoration;
• case_sensitive tells the utility whether the case of characters in the search pattern is important;
• using size is determined maximum size recoverable files;
• V header and optional footer the sequences of the file header and its bottom part are described, respectively.

For example, a template definition for JPG files might look like this:

Jpg y 200000000 \xff\xd8\xff\xe0\x00\x10 \xff\xd9

After you have made the necessary changes to configuration file Scalpel and prepare an empty (required!) directory to save the found files, you can start the search and recovery process:

# scalpel -o ~/recovered /dev/sdb1

where using the option "-o" The path to the directory for saving the found files is determined. The utility's operating process is usually very long, since it scans the entire device, so take a moment and go for a walk outside, fresh air I haven't bothered anyone yet ;)

After Scalpel completes its work, examine the contents of the output directory to see if it contains the files you need.

Conclusion

Few people would like to find themselves in a situation where important data is accidentally deleted or damaged. And although Linux offers tools for recovering lost data; there is little pleasure in this process. Therefore, always be extremely careful about your data and how you work with it. And, of course, do not forget about timely treatment - an old and proven method that has saved more than one thousand nerve cells from certain death.

The fsck program is used to check file systems and correct file system errors if any are found. The main requirement for checking a file system is that the file system must be unmounted. Running f century on an already mounted file system can lead to its destruction - then even fsck will not help. The fsck program can be used to check file systems that are supported by the Linux kernel.
The program call format is as follows:
sudo fsck [options] [filesystem]

Parameters, like the file system, can be omitted. If you do not specify a file system, the program will begin checking all file systems listed in the /etc/fstab file. This is highly undesirable because these file systems may be mounted, possibly causing the file system to become corrupted.

The file system check sequence should be as follows:
1. Unmount the file system.
2. Run f sck to check it.

For example, to check the file system of the /dev/hda5 partition, first unmount it and then run f sck:
sudo -i
# umount /dev/hda5
# fsck /dev/hda5

But sometimes we cannot unmount the file system, for example when we need to check the root file system. In this case, you need to do the following:
1. Reboot into single-user mode.
2. Remount the root file system in read-only mode.
3. Check the file system.

To reboot in single-user mode, reboot the system (reboot command), and when booting, pass the single parameter to the kernel.
In single-user mode, as you would expect, only one user can work - root.
All services are turned off, so nothing should interfere with checking the file system. To remount the file system, enter the command:
# mount -o remount th -t ext3 /
The -o option to the mount command allows you to specify various options. In this case, we specify the remount and go options, which means remount in read-only mode. The -t parameter specifies the file system type - ext3, and the last parameter is the root file system (/).