A program that searches for similar files. Duplicate files on your computer - why do you need them?

Good day.

Statistics are an inexorable thing - for many users hard drives Sometimes there are dozens of copies of the same file (for example, pictures, or music track). Each of these copies, of course, takes up space on the hard drive. And if your disk is already “filled” to capacity, then there can be quite a lot of such copies!

Cleaning duplicate files manually is not a rewarding thing, which is why I want to collect in this article programs for finding and removing duplicate files (even those that differ in file format and size from each other - and this is a rather difficult task !). So…

List of programs for finding duplicates

1. Universal (for any files)

They search for identical files by their size (checksums).

Under universal programs, I understand, those that are suitable for searching and deleting duplicates of any type of file: music, movies, pictures, etc. (below in the article, “its own” more accurate utilities will be given for each type). Most of them all work according to the same type: they simply compare file sizes (and their checksum), if among all the files there are identical ones in this characteristic - they show you!

Those. thanks to them you can quickly find it on disk full copies(i.e. one to one) files. By the way, I’ll also note that these utilities work faster than those that are specialized for a specific type of file (for example, image search).

DupKiller

I put this program in first place for a number of reasons:

• supports simply a huge number of different formats in which it can search;
• high speed;
• free and with Russian language support;
• very flexible settings for searching for duplicates (search by name, size, type, date, content (limited)).

Duplicate Finder

This utility, in addition to searching for copies, also sorts them as you please (which is very convenient when there are an incredible number of copies!). Also add byte-by-byte comparison and reconciliation to the search capabilities checksums, deleting files with zero size (and empty folders too). In general, this program does a pretty good job of finding duplicates (both quickly and efficiently!).

Those users who are new to English will feel a little uncomfortable: Russian is not in the program (maybe it will be added later).

Glary Utilities

In general, this is not one utility, but a whole collection: it will help you delete “junk” files, set optimal settings on Windows, defrag and clean hard drive etc. Including this collection there is a utility for searching for duplicates. It works relatively well, which is why I will recommend this collection (as one of the most convenient and universal - as they say, for all occasions!) once again on the pages of the site.

2. Programs for finding duplicate music

These utilities will be useful to all music lovers who have a decent collection of music accumulated on their disk. I’m picturing a fairly typical situation: you download various music collections (100 best songs October, November, etc.), some of the compositions are repeated in them. It is not surprising that, having accumulated 100 GB of music (for example), 10-20 GB can be copies. Moreover, if the size of these files in different collections were the same, then they could be deleted by the first category of programs (see above in the article), but since this is not the case, then these duplicates cannot be found by anyone except your “hearing” And special utilities (which are presented below).

Music Duplicate Remover

The result of the utility.

This program differs from others, first of all, in its quick search. It searches for duplicate tracks by their ID3 tags and sound. Those. she will listen to the composition for you, remember it, and then compare it with others (thus doing a huge amount of work!).

The screenshot above shows the result of her work. She will present her found copies to you in the form of a small tablet, in which a percentage similarity figure will be assigned to each track. In general, quite convenient!

A

Found duplicate MP3 files...

This utility is similar to the one above, but it has one undoubted advantage: the presence of a convenient wizard who will guide you step by step! Those. a person who launches this program for the first time will easily figure out where to click and what to do.

For example, in my 5000 tracks in a couple of hours, I managed to find and delete several hundred copies. An example of how the utility works is shown in the screenshot above.

3. To search for copies of pictures, images

If you analyze the popularity of certain files, then the pictures will probably not lag behind the music (and for some users they will surpass them!). It’s hard to imagine working on a PC (and other devices) without pictures! But searching for images with the same image on them is a rather difficult (and long) task. And, I must admit, there are relatively few programs of this kind...

ImageDupeless

A relatively small utility with fairly good performance in finding and eliminating duplicate images. The program scans all the images in the folder and then compares them with each other. As a result, you will see a list of pictures that are similar to each other and you will be able to make a conclusion about which one to keep and which one to delete. It is very useful, sometimes, to thin out your photo archives.

ImageDupeless example

By the way, here is a small example of a personal test:

• experimental files: 8997 files in 95 directories, 785MB (archive of pictures on a flash drive (USB 2.0) - gif and jpg formats)
• gallery occupied: 71.4MB
• creation time: 26 min. 54 sec.
• time for comparison and output of results: 6 min. 31 sec.
• result: 961 similar image in 219 groups.

Image Comparer

I have already mentioned this program on the pages of the site. Represents the same small program, but with pretty good image scanning algorithms. There is a step-by-step wizard that starts when you first open the utility, which will guide you through all the “thorns” of first setting up the program to search for duplicates.

By the way, just below is a screenshot of the utility’s operation: in the reports you can view even small details, where the pictures are slightly different. In general, it’s convenient!

4. To search for duplicates of films and videos

Well, the last popular file type that I would like to dwell on is video (films, videos, etc.). If once before, having a 30-50 GB disk, I knew in which folder where and what movie takes up how much (and they were all against each other), then, for example, now (when disks have become 2000-3000 GB or more) - they are often found the same videos and films, but in different quality(which can take up quite a lot of hard drive space).

Very convenient function V CCleaner application is to find duplicate files. Very often, there are files on your computer that are identical in date, size and name. Of course, some of them are needed, and some may have been created accidentally or downloaded several times from the Internet. All these files eventually accumulate, free space becomes less and less, and, as a result, the computer begins to slow down. Therefore, from time to time, you need to get rid of such files. If you are an advanced PC user, then you will not have any difficulty finding the files CCleaner duplicates which ones to delete, but if you are new to this matter, then we will help you figure it out.

What files should not be deleted

Before we start searching for duplicates and deleting them, let's look at whether it is possible to delete duplicate files using Cyclener? To begin with, I would like to note that the program will not allow you to delete absolutely all copies of a file. One of them must remain untouched. Further, we do not recommend deleting system files. It is quite normal for them to have duplicates. Typically, system files are located on drive C in the Windows folder.

Files that can be deleted

Typically, a computer consists of several partitions (disks). The amount of information that is stored on each of them is most likely impressive. There are pictures, music, videos, photographs, and much more. Some of the duplicate files could have been duplicated by the user by mistake, for example, due to forgetfulness, the file was saved in different sections. Some files may have been downloaded from the Internet several times, etc. And when the program finds such files, you can safely delete them from your computer.

Find duplicates

In the " Service"there is a section" Search for duplicates».

In this section, at the user's discretion, you can set search criteria. You can search for duplicates by any one search parameter: by size, by date, by name and by content, or by several parameters at the same time, marking them with checkboxes.

You can also define the files that need to be skipped. There are several options here:

• Zero size files;
• Files that are read-only;
• Hidden files;
• System files;
• Files whose size does not exceed the megabyte size you specify;
• Files larger than the specified megabyte size.

In the " Inclusions» you can specify the places where the search will be carried out. To scan a specific folder, you must select " Add" The following window will appear

Click on the button " Review" and select the desired folder, indicating the path to it.

In the " Exceptions» you can specify those folders that should not be affected during the search.

You can add them similarly to the “ Inclusions" Click " Review" and select this folder.

After setting all the search parameters, click on the button “ Find».

Duplicate search results

After searching, the results will be shown in table form.

It will indicate the file names, locations where they are located, their sizes and creation dates.

To remove duplicates, check the boxes next to them. If you right-click on any file, a context menu will appear.

After you have selected all the duplicates that you want to delete, click on the “ Delete».

Conducting effective spring cleaning on the computer's disk space, along with the use of programs for automatic Windows cleaning, still requires manual work to remove unnecessary files and duplicate files. Designed to track unnecessary files special programs– analyzers disk space, they help filter the contents of computer disks according to certain criteria (in particular, weight) so that the user can decide whether to delete files or leave them. To find duplicate files there is also special software- either in the form individual programs or small utilities, or as part of complex software for cleaning and Windows optimization. Below we will look at five programs for finding duplicate files. Completely included in the top five free programs with Russian language support.

When searching for duplicate files on system disk It is better to indicate not the entire partition, but only individual folders where they are stored. user files. Duplicates found in workers Windows folders, cannot be deleted. If you find some heavy folders or files with unfamiliar names on your C drive, it is recommended that you get help about them on the Internet.

1.AllDup

AllDup is equipped with many different options for fine tuning search for duplicate files. Customizable options include selecting a comparison method, search criteria, priority for checking duplicates, enabling exclusion filters, including content archive files etc. It is even planned to change the design themes and individual settings interface. The program is good, however, with a somewhat ill-thought-out interface. On its toolbar, all the tabs - even basic operations, even additional functions– are listed as equivalent. And to help the user who launched AllDup for the first time master the specifics of the program, its creators equipped the interface with a floating widget in the form quick guide, which step should be taken after which. In the “Source Folders” tab, you specify the search area - disk partitions, connected devices, or individual user folders on the system drive C.

Next, in the “Search Method” tab, set the search criteria. Here you can specify the search for duplicates and add extension, size, content, etc. to the preset search criteria for file names.

Duplicate search results can be sorted by size, path, file modification date, etc. Detected files can be deleted, their location can be opened in Windows Explorer, apply other actions provided for by the program.

To return to the analysis of AllDup duplicate search results at a later time, but without waiting for the scan to complete, the current results can be saved to a program file format or exported to TXT files and CSV.

AllDup has portable version, which does not require installation into the system.

2. Duplicate Cleaner

One more functional program for a highly customizable search for duplicate files – Duplicate Cleaner. Duplicate program Cleaner exists in two editions - paid Pro and free. Although the latter is limited by the unavailability of some functions, its capabilities will be sufficient for effective search duplicates. Duplicate Cleaner Free allows you to set search criteria by file name, content, size, and creation date. Additional search criteria are provided for audio file data, as well as filtering by content types and file extensions. All these points are configured in the first tab of the program “Search Criteria”.

In the second tab of the program – “Scan path” – the search area is selected.

In the search results window, duplicates can be sorted, deleted, their location can be opened in Explorer, and other program options can be applied to them.

Duplicate Cleaner search results can be exported to a CSV tabular data format file. Data export is carried out both for the entire list of duplicates and for only files marked by the user. Among the advantages of Duplicate Cleaner are: user-friendly interface and thoughtful organization.

3. DupeGuru

DupeGuru is the simplest duplicate file search engine for those who do not have the time or desire to master all the intricacies of functional, highly specialized programs such as those discussed above. At the bottom of the small program window, a search area is selected and scanning starts.

Duplicate search results can be sorted by location path and file size. Context menu DupeGuru search results contain only the operations necessary to work with found duplicates.

Search results are saved to a program file or exported to HTML.

DupeGuru is a cross-platform program, but only its older versions are adapted for Windows. The program installer for Windows 7 offered on the official website is also suitable for system versions 8.1 and 10.

4. CCleaner

At one of the stages of its improvement, the most popular cleaner for Windows, CCleaner, received a function to search for duplicate files. You can use this function in the “Service” section. To search for duplicates, search criteria are available by name, creation date, file contents, and file size.

Like the previous reviewer, CCleaner program in the duplicate search results environment it is not particularly rich in functionality, but basic operations are present. This, in particular, sorts search results and deletes files.

5. Glary Utilities 5

Comprehensive program for cleaning and optimizing Windows Glary Utilities 5 among the arsenal contains a utility for finding duplicate files. The same utility, if desired, can be downloaded separately from the official website of the program if the full power of the software package is not required.

The duplicate finder included in Glary Utilities 5 is simple, but convenient and customizable. You can start scanning duplicates immediately by selecting only the search area - disk partitions, removable devices or separate folders. You can further refine your search by clicking the “Options” button.

The options configure the criteria for searching for duplicates - by name, by size, by time of file creation. In some cases of thorough cleaning of disk space, you can change the preset selection of searching only among common file types to scanning all types.

The search results provide only the necessary operations for working with duplicates, in particular, deleting and opening the placement path in Windows Explorer.

For the convenience of the user, Glary Utilities 5 catalogs found duplicates by content type - documents, pictures, videos, programs, etc. For each type, the total weight of found files is displayed.

Have a great day!

Sometimes in everyday computer activities the task arises of finding duplicate files. There can be many reasons for this: lack of space on the hard drive, attempts to reduce entropy in your files, deal with those dumped in different times photographs from the camera and many other necessary cases.

You can find it online large number programs that allow you to search for duplicate files. But why look for any programs if a smart tool for such work is usually always at hand. And this tool is called Total Commander (TC).

In this article I will show all the methods based on Total Commander versions 8.5 , in this version, the search for duplicate files has become very rich in functionality.

!!!A small important digression. What do you mean by duplicate file? Two files are IDENTICAL only if they are exactly the same bit by bit. Those. Any information in a computer is represented by a sequence of zeros and ones. So, files match only when they completely match the sequence of zeros and ones that make up these files. All talk about how you can compare two files on any other basis is deeply erroneous.

TC has two, essentially different, methods for finding duplicate files:

• Synchronize directories;
• Search for duplicates;

Their features and applications are best illustrated with examples.

1.Directory synchronization.

This method is used when your two folders being compared have an identical structure. This usually happens in many cases, here are a few of them:

• Have you regularly archived your work folder? After some time, you need to find out which files have been added or changed since the archive was created. You unpack the entire archive into a separate folder. The folder structure in it practically coincides with the working one. You compare two folders “original” and “restored from archive” and easily get a list of all changed, added or deleted files. A couple of simple manipulations - and you remove from the recovered folder all the duplicate files that are in the working one.
• You are working in a folder on network drive and regularly make a copy to yourself local disk. In time your working folder has become quite large and the time spent on a complete copy has become very large. In order not to copy the entire folder each time, you can first compare it with the backup one and copy only those files that have been changed or added, and also delete them in backup folder files that were deleted from the main one.

Once you get the hang of it and feel the full power of this method, you yourself will be able to come up with thousands of situations where the directory synchronization method will be of great help to you in your work.

So, how does everything happen in practice? Let's get started.

Let's assume we have a main folder "Working", which contains the files with which you are working. And there is a folder "Archive", which contains an old copy of the folder "Working". Our task is to find duplicate files in both folders and remove them from the folder "Archive".

Open TC. In the right and left panels, open the folders being compared:

Press menu “Commands” - “Synchronize directories...”

The directory comparison window opens

Next we need to set the comparison parameters. Put a tick in the parameters “with subdirectories”, “by content”, “ignore date”

• "with subdirectories"— files in all subdirectories of the specified folders will be compared;
• "by content"- this is the key option that forces TC to compare files BIT by BIT!!! Otherwise, files will be compared by name, size, date;
• "ignore date"- this option forces TC to show differing files without trying automatic detection directions for future copying;

!!! Only files with the same names will be compared!!! If the files are identical, but they have a different name, then they will not be compared!

Press the button "Compare". Depending on the size of the files, the comparison can take a very long time, do not be alarmed. Eventually the comparison will end and the result will be displayed in the bottom status line (section 1 in the figure):


If the buttons in the “Show” section (section 2 in the figure) are pressed, then you will see the comparison result for each file.

— this button enables the display of files that are in the left panel, but not in the right;

— this button enables the display of identical files;

— this button enables the display of differing files;

— this button enables the display of files that are in the right panel, but not in the left;

If you initially have all display buttons turned off, then the result of the comparison can only be assessed by the status bar (section 1 in the figure above), in in this case we see that 11 files were compared, of which 8 files are the same, 2 files are different, and there is also a file in the left panel that is not in the right panel.

To complete our task, it is necessary to leave the display of only identical (identical) files, so we turn off all other display buttons


Now we only have identical files left, and we can safely delete them in the folder "Archive". To do this, select all files. The easiest way to do this is by pressing the universal combination CTRL+A. Or first select the first line with the mouse, then press the key on the keyboard SHIFT and without releasing it, select the last line with the mouse. As a result, you should get something like this:

The final step is to right-click on any line and select the item in the menu that opens "Delete on the left"

TC kindly asks us about our desire,

and if we press "YES" then it deletes all marked files in the folder "Archive".

After this, the two folders are automatically compared again. If you do not need a repeated comparison, the process can be interrupted by clicking on the button "Abort" or press a key ESC on the keyboard. If the repeated comparison was not interrupted, and we turned on all the display buttons, then we will see a window like this

All. The task has been completed. All identical files found and deleted in the folder "Archive".

Educational video on the topic

2.Search for duplicates.

Fundamental difference this method from the directory synchronization method is that TC ignores the names of the files being compared. In fact, it compares each file with each, and shows us identical files no matter what they are called ! This search is very convenient when you do not know either the folder structure or the names of the files being compared. In any case, after searching for duplicates, you will receive an exact list of identical files.

I will demonstrate finding duplicates using one practical task, finding duplicates of personal photos. Quite often you dump photos from your digital gadgets. Often the situation gets confused, something is reset many times, something is skipped. How to quickly delete files that have been dropped multiple times? Very simple!

Let's get started.

Let's say you always dump all your photos into a folder "PHOTO" on drive D. After all the resets, the folder looks something like this:

As you can see, some files are located in folders named by the date of shooting, some are dropped to the root of the folder "_New" And "_New1"

To start searching for duplicates, open the folder in which we will search in any TC panel. In our case this is the folder "PHOTO"

Next, press the key combination on the keyboard ALT+F7 or select from the menu “Commands” - “Search files”

The standard TC search window opens. String "Search files:" leave it empty, then all files will be compared.

Then go to the bookmark "Additionally" and check the boxes “Search for duplicates:”, “by size”, “by content” and press "Start Search".

The search can take a VERY long time, do not be afraid of this, since there are a huge number of comparisons of a large volume of files. At the same time, the progress percentage is shown in the status bar

When the search ends, a search results window will open, in which we press the button "Files to panel"

In the search window and in the panel window, identical files are collected in sections separated by dotted lines

Each section displays the file name and full path to the file. The names of IDENTICAL files can be completely different!
In this case, it is clear that the same photograph was recorded THREE times, twice under the same name( IMG_4187.JPG) and the third time this photograph was recorded under a completely different name ( IMG_4187_13.JPG).

Next, it remains to select unnecessary identical files and delete them. This can be done manually by selecting each file by pressing a key Ins. But it takes a long time and is not effective. There are better and faster ways.

So our task is to remove duplicate files in folders "_New" And "_New1".
To do this, click on additional keyboard, big key on the right [+] . Typically, using this key in TC, files are selected by mask. The same operation can be done through the menu “Selection” - “Select group”

After a long, constant use of a computer, large amounts of data, i.e., all sorts of photographs, videos, films, music, documents, etc., accumulate on its disks, whatever one may say. When data takes up a lot of space, this is normal, for example, I myself have more than 600 GB of necessary data, and for others it’s even more. But very often duplicate files take up too much space.

Such files can appear when, for example, you transfer them from somewhere to a new location on the disk, forgetting that you already have such files on this disk. And it’s okay if there are a lot of duplicates of all sorts of documents, but when there are a lot of duplicate photos, music and especially videos, then this, as a rule, will take up a lot of your disk space. I recently checked and found that duplicates are eating up about 100 GB. on the hard drive, which, in my opinion, is quite a lot :)

In this article, I will show you an easy way to find all duplicate files in Windows on your drives, so you can easily check them and quickly delete everything you don't need.

In Windows, unfortunately, there are no normal built-in tools for finding duplicate files. There is an option to do this via command line PowerShell, but this is very inconvenient, especially for beginners it will be difficult. Therefore it is easier to use third party programs. One of these is called AllDup. It is completely free, available in Russian, supported by everyone operating systems Windows is finally quite easy to use.

Downloading and installing the AllDup program

The program can be downloaded for free from the official AllDup website. Below is the link to the download section:

The program is available in two versions: regular installation and portable (Portable). The portable version is different in that it does not require installation on a computer, i.e. the program can be launched directly from the downloaded folder.

To download, click the button “Server #1”, or “Server #2” or “Server #3” (if the first button does not download, spare servers are given) under the required version programs.

Direct links to download the latest version (March 2017) of AllDup: standard version, portable version. For latest versions always refer to the official AllDup website!

Installing the program is very simple, one might say, it consists of successive clicks “Next”; no special settings need to be made. That's why this process I won't consider it.

Learn more about the nuances of installing programs for Windows

Finding duplicates using AllDup

After installing the program, run it. The main window for search settings will open:

Setting up a search includes several steps:

These are all the main stages of setting up a search; the rest can be omitted.

Now, to start searching for duplicates, click the “Search” button at the top of the AllDup window:

The search process will begin.

How more files in the folders you specified is located on your disks, the longer the search will take.

After the search is completed, the program will display the found files with duplicates in the form of a table.

The first thing that is better to do right away is to save the search results, because if you now close this window with the results, then you will have to perform the search again. To save, click the button with the image of a floppy disk, or select top menu“Search Result” and click “Save Search Result”.

Now, even if you turn off your computer, and then launch the program again, you will be able to get to the search results again.

You can sort the search results by different parameters by clicking on the column headings in the table. The most useful sorting criterion, in my opinion, is file size. Therefore, if you want the largest files found to be displayed at the top of the table, then click on the “Size (Bytes)” column.

The next thing that is best to configure for ease of viewing results is the displayed size. Initially, the program shows the file size in bytes, which is not very convenient. It is better to display in megabytes or even gigabytes. To do this, click the button marked in the screenshot below (1), then check one of the options (2):

Now I’ll dwell on how to actually use the search results, how to view and remove unnecessary duplicates...

The program divides the found duplicates into so-called groups. One group is all found copies of the same file, including the original (it will also be displayed in this group).

To view duplicates of one of the groups, you need to open it by clicking on the arrow. Example:

Once you have expanded a specific group, you can check what kind of file it is by opening it. To do this, simply double-click on the file in the group or right-click and select “Open file”. The file will be opened in standard program Windows, through which you usually open all files of the selected type.

To delete duplicates, check them, right-click and select one of the options: delete file in Windows Recycle Bin or permanent deletion.

Accordingly, do not delete all files from the group, because this way you will delete both duplicates and the original at once! For example, if there are 3 files in a group, then by deleting 3 at once, you will delete both the original and 2 duplicates. In this case, to keep only a single copy of the file, you need to remove 2 files from the group.

This way you can check each group separately and remove duplicates. But if a lot of information has been found, it can be done simpler. Make sure that the program automatically selects all files in each group except one (i.e., only duplicates), after which you can get rid of all duplicates at once, or before that, go through and double-check whether everything marked is exactly to be deleted.

To automatically mark duplicates, go to the “Select” menu (1) and check and enable one of the options there (2), for example, “Select all files except the first file.”

As a result, the program will select 2 duplicates in each group, and leave the first file in the list unselected. That is, in this way you will mark 2 duplicates, and the original will remain unmarked. Or you can use the “Select” menu to try other options that are convenient for you.

Once the program has marked the files, you can double-check your selection if required. And to quickly delete everything unnecessary or perform some other action, click the button marked in the screenshot below:

In the window that opens, you will see the total volume of the selected files, i.e. how much space the found duplicates take up and the number of selected files. At the bottom you need to select an action on the selected files. You can delete files through the recycle bin, delete them permanently (the “Delete files” item), copy or move files to a folder, and also rename the found duplicates. If you are sure that the marked files are duplicates and you no longer need them, then it is easier to delete them, but in any case, the choice is yours.

So choose required action(1) and click OK (2). You don't have to configure anything else here.

After this, the program will perform the action you selected on the previously marked files!

That's the whole process :) To exit the search results, simply close this window. If you have saved your search results, then if you need this result again, you can get to it through the “Search Result” section (1) in the main program window. The results you saved will be displayed in the table (2). To open desired result just double click on it.

Conclusion

AllDup - very convenient program to find duplicates of your files on your computer. In fact, there is nothing superfluous in the program; it has all the necessary tools, filters and parameters for quickly processing a large volume of found duplicates. Of course, there are similar programs that probably also do their job well. So far I have only tried AllDup and I don’t see any point in changing it yet.