How to Find and Delete Duplicate Files in Mac OS X

Duplicate files such as documents and photos can occupy a surprising amount of space on your Mac’s hard disk and you might not even realise they are there. Perhaps you’ve copied files into several folders and external USB drives, and additional copies could exist in the Mail downloads folder. You might even find a few gigabytes of duplicate content in iTunes, normally because several albums from the same artist might contain the same tracks (e.g. an album and a greatest hits).

It’s therefore a good idea to get rid of those unwanted duplicates which are just clogging up your system, especially if you’re a bit low on disk space.

There are several ways to find and delete duplicate files in Mac OS X:

• Use a third-party application
• Manually using Finder
• Removing duplicates songs from iTunes
• Use a Terminal command or script

iTunes also has the ability to find duplicate items, which is covered later in this tutorial.

METHOD 1: Use a Third-Party Application

There are dozens of apps available in the Mac App Store which can find and remove duplicate files. Some are free whereas others charge a small fee. They all do pretty much the same thing but some have much more comprehensive features and allow you to filter by filename only or the actual content. For example, it’s possible that multiple identical files have different filenames – utilities such as those mentioned below can help to analyse the contents of files on a byte level to determine if they really are duplicates.

The apps below are just a few examples that can be found in the Mac App Store. Many similar apps can be found on the Internet, but it’s safe to assume that applications in the App Store are legitimate and don’t contain any malware.

• ACDSee Duplicate Finder
• Araxis Find Duplicate Files
• Chipmunk
• Gemini: The Duplicate Finder
• Search Duplicate Files

METHOD 2: Use Finder to Locate Duplicates

Duplicate files can be found quite easily using Finder. The process is rather slow because you need to manually decide which ones to keep and which to delete, but with just a little time and patience you can locate most duplicate items.

Follow the method below:

1. Open a new Finder window
2. In the search field, enter the wild card asterisk symbol
3. Make sure the Size and Kind columns are displayed
4. Filter the results by Kind, and you will see the list of similar items

This method is fairly accurate because it shows identical files listed by name, type, and size. Just delete any files that you don’t want to keep but make sure they really are duplicates first. Unfortunately, this method doesn’t let you determine whether the files are true byte-level copies but there is enough information presented to be able to make an informed decision. The main drawback with this method is the amount of time it takes to go through the items one at a time.

Using Finder to find duplicate files

A Note on How to Remove Duplicate Items in iTunes

Fortunately iTunes provides the ability to check for duplicate files, as follows:

1. Make sure you’ve highlighted the relevant item in your iTunes library (such as Music, Films, Podcasts, etc)
2. In the File menu, select Display Duplicates

Any duplicate items found will be displayed in the main iTunes window, allowing you to simply delete them as normal.

How to find duplicate items in iTunes

METHOD 3: Use a Terminal Command or Script

It’s possible (but certainly not recommended for novices) to use “shell commands” in Terminal to list and delete duplicate files. Generally speaking, it’s considerably easier to use a third-party application but some people prefer the flexibility that Terminal provides.

Follow the steps below to view the list of duplicate files.

1. Open Terminal from Spotlight or the Utilities folder
2. Change to the directory (folder) you want to search from (including sub-folders) using the cd command. At the command prompt type cd <folder name> for example cd ~/Documents to change directory to your home Documents folder
3. At the command prompt, type the following command: find . -size 20 \! -type d -exec cksum {} \; | sort | tee /tmp/f.tmp | cut -f 1,2 -d ‘ ‘ | uniq -d | grep -hif – /tmp/f.tmp > duplicates.txt
4. This method uses a simple checksum to determine whether files are identical. The names of duplicate items will be listed in a file named duplicates.txt in the current directory. Open this to view the names of identical files

There are now various ways to delete the duplicates. To delete all the files in the text file, at the command prompt type: while read file; do rm “$file”; done < duplicates.txt. Note: with this method, you’ll need to copy one of each duplicate item to another folder before you enter the command because it deletes every file listed.

There are more complex but useful Terminal commands and scripts available on the Internet which can automate the process and ensure that for every duplicate the original is kept, but it’s only really recommended for experts as there is a lot of scope for things to go wrong.