Why duplicate emails happen with pop3

POP3 email protocol is rapidly becoming an artifact of the past. With many now accessing their email over mobile devices they are turning to IMAP to handle their email.

The POP3 protocol has some major differences with the IMAP protocol. The biggest one being that the IMAP protocol works much like the web, your email is stored on the server all the time and when your email client wants to read it it downloads a copy right then and shows it to you. The POP3 protocol, your email client checks the server for new email than downloads the entire message to your computer making it available offline, however this does not leave a copy for other devices such as mobile phones.

The biggest pitfall of POP3 is the implementation among email clients. For example, in Microsoft Outlook typically it is configured to leave a copy on the server for a certain amount of time. This is what causes the duplicate email trouble. Outlook downloads the email and stores information about that email on your computer so that when it goes to download email again it can try to skip that one and prevent a duplicate download.

This behavior of leaving a copy on the server can cause trouble. If outlook gets confused about which messages it has already seen it could download another copy of a message that it has already downloaded, creating a duplicate copy.

This problem is magnified when you try to modify certain settings with your email provider or if you migrate the email to a different server.

When the copies left on the server that Outlook has already downloaded are migrated to another server, Microsoft Outlook says "hey look, this message looks new" and downloads all the email copies again from the new server. This becomes a problem because it has already downloaded the emails once and it is just confused so it downloads them all again from the new server.

There isn't much we can do to combat the problem with duplicates and migration except for a couple work arounds:

  1. Before migration you can sync all your email at a known time, and than the systems admin can remove all email copies that still remain on the server before that set time. This basically works around the duplicates by simply not migrating any duplicates. The old email copies that have already been downloaded are deleted and not migrated, therefore when Outlook connects to the new server there are no old email messages for it to be confused by. This method has one drawback in that when we remove the old emails before a certain time and just migrate new emails that may pop in in the meantime, if Outlook downloads the new email copy from the old server as we migrate it over to the new, there will be some overlap and some duplicates will be there. This is drastically improved though from having hundreds duplicates, down to a few that came in during the migration timeframe.
  2. This second option is not really an elegant one, but it is an option. In this option we migrate all copies of emails on the server and do not remove any that could potentially be duplicates. Once Outlook is switched over to the new server it will download all the emails again and potentially create hundreds of duplicates. The Outlook user will than have to deal with his duplicate emails either by deleting them by hand or by installing a plugin to try and help with removal of the duplicates.

As you can see this problem really only exists with POP3 email, IMAP protocol always leaves it's mail on the server and doesn't retrieve it's own local copies that could potentially create a situation with duplicate emails. IMAP only has a cache of current emails against what the server has stored. IMAP is a much better protocol when using multiple devices for email.


Be the first to leave a comment on this post.

Leave a comment

To leave a comment, please log in / sign up