IMAP, tracking moved messages using message id

1
votes

We are developing a mail client written in Java. It has same functionalities like Outlook or Thunderbird, etc. It communicates with the mail server directly. Additionally, our business rules demand that we store all messages in our database and messages should be kept synchronised always. I know that is not very suitable for IMAP, but we must keep everything in our database. Question arises, how to track an IMAP message moved from folder A to folder B? How can we get informed about that? If you remove a message from A, it is deleted from A and it created newly in B, as a result: The UID value of the message is changed. Can we rely on the MessageID found in the headers? I checked some mails servers and see that the message id in the headers remain unchanged. But i have read somewhere, that the messageids can be empty depending on the mail server.

  • Are the MessageID in headers always set, can be cases or mailservers that they leave it blank?
  • Are the MessageID value in headers unique in an IMAP folder?
  • Is it possible that it gets changed when message is moved or folders UIDVALIDITY changed?

  • What about setting a custom header during fetch? When I add a non-standart header name value pair, will it be kept on the mail server or is it possible that non-standart mail heraders will be deleted by mail server? Is it a bad idea applying a non-standart header value?

    IMAPMessage m;
m.setHeader("myHeader", "myValue");

  • There were some suggestions in stackoverflow, it is said to generate a hash including messageId and other parameters such as sender, subject etc, is it a safe approach? We can get conflicts if there is no unique MessageID is provided or no MessageID is provided.

1
javaemailimapmovemessageid
No, you can't count on messageids being unique or present. You can't add a header as imap messages are immutable. - Max
Your best bet is to go with your suggestions you mention at the end - create a hash that combines sender, receiver, subject, date and as many other unique fields as possible. - Fallso
And beware, it's pretty simple to create a copy of a message (eg, IMAP COPY command), so even all of that can still lead to duplicates. Messages themselves aren't unique. - Max
what about setting custom flags? do they remain same when message moved between folders? - benchpresser

1 Answers

0
votes

There are three things you can do.

First, message-id. You can rely on the message-id being present and unique these days if your mode of failure is good enough. In your case, if the message-id is not there and a message is moved, is the failure just that you waste space in the database and/or download the message twice? The wasted space will be small these days.

Second, x-gm-msgid. That's a gmail-specific feature, a 63-bit number that never changes. If two messages have the same x-gm-msgid, they are the same.

Third, the COPYUID response code tells you about moves, but only applies when you do the moving, not when someone else does.

Put together, these should give you a fairly good understanding of how the user's mailboxes change.