File Formats – Everything MDaemon

I managed to dig up some information on the IMAP.MRK file, for anyone brave enough to want to modify it pragmatically. The format is actually fairly simple, one header followed by zero or more message records.

If a MSG file has no corresponding record, MDaemon will update the IMAP.MRK file the next time an IMAP client or WorldClient user touches the folder, and the message will be treated as UNSEEN and UNREAD

The header is defined as follows:

struct IMAPMrkHeader
{
unsigned HeaderVersion;
unsigned UIDValidity;
unsigned UIDNext;
unsigned LastWriteCounter;
unsigned Filler0;
unsigned Filler1;
unsigned Filler2;
unsigned Filler3;
unsigned CRLF;
};

Each MSG file will have one record, which is defined as follows:

struct IMAPMrkMessage {
char Filename[MAX_IMAP_FILENAME];
unsigned char Flags;
unsigned UID;
unsigned Size;
time_t Date;
};

#define FLAG_SEEN 32
#define FLAG_ANSWERED 16
#define FLAG_FLAGGED 8
#define FLAG_DELETED 4
#define FLAG_DRAFT 2
#define FLAG_RECENT 1
#define MAX_IMAP_FILENAME 23
#define IMAP_RECORD_SIZE 36

So what do all those fields mean?

HeaderVersion is a tag to identify what file format the IMAP.MRK is using. The current value is 1. UIDValidity is the IMAP UID Validity for the folder. UIDNext is the UID that will be used for the next message added to the folder.
LastWriteCounter is a value that changes whenever something has changed in the file. This was added when IDLE support was added to the IMAP server, so that there’s a quick way to check for changes (by comparing this one value rather than doing a compare of the entire file’s contents).When changing records in the file, lock the IMAP folder, increment the LastWriteCounter, change the records, then unlock the folder.
UIDNext is the next UID to be assigned. To add records, lock the IMAP folder, increment the LastWriteCounter, and for each record you’re adding, use the current UIDNext value as its UID and then increment UIDNext.

So how do you lock a folder?

Lock an IMAP folder by creating a file “IMAP-foldername-email.lck”, where foldername is a “filename-safe” version of the folder name (” and ‘/’ characters replaced with ‘_’) and email is the owner’s email address (use “public” for public folders). If the lock file already exists, something is currently operating on the IMAP.MRK so you should wait until it is unlocked before you lock it and continue.

Hope this helps someone.

The config.xml file stores a number of Outlook Connector related account settings. This file is not formally documented, but since it’s in XML format, most of the items are self-explanatory. Here is a breakdown of the options.

<customCache>0</customCache>

Whether or not you use a custom localcache.db location.

0 – Use default location
1 – Use custom location

<localCacheDir />

If you use a custom localcache.db location, this is the location where it is stored.

<receiptResponse>2</receiptResponse>

Whether or not to send a return-receipt when requested. You can use any number from 0-2, with the following meanings:

0 – Always send a response
1 – Never send a response
2 – Poompt me before sending a response

<ServerTimeout>0</ServerTimeout>

Timeout in seconds for socket communication. I’m not sure if it’s just for IMAP or also for SMTP. The default is 0, which means use the default OS behavior (which I think is no timeout). There is no GUI for this setting.

<auto_send_recv>0</auto_send_recv>
<auto_send_recv_timeout>1</auto_send_recv_timeout>
<SendImmediately>1</SendImmediately>

Outlook 2000 does not have a built-in automatic send/receive ability, so OC provides one. These settings are for whether it’s enabled, and how much time (in minutes) to wait between them. These settings can be set in the GUI on Outlook 2000. It is not used in other versions of Outlook.

<showProgressDialog>1</showProgressDialog>

Whether or not to show a progress indicator dialog when loading messages.

0 – Show no dialog
1 – Show the dialog

<progressDialogThreshhold>50</progressDialogThreshhold>

If you are showing a progress indicator, this setting controls how many messages need to be queued for Outlook Connector to show a progress indicator as messages are being downloaded.

<AccountDisplayName></AccountDisplayName>

This controls what name appears in the “root” of the mailbox in Outlook. This is especially useful if you have multiple accounts and want to easily tell them apart. This is available in the GUI.

<IncomingServer></IncomingServer>
<IncomingPort>993</IncomingPort>

The IMAP server hostname and port number.

<UseIncomingSSL>1</UseIncomingSSL>

Whether or not to use SSL for IMAP sessions.

0 – Don’t use SSL
1 – Require SSL.

<EmailAddress></EmailAddress>

The email address Outlook Connector uses in the From field (and possibly other places internally)

<Organization></Organization>

Your “organization” is sent along with email you send.

<YourName></YourName>

This is the name displayed in the From field (And possibly other places internally)

<UserName></UserName>
<Password></Password>

The IMAP username and password. Note that the password field is encrypted using reversable encryption, which means that although it’s not visible here, it’s possible a skilled attacker could recover the password.

<RememberPassword>1</RememberPassword>

Outlook Connector can optionally save passwords between sessions.

0 – Ask for the password each session.
1 – Remember the password

<OutgoingServer></OutgoingServer>
<OutgoingPort>587</OutgoingPort>

The SMTP server hostname and port number. Typical options are:

25 – The default
366 – One of MDaemon’s alternate SMTP ports (ODMR)
587 – Another of MDaemon’s alternate SMTP ports (MSA, requires AUTH)
995 – Dedicated SSL port.

<UseOutgoingSSL>0</UseOutgoingSSL>
<UseSMTP_TLS>1</UseSMTP_TLS>

SMTP server SSL settings. Set both to zero to not use SSL at all, set “UseOutgoingSSL” to 1 to use a dedicated SSL port (995). Set “UseSMTP_TLS” to 1 to connect with an unencrypted session and switch over to SSL using the STARTTLS command.

<UseIMAPAuthForSMTP>1</UseIMAPAuthForSMTP>

Use the IMAP authentication information for SMTP as well.

0 – SMTP uses it’s own authentication information
1 – SMTP uses the IMAP authentication information.

<UseSMTP_Auth>0</UseSMTP_Auth>

This switch controls whether or not SMTP authentication is required.

<SMTPUserName></SMTPUserName>
<SMTPPassword></SMTPPassword>

SMTP username and password, if you are using custom SMTP authentication information. If <UseIMAPAuthForSMTP> is set to 1, these fields are ignored.

<RememberSmtpPassword>0</RememberSmtpPassword>

Outlook Connector can optionally save passwords between sessions.

0 – Ask for the password each session.
1 – Remember the password Ignored if you are using your IMAP password.

<EnglishFoldersOnServer>0</EnglishFoldersOnServer>

Override custom language and use English names for the built-in folders (Inbox, Contacts, Calendar, Tasks, Journal, Notes, Deleted Items, Sent Items, etc)

<ShowAllFolders>0</ShowAllFolders>

Whether to show all folders, or only IMAP subscribed folders. Note that you cannot hide the built-in folders.

0 – Show only subscribed folders.
1 – Show all folders.

<SendReceiveAllFolders>0</SendReceiveAllFolders>

Controls whether you want to send/receive all folders, or only selected folders.

0 – Only selected folders
1 – All Folders

Note: All folders will be synchronized when accessed, this only changes the behaviour of the Send/Receive Mail feature in Outlook (whether by manual mail check, or scheduled check)

<SendReceiveFolders>

This is a binary field and defined, it controls which folders will be checked for new mail.

Although this field is not documented, you can copy it from one machine to another if the folder list is identical. This is useful for an initial deployment, when you want many folders synchronized regularly.

<DownloadHeadersOnly>0</DownloadHeadersOnly>

This setting tells Outlook Connector whether or not to download full message bodies, including attachments, when they are first downloaded from the server.

0 – Download entire messages, including attachments, as soon as they are discovered.
1 – Download headers immediately, only download message bodies as they are needed.

Note that Offline users will probably want to download entire messages. However, downloading entire messages will result in a significantly larger database if your users have mail they don’t open (for example, mail already stored on the server before Outlook Connector was set up)

<UsePersistentConnection>1</UsePersistentConnection>

This controls whether Outlook Connector will leave the IMAP connection open at all times, or open and close connections repeatedly.

0 – Close the connection when not in use.
1 – Leave the connection open.

Note that you’ll nearly always want to leave this at 1, setting this to 0 will actually create more load on the server, as well as cause user delays. IMAP was designed to leave connections open for an extended period of time.

<LastBackup>0</LastBackup>
<BackupEverySession>1</BackupEverySession>
<DaysBetweenBackupProgression>3</DaysBetweenBackupProgression>
<NumberOfBackups>0</NumberOfBackups>

Outlook Connector can create a backup copy of LocalCache.db at shutdown. This functionality was added a long time ago and has never been tested by Alt-N, so I don’t know if it works or not. There is no GUI for these settings. Also note that it does not backup the attachments folder, and that restoring will cause data to get out of sync.

LastBackup – Timestamp of when the last backup happened.
BackupEverySession – Backup every time outlook closes?
DaysBetweenBackupProgression – How many days between backups, if not backing up every session.
NumberOfBackups – How many backups to create. 0 (the default) = none.

I don’t know what the filenames of the backups would be or where they are stored. If you use this feature, please don’t file bug reports with Alt-N, this is entirely undocumented, unsupported, and due to the inconsistent state your attachments folder will be left in, it’s probably going to cause more problems then you think it might solve.

Tag Archives: File Formats

IMAP.MRK file format

config.xml

messages.idx file format