Ticket #294 (closed Bugs: fixed)

Opened 9 years ago

Last modified 9 years ago

IMAP implementation does not handle untagged responses correctly

Reported by: octo Owned by: octo
Priority: major Milestone: 2.0
Component: LogicMail Version: 2.0.0
Keywords: Cc:
Blocked By: #277 Blocking:


The work for this is being done in tandem with #277, since there is a lot of overlap. However, since this aspect is causing the bulk of the work, I've decided to create a separate ticket to document it.

If anyone leaves LogicMail continuously connected to an IMAP server while also accessing it with other mail clients, and has been seeing strange/quirky behavior within LogicMail, this bug is the most likely cause.

Current IDLE/NOOP behavior only pays attention to an untagged RECENT response, and triggers a FETCH based on UIDNEXT accordingly. Furthermore, unrequested untagged responses during a variety of other IMAP commands are completely unhandled and may actually cause LogicMail to fail in parsing the results of those commands.

The following untagged IMAP responses need to be correctly handled in all the cases where they may be sent unrequested:

  • EXISTS - Indicates a change in mailbox message count
  • RECENT - Indicates new messages
  • EXPUNGE - Indicates removed messages
  • FETCH - Indicates a change in message flags

This problem is especially likely to cause issues during UID FETCH and UID STORE, both of which are frequently used by LogicMail and may have these untagged responses mixed in with their expected output.

Change History

comment:1 Changed 9 years ago by octo

  • Blocked By 277 added

(In #277) [831] attempts to fix this by implementing a separate read thread for IMAP IDLE mode, which does a blocking read in a loop that is only broken by an end of IDLE or a connection error.

comment:1 Changed 9 years ago by octo

  • Owner set to octo
  • Status changed from new to accepted
  • Blocked By 277 removed

[831] adds initial support for this. It is a massive changeset, and still needs extensive testing before this ticket can be closed.

Since most of these unrequested untagged responses reference messages purely by index, a core part of this change involves having the IMAP client implementation maintain its own index-to-MessageToken map. This map already has extensive unit tests written against it, to ensure that it correctly updates when EXPUNGE responses are received. However, there are a number of cases that can cause this map to become invalid, and extensive higher-level testing needs to be done around those cases.

It has also been observed that the initial code in [831] does not cause message gap information in the data model (and UI) to be correctly updated. This data is used to determine where "Load more messages..." UI elements should be placed on the mailbox screen.

comment:2 Changed 9 years ago by octo

  • Blocked By 277 added

(In #277) The core IMAP IDLE issue has been resolved, so I am closing this ticket. The remaining issues will fall under #294, as they all related to correct handling of untagged server responses and related mailbox state.

comment:3 Changed 9 years ago by octo

[832] fixes some additional expunge issues, including cache cleanup and notifying the data model of message index changes.

Before this ticket can be closed, some additional testing and fixing has to be done around the various cases that could cause the index map to become invalidated.

comment:4 Changed 9 years ago by octorian

  • Status changed from accepted to closed
  • Resolution set to fixed

In [846]:

Added support for notifying the folder request handler of invalidated mailbox state and new messages, as detected during a folder selection. (fixes #294)

Note: See TracTickets for help on using tickets.