Ticket #277 (closed Bugs: fixed)

Opened 9 years ago

Last modified 9 years ago

IMAP IDLE may fail to detect new messages

Reported by: octo Owned by: octo
Priority: major Milestone: 2.0
Component: LogicMail Version: 2.0.0
Keywords: Cc:
Blocked By: Blocking: #283, #294

Description

This has been reported with Gmail via a 4.6 device, and has been reproduced with Gmail on a 6.0 device.

Steps to reproduce:

  • Set device log level to "Debug" and enable "Connection debugging"
  • Connect to a Gmail IMAP account, selecting the INBOX
  • Send a message to the Gmail account's address
  • Wait 5-6 minutes, to ensure the IMAP keepalive loop has a chance to run
  • Observe that no new messages are detected

Notice the following network transcript (or similar) in the device event log:

[SEND] A7 IDLE
...nothing for 5 minutes...
[SEND] DONE
[RECV] + idling
[RECV] * 151 EXISTS
[RECV] * 151 EXISTS
[RECV] A7 OK IDLE terminate
[SEND] A8 NOOP
[RECV] A8 OK Success
[SEND] A9 IDLE

This is actually two related issues:

  1. LogicMail is not receiving the IDLE new-message notifications while it is in IDLE mode
  2. Those IDLE notifications are received when exiting IDLE mode, but are ignored.

IMAP IDLE with Gmail was also tested directly using "openssl s_client", and appeared to behave normally.

Change History

comment:1 Changed 9 years ago by octo

This may be a little more difficult to solve than originally thought. The current IDLE logic does a periodic check of the available() property of the socket's input stream, and reads in the untagged responses if it is non-zero. This approach seems to work in some cases (like my test servers), but not in other cases (like Gmail). Ultimately, it may prove to be an unreliable approach altogether.

Since the BlackBerry does not support conventional non-blocking I/O, the correct solution may be to completely separate read and write operations (at least for IMAP IDLE) onto separate threads. One thread will handle the write operations to begin and end IDLE mode. The other thread will be continually blocked on a read(), until it gets an untagged response or an end-of-IDLE confirmation.

comment:2 Changed 9 years ago by octo

  • Blocking 283 added

comment:3 Changed 9 years ago by octo

  • Priority changed from normal to major
  • Status changed from new to accepted

comment:4 Changed 9 years ago by octo

  • Blocking 294 added

[831] attempts to fix this by implementing a separate read thread for IMAP IDLE mode, which does a blocking read in a loop that is only broken by an end of IDLE or a connection error.

comment:5 Changed 9 years ago by octo

  • Blocking 294 removed

(In #294) [831] adds initial support for this. It is a massive changeset, and still needs extensive testing before this ticket can be closed.

Since most of these unrequested untagged responses reference messages purely by index, a core part of this change involves having the IMAP client implementation maintain its own index-to-MessageToken map. This map already has extensive unit tests written against it, to ensure that it correctly updates when EXPUNGE responses are received. However, there are a number of cases that can cause this map to become invalid, and extensive higher-level testing needs to be done around those cases.

It has also been observed that the initial code in [831] does not cause message gap information in the data model (and UI) to be correctly updated. This data is used to determine where "Load more messages..." UI elements should be placed on the mailbox screen.

comment:6 Changed 9 years ago by octo

  • Status changed from accepted to closed
  • Resolution set to fixed
  • Blocking 294 added

The core IMAP IDLE issue has been resolved, so I am closing this ticket. The remaining issues will fall under #294, as they all related to correct handling of untagged server responses and related mailbox state.

Note: See TracTickets for help on using tickets.