Closed Bug 925480 Opened 11 years ago Closed 10 years ago

[email/IMAP] compensate for gmail reporting INBOX as "Inbox" but reporting subfolders under "INBOX"

Categories

(Firefox OS Graveyard :: Gaia::E-Mail, defect)

defect
Not set
normal

Tracking

(tracking-b2g:+, b2g-v2.1 verified, b2g-v2.2 verified)

VERIFIED FIXED
2.1 S9 (21Nov)
tracking-b2g +
Tracking Status
b2g-v2.1 --- verified
b2g-v2.2 --- verified

People

(Reporter: pancake, Assigned: asuth)

References

Details

(Whiteboard: [priority])

Attachments

(1 file, 1 obsolete file)

Attached image 2013-10-10-22-45-06.png (obsolete) —
User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:27.0) Gecko/20100101 Firefox/27.0 (Beta/Release)
Build ID: 20131005030203

Steps to reproduce:

Use a GMail account on the mail client of fxos (current git 1.3).  (didnt tried with plain imap servers like dovecot)


Actual results:

the list of mails in the inbox is listed properly, but when you click on a mail, the body is never retrieved. it shows an infinite circle animation, but never gets the text. I can reproduce this bug for at least 1.2 on gp-peak. Rebooting or reconfiguring the mail account doesn't fixes anything.


Expected results:

Got the mail body. and be able to read it :)
Attachment #815571 - Attachment is obsolete: true
Marking WFM based on the reporter duping to a (now-)fixed bug and current belief that gmail works pretty well.
Status: UNCONFIRMED → RESOLVED
Closed: 10 years ago
Resolution: --- → WORKSFORME
Same problem for me using 1.4.0, commit 56f8ff83, date 2014-03-15 00:48:57.
Maybe related: http://pastebin.com/NfSXVHAs

I/GeckoDump(  783): ERR: indexedDB error: DataCloneError from request as part of readonly transaction on object store "bodyBlocks"
I/GeckoDump(  783): ERR: onerror reporting: DataCloneError @ app://email.gaiamobile.org/js/ext/mailapi/main-frame-setup.js : 4749
E/GeckoConsole(  783): [JavaScript Error: "DataCloneError" {file: "app://email.gaiamobile.org/js/ext/mailapi/main-frame-setup.js" line: 4749}]
E/GeckoConsole(  783): [JavaScript Error: "DataCloneError: The object could not be cloned."]
I/GeckoDump(  783): ERR: Problem handling message type: batchSlice [Exception... "Unexpected error"  nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)"  location: "JS frame :: app://email.gaiamobile.org/js/ext/mailapi/main-frame-setup.js :: exports.ContactCache.resolvePeep :: line 902"  data: no] 
I/GeckoDump(  783):  undefined
I/GeckoDump(  783): ERR: onerror reporting: Error: Problem handling message type: batchSlice [Exception... "Unexpected error"  nsresult: "0x8000ffff (NS_ERROR_UNEXPECTED)"  location: "JS frame :: app://email.gaiamobile.org/js/ext/mailapi/main-frame-setup.js :: exports.ContactCache.resolvePeep :: line 902"  data: no] 
I/GeckoDump(  783):  undefined @ app://email.gaiamobile.org/js/ext/mailapi/main-frame-setup.js : 2227
Also, http://pastebin.com/AvX1g6qW

I/Gecko   (  804): WLOG: Email knows that it is: offline and previously was: online
I/Gecko   (  804): WWAR: Conn steady error: [object Object] on imap.googlemail.com 993
I/Gecko   (  804): WLOG: runOp_end(do: {"type":"downloadBodies","longtermId":"0/62","lifecycle":"do","localStatus":"done","serverStatus":"doing","tryCount":0,"humanOp":"downloadBodies","messages":[{")
I/Gecko   (  804): 
I/Gecko   (  804): WERR: Connect error: unresponsive-server formal: Error: Unable to connect. Reason: [object Object] on imap.googlemail.com 993
Thank you very much for the logs and precise commit info, those are invaluable!  I just got another report of DataCloneErrors yesterday too, which is an entirely new problem as far as I know, but would absolutely explain this problem.

The latter log snippet makes sense and is not a problem.  The internet connection dropped (perhaps you're on wi-fi and the device was not charging and the screen turned off), so we lost the connection and when we tried to automatically retry we found the connect failed immediately.

I unfortunately think I have a good idea of the source of the problem DataCloneError, I'll look into that now.
Status: RESOLVED → REOPENED
Ever confirmed: true
Resolution: WORKSFORME → ---
Andrew, is this bug algo related?
https://gist.github.com/radare/9609076

Check lines 33-...
(In reply to CodingFree from comment #7)
> Andrew, is this bug algo related?
> https://gist.github.com/radare/9609076
> 
> Check lines 33-...

The "box.attribs is undefined" error occurs during folder sync and is either unrelated or indirectly related to your problem in comment 4 (by way of altering timing).  Historically we had a problem with the Courier IMAP server delivering folders out of order but we fixed that.  It's conceivable we have a problem if you have server ambiguity with namespaces where weird things like "INBOX.INBOX" happen.  What IMAP server are you using?
This log is mine, I'm using GMail imap server. I have several folders, because some mail clients create them, for example: Sent, Sent Messages, Trash, [GMail]Trash, .. is this the problem? i never had any issue with that in any other mail client. But email body is never fetched, i dont know if it's related to this WERR
A question about your comment 4 problem; around when the problem occurred, did you view any messages in the Inbox with embedded images?  Like where you have to hit the bar across the bottom of the message to "download embedded images"?  Also related, do you force-kill the e-mail app a lot or do you know if your device does a lot of OOM killing?

Also, are you using periodic sync?  If so, what interval, and using what type of network?

I'm trying to re-create the DataCloneError and having trouble doing so.
It doesnt matters which kind of contents the mail have. I always get an infinite loading gif. Yep i have killed the email client several times, trying to get it working, I always use manual sync.

I have configured my secondary GMail account and this one doesnt seems to cause any issue. It works fine, it fetches body of mails fine... so i guess its related to the contents of my imap account. I can't give you access to my account :P but I can do other tests.. anyway that error message should be fixed.
Surprisingly after configuring that secondary gmail account, the first one started working as fine. So now I cant reproduce the issue O_o
(In reply to pancake from comment #9)
> This log is mine, I'm using GMail imap server. I have several folders,
> because some mail clients create them, for example: Sent, Sent Messages,
> Trash, [GMail]Trash, .. is this the problem? i never had any issue with that
> in any other mail client. But email body is never fetched, i dont know if
> it's related to this WERR

Hm, yes, actually, that could prevent body downloads, yes, since it appears it breaks the job queue for the account.  There must be a folder hierarchy thing that upsets the e-mail app.

If you're on linux and have openssl installed / can install it, maybe you could do the following so I can get a dump of your folder structure and figure out what is breaking us?

1) type: openssl s_client -crlf -connect imap.gmail.com:993
2) type: a1 LOGIN <username> <password>
3) (you get logged in hopefully)
4) type: a2 XLIST "" "*"
5) copy and paste that what you get back, or email it to me at asuth@mozilla.com depending on how sensitive you think the data is.  If there are tag names/etc. in there that you are uncomfortable with me seeing, you should probably be able to safely search/replace such instances.  Or if something just jumps out at you as being wrong.

This is what my test account's results look like:

a2 XLIST "" "*"
* XLIST (\HasNoChildren \Inbox) "/" "Inbox"
* XLIST (\Noselect \HasChildren) "/" "[Gmail]"
* XLIST (\HasNoChildren \AllMail) "/" "[Gmail]/All Mail"
* XLIST (\HasNoChildren \Drafts) "/" "[Gmail]/Drafts"
* XLIST (\HasNoChildren \Important) "/" "[Gmail]/Important"
* XLIST (\HasNoChildren \Sent) "/" "[Gmail]/Sent Mail"
* XLIST (\HasNoChildren \Spam) "/" "[Gmail]/Spam"
* XLIST (\HasNoChildren \Starred) "/" "[Gmail]/Starred"
* XLIST (\HasNoChildren \Trash) "/" "[Gmail]/Trash"
* XLIST (\HasNoChildren) "/" "test_compose_reply_all"
* XLIST (\HasNoChildren) "/" "test_empty_sync"
* XLIST (\HasNoChildren) "/" "test_initial_full_sync"
* XLIST (\HasNoChildren) "/" "test_multiple_ranges"
* XLIST (\HasNoChildren) "/" "test_saturated_sync"
a2 OK Success
We only try and synchronize folders once per day.  It's possible that we managed to fail in such a way that we are no longer trying to synchronize the folders.
ping? Did you got the mail?
Just found it in my mail quarantine.  Thanks for the ping!  Usually postini doesn't mess up.

The problem appears to be this pattern:
* XLIST (\HasChildren \Inbox) "/" "Inbox"
* XLIST (\HasNoChildren) "/" "INBOX/anyfolder"
Summary: mail client never fetches body → [email/IMAP] compensate for gmail reporting INBOX as "Inbox" but reporting subfolders under "INBOX"
I want to escalate this into our priority backlog but am missing the ability to set backlog, so asking for 2.0?.
blocking-b2g: --- → 2.0?
Whiteboard: [priority]
blocking-b2g: 2.0? → backlog
See Also: → 1012222
Note: the workaround for this would be to delete any labels that are under Inbox.
Any update on this issue?
It is our hope to address this during the 2.0 stabilization phase that will take place over the next ~6 weeks.
Current adb log I get is this:

I/Gecko   ( 1458): WERR: Explosion while processing data TypeError: box.attribs is undefined
I/Gecko   ( 1458): WERR: Stack: properties._determineFolderType@app://email.gaiamobile.org/js/ext/mailapi/composite/configurator.js:3654:1
I/Gecko   ( 1458): walkBoxes@app://email.gaiamobile.org/js/ext/mailapi/composite/configurator.js:3785:13
I/Gecko   ( 1458): properties._syncFolderComputeDeltas@app://email.gaiamobile.org/js/ext/mailapi/composite/configurator.js:3813:5
I/Gecko   ( 1458): processResponse@app://email.gaiamobile.org/js/ext/mailapi/imap/probe.js:873:9
I/Gecko   ( 1458): processData@app://email.gaiamobile.org/js/ext/mailapi/imap/probe.js:1156:5
I/Gecko   ( 1458): ImapConnection.prototype.connect/<@app://email.gaiamobile.org/js/ext/mailapi/imap/probe.js:566:9
I/Gecko   ( 1458): EventEmitter.prototype.emit@app://email.gaiamobile.org/js/ext/mailapi/worker-bootstrap.js:15624:9
I/Gecko   ( 1458): NetSocket.prototype._ondata@app://email.gaiamobile.org/js/ext/mailapi/composite/configurator.js:4124:3
I/Gecko   ( 1458): NetSocket/routerInfo<@app://email.gaiamobile.org/js/ext/mailapi/composite/configurator.js:4046:5
I/Gecko   ( 1458): receiveInstanceMessage@app://email.gaiamobile.org/js/ext/mailapi/worker-bootstrap.js:13879:5
I/Gecko   ( 1458): receiveMessage@app://email.gaiamobile.org/js/ext/mailapi/wor
I/GeckoDump( 1458): ERR: onerror reporting: TypeError: box.attribs is undefined @ app://email.gaiamobile.org/js/ext/mailapi/composite/configurator.js : 3654

```
The switch to email.js libraries (bug 885110) is going to address the fatal exception issue, but we will have some weirdness with the second inferred inbox.  I've filed https://github.com/whiteout-io/browserbox/issues/18 to track completely correctly fixing the issue.  We should mark this bug fixed when the upstream bug is fixed (possibly by us) and we've taken the fix.
[priority] --> tracking-b2g:+ conversion
tracking-b2g: --- → +
Assignee: nobody → bugmail
Target Milestone: --- → 2.1 S8 (7Nov)
Target Milestone: 2.1 S8 (7Nov) → 2.1 S9 (21Nov)
Depends on: 1097421
Fixed as part of bug 1097421's omnibus patch. Many servers were experiencing more than one bug, and we had experienced some upstream drift, and especially for uplift tracking, it made sense to land these together.  See the gaia-email-libs-and-more pull request's specific commits for the specific fixes and tests.  I am planning to request v2.1 uplift shortly after performing v2.1 testing.
Status: REOPENED → RESOLVED
Closed: 10 years ago10 years ago
Resolution: --- → FIXED
Batching marking v2.1 fixed since the uplift of bug 1097421 occurred on Dec 9th, thereby fixing all dependent bugs of bug 1097421 on v2.1.  Uplift commit comment is https://bugzilla.mozilla.org/show_bug.cgi?id=1097421#c6
Attached video video
This problem is verified pass on latest build of Flame 2.1/2.2
See attachment: Verify_video.MP4
Rate: 0/20

Flame 2.1 build:
Gaia-Rev        77c57eb8a985d5cbd34a597fb1b978ba6e205af6
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g34_v2_1/rev/4c28bb3be0c6
Build-ID        20150121001510
Version         34.0
Device-Name     flame
FW-Release      4.4.2
FW-Incremental  eng.cltbld.20150121.034530
FW-Date         Wed Jan 21 03:45:41 EST 2015
Bootloader      L1TC000118D0


Flame 2.2 build:
Gaia-Rev        e4f9b5da3751798f9cc5d95f302c30722cc11fca
Gecko-Rev       https://hg.mozilla.org/releases/mozilla-b2g37_v2_2/rev/75a462a58d7a
Build-ID        20150121002607
Version         37.0a2
Device-Name     flame
FW-Release      4.4.2
FW-Incremental  eng.cltbld.20150121.040751
FW-Date         Wed Jan 21 04:08:02 EST 2015
Bootloader      L1TC000118D0
Status: RESOLVED → VERIFIED
blocking-b2g: backlog → ---
You need to log in before you can comment on or make changes to this bug.

Attachment

General

Creator:
Created:
Updated:
Size: