Vlad Lucaci, Desktop QA (:vlucaci) Reporter
	Description • 6 years ago

[Affected versions]:
-62.0b17, 63.0a1 (2018-08-13)

[Affected platforms]:
- Windows 10 x64, Windows 8.1 x64

[Steps to reproduce]:
1.Launch FF.
2. Go to weather.com

[Expected result]:
-the page is loaded without any issues.

[Actual result]:
-Page is loading a "The page isn't redirecting properly" error

[Regression range]:
-https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=23c2ec5544b9e0a74a047b87b594e4c36a8fe95c&tochange=f97a056ae6235de7855fd8aaa04fb1c8d183bd06

[Additional notes]:
-No errors are displayed inside the DevTools Console
-This issue is not reproducible on macOS or Ubuntu platforms.
-This issue is reproducible on both x32 and x64 builds.
-This issue reproduces even with fresh profiles.

	Vlad Lucaci, Desktop QA (:vlucaci) Reporter
	Comment 1 • 6 years ago

After further investigation, we have discovered that this issue is reproducible on certain builds of Windows:

This issue is reproducible with 100% repro rate on:

-Windows 8.1 x64 - 6.3 Build 9600
-Windows 10  x64 - Version 1709 (16299.431)

This issue is NOT reproducible on :

-Windows 10 x64 - Version 1803 (17134.165)
-Windows 10 x64 - Version 1803 (17134.191)

Another thing worth mentioning, is that this issue does not reproduce on the affected platforms with longer url's from the same domain. Example:

https://weather.com/weather/radar/interactive/l/USOH1375:1:US?layer=radar&overlays=currtrafficspeed

	Chris Peterson [:cpeterson]
	Comment 2 • 6 years ago

(In reply to Vlad Lucaci (:vlucaci) from comment #0)
> [Regression range]:
> -https://hg.mozilla.org/mozilla-central/pushloghtml?fromchange=23c2ec5544b9e0a74a047b87b594e4c36a8fe95c&tochange=f97a056ae6235de7855fd8aaa04fb1c8d183bd06

The regression range is surprising because it points to Firefox 51 from 2016. I don't see any obvious suspects in the regression range.

	Chris Peterson [:cpeterson]
	Comment 3 • 6 years ago

Honza, I think this is a networking bug that only affects some Windows versions. How do we get this bug reviewed by the Networking team's bug triage?

Vlad tested weather.com using a VPN and found that, on Windows 8.1 and Windows 10 version 1709 (build 16299.431):

1. weather.com loads correctly when VPN'ing from Romania into the UK or US.
2. weather.com shows a "The page isn't redirecting properly" error when VPN'ing from Romania into Sweden, Germany, France, Netherlands, Switzerland, Italy, or Romania.

weather.com loads correctly when using Windows 10 version 1803 (build 17134.165) or version 1803 (build 17134.191).

	Romain Testard [:RT]
	Comment 4 • 6 years ago

For info I tested on High Sierra 10.13.6 and reproduced the issue with Firefox 61.0.2 from France
Screencap: http://recordit.co/1AvGPThfKT
https://weather.com/weather/today/l/48.86,2.35?par=google loads fine but https://weather.com does not

	Honza Bambas (:mayhemer)
	Comment 5 • 6 years ago

Sorry for late reply here.

I will ask anyone able to reproduce - please provide logs:
https://developer.mozilla.org/en-US/docs/Mozilla/Debugging/HTTP_logging

don't forget to note the URL that is failing for you.  the more details the better.  thanks.

	Vlad Lucaci, Desktop QA (:vlucaci) Reporter
	Comment 6 • 6 years ago

Hello Honza,

The URL that is failing is : weather.com

This issue is reproducible with 100% repro rate on:

-Windows 8.1 x64 - 6.3 Build 9600
-Windows 10  x64 - Version 1709 (16299.431)
-macOS 10.13.6

This issue is NOT reproducible on :

-Windows 10 x64 - Version 1803 (17134.165)
-Windows 10 x64 - Version 1803 (17134.191)

Another thing worth mentioning, is that this issue does not reproduce on the affected platforms with longer url's from the same domain. Example:

https://weather.com/weather/radar/interactive/l/USOH1375:1:US?layer=radar&overlays=currtrafficspeed

Attached you can find the debugging logs:
https://drive.google.com/file/d/1YB65HKqWYE3EZc4yoMO9I31BborI3qKz/view?usp=sharing

	Honza Bambas (:mayhemer)
	Comment 7 • 6 years ago

This is a cookie timing issue and it's a server side issue -> invalid

Not locally reproducible for me.

I can see the following in the provided (failing) log:
- navigation to http://weather.com/, no cached entry -> 301 Moved Permanently, Location: https://weather.com/
- redirecting to https://weather.com/, no cached entry -> 302 Found, location: https://weather.com/
- redirecting again to https://weather.com/ (same url), having an existing cache entry with no-cache, hence revalidating with the server -> 302 Found, location: https://weather.com/

And since then we just loop.

This is a cookie bug, I can see Bug 368964 in the regression range, and I can also see a difference in cookies and their expiration timings we get from the server (or a balancer) and that we then send back on the second redirect to https version of the page:



* Good case (no loop, local attempt log):
2018-08-23 15:56:30.725000 UTC - [Parent 8576: Socket Thread]: I/nsHttp http response [
  date: Thu, 23 Aug 2018 14:42:04 GMT
  set-cookie: speedpin=4G; expires=Thu, 23-Aug-2018 16:26:30 GMT; path=/; domain=.weather.com; secure
  set-cookie: Goto=Redirected; expires=Thu, 23-Aug-2018 15:56:35 GMT; path=/; domain=.weather.com; secure
  set-cookie: o=deleted; path=/; domain=.weather.com; secure
  set-cookie: ci=TWC-Locale-Group=US&X-Origin-Hint=Goto-Prod&TWC-GeoIP-Country=CZ&TWC-Privacy=gdpr; path=/; domain=.weather.com; secure

we then send
  Cookie: speedpin=4G; Goto=Redirected; o=deleted; ci=TWC-Locale-Group=US&X-Origin-Hint=Goto-Prod&TWC-GeoIP-Country=CZ&TWC-Privacy=gdpr


>>> 2018-08-23 15:56:30.725000 UTC < 23-Aug-2018 15:56:35 GMT



* Broken (your log):
2018-08-23 15:42:05.268000 UTC - [2600:Socket Thread]: I/nsHttp http response [
  date: Thu, 23 Aug 2018 15:56:30 GMT
  set-cookie: speedpin=4G; expires=Thu, 23-Aug-2018 15:12:04 GMT; path=/; domain=.weather.com; secure
  set-cookie: Goto=Redirected; expires=Thu, 23-Aug-2018 14:42:09 GMT; path=/; domain=.weather.com; secure
  set-cookie: o=deleted; path=/; domain=.weather.com; secure
  set-cookie: ci=TWC-Locale-Group=US&X-Origin-Hint=Goto-Prod&TWC-GeoIP-Country=RO&TWC-Privacy=gdpr; path=/; domain=.weather.com; secure

we send back
  Cookie: o=deleted; ci=TWC-Locale-Group=US&X-Origin-Hint=Goto-Prod&TWC-GeoIP-Country=RO&TWC-Privacy=gdpr


>>> 2018-08-23 15:42:05.268000 UTC > 23-Aug-2018 14:42:09 GMT  <--- we expire the "Goto=Redirected" cookie and don't send it back


Note that the Date response header is in the past relative to the actual time for the good case and in the future for the broken case.  The server/balancer(s) has(ve) clearly time sync issues.

	Chris Peterson [:cpeterson]
	Comment 8 • 6 years ago

Is this redirect problem reproducible in Chrome and Edge? If not, is weather.com serving them different content or do they handle cookie expiration differently?

If the site didn't work in any browser in Sweden, Germany, France, Netherlands, Switzerland, Italy, or Romania, you would think someone at weather.com would notice. :)

Should we reach out to weather.com?

	Vlad Lucaci, Desktop QA (:vlucaci) Reporter
	Comment 9 • 6 years ago

:cpeterson 

This issue does not reproduce on: 
Edge 41.16299,
Internet Explorer 11.16299, 
Google Chrome 68.0 
on the same affected machine on which FF does not work.

The content seems to be the same one on all browsers.

	Honza Bambas (:mayhemer)
	Comment 10 • 6 years ago

(In reply to Vlad Lucaci (:vlucaci) from comment #9)
> :cpeterson 
> 
> This issue does not reproduce on: 
> Edge 41.16299,
> Internet Explorer 11.16299, 
> Google Chrome 68.0 
> on the same affected machine on which FF does not work.
> 
> The content seems to be the same one on all browsers.

Can you please check the date, set-cookie response headers and cookie request headers in those browsers and post them here?  I can't repro the issue.  If the timing issue is there as well and they treat it differently than us, we may want to back bug 368964 out or do further workarounds.

cc'ing Ehsan who reviewed that patch.

	Vlad Lucaci, Desktop QA (:vlucaci) Reporter
	Comment 11 • 6 years ago

Hello Honza

I have accessed weather.com on I Edge 41 and Chrome 68 and pulled a HAR log file for both browsers when accessing the website. Hope it helps:

https://drive.google.com/file/d/1OY--hvdbkLTSKIvvVVUnaEwJQ0HM_Br8/view?usp=sharing
https://drive.google.com/file/d/1o08Ly1GD9UIHhEc3_PYVFAruiAlYlrKE/view?usp=sharing

	Honza Bambas (:mayhemer)
	Comment 12 • 6 years ago

(In reply to Vlad Lucaci (:vlucaci) from comment #11)
> Hello Honza
> 
> I have accessed weather.com on I Edge 41 and Chrome 68 and pulled a HAR log
> file for both browsers when accessing the website. Hope it helps:
> 
> https://drive.google.com/file/d/1OY--hvdbkLTSKIvvVVUnaEwJQ0HM_Br8/
> view?usp=sharing
> https://drive.google.com/file/d/1o08Ly1GD9UIHhEc3_PYVFAruiAlYlrKE/
> view?usp=sharing

Thanks.

The Edge HAR can't be loaded on [1].

Can you please tell me at what EXACT time (GMT or your local) have you captured these logs?

Next time would be good to provide Chrome and Firefox HARs from nearly the same time, for compare.

Other thing I would like you to try is to change [2] Chrome's UA to Firerox UA for which you can reproduce the problem.  (UA = the User-Agent request header)

Thanks!


[1] https://toolbox.googleapps.com/apps/har_analyzer/
[2] https://winaero.com/blog/change-user-agent-chrome/

	Vlad Lucaci, Desktop QA (:vlucaci) Reporter
	Comment 13 • 6 years ago

Hello Honza,

I have re-taken the HARs and this time I will also provide time logs:

Chrome Archive: https://drive.google.com/open?id=1J6DA8Ic8qOQXeP-AVybsH7Qq8dRAEEZ1
Accessed at 17:18:20 Romania Time

Mozilla Archive: https://drive.google.com/open?id=1IrMP53nurpebCTaUktxc0THSMB0kHhxT
Accessed at 17:18:40 Romania Time

Chrome's UA to FF UA: https://drive.google.com/open?id=1JSLWcLRLDbPbn0spoUVnzdAFAI0W420w
Accessed at 3:04 Romania Time

Worth mentioning that for the latter, changing the UA of Chrome to FF loaded weather.com without any issues .

Hope it helps.

	Honza Bambas (:mayhemer)
	Comment 14 • 6 years ago

(In reply to Vlad Lucaci (:vlucaci) from comment #13)
> Chrome's UA to FF UA:
> https://drive.google.com/open?id=1JSLWcLRLDbPbn0spoUVnzdAFAI0W420w
> Accessed at 3:04 Romania Time

Are you sure about the time?

	Honza Bambas (:mayhemer)
	Comment 15 • 6 years ago

Another discrepancy I've found is that you use for UA simulation a wrong header value:

Mozilla/5.0 (Windows NT 10.0; WOW64; rv:46.0) Gecko/20100101 Firefox/46.0

It has to be:

Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:62.0) Gecko/20100101 Firefox/62.0


The reason this may be a cause you still don't see the bug is that the Firefox behavior change (bug 368964, 51) that they may be trying to workaround will only trigger for version 51+.  You are claiming to be 46.

There is definitely some UA sniffing involved because Firefox receive a different set of cookies, as well as Chrome when claiming to be Firefox (the speedpin=4G cookie)

Can you please retest Chrome with the correct Firefox UA and submit a HAR again?

	Honza Bambas (:mayhemer)
	Comment 16 • 6 years ago

Chrome Archive: https://drive.google.com/open?id=1J6DA8Ic8qOQXeP-AVybsH7Qq8dRAEEZ1

request made at 14:18:20.921
date Thu, 30 Aug 2018 14:18:19 GMT

set-cookie Goto=Redirected; expires=Thu, 30-Aug-2018 14:18:24 GMT; path=/; domain=.weather.com; secure

==> the cookie expires in 5 secs in the future.

------------------

Mozilla Archive: https://drive.google.com/open?id=1IrMP53nurpebCTaUktxc0THSMB0kHhxT

request made at 14:18:27.497
date Thu, 30 Aug 2018 14:18:26 GMT

set-cookie speedpin=4G; expires=Thu, 30-Aug-2018 14:48:26 GMT; path=/; domain=.weather.com; secure
set-cookie Goto=Redirected; expires=Thu, 30-Aug-2018 14:18:31 GMT; path=/; domain=.weather.com; secure

==> the cookie expires in 5 secs in the future.  *we calculate the expiration wrong!*

------------------

Chrome's UA to FF UA: https://drive.google.com/open?id=1JSLWcLRLDbPbn0spoUVnzdAFAI0W420w

request made at 12:30:13.479
date Thu, 30 Aug 2018 12:30:12 GMT

set-cookie speedpin=4G; expires=Thu, 30-Aug-2018 13:00:12 GMT; path=/; domain=.weather.com; secure
set-cookie Goto=Redirected; expires=Thu, 30-Aug-2018 12:30:17 GMT; path=/; domain=.weather.com; secure

==> same as before.



Vlad, please disregard my previous comment.  Thanks for the logs, looking more carefully inside them seems like a confirmation we have a bug.



Ehsan, the problem is following (all times UTC):

- we send a request at 30 Aug 2018 14:18:27.497 to https://weather.com/

- we get the following response + headers:

302 Found
location https://weather.com/ (the same URL)
date: Thu, 30 Aug 2018 14:18:26 GMT
set-cookie: speedpin=4G; expires=Thu, 30-Aug-2018 14:48:26 GMT; path=/; domain=.weather.com; secure
set-cookie: Goto=Redirected; expires=Thu, 30-Aug-2018 14:18:31 GMT; path=/; domain=.weather.com; secure
set-cookie: o=deleted; path=/; domain=.weather.com; secure
set-cookie: ci=TWC-Locale-Group=US&X-Origin-Hint=Goto-Prod&TWC-GeoIP-Country=RO&TWC-Privacy=gdpr; path=/; domain=.weather.com; secure

- we do another request (redirected) to https://weather.com/ (same URL) at 15:18:27.671, with headers:
Cookie o=deleted; ci=TWC-Locale-Group=US&X-Origin-Hint=Goto-Prod&TWC-GeoIP-Country=RO&TWC-Privacy=gdpr

apparently, we treat the Goto=Redirected cookie as expired, even though it's expiration time against the request time and the date: header value is 5 seconds in the future.

other issue could be we discard the cookies with expiration.

note that this is strongly dependent on the timezone.  I can't reproduce in Prague (gmt+2) but Vlad can repro in Romania (gmt+3)

- we get:

302 Found and a loop starts because we never send back the Goto=Redirected cookie

	Honza Bambas (:mayhemer)
	Comment 17 • 6 years ago

Vlad, can you please produce whatever MOZ_LOG and look at the timestamp and compare against the local time setting of the machine?


It somewhat looks like DST or time zone is badly handled here (probably by us?).  Looking again into the log from comment 6, PR_Now used for writing timestamps to the log (with params {GMT offset=0,DST offset=0}) says 15:42:05 (UTC) but the set-cookie header expiration is 14:42:09 GMT.  Hence, it may appear to us as in the past.  the date: resp header is shifted the same way.

With a local test on Win10 I can see:
- PR_Now() writes to the log UTC (16:23:41) while my local time is UTC+2, all of it is correct (PRG is UTC+1 and we are on the summer time, hence also DST+1)
- the server responses with:
 1) Goto=Redirected; expires=Mon, 03-Sep-2018 16:23:45 GMT, which is the correct time
 2) date: Mon, 03 Sep 2018 16:23:40 GMT, also correct

In the log from comment 6, the server dates are shifted by one hour to the past against what PR_Now() returns.

Why PR_Now?  Because that is what we are using [1] to compare cookie absolute expiration time.

This could also be a clue to why only some (older platforms) manifest this issue.  There could be a system API bug wrongly returning the UTC time.

[1] https://searchfox.org/mozilla-central/rev/721842eed881c7fcdccb9ec0fe79e4e6d4e46604/netwerk/cookie/nsCookieService.cpp#3450
[2] https://searchfox.org/mozilla-central/rev/721842eed881c7fcdccb9ec0fe79e4e6d4e46604/nsprpub/pr/src/md/windows/ntmisc.c#323-325

	Vlad Lucaci, Desktop QA (:vlucaci) Reporter
	Comment 18 • 6 years ago

Hello Honza,

I have produced another log and compared the timestamp from the log and the local time setting of the machine.

I have noticed that the log entries were generated at 12:06 UTC while on the local machine I have UTC+02:00-Athens,Bucharest (14:06PM)so your suspicion seem to check out. 

https://drive.google.com/open?id=15aXlIsNsNR8NcXJTmoLax7hMcWn1ZtSQ

	(no longer active)
	Comment 19 • 6 years ago

I think Honza's suspicion is completely correct!  That code should really be using GMT time.  It is astonishing that we have not discovered this problem so far...

	u408661
	Comment 20 • 6 years ago

Honza, you seem to understand what's going on here & the urgency pretty well. Can you triage appropriately?

	Honza Bambas (:mayhemer)
	Comment 21 • 6 years ago

According comment 1 (thanks for it, Vlad!) this may have been a Windows API bug that has been recently fixed.  

I somewhat tend to close this again as WONTFIX as this has been fixed in Win10 1803.  Users should update their machines, unless this manifests on the non-targeted windows update branches (for organizations).

Vlad, do can you test this on non-targeted Windows branches?


Something tells me to work this around would mean to 1) really understand the bug, 2) likely to detect Windows version (range(s)?) and use some kind of a correction.

OTOH, some assessment of what the hell is/was going on here may be appropriate, just in case.

P2, -> me.

	Honza Bambas (:mayhemer)
	Comment 22 • 6 years ago

comment 21

	Vlad Lucaci, Desktop QA (:vlucaci) Reporter
	Comment 23 • 6 years ago

Hello Honza,

Unfortunately I am unable to test this issue on a non-targeted windows branch because we do not own such a work station.

However, we must also take into account that a large majority of users avoid updating the Windows version to 1709 from 1803 due to different performance/security reasons and may encounter this issue.

	David Balažic
	Comment 24 • 6 years ago

An info point: I have this issue under Windows 7 Enterprise SP1 64 bit

Affected:
 Firefox 62.0.2 (64 bit)
 IE 11

Not affected:
Chrome Version 69.0.3497.100 (Official Build) (64-bit)


FF and IE fail to load weather.com, but do load the longer URL from comment 1

Chrome loads both.

Location: Slovenia

	Pascal Chevrel:pascalc (PTO until April 26)
	Comment 25 • 6 years ago

Jan, any progress on this one? Should we reprioritize since this is marked as a P2 but we have shipped multiple versions of Firefox with this bug? Thanks

	Honza Bambas (:mayhemer)
	Comment 26 • 6 years ago

No progress, sorry.  This is not that much a Firefox but a bug in Windows API regarding wall clock, it's been fixed on newer version of the OS.

I won't have time to look deeply into it, even though we still may support that Windows version.  Also, this is more a NSPR bug than a Necko bug, actually.

	Liz Henry (:lizzard) (relman/hg->git project)
	Comment 27 • 5 years ago

Too late to fix in 64. Marking this issue as fix-optional for 65; if you land a patch in nightly and think it's low-risk for beta, please request uplift.