Periodic "An Unexpected Error Occurred" when browsing reports and comments
Categories
(Socorro :: Webapp, defect, P2)
Tracking
(Not tracked)
People
(Reporter: marcia, Assigned: willkg)
Details
Attachments
(8 files)
53.98 KB,
image/png
|
Details | |
68.38 KB,
image/png
|
Details | |
102.70 KB,
image/png
|
Details | |
53 bytes,
text/x-github-pull-request
|
Details | Review | |
53 bytes,
text/x-github-pull-request
|
Details | Review | |
53 bytes,
text/x-github-pull-request
|
Details | Review | |
53 bytes,
text/x-github-pull-request
|
Details | Review | |
53 bytes,
text/x-github-pull-request
|
Details | Review |
Comment 1•7 years ago
|
||
Reporter | ||
Comment 2•7 years ago
|
||
Comment 3•7 years ago
|
||
Reporter | ||
Comment 4•7 years ago
|
||
Assignee | ||
Comment 5•7 years ago
|
||
Comment 6•7 years ago
|
||
Assignee | ||
Comment 7•7 years ago
|
||
Comment 8•7 years ago
|
||
Comment 9•7 years ago
|
||
Assignee | ||
Comment 10•7 years ago
|
||
Assignee | ||
Comment 11•7 years ago
|
||
Comment 12•7 years ago
|
||
Updated•7 years ago
|
Assignee | ||
Comment 13•7 years ago
|
||
Assignee | ||
Comment 14•7 years ago
|
||
Comment 15•7 years ago
|
||
Assignee | ||
Comment 16•7 years ago
|
||
Comment 17•7 years ago
|
||
Reporter | ||
Comment 18•6 years ago
|
||
Assignee | ||
Comment 19•6 years ago
|
||
Reporter | ||
Comment 20•6 years ago
|
||
Assignee | ||
Comment 21•6 years ago
|
||
Assignee | ||
Comment 23•6 years ago
|
||
Unassigning myself from this. I haven't been able to reproduce it, but that doesn't mean it's not still a problem. I don't have any good ideas on how to go forward.
Comment 24•6 years ago
|
||
Add some more XHR endpoints to exempt from session renewal checks.
Comment 25•6 years ago
|
||
jwhitlock merged PR #4957: "bug 1473068: Expand OIDC_EXEMPT_URLS for more XHR usage" in 2f2bb3c. (r=willkg).
Comment 26•6 years ago
|
||
The latest changes, which will be deployed soon, should eliminate this issue when loading most tabs in the signature report. The aggregations and graph tabs are still vulnerable, but will require deeper changes, such as in the external library mozilla-django-oidc.
Please continue to report when this happens. Screenshots and URLs are helpful.
Reporter | ||
Comment 27•6 years ago
|
||
(In reply to John Whitlock [:jwhitlock] from comment #26)
The latest changes, which will be deployed soon, should eliminate this issue when loading most tabs in the signature report. The aggregations and graph tabs are still vulnerable, but will require deeper changes, such as in the external library mozilla-django-oidc.
Please continue to report when this happens. Screenshots and URLs are helpful.
Thanks John. I have continued to see this issue since I am in Socorro a good portion of the day. I will mention in Channel Meeting today to see if anyone else besides me sees the issue.
Comment 28•6 years ago
|
||
The changes are deployed. If our theory is correct, on signature reports the tabs Summary, Reports, Bugzilla, Comments, and Correlations should not show "An Unexpected Error Occurred". Please report if it does. The tabs Aggregations and Graphs may continue showing the bug. We're discussing the best way to apply the fix to these as well.
Assignee | ||
Comment 29•6 years ago
|
||
Given that no one has said anything here, I think we're probably fine now.
If there are still issues, please reopen this with the urls you had problems with.
Assignee | ||
Comment 30•6 years ago
|
||
Oh, whoops--I missed comment #26.
Outstanding:
The aggregations and graph tabs are still vulnerable, but will require deeper changes, such as in the external library mozilla-django-oidc.
Assignee | ||
Comment 31•6 years ago
|
||
We're not going to get to this any time soon, so I'm bumping it down to P3.
Comment 32•6 years ago
|
||
I agree on P3, the error should be trigger much less often now that most API endpoints are exempted.
I've proposed two changes to the upstream project mozilla-django-oidc
, which are different approaches to exempting a set of URLs. If either is merged and released, we can update Socorro's configuration for the remaining vulnerable views.
- https://github.com/mozilla/mozilla-django-oidc/pull/310 - Add a new field for prefix path exceptions
- https://github.com/mozilla/mozilla-django-oidc/pull/311 - Update existing field to allow a trailing "*" for wildcard-style matches
Assignee | ||
Comment 33•5 years ago
|
||
Bumping this to P2. Gabriele is hitting it periodically.
Comment 34•5 years ago
|
||
Gabriele, do you recall which URLs you are seeing this on? I'd expect the aggregations or graphs tabs, but not on the other ones.
(and sorry if this is the wrong Gabriele!)
Comment 35•5 years ago
|
||
Middle-clicking a few links here so that the tabs would be opened in rapid sequence always triggered this issue for me... but now it doesn't anymore. I'm really puzzled, I'm sure I've run into this issue just a couple of weeks ago.
Comment 36•5 years ago
|
||
Thanks. This doesn't sound like the issue addressed in June 2019 - the XHR URLs on that page are /search/results/
and /search/fields
, which have been in the OIDC_EXEMPT_URLS
fields forever.
The issue may have been a load on the Elastic Search server, potentially from a different user. Will and Brian have been adjusting ES in the past few weeks, and maybe their mitigations helped as well.
If it happens again (May 2021?), please post the URL and when you saw it.
Comment 37•5 years ago
|
||
I just hit this again: I had three tabs open on various crashes and I restarted Firefox. Upon restart Firefox tried to reload the three tabs at the same time and they all returned the error page.
Assignee | ||
Comment 38•5 years ago
|
||
John: Do you think this bug is a manifestation of the problem discussed here?: https://github.com/mozilla/mozilla-django-oidc/pull/345
Comment 39•5 years ago
|
||
It's hard to tell without more information, like the URLs on https://crash-stats.mozilla.org that were displaying an error page or errors in the developer tools. I also have reservations about the proposed patch on PR 345.
The links from https://dbaron.org/mozilla/crashes-by-build go to /search pages, and make XHR requests to /search/results and /search/fields. I having trouble finding the errors returned from those pages. I can only find 11 requests with https://dbaron.org/... as the referrer, with 302s (for redirect to login) at the start of the session, and 200s (success) for the later requests when the bug was filed.
Assignee | ||
Comment 40•4 years ago
|
||
https://github.com/mozilla-services/socorro/pull/5561 updated mozilla-django-oidc to pick up the changes John made to exempt urls configuration and also better session handling. We theorize both those fixes should help with this bug.
John has switched to MLS full-time, so I'm going to grab this from him and look into it soon.
Assignee | ||
Comment 41•4 years ago
|
||
Assignee | ||
Comment 42•4 years ago
|
||
willkg merged PR #5565: "bug 1473068: add signature urls to OIDC_EXEMPT_URLS" in c170a9e.
I can't reproduce the issue, so I'm going to do a light test on stage to make sure it didn't break anything, then push it to prod.
Assignee | ||
Comment 43•4 years ago
|
||
I pushed the latest changes to prod in bug #1664250.
Can anyone still reproduce this?
I'll let it sit for a week and then close it if I haven't heard from anyone.
Assignee | ||
Comment 44•4 years ago
|
||
It's been a week. I don't see any new instances of the problem in Sentry and no one has said anything, so I'm marking this FIXED!
Description
•