Closed Bug 1408727 Opened 7 years ago Closed 7 years ago

Some sites not working in just Firefox: Invalid OCSP signing certificate in OCSP response. Error: SEC_ERROR_OCSP_INVALID_SIGNING_CERT

Categories

(Core :: Security, defect)

Product:
Core
Component:
Security
Type:
defect
Priority:
Not set
Severity:
normal

Tracking

()

Status:
RESOLVED DUPLICATE of bug 1368868

People

(Reporter: achavez, Unassigned)

References

Details

Attachments

(2 files)

error SEC_ERROR_OCSP_INVALID_SIGNING_CERT
262.74 KB, application/pdf
Details
ssllab report
421.30 KB, application/pdf
Details 
<Caspy7> ashlee: is the OCSP signing issue high priority at the moment? Is this a Firefox problem or a server problem?

other browsers are unaffected as far as I know

check this out in Firefox, see if it loads https://www.cbsnews.com/news/adam-sandler-saves-day-for-letterman/

someone here, the top comment, is a firefox sumo helper I believe and indicates the issue is at least known... https://www.reddit.com/r/firefox/comments/76gkhu/firefox_stopped_showing_images_from_ebay_chrome/ 

another user with another server issue, same error and https://www.reddit.com/r/firefox/comments/76hosc/firefox_56_x64_not_opening_amazonde_ocsp_error/
Flags: needinfo?(past)
I was able to load https://www.cbsnews.com/news/adam-sandler-saves-day-for-letterman/ and my colleague, daveio was able to load the site as well.

A few minutes after reporting the issue, Caspy7 was able to load the site. 

Wanted to file this bug for tracking purposes in case others report of the same issue.

Thanks!
Sounds like a transient issue with the site, so I am resolving this. I will also needinfo David in case he thinks there is anything else to do here.
Status: NEW → RESOLVED
Closed: 7 years ago
Component: General → Security
Flags: needinfo?(past) → needinfo?(dkeeler)
Product: Firefox → Core
Resolution: --- → WORKSFORME
> Sounds like a transient issue with the site

I'm curious why this issue only affected Firefox.
Still seeing reports of this.
One on IRC just now for https://admission.ignou.ac.in/ which also doesn't work for me (currently).
Another from reddit for https://www.asrock.com/ (ditto)
And an additional "me too" comment in that reddit post https://redd.it/76hosc

These all seem to work in Chrome. This gives users the impression that something is wrong with Firefox. Is there a good action or mitigation going forward?

In response to the IRC link someone said:
> The OCSP response for that site says "Next Update: Oct 16 00:14:59 2017 GMT" so it ought to start working then

More info on that link: https://www.ssllabs.com/ssltest/analyze.html?d=admission.ignou.ac.in

Can anyone provide insight here? Who is at fault? Is there anything we can/should do?
Two aspects here:

(1) Whose fault:

For nearly all of the problem sites during the time Firefox could not connect, SSLLabs showed this in the last table on the report:

OCSP stapling: Invalid - Failed to validate response

And they all have:

Issuer: Symantec Class 3 Secure Server CA - G4

Fingerprint SHA256: eae72eb454bf6c3977ebd289e970b2f5282949190093d0d26f98d0f0d6a9cf17
Pin SHA256: 9n0izTnSRF+W4W4JTq51avSXkWhQB8duS2bxVLfzXsY=

CRL: http://ss.symcb.com/ss.crl
OCSP: http://ss.symcd.com

Perhaps Symantec's servers messed up a bunch of verifications that still have not completely worked their way out of the system?

(2) How to better handle this:

I don't know why the stapled response was invalid. Since some of these sites still give the error, someone who knows how to extract SSL exchange details probably can pinpoint the problematic response more precisely.

Let's assume we know what we're looking for. What should Firefox do differently? For example:

* Automatically double-check the CRL/OCSP independently of the stapled response -- this would be most seamless.
* Ask the user if they want to double-check the CRL/OCSP independently of the stapled response -- this would be a smidge more privacy-respecting but it's hard to know whether users care about issuers having an OCSP check as a data point.
Based on further reports on IRC, my previous comments about new reports as well as dups, this is still an issue. Reopening.
Status: RESOLVED → REOPENED
Resolution: WORKSFORME → ---
Summary: User reports issues with sites loading in Firefox browser specifically → Some sites not working in just Firefox: Invalid OCSP signing certificate in OCSP response. Error: SEC_ERROR_OCSP_INVALID_SIGNING_CERT
I have the same issue. It started yesterday (Sunday) The morning was fine the afternoon no go. I have tested this in multiple browsers on multiple desktops ONLY Firefox gives an issue

When checking the server it is sending the correct certificates including the intermediate certificate. this appears to be a Firefox only problem.

Manual checking using openssl says the cert is fine.

SSLLabs says the cert is fine

    Path #1: Trusted
    1 	Sent by server 	XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
    Fingerprint SHA256: 9101a290b8acaa0d433948dcf88abc65b9e9e8560807cd6214b6771d488af9d6
    Pin SHA256: mhqQUl1u7buPywI4yZAfDsJnd+j0ajapemNkRWz13ec=
    RSA 2048 bits (e 65537) / SHA256withRSA
    2 	Sent by server 	Symantec Class 3 Secure Server CA - G4
    Fingerprint SHA256: eae72eb454bf6c3977ebd289e970b2f5282949190093d0d26f98d0f0d6a9cf17
    Pin SHA256: 9n0izTnSRF+W4W4JTq51avSXkWhQB8duS2bxVLfzXsY=
    RSA 2048 bits (e 65537) / SHA256withRSA
    3 	In trust store 	VeriSign Class 3 Public Primary Certification Authority - G5   Self-signed	
    Fingerprint SHA256: 9acfab7e43c8d880d06b262a94deeee4b4659989c3d0caf19baf6405e41ab7df
    Pin SHA256: JbQbUG5JMJUoI6brnx0x3vZF6jilxsapbXGVfjhN8Fg=
    RSA 2048 bits (e 65537) / SHA1withRSA
    Weak or insecure signature, but no impact on root certificate


Again this is a Symantec problem but why is it only effecting Firefox?
(In reply to Chris from comment #11) 
> When checking the server it is sending the correct certificates including
> the intermediate certificate. this appears to be a Firefox only problem.
> 
> Manual checking using openssl says the cert is fine.

This is a problem with OCSP stapling. Something is wrong with the verification of non-revocation of the certificate.

Does OpenSSL allow you to gain any insight on that?

On the SSLLabs report, look at the last table on the page for the OCSP Stapling line. Usually it indicates INVALID for the problem hosts, although that did not appear with the eBay image server.
As I stated ssllabs did not indicate any issues. OCSP Stapling was listed as a plain "Yes"

The problem has cleared now. It was evident for a little over a day.
Sorry must take that back. was looking at the wrong ssllabs response. it was listed as "OCSP stapling 	Invalid   Failed to validate response"

So the take away is the problem was with Symantec (Doh like they don't have enough issues already) and that Firefox is much more strict and immediately flagged the error yet IE and Chrome let it slide...
> So the take away is the problem was with Symantec and that Firefox is much more strict and immediately flagged the error yet IE and Chrome let it slide...

That's what I was afraid of.

So as I understand, Firefox was not in the wrong here and enforcement was proper.

This presents the question of whether enforcement should change at all. What's Chrome & IE's logic when they let it slide (for X number of days?)?

My biggest rub here is that this ends up making Firefox look bad every time it happens and people blame the browser - after all, it doesn't happen in Chrome.
Attached file ssllab report — 

    
    

    
  
this problem affects and sites with server certs issued by other CA
see my attachments - https://e-postbank.bg can't be opened with Firefox, but its ssl cert is from GeoTrust
the error is SEC_ERROR_OCSP_INVALID_SIGNING_CERT and ssllabs  shows 
Revocation status - Good (not revoked) 
OCSP stapling - Invalid - Failed to validate response

    
    

    
  
We are also experiencing this problem now, with Firefox 56.0.1 and 52.4.1 ESR.

openssl reports that everything is fine with the server, this also seems to work fine in all other browsers, so I would gues it is a bug in firefox.

    
    

    
  
Should also mention that we are using a certificate from thawte, Inc.

The openssl command I tested with was openssl s_client -connect <server>:443 -tls1 -tlsextdebug -status

Some of the output:
OCSP response:
======================================
OCSP Response Data:
    OCSP Response Status: successful (0x0)
    Response Type: Basic OCSP Response
    Version: 1 (0x0)
    Responder Id: 25AF2A14892709EDFEDEF97D37309C299EED37D8
    Produced At: Oct 10 19:14:46 2017 GMT
    Responses:
    Certificate ID:
      Hash Algorithm: sha1
      Issuer Name Hash: 6F765B8601CFE5631D5B5C50679653F5BA060798
      Issuer Key Hash: C24F4857FCD14F9AC05D387D0E05DBD92EB55260
      Serial Number: 0C3F1BA58652D6EE12B9D093AD5A0802
    Cert Status: good
    This Update: Oct 10 19:14:46 2017 GMT
    Next Update: Oct 17 19:14:46 2017 GMT

    
    

    
  
The issue here appears to be that Symantec is issuing OCSP response whose validity period ends after the OCSP responder certificate expires. This results in them OCSP responses not validating.

These sites are using OCSP stapling, and their server software has a (probably very common) bug where it does not refresh the OCSP staple in this case.

I'm not sure there's anything Firefox can do in these cases; servers need to be more resilient to OCSP servers giving them responses like this, and OCSP servers _really_ shouldn't issue responses whose validity exceeds their own.
See Also:  → 1368868

    
    

    
  
(In reply to Alex Gaynor [:Alex_Gaynor] from comment #21)

Is there an official bug addressed to Symantec about this "bad" OCSP responses?

Do you know which web servers are affected by this "very common bug". Is there an official bug addressed to them?
Based on report from http://toolbar.netcraft.com/site_report
there are different web servers affected - IIS, AkamaiGHost, cloudflare-nginx ...

btw - I can't find any bug report in MSDN related to OCSP stapling

    
    

    
  
(In reply to Caspy7 from comment #5)
> Can anyone provide insight here? Who is at fault? Is there anything we can/should do?

First, the situation should be explained to the user. What is the reason for the error/message?

I have used the description from OCSP Stapling in Firefox [blog.mozilla.org. 29.07.2013] as basis. It contains pictures that illustrates the process with a explaining text completes. I think it were a good idea to create a site under support.mozilla.org and add a link to this site with a a short explanation. 

I do not know whether it is possible to give the user at this point the option to temporarily disable the OCSP Stapling and directly send a request to the CA.

    
    

    
  
> Do you know which web servers are affected by this "very common bug". Is there an official bug addressed to them?

Here's some more detail on how Apache and Nginx are broken with respect to OCSP Stapling: https://blog.hboeck.de/archives/886-The-Problem-with-OCSP-Stapling-and-Must-Staple-and-why-Certificate-Revocation-is-still-broken.html

Also: https://gist.github.com/sleevi/5efe9ef98961ecfb4da8 and https://blog.cloudflare.com/high-reliability-ocsp-stapling/.

The short version is: When OCSP Stapling was introduced, it was thought of as a privacy and performance feature, so servers treated it as optional / best effort. However, as I understand it, Firefox's implementation was done with the newer Must Staple standard in mind, and so chooses to be stricter than most other implementations, as a way to help ferret out bugs in server implementations before / while sites roll out Must Staple. This is one such bug.

Apache bug: https://bz.apache.org/bugzilla/show_bug.cgi?id=57121
Nginx bugs: https://trac.nginx.org/nginx/ticket/812
  https://trac.nginx.org/nginx/ticket/990

    
    

    
  
(In reply to Jacob from comment #24)
> The short version is: When OCSP Stapling was introduced, it was thought of
> as a privacy and performance feature, so servers treated it as optional /
> best effort.

As far as I could tell from the rfc and its historical versions, the spec included language about clients aborting the handshake if the stapled revocation information is inadequate from the first draft. I have no insight as to if or why server authors considered it optional/best effort except perhaps that historically OCSP has essentially been best-effort.

In any case, any changes we make to firefox will likely happen in bug 1368868, so I'll mark this as a duplicate of that.
Status: REOPENED → RESOLVED
Closed: 7 years ago → 7 years ago
Flags: needinfo?(dkeeler)
Resolution: --- → DUPLICATE

          You need to log in
          before you can comment on or make changes to this bug.
        






  

    
  







  

    

      
Attachment

      

      
      
      
      
    

    

      

        

          

            
General

            

              Creator: 



            

            
Created: 

            
Updated: 

            
Size: