feluchs Reporter
	Description • 6 years ago

User Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/63.0.3239.84 Chrome/63.0.3239.84 Safari/537.36

Steps to reproduce:

add following line to opensearch document; referenced in root html head section
<Url type="application/json" rel="suggestions" ...></Url>


Actual results:

firefox doese not allow such search engine; and if it doese it doese not use suggestion service; firefox expects "application/x-suggestions+json"


Expected results:

As  can be seen here: http://www.opensearch.org/Specifications/OpenSearch/1.1#Url_rel_values
under the "The "Url" element" headline. Multiple Examples for implementation "suggestion" are given. Look at "Example of a Url element describing a request for search suggestions over JSON:".
while reference is showing type="application/json" ... firefox doese not except it. Firefox doese except => application/x-suggestions+json. In my opinion, this is counterproductive. Not only is this not an accepted IANA standard, it is also not compliant with the opensearch standard.

Also: Could not verify that installed opensearch search engines are updated when source xml on server is changed. Server logs dont show any request after initial fetch!

	Mark Banner (:standard8) Assignee
	Comment 1 • 6 years ago

Having read through the specification, and the suggestions extension (http://www.opensearch.org/Specifications/OpenSearch/Extensions/Suggestions), I believe the main specification's example is wrong.

The suggestions specification explicitly references the response type:

```
The following type is used to indicate that the response will contain search suggestions:

    application/x-suggestions+json 
```

The main specification gives the example of 'application/json' as you say, but I believe that is wrong and definitely conflicts with the definition in the suggestions spec.

I'm trying to report this back to opensearch, but their website won't allow me to log in, so I'm trying via the email list.

	feluchs Reporter
	Comment 2 • 6 years ago

(In reply to Mark Banner (:standard8) from comment #1)
> Having read through the specification, and the suggestions extension
> (http://www.opensearch.org/Specifications/OpenSearch/Extensions/Suggestions),
> I believe the main specification's example is wrong.
> 
> The suggestions specification explicitly references the response type:
> 
> ```
> The following type is used to indicate that the response will contain search
> suggestions:
> 
>     application/x-suggestions+json 
> ```
> 
> The main specification gives the example of 'application/json' as you say,
> but I believe that is wrong and definitely conflicts with the definition in
> the suggestions spec.
> 
> I'm trying to report this back to opensearch, but their website won't allow
> me to log in, so I'm trying via the email list.

Hi Mark,

were you successful in the attempt to report this bug to the 'opensearch' website? Did you get any new information whether the 'type' could/should be set to 'application/json'?

feluchs

	Mark Banner (:standard8) Assignee
	Comment 3 • 6 years ago

There was a discussion here, although action seems to have stalled a bit...

https://groups.google.com/d/msg/opensearch/24v1tPcrVe8/5nyYJ6rECQAJ

	Mark Banner (:standard8) Assignee
	Comment 4 • 6 years ago

They've now moved the site: https://groups.google.com/d/msg/opensearch/oUnQz1X3Ugk/MjMuyg3DCAAJ

I've submitted a PR for the correction:

https://github.com/dewitt/opensearch/pull/1

Note: I probably won't update this bug any more. Please follow along there.

	feluchs Reporter
	Comment 5 • 6 years ago

great. thanks for all your help.

	Mark Banner (:standard8) Assignee
	Comment 6 • 6 years ago

Following along from the PR, we've had a few discussions and we're proposing to support both. There's an intent to ship here:

https://groups.google.com/d/msg/mozilla.dev.platform/mlNx35Xjw0U/AOVtEv41BgAJ

	Mark Banner (:standard8) Assignee
	Comment 7 • 6 years ago

Eric, what needs documenting here? As far as I know we don't have anything referencing or describing OpenSearch on developer.mozilla.org. We will try and get the OpenSearch specs updated when we do this though...

	Eric Shepherd [:sheppy]
	Comment 8 • 6 years ago

We do, in fact, but the default search options on MDN leave it out because none of its tags are included in the set of tags that show up in search results by default. I have updated MDN's settings (to create a new "Web Standards" tag that will show up in search by default) and have added that tag to our OpenSearch page at https://developer.mozilla.org/en-US/docs/Web/OpenSearch. This change needs to propagate through our CDN but should start showing up soon.

	Mark Banner (:standard8) Assignee
	Comment 9 • 6 years ago

(In reply to Eric Shepherd [:sheppy] from comment #8)
> We do, in fact, but the default search options on MDN leave it out because
> none of its tags are included in the set of tags that show up in search
> results by default. I have updated MDN's settings (to create a new "Web
> Standards" tag that will show up in search by default) and have added that
> tag to our OpenSearch page at
> https://developer.mozilla.org/en-US/docs/Web/OpenSearch. This change needs
> to propagate through our CDN but should start showing up soon.

Great, thank you!

	feluchs Reporter
	Comment 10 • 6 years ago

Hi Mark and Eric,
I think it's good that you take care of this.

However, I believe that a greater acceptance range of MIME types is not a good solution, even though the motives are clear to me. This approach leads in my opinion only to softening of existing standards. I would argue that we should drop 'application/x-suggestions+json' completely in favor of 'application/json'. Introducing 'application/x-suggestions+json' was already a mistake.

[+] Format/Schema used to structure server provided suggestions is valid JSON (https://www.json.org/json-de.html)
[+] IETF RFC6648 is deprecating MIME types with prefix 'x-' (https://tools.ietf.org/html/rfc6648)
[+] 'application/x-suggestions+json' would imply that code is execute on the server (https://pagerangers.com/seo-handbuch/onpage/allgemein/was-ist-ein-mime-typ)
[+] Clearer format announcement
[-] Maybe missing compatible server implementations

Alternatively, a motion could be submitted for 'application/suggestions+json', as this is apparently not yet occupied (https://www.iana.org/assignments/media-types/media-types.xhtml). Personally, I do not consider this solution desirable.

[+] MIME type with fixed defined schema for search suggestions
[+] Recognizability of the content without a context
[-] A further MIME standard without real gain over 'application/json'
[-] Additional creation effort and maintenance needed
[-] Compatibility issues with browsers already expecting 'application/json' today

	Mark Banner (:standard8) Assignee
	Comment 11 • 6 years ago

(In reply to feluchs from comment #10)
> However, I believe that a greater acceptance range of MIME types is not a
> good solution, even though the motives are clear to me. This approach leads
> in my opinion only to softening of existing standards. I would argue that we
> should drop 'application/x-suggestions+json' completely in favor of
> 'application/json'. Introducing 'application/x-suggestions+json' was already
> a mistake.

From a site compatibility viewpoint, we need to support both on the Firefox side. If we switched to application/json, we'd break compatibility with many sites straight away.

For the specification itself, we can propose that application/json becomes the new standard with application/x-suggestions+json being marked as deprecated.

	Mark Banner (:standard8) Assignee
	Comment 13 • 6 years ago

feluchs: I think the patch is working correctly, though I've a little more testing to do.

Could you check one of the builds and see if it works for you? They are based on the latest Firefox nightly, so you might want to set up a new profile to test with.

Linux (64bit): https://queue.taskcluster.net/v1/task/SmLLJgyqS7KPAC3m02_lKQ/runs/0/artifacts/public/build/target.tar.bz2
Mac: https://queue.taskcluster.net/v1/task/DIyLlDigT0eupXzDHpJaXA/runs/0/artifacts/public/build/target.dmg
Windows: https://queue.taskcluster.net/v1/task/H4j5dOECSSGpDA1RWNWYFg/runs/0/artifacts/public/build/install/sea/target.installer.exe

	feluchs Reporter
	Comment 14 • 6 years ago

Mark: Unfortunately, I currently have no test environment and pinched under a big pile of work. I try to get one in the next few days and can then run a few tests. Really great that you have stuck through and have now fixed this bug. Thank you very much!

	Mark Banner (:standard8) Assignee
	Comment 15 • 6 years ago

Ok, I just set up a local test with `<Url type='application/json' rel='suggestions' ...` in the xml, and it seemed to work fine.

I also tested returning the suggestions as a text/html content type (i.e. something not application/json), and it still works ok (we don't seem to care what type, as long as the file is json itself - but that's probably OK, as the type is specified in the url definition).

	Mike de Boer [:mikedeboer]
	Comment 16 • 6 years ago

Comment on attachment 9003190 [details]
Bug 1425827 - Allow "application/json" as well as "application/x-suggestions+json" as MIME types for Suggestions in OpenSearch

Mike de Boer [:mikedeboer] has approved the revision.

	Pulsebot
	Comment 17 • 6 years ago

Pushed by mbanner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/a6f19768aa91
Allow "application/json" as well as "application/x-suggestions+json" as MIME types for Suggestions in OpenSearch r=mikedeboer

	Andrei Ciure[:aciure]
	Comment 18 • 6 years ago
bugherder

https://hg.mozilla.org/mozilla-central/rev/a6f19768aa91

	Chris Mills [:cmills]
	Comment 19 • 6 years ago

I've documented this by:

mentioning that this new MIME type is now accepted as a search URL type, and that it is an alias of application/x-suggestions+json; see https://developer.mozilla.org/en-US/docs/Web/OpenSearch

Adding a note to the Fx63 rel notes: https://developer.mozilla.org/en-US/docs/Mozilla/Firefox/Releases/63#Other

Does that sound OK?