Closed Bug 1175123 Opened 9 years ago Closed 9 years ago

keyword trigger suggestion provider

Categories

(Input Graveyard :: Submission, defect, P3)

Product:
Input Graveyard
Component:
Submission
Type:
defect

Tracking

(Not tracked)

Status:
RESOLVED FIXED

People

(Reporter: willkg, Assigned: willkg)

References

Details

(Whiteboard: u=user c=suggestions p=5 s=input.2015q3) 

Fjord has a suggestions framework that allows us to build providers that suggest links the user might be interested in based on the feedback the user has just left.

So far we have a single provider that queries SUMO for kb articles that might be relevant to the user's problems as indicated by their feedback.

There are a couple of additional use cases we'd like to cover:

1. Sees trigger words in the feedback and asks the user to complete a survey for more information.

2. Sees trigger words in the feedback and tells the user to go read our hot-fix workaround article.

This bug covers implementing a trigger-keyword-based provider.

We want to make it as simple as possible for now. We can enhance it later as new use cases/needs arise.
This is going to require a db table for trigger-keyword suggestions. Off the top of my head, it should have the following columns:

Data:

1. id
2. title
3. description
4. url
5. is_enabled
6. priority -- determines order of links from the trigger-keyword-based provider

Matching specification:

7. locales -- which locales are affected
8. products -- which products are affected (many-to-many to Product table)
9. keywords -- newline separated list of key words and phrases to match on, all lowercase, gets turned into a regex

We'd be doing "AND" operations for the matching requirements. So something that had locales "en-US" and products "Firefox" and keywords "rc4" would only trigger on feedback that had *all* of those things.


Additionally, I think we want create, edit and delete pages for these in the analyzer's area.
Talked with Matt on IRC about the matching specification. Here are some examples of how I think it would work:

Rule1:
   locales=en-US
   products=Firefox
   keywords=rc4

This would match all feedback where the locale was "en-US" AND the product was "Firefox" and the word "rc4" appeared in the description.


Rule2:
   locales=
   products=Firefox OS
   keywords=whatsapp

This would match all feedback from any locale AND the product was "Firefox OS" and the word "whatsapp" appeared in the description.


Rule3:
   locales=es,pt
   products=Firefox,Firefox for Android
   keywords=word1,word2,word3,"this is a phrase"

This would match all feedback where the locale was "es" OR "pt" AND the product was "Firefox" OR "Firefox for Android" AND the word "word1", "word2", "word3" OR phrase "this is a phrase" appeared in the description.


That's moderately flexible and should let us express the situations I can imagine will come up. Plus it's straight-forward to implement.

If we discover we need something more complex, we'll need a "for realz" rules engine and we'll have to rewrite this.
Gregg mentioned being able to filter on versions would be good, too. For this to be useful, Input would need to be capturing good version information and I'm not sure it does. Need to look into this.
Also, after some more discussion on IRC, I'm going to name these trigger rules. The explanation being:

    A trigger rule is a set of requirements which when satisfied
    for a given feedback will produce a suggested link.
This would be helpful. One nice thing about doing it now is that it'll be all ready for the next release which is in two weeks in case we need to use it.

Given that, I'm grabbing it to work on.
Assignee: nobody → willkg
Status: NEW → ASSIGNED
Priority: -- → P3
We talked on IRC about being able to track:

1. how many people received trigger keyword suggestion x?
2. how many people clicked on trigger keyword suggestion x?

Additionally, Gregg said for surveys and other things like that, it's super helpful to have some metadata about the user's feedback request like locale, product, version, etc.

I think that means we need a redirector that allows for templated urls. Gregg suggested using the Firefox TemplateUrl scheme where {LOCALE} gets replaced with the locale. We could probably just go with Python formatting and do {locale}, {product}, etc.

Anyhow this makes it much easier on the survey end to do tailoring of things to make them work better.
I got about 60% of the way done with this, but I think I need to rework parts of it and today is the last day I can deploy for a while. Given that, I won't be able to finish this until mid-July.
Oops--said all that, but also meant to bump it to next quarter.
Whiteboard: u=user c=suggestions p= s=input.2015q2 → u=user c=suggestions p= s=input.2015q3
Converting this to a tracker bug. It's getting a bit big for one bug.

Also, I created a project plan: https://wiki.mozilla.org/Firefox/Input/Keyword_suggester
Summary: keyword trigger suggestion provider → [tracker] keyword trigger suggestion provider
Talked with webcompat. We also want a yes/no checkbox for "must have url".
Making this a 5 point bug. I've done a bunch of work on it already and I have a bunch more work to do to account for new requirements.
Whiteboard: u=user c=suggestions p= s=input.2015q3 → u=user c=suggestions p=5 s=input.2015q3
Still working on this. I've got another day or two to go, I think. Then it'll be ready for a PR.

I'm pushing off the "test the trigger" thing until after I land the bulk of it. I have to mull over how it should work UX-wise. While it's important, I think we can launch without it. If it turns out we *really* need it, then no one will use it until the testing piece is in.
Untrackerifying this bug since I'm doing it all under this bug number.
Summary: [tracker] keyword trigger suggestion provider → keyword trigger suggestion provider
In a PR: https://github.com/mozilla/fjord/pull/643

There are some outstanding things to do which I'll spin off into different bugs:

1. list of valid locales for the locale field
2. templated urls
3. trigger rule tester

After the pull request gets reviewed, I'll push it to stage and put it through more testing. If it's good to go, then we can push it to production.

We can enable/disable it in settings, so if worse comes to worse, it's a quick deploy to disable it.
Landed in master: https://github.com/mozilla/fjord/commit/9556e2debe1370fdbea4079ac689c3394a5325ea

Still outstanding:

1. templated urls

2. way to see list of valid locales (maybe even switch the form field used to one that lets us choose (except we don't have a definitive list of all locales :( ))

3. trigger rule tester


I'll push the code to stage and test there. Then to prod where it'll sit unused for a bit. Then I'll write up the outstanding bugs and close this out.
I wrote up bugs for the outstanding work.

I pushed this to stage and started more thorough testing. First off, it's really hard without a trigger rule tester. Regardless, I found a bug where creating a trigger for "best" won't match "firefox is the best!" because of the "!" at the end. I need to redo the regex that's getting built for keyword matching.

I'll do that work in this bug.
Pushed this to prod just now. Marking as FIXED.
Status: ASSIGNED → RESOLVED
Closed: 9 years ago
Resolution: --- → FIXED
Product: Input → Input Graveyard
You need to log in before you can comment on or make changes to this bug.