Karen Ward [:kward] Reporter
	Description • 10 years ago

>> Feature Request Title:
Tracking Pre-Load Apps

>> Description of feature, or problem to be solved
Today there is no way to track which apps have been pre-installed on Firefox OS branded devices.  This information is important to Mozilla because:
a) Apps Bus Dev team needs it to be able to share with developers.  (Developers are very happy to have the preferred positioning and will be more motivated to enhance their apps based on this info).
b) Marketplace stats do not include info about installation of pre-loads, only those installed by end users after first run.
c) Mozilla has revenue agreements with some 3rd party app developers and this information is important to collecting revenue.

>> Impact of implementing the feature/solution
Apps Bus Dev will be better equipped to motivate key partners to enhance their HTML5 apps.
Moz will be able to collect revenue from those 3rd party developers whom we have a rev share agreement with.

>> Impact of NOT implementing the feature/solution
As we scale, we will have no way of identifying which apps are pre-loaded.

>> Date required
This feature should be considered part of Mozilla's top goals to scale FFOS in 2014.  The sooner this feature can be implemented the better.

	Karen Ward [:kward] Reporter
	Comment 1 • 10 years ago

- See related link: https://wiki.mozilla.org/FirefoxOS/Metrics
- There are no privacy issues related to this request. See: https://www.mozilla.org/en-US/privacy/firefox-os/#top
- Once this feature is available a monthly report will be needed.  Report should contain:  
  > App Name, Marketplace URL, MCC, MNC and cumulative count of preinstalls

	Thomas Elin [:relaas]
	Comment 2 • 10 years ago

Thanks much for putting this together Karen!

Our Apps BD team is constantly being asked for this information. Partners like YT, FB, Zeptolabs, EA, etc are asking for this information while negotiating with us on next steps and more listed apps. 

Adding to the impact of NOT implementing this feature...
 
1. By NOT providing distribution numbers to our Preload app partners we are leaving them with only 1 method of retrieving activity metrics; Marketplace installs. These numbers are dramatically under reported as it only captures installs for devices that did not already have the app pre-installed and/or re-installs.

2. Not implementing this feature will continue to hamstring our BD Partner negotiations for additional content from Preload app partners. Keep in mind we can get many of our partners to trust the platform and try it out. However, if we cannot deliver tangible results it's very difficult to expand the relationship and/or drive improvements to the existing app.

	Desigan Chinniah [:cyberdees] [:dees] [London - GMT]
	Comment 3 • 10 years ago

'Thirding' this feature request — talking from experience of holding relationships with a number of Top Tier Must Have App partners. 

Once pre-loaded they have an assumed expectation that they will be able to get detailed stats on how these deals works. This includes but is not limited too:
-the usage they get out of pre-loads
-the revenue they generate per pre-loads
-the ability to see which carrier deal/s work better than others
-the ability to identify which market locale pre-loads work better than others
-have insight into exactly what devices )oem, specs etc) that they are pre-loaded on and which work better than others

	Peter Dolanjski [:pdol]
	Comment 4 • 10 years ago

Marshall, this could use your assessment on the difficulty of implementation.  
What would be involved?

	Marshall Culpepper [:marshall_law] Assignee
	Comment 5 • 10 years ago

This could be something we collect as part of the FTU (first time use) ping.

Getting a list of apps and their URLs at activation time should be pretty basic, IIRC there is an API that already does this, so integration would be easy.

	Mika
	Comment 6 • 10 years ago

I opened a legal/privacy bug (#1029868) to document the email conversation about this Bug and specify the scope of the request that's approved (preinstalls).

-mika

	Marshall Culpepper [:marshall_law] Assignee
	Comment 7 • 10 years ago

One concern I have about this is a full list of app manifest URLs can easily reach above 1KB by itself. I hacked up an example on the FxOS simulator and got a JSON string 1532 bytes in length, representing 32 apps.

Here's the code: https://gist.github.com/marshall/1bb4ee1aee30ec723fc6

For reference, the existing FTU ping aims to be at or under 1KB including headers, so this would more than double the size of the payload.

Some ways to mitigate this:

* Combine gzip compression and base64 encoding. This turned my 1532 byte JSON into 404 bytes. The amount of work to do this in gaia would probably be more than the feature itself, and as far as I can tell, it might even require exposing code from Gecko to achieve the compression (though I plan to research it to verify)

* Detect any gaia domain apps and just use their subdomain (or app name) for short. Anything else can use a simple URL shortening pass that replaces things like app://, https://, and manifest.webapp with simple tokens. I achieved 400 bytes using this method in my "Apps JSON" example in the gist. A real solution would require smarter URL parsing, but barring that I think this is actually doable with minimal overhead on the payload size

	Ravi Dandu
	Comment 8 • 10 years ago

Marshall,  Your idea of parsing URL's and using subdomain (for app name), to reduce the data looks good. Lets share an example list with Marketplace team to see if that example short app name list will suffice.

	Thomas Elin [:relaas]
	Comment 9 • 10 years ago

It's been awhile since an update was posted on this thread. 

@Marshall- were you able to developer a short list of devices and their corresponding apps as an example? If so, please forward that over asap.

If not, when do you think I could expect to see that list?

Cheers
Thomas

	Thomas Elin [:relaas]
	Comment 10 • 10 years ago

oops meant to NI Marshall

	Marshall Culpepper [:marshall_law] Assignee
	Comment 11 • 10 years ago

(In reply to thomas elin from comment #9)
> It's been awhile since an update was posted on this thread. 
> 
> @Marshall- were you able to developer a short list of devices and their
> corresponding apps as an example? If so, please forward that over asap.

To do that, I'd need to implement this feature, and then we'd need to collect and report the data server side.

> If not, when do you think I could expect to see that list?

Considering we haven't slated this for a release yet, and development for 2.1 has already begun (and my commitment is pretty booked), the feature would need to be targetted at 2.2.

	Desigan Chinniah [:cyberdees] [:dees] [London - GMT]
	Comment 12 • 10 years ago

Thanks, Marshall. How can we 200% ensure that this indeed is on the 2.2 roadmap? Who will be making the call?

We're hurting by not being able to report on this data.

	Marshall Culpepper [:marshall_law] Assignee
	Comment 13 • 10 years ago

(In reply to Desigan Chinniah [:cyberdees] [:dees] [London - GMT] from comment #12)
> Thanks, Marshall. How can we 200% ensure that this indeed is on the 2.2
> roadmap? Who will be making the call?

It looks like the feature flags for 2.2 don't exist yet, but from my perspective I should have bandwidth for this in 2.2. Tagging Hema who might know better how to properly tag this to make sure it's scheduled / prioritized for 2.2..

	Hema Koka [:hema]
	Comment 14 • 10 years ago

NI Ravi to add it to 2.2 roadmap and tagging this to be considered for 2.2

	:Harald Kirschner :digitarald
	Comment 15 • 10 years ago

[Blocking Requested - why for this release]: As Dees said, not having this prevents preload agreements, as partners need metrics and numbers to make a business case for preloads. Per :marshall_law this is low effort. 2.1 would be reasonable while 2.2 is a must.

	Ravi Dandu
	Comment 16 • 10 years ago

Lets put it in for 2.2. Our 2.1 bucket is full, so 2.2 is the natural release cycle.

	Kai-Chih Hu [:khu]
	Comment 17 • 10 years ago

Use feature-b2g:2.2? rather than just 2.2.

	Kai-Chih Hu [:khu]
	Comment 18 • 10 years ago

(In reply to rdandu from comment #16)
> Lets put it in for 2.2. Our 2.1 bucket is full, so 2.2 is the natural
> release cycle.

I removed it from 2.2, because I did not see it from our proposed feature list.

	Desigan Chinniah [:cyberdees] [:dees] [London - GMT]
	Comment 19 • 10 years ago

This feature has been pushed back a number of releases now — and feedback is that it is low effort. It is super important to our partners that are already on the content ecosystem and are being pre-loaded, not to mention those that are in discussions to become pre-loaded with our carrier partners as well as new ones that we hope to attract going forward.

Thomas — how can we get this back into 2.2, please.

	Thomas Ho [:tho]
	Comment 20 • 10 years ago

Dees, Ravi raised the apps usage ping late, and I told him clearly that I'm happy to have this included, as long as engineering confirmed that it's trivial/low effort and low risk, and it needs to be agreed by Faramarz. Thanks.

	David Almström
	Comment 21 • 10 years ago

Ravi,
please confirm that you are able to add this. As we've discussed in the past, this is quite important to add as we also have contractual agreements with some partners on revenue share (albeit low).

I'm happy to get on a call to once again 'enlight' you why this is needed. As Thomas points out, if easy - get it in :) And it should be fairly easy. FTU ping of what apps are installed on the device.

Marketplace can take care of analyzing the data but we need to get the data. See this is part of new Metrics' project.

	[:fabrice] Fabrice Desré
	Comment 22 • 10 years ago

Can anyone cc me to bug 1029868 ? thanks!

	Ravi Dandu
	Comment 23 • 9 years ago

Marshall, this was planned for 2.2, can you confirm it can be done in the timeframe for 2.2 (Branch: Jan 12, FL: Feb 23th)

	Ravi Dandu
	Comment 24 • 9 years ago

Fabrice, have cc'ed you to bug 1029868.

	Marshall Culpepper [:marshall_law] Assignee
	Comment 25 • 9 years ago

Yes this should be able to make it in for 2.2, is there anything we need to do to make sure it's accounted for?

	Marshall Culpepper [:marshall_law] Assignee
	Comment 26 • 9 years ago

This patch adds a "preinstalled" object to the existing FTU ping payload, that follows the simple format:

"preinstalled": {
  "MANIFEST_URL": "APP_NAME"
}

Because there are so many preinstalled apps, and we try to focus on keeping the FTU payload size small, I'm replacing many common strings in the manifestURL with easily replacable tokens:

{A} -> app:/
{H} -> http:/
{S} -> https:/
{G} -> gaiamobile.org
{M} -> marketplace.firefox.com
{m} -> manifest.webapp

	Marshall Culpepper [:marshall_law] Assignee
	Comment 27 • 9 years ago

Here's a sample FTU ping payload with preinstalled apps added on my Flame:

{
    "activationTime": 1423080960004,
    "app.update.channel": "default",
    "devicePixelRatio": 1.5,
    "deviceinfo.firmware_revision": "",
    "deviceinfo.hardware": "qcom",
    "deviceinfo.os": "3.0.0.0-prerelease",
    "deviceinfo.platform_build_id": "20150204135133",
    "deviceinfo.platform_version": "38.0a1",
    "deviceinfo.product_model": "flame",
    "deviceinfo.software": "Boot2Gecko 3.0.0.0-prerelease",
    "icc": null,
    "locale": "en-US",
    "network": null,
    "pingID": "fa1d1125-cc48-44b2-8f8f-48e8aef17744",
    "pingTime": 1423087433262,
    "preinstalled": {
        "{A}/addon1.{G}/{m}": "First Addon",
        "{A}/addon2.{G}/{m}": "Second Addon",
        "{A}/bluetooth.{G}/{m}": "Bluetooth Manager",
        "{A}/bookmark.{G}/{m}": "Bookmark",
        "{A}/bookmarks-reader.{G}/{m}": "Bookmarks Reader",
        "{A}/calendar.{G}/{m}": "Calendar",
        "{A}/callscreen.{G}/{m}": "Callscreen",
        "{A}/camera.{G}/{m}": "Camera",
        "{A}/clock.{G}/{m}": "Clock",
        "{A}/collection.{G}/{m}": "Smart Collections",
        "{A}/communications.{G}/{m}": "Communications",
        "{A}/contacts-ds-provider1.{G}/{m}": "Google Contacts DS",
        "{A}/contacts-ds-provider2.{G}/{m}": "LI Contacts DS",
        "{A}/contacts-manager.{G}/{m}": "ContactsManager",
        "{A}/costcontrol.{G}/{m}": "Usage",
        "{A}/default_theme.{G}/{m}": "Default Theme",
        "{A}/demo-hci-event.{G}/{m}": "HCI Event Demo",
        "{A}/download.{G}/{m}": "Downloads",
        "{A}/ds-test.{G}/{m}": "Device Storage Test",
        "{A}/email.{G}/{m}": "E-Mail",
        "{A}/emergency-call.{G}/{m}": "EmergencyCall",
        "{A}/findmydevice.{G}/{m}": "Find My Device",
        "{A}/fl.{G}/{m}": "Purchased Media",
        "{A}/fm.{G}/{m}": "FM Radio",
        "{A}/ftu.{G}/{m}": "FTU",
        "{A}/gallery.{G}/{m}": "Gallery",
        "{A}/geoloc.{G}/{m}": "Geoloc",
        "{A}/homescreen.{G}/{m}": "Homescreen",
        "{A}/keyboard.{G}/{m}": "Built-in Keyboard",
        "{A}/membuster.{G}/{m}": "Membuster",
        "{A}/music.{G}/{m}": "Music",
        "{A}/music2.{G}/{m}": "Music2",
        "{A}/network-alerts.{G}/{m}": "Network Alerts",
        "{A}/nfc-api-test.{G}/{m}": "NFC API tests",
        "{A}/operatorvariant.{G}/{m}": "OperatorVariant",
        "{A}/pdfjs.{G}/{m}": "PDF Viewer",
        "{A}/privacy-panel.{G}/{m}": "Privacy Panel",
        "{A}/ringtones.{G}/{m}": "Ringtones",
        "{A}/search.{G}/{m}": "Browser",
        "{A}/settings.{G}/{m}": "Settings",
        "{A}/share-receiver.{G}/{m}": "Share Receiver",
        "{A}/sharedtest.{G}/{m}": "Shared Tests",
        "{A}/sheet-app-1.{G}/{m}": "Sheet app 1",
        "{A}/sheet-app-2.{G}/{m}": "Sheet app 2",
        "{A}/sheet-app-3.{G}/{m}": "Sheet app 3",
        "{A}/sms.{G}/{m}": "Messages",
        "{A}/system.{G}/{m}": "System",
        "{A}/template.{G}/{m}": "Template",
        "{A}/test-agent.{G}/{m}": "Test Agent",
        "{A}/test-container.{G}/{m}": "Test Container",
        "{A}/test-findmydevice.{G}/{m}": "Test Find My Device",
        "{A}/test-fxa-client.{G}/{m}": "Test FxA Client",
        "{A}/test-iac-publisher.{G}/{m}": "Test IAC Publisher",
        "{A}/test-iac-subscriber.{G}/{m}": "Test IAC Subscriber",
        "{A}/test-ime.{G}/{m}": "IME Tests",
        "{A}/test-otasp.{G}/{m}": "Test OTASP",
        "{A}/test-receiver-1.{G}/{m}": "Test receiver#1",
        "{A}/test-receiver-2.{G}/{m}": "Test Receiver#2",
        "{A}/test-receiver-inline.{G}/{m}": "Test receiver (inline)",
        "{A}/test-shared-css.{G}/{m}": "Test Shared CSS",
        "{A}/test-wappush.{G}/{m}": "Test Wap Push",
        "{A}/theme-test-1.{G}/{m}": "Test theme 1",
        "{A}/theme-test-2.{G}/{m}": "Test theme 2",
        "{A}/theme-test-3.{G}/{m}": "Broken theme 3",
        "{A}/uitest-privileged.{G}/{m}": "UI tests - Privileged App",
        "{A}/uitest.{G}/{m}": "UI tests",
        "{A}/verticalhome.{G}/{m}": "Homescreen",
        "{A}/video.{G}/{m}": "Video",
        "{A}/wallpaper.{G}/{m}": "Wallpaper",
        "{A}/wappush.{G}/{m}": "WAP Push manager",
        "{H}/inapp-pay-test.paas.allizom.org/{m}": "In-app Payment Tester",
        "{H}/mochi.test:8888/{m}": "Mochitest",
        "{S}/{M}/app/1007e041-7d37-4eb7-b445-ff077a2bba42/{m}": "Stage",
        "{S}/{M}/app/965bbfd7-936d-451d-bebf-fafdc7ce8d9e/{m}": "Dev",
        "{S}/{M}/packaged.webapp": "Marketplace"
    },
    "screenHeight": 569,
    "screenWidth": 320,
    "ver": 3
}

	Ravi Dandu
	Comment 28 • 9 years ago

Kevin,
    Please pull into 2.2. Information on pre-installed apps is important to Mozilla because:
a) Apps Bus Dev team needs it to be able to share with developers.  (Developers are very happy to have the preferred positioning and will be more motivated to enhance their apps based on this info).
b) Marketplace stats do not include info about installation of pre-loads, only those installed by end users after first run.
c) Mozilla has revenue agreements with some 3rd party app developers and this information is important to collecting revenue.

	[:fabrice] Fabrice Desré
	Comment 29 • 9 years ago

Comment on attachment 8559408 [details] [review]
Report preinstalled apps - v1

Do we really need to send the list of gaia apps installed?

	Marshall Culpepper [:marshall_law] Assignee
	Comment 30 • 9 years ago

(In reply to Fabrice Desré [:fabrice] from comment #29)
> Do we really need to send the list of gaia apps installed?

Good catch, it seems like mostly on 3rd party preinstalled apps are needed from the requirement.. I'll update the patch before landing

	Michael:mtreese
	Comment 31 • 9 years ago

https://bugzilla.mozilla.org/show_bug.cgi?id=1109426

	Marshall Culpepper [:marshall_law] Assignee
	Comment 32 • 9 years ago

I've removed Gaia apps from the preinstalled listing, and also the shortening of the URLs to avoid complexity in the testing and impact of this patch. We can revisit shortening URLs if it data size becomes an issue, but removing Gaia apps substantially decreases the number of strings sent in the payload, so I'm less worried about it at this point.

	Marshall Culpepper [:marshall_law] Assignee
	Comment 33 • 9 years ago

Comment on attachment 8559408 [details] [review]
Report preinstalled apps - v1

Hey Fabrice, I just realized you marked this with feedback+. I think I've addressed your question about Gaia apps in the newest commit (I've removed them completely which actually does make more sense). I'll hold back on landing until I have your r+..

	[:fabrice] Fabrice Desré
	Comment 34 • 9 years ago

Comment on attachment 8559408 [details] [review]
Report preinstalled apps - v1

lgtm, thanks.

	Stephany Wilkes
	Comment 35 • 9 years ago

Giving this a 2.2+ in B2G triage based on the reasons stated in comment #28.

	Marshall Culpepper [:marshall_law] Assignee
	Comment 36 • 9 years ago

master - https://github.com/mozilla-b2g/gaia/commit/6a5f674dbf93d47f43f7e8e85bb584703c122ca0

	Marshall Culpepper [:marshall_law] Assignee
	Comment 38 • 9 years ago

Comment on attachment 8567152 [details] [review]
[gaia] marshall:bug1021173_preinstalledApps_v2.2 > mozilla-b2g:v2.2

Approval to land preinstalled app report in FTU ping for v2.2
[Bug caused by] (feature/regressing bug #): Bug 1021173
[User impact] if declined: We won't see which partner apps are preinstalled in FxOS devices
[Testing completed]: Unit tests, manual tests in production mode
[Risk to taking this patch] (and alternatives if risky): This increases the size of the FTU ping by around 50-100 bytes depending on the app URLs involved
[String changes made]: None

	Ryan VanderMeulen [:RyanVM]
	Comment 39 • 9 years ago

v2.2: https://github.com/mozilla-b2g/gaia/commit/db973fbdbc6f3a6b6fe8d8c40146b28fa9253e43

	Shing Lyu [:shinglyu]
	Comment 40 • 9 years ago

I notice that "https://marketplace.firefox.com/packaged.webapp": "Marketplace" is in the preinstalled list, although it is not a valuable information for partners. But I guess it's ok, since the incentive to skip gaiamobile.org apps is to reduce the size of the payload.