Open Bug 1475334 Opened 6 years ago Updated 3 months ago

Deprecate Symbolication v4 endpoint

Categories

(Eliot :: Symbolication, task, P2)

Tracking

(Not tracked)

People

(Reporter: peterbe, Unassigned)

References

Details

The old v4 endpoint is discouraged. It requires you to send one stack at a time. With the v5 endpoint you can send batches of multiple stacks in one big JSON blob which performs better as a whole because there's less network and internally, the extraction of symbols from the Redis store can potentially reuse symbols in the intersection of modules mentioned in each job's `memoryMap`. 

The new URL everyone should be using is:

  https://symbols.mozilla.org/symbolicate/v5 

The old URL that we should deprecate is:

  https://symbols.mozilla.org/symbolicate/v4

Moving bugs to Tecken product.

Component: Tecken → Symbolication
Product: Socorro → Tecken

I added a graph to our dashboard showing version usage. As near as I can tell, only v4 is in use--v5 isn't used anywhere.

Next step here is to redo the symbolication docs so they more clearly document the v4 and v5 APIs and walk through migrating from v4 to v5. Once that's done and we have something to point to, we need to figure out what's using v4 and get those things updated.

Making this a P3 for now.

Priority: -- → P3

Here's a list of things using the v4 API:

We can update that code to use v5. Then since they're part of Firefox, we'll have to wait until there aren't any supported versions of Firefox that have the v4-using code and we can nix v4.

I'm bumping this up to P2 because the sooner we do this, the sooner it'll expire from everything.

Priority: P3 → P2
Assignee: nobody → willkg
Status: NEW → ASSIGNED

In Matrix today:

willkg: i think chutten said i could remove the about:telemetry use. that's not in the bug, so maybe i have additional notes about this elsewhere.
chutten: Yes. about:telemetry is very infrequently used (< 200 pageviews per... week? release? Very few over fairly long anyway) and the folks that do use it tend to be mozilla FTEs who have access to workarounds if you remove something. (Might be nice to put the instructions for the workaround in about:telemetry itself, but that's probably not crucial.)

I think the telemetry code is already dead code and can be ignored for the purposes of removing the v4 endpoint. I don't see anything that produces capturedStacks telemetry, so the relevant section will never be shown on about:telemetry. https://searchfox.org/mozilla-central/search?q=capturedStacks&path=&case=true&regexp=false
I've filed bug 1730186 about this.

I went through and documented uses of the symbolication api versions in the Tecken Symbolication API proposal in September 2020. It's probably the case it needs to be updated, but copying it here so I remember next time I'm looking for it.

about:telemetry

about:telemetry uses the v4 symbolication API. This is shipped as part of Firefox.

https://searchfox.org/mozilla-central/source/toolkit/content/aboutTelemetry.js

mozgeckoprofiler

mozgeckoprofiler uses the v4 symbolication API and has code to degrade to a non-existent v3 version. This is shipped as part of Firefox and runs in various test scenarios.

https://searchfox.org/mozilla-central/source/testing/mozbase/mozgeckoprofiler/mozgeckoprofiler/symbolicationRequest.py

Firefox devtools profiler

The Firefox devtools profiler uses the v5 symbolication API to symbolicate profiles. They don't send multiple jobs--they send single requests.

Their single requests can contain stacks with 10-15K items in it because they're not really symbolicating a stack, they're symbolicating addresses in a library.

https://github.com/firefox-devtools/profiler/blob/75ac93cc247f6fc538dbd328027e5d23e0ce8ed2/src/profile-logic/symbol-store.js#L111-L112

https://github.com/firefox-devtools/profiler/blob/c4f68f423c7035a094952311addd0af5bec209e7/src/profile-logic/mozilla-symbolication-api.js#L121

Probabilistic Heap Checker (PHC)

PHC has two modes. One mode does the symbolication locally by parsing sym files and doing sym lookups using that data. The other mode does symbolication using the v5 symbolication API. It's not using the batch feature of the v5 API.

They're currently symbolicating locally because they need line numbers which the symbolication API doesn't support.

https://github.com/choller/phc-tools

I'm not going to work on this any time soon so unassigning.

Assignee: willkg → nobody
Status: ASSIGNED → NEW

Moving bugs into new Eliot product.

Product: Tecken → Eliot

From bug #1825806 on 2023-12-21:

agent count
Python-urllib/3.9 306,121
Python-urllib/3.7 157,507
Python-urllib/3.8 1,928
Python-urllib/3.11 32

Those user agents don't tell us anything about who's using the v4 api.

I think the next step is to remove the v4 api endpoint and have it return an HTTP error (404?) with a message along these lines:

{"error": "this api endpoint has been deprecated; please switch to the symbolication v5 api <docs link>"}
You need to log in before you can comment on or make changes to this bug.