Closed Bug 1746747 Opened 2 years ago Closed 2 years ago

[mozcrash] use build symbols on disk where possible

Categories

(Testing :: Mozbase, defect)

Default
defect

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: willkg, Assigned: Gankra)

References

Details

Attachments

(2 files)

I was working on bug #1594515 when I discovered that mozcrash runs minidump-stackwalk such that minidump-stackwalk fetches all the symbols from symbols.mozilla.org rather than using symbols from the symbols path.

https://bugzilla.mozilla.org/show_bug.cgi?id=1594515#c25

I claim this is not what we want. What we want is for mozcrash to use the symbols that were created with the build and use symbols.mozilla.org for system symbols. This will work in situations where symbols were not uploaded to symbols.mozilla.org and possibly speed up the tests a little (downloading the symbols from the build artifacts as a big zip vs. downloading individual symbols files from symbols.mozilla.org and wherever the symbols live (AWS S3 for now, but GCP GCS soon)).

My guess is that the whole "ondemand" thing isn't working right in some way. Either it's not downloading the build symbols at all, or it downloads them but doesn't unzip them, or it's putting them in a different place than it tells minidump-stackwalk about.

I'm game for debugging further and fixing it.

It'd be great if someone who knows more about or owns mozcrash could confirm this bug (you can look at my comments, try runs, and logs from bug #1594515) and give me the green light to fix it first.

The severity field is not set for this bug.
:gbrown, could you have a look please?

For more information, please visit auto_nag documentation.

Flags: needinfo?(gbrown)
Severity: -- → S4
Flags: needinfo?(gbrown)
See Also: → 1735910

:glandium - Recall that you added symbols.mozilla.org to the (automation) minidump-stackwalk command line, in bug 1735910. Your comment there says "Fallback to the symbols server" whereas this bug suggests the symbols server is being used preferentially. Can you clarify?

Flags: needinfo?(mh+mozilla)

We switched to a new minidump-stackwalk in rust, that may have a different behavior.

Flags: needinfo?(mh+mozilla) → needinfo?(a.beingessner)

At least on paper, the new implementation wires up the local symbols:

https://searchfox.org/mozilla-central/rev/5ce782034b736293836a2850ab698e149ca04657/testing/mozbase/mozcrash/mozcrash/mozcrash.py#389-393

I can confirm the new implementation preferentially uses any local symbols it has over http. That said, I do not know if the new implementation and the old implementation disagree on the "format" of these symbols (or if they've just been broken forever).

I previously tried to make sense of where all the inputs from mozcrash.py come from, but the set of depenencies was too byzantine for me to make sense of. My gut is to add some debug printing to mozcrash.py to inspect the contents of the local symbols.

Flags: needinfo?(a.beingessner)

Checking the logs of a crash we have the line:

[task 2021-12-17T22:25:28.545Z] 22:25:28     INFO - mozcrash Copy/paste: /builds/worker/fetches/minidump_stackwalk/minidump_stackwalk --symbols-url=https://symbols.mozilla.org/ --human /builds/worker/workspace/build/tmpjug9jf6h.mozrunner/minidumps/28fdbde5-db71-475c-3dd3-7a5015a24132.dmp /tmp/tmpkzliztex

So we are indeed passing in something as local symbols -- /tmp/tmpkzliztex. Gonna add some logging to inspect the structure of that.

Assignee: nobody → a.beingessner

Not sure if this is relevant, but....

aiui opt and debug tasks handle symbol downloads differently, depending on download_symbols:
https://searchfox.org/mozilla-central/rev/435a77f1a1aaf1a78d30a2aaa81c6158a2f83dba/testing/mozharness/mozharness/mozilla/testing/testbase.py#522
Debug tasks download and extract symbols at task startup, and opt only downloads and extracts them when there is a crash.

Here are some crashes:
https://treeherder.mozilla.org/jobs?repo=try&revision=6bfa2451fe6a60ae9041ba0f2e36784b96d02ba7

In the debug case I see:

[task 2022-01-03T18:14:34.900Z] 18:14:34     INFO - Downloading and extracting to /builds/worker/workspace/build/symbols these dirs * from https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Et1p71iuTea6bB6ZBc_Mug/artifacts/public/build/target.crashreporter-symbols.zip
...
[task 2022-01-03T18:19:34.024Z] 18:19:34     INFO - mozcrash Copy/paste: /builds/worker/fetches/minidump_stackwalk/minidump_stackwalk --symbols-url=https://symbols.mozilla.org/ --human /tmp/tmpbxayigg0.mozrunner/minidumps/70e257c5-04bb-0e77-0356-e6315b8be504.dmp /builds/worker/workspace/build/symbols

Hopefully target.crashreporter-symbols.zip was extracted to /builds/worker/workspace/build/symbols.

In the opt case I see:

[task 2022-01-03T18:19:18.964Z] 18:19:18     INFO - mozcrash Downloading symbols from: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AA10PRfZQJybAkvAXeyWEg/artifacts/public/build/target.crashreporter-symbols.zip
[task 2022-01-03T18:19:23.863Z] 18:19:23     INFO - mozcrash Copy/paste: /builds/worker/fetches/minidump_stackwalk/minidump_stackwalk --symbols-url=https://symbols.mozilla.org/ --human /tmp/tmp_s12nra0.mozrunner/minidumps/6c78385a-04cb-23b6-0793-0d407ff55c17.dmp /tmp/tmp6jhmca8d

Hopefully target.crashreporter-symbols.zip was extracted to /tmp/tmp6jhmca8d.

This part of the code certainly seems dubious:

https://searchfox.org/mozilla-central/rev/080e18fa4748456003164f58b0d925b8c3826a67/testing/mozbase/mozcrash/mozcrash/mozcrash.py#253-271

        # If no symbols path has been set create a temporary folder to let the
        # minidump stackwalk download the symbols.
        if not self.symbols_path:
            self.symbols_path = tempfile.mkdtemp()
            self.remove_symbols = True

If there's no symbols it just makes an empty tempdir and passes that in. I think that code is trying to set --symbols-tmp/--symbols-cache but passing it as a symbols-path (which mdsw can create for itself if you don't supply them).

The other path in this function does correctly unzip and pass the unzipped dir as symbols-path. If that path is taken you get an info with "Downloading symbols from: ...". Not sure about the "Downloading and extracting to..." path, that seems to happen outside of mozcrash.py.

Still trying to get a crash in try with extra logging

At least for this crash, the local symbols seem perfectly cromulent: https://treeherder.mozilla.org/logviewer?job_id=364047955&repo=try&lineNumber=2413

 Copy/paste: /builds/worker/fetches/minidump_stackwalk/minidump_stackwalk --symbols-url=https://symbols.mozilla.org/ --verbose=trace --human /tmp/tmp4ehgbhgm.mozrunner/minidumps/31729052-5e5e-8b20-6eff-8c335a60a875.dmp /tmp/tmpe4zc2ku2

Directory Spider:

 /tmp/tmpe4zc2ku2
   TestEndian
     D3B0178D38D2032D1889DEDCCDDA5ED00
       file: TestEndian.sym
   BadCertAndPinningServer
     EDDAF3A03CDDD2BAF00E9E1EBCBA1BD60
       file: BadCertAndPinningServer.sym
   TestCasting
     C74F705AB09ACC85843EB4BA82EE4D770
       file: TestCasting.sym
   TestResult
     2A32B5E0C0DF3B7A3CB88AE741B009220
       file: TestResult.sym
   TestSPSCQueue
     1BBD5342B66310FFD64155EFA21FC1320
       file: TestSPSCQueue.sym
   updater
     8694DC93A93A4B059DFFA446D0FF6D7D0
       file: updater.sym
   certutil
     7A7ACC2255B1BCB6CA22F5633EC16BE70
       file: certutil.sym
   TestVector
     E7925BED6C69B5F7A0AC9F0319FED0450
       file: TestVector.sym
   TestTypeTraits
     E2ACE2A49F613A77407496F6FB3A9EC70
       file: TestTypeTraits.sym
   TestEnumSet
     2E76757DD232F6EFA92FE7678D7F5C6F0
       file: TestEnumSet.sym
   TestEnumeratedArray
     591BC98C869878505A3E42EACF36CB490
       file: TestEnumeratedArray.sym
   logalloc-replay
     EA2048FD8FB9B8B3599DD5A23FEDE2FC0
       file: logalloc-replay.sym
   xpcshell
     880E503B525B7DFBF75D754F09E5EF830
       file: xpcshell.sym
   libsoftokn3.so
     56963F22D8EE32187D539D851121F95A0
       file: libsoftokn3.so.sym
   TestBufferList
     E2E8DD0BFBE769856FFB03C2F41E80060
       file: TestBufferList.sym
   TestRange
     28FF631E80CC7587585F689E75D006CA0
       file: TestRange.sym
   TestCountPopulation
     6E5D77C766E5AEAAD135F1B5C18A77840
       file: TestCountPopulation.sym
   TestHashTable
     5486884580B28B5677444B0C4A570ED00
       file: TestHashTable.sym
   TestJSONWriter
     2C43ED64DB182DD227AA4E331ABE11DD0
       file: TestJSONWriter.sym
   TestRefPtr
     F2C3CB9C0E6F8497C1C196BEB1605E260
       file: TestRefPtr.sym
   js
     AE3727507BE769174B613236426EB23D0
       file: js.sym
   libnspr4.so
     80082CFFCFA17587B5F9F7BA16D3C2660
       file: libnspr4.so.sym
   TestMaybe
     A1DCFA82513FDF07186CB685BE0380610
       file: TestMaybe.sym
   TestQuickReturn
     AF75F7944A131E7DC1A93C153E33AE630
       file: TestQuickReturn.sym
   TestTuple
     ECF34ABB23321BB57FFD146CFEDDAC730
       file: TestTuple.sym
   OCSPStaplingServer
     639277BB334DCE6718AC46FEADB6DD760
       file: OCSPStaplingServer.sym
   libplc4.so
     CF55E15651D5AF1004F010B338DF45760
       file: libplc4.so.sym
   gdb-tests
     787213784CE364C31712F9D9743520480
       file: gdb-tests.sym
   TestAlgorithm
     A782ACEEAA9F47519CFE1ECAFC3B53AB0
       file: TestAlgorithm.sym
   TestRollingMean
     D1B60AB935EEFCB4F0857D4012E043740
       file: TestRollingMean.sym
   TestSmallPointerArray
     ABE7C0B6FCB8D32029B829C9B9E350110
       file: TestSmallPointerArray.sym
   firefox-bin
     47089C6E4BD0E4699F66D1023E57EA6D0
       file: firefox-bin.sym
   TestMathAlgorithms
     A86F85AC0022C2B8D11B82151CE1AAF40
       file: TestMathAlgorithms.sym
   pingsender
     9A9F72805CAB57870B4E93E7585F63540
       file: pingsender.sym
   screentopng
     1779F8652130D766EA8A1071067FBE390
       file: screentopng.sym
   TestMemoryPressureWatcherLinux
     9AE8EFD68CCC37BD2CE47840DC7CBABF0
       file: TestMemoryPressureWatcherLinux.sym
   TestTextUtils
     30AC34CC1A23C4196DB0254EA06BB93D0
       file: TestTextUtils.sym
   libnssutil3.so
     FE018AFAB9B1E16F25BA86F57A82906C0
       file: libnssutil3.so.sym
   TestArguments
     6ED970CE75D6D24D76D8633DA7A7D5600
       file: TestArguments.sym
   TestNotNull
     FECF5ED5266CF3AFD2FC293CFB706F8E0
       file: TestNotNull.sym
   TestUnicodeArguments
     F8A9835F3937FC2B9E118ED1CD0946280
       file: TestUnicodeArguments.sym
   TestBinarySearch
     68076E5915D0711C4845F9B11A68C9190
       file: TestBinarySearch.sym
   liblgpllibs.so
     9AFC395B4C85B792188289AB343ACFE50
       file: liblgpllibs.so.sym
   plugin-container
     A81E2938C444833714027EBFD0D831150
       file: plugin-container.sym
   libfreeblpriv3.so
     EAC43F4AECA9395DF2F32A73FBBF5B9C0
       file: libfreeblpriv3.so.sym
   TestUtf8
     D555483A1252947C791B2F3B7069DE550
       file: TestUtf8.sym
   libmozavutil.so
     F4361D966AC3AC49BDC6E70EAA268A980
       file: libmozavutil.so.sym
   libclearkey.so
     C3968C5726D2A4F42E9175FA148CC0030
       file: libclearkey.so.sym
   TestEnumTypeTraits
     B677404EDA92BD8CDA3445EBDCFC92790
       file: TestEnumTypeTraits.sym
   TestThreadSafeWeakPtr
     AA6196E018084A799E2A06CC754898020
       file: TestThreadSafeWeakPtr.sym
   libmozsandbox.so
     F28D515714C5AF18752869A8122280CC0
       file: libmozsandbox.so.sym
   minidump-analyzer
     EE45ED6D53EA2E852193AB9B5D3DDCE40
       file: minidump-analyzer.sym
   libxul.so
     61CD6124B55230FDE7165367012750660
       file: libxul.so.sym
     F86007E5B8EB0DD191CCD964267311440
       file: libxul.so.sym
   libmozgtk.so
     291346C71689D358C6F5D68928AC3FFD0
       file: libmozgtk.so.sym
   TestNonDereferenceable
     58770F76D35D00840ACC5F89EF562CD10
       file: TestNonDereferenceable.sym
   TestDoublyLinkedList
     6C2C4E0E4B8F839AA7164296E34D84230
       file: TestDoublyLinkedList.sym
   TestArray
     9A1B017EBC41E48464F605AC325F1BB30
       file: TestArray.sym
   libnss3.so
     62EBD51AD7D1CF515C041D9A835C5DE90
       file: libnss3.so.sym
   TestArrayUtils
     61E091E6AB00C71CAAEE55F1FAF742A40
       file: TestArrayUtils.sym
   TestSHA1
     0554B7A0EC539742EE9BD22D324EB6540
       file: TestSHA1.sym
   TestPRIntN
     70035B38A3E224BB20FD5CD51E8C3E300
       file: TestPRIntN.sym
   TestDefineEnum
     E544DC6A94BB15B319E6B4FE64D10D010
       file: TestDefineEnum.sym
   rapl
     83FE5A0D77C1F03C9ED3BD909DA675B30
       file: rapl.sym
   firefox
     96806C35ED072D4F99C4FE082AE8E79C0
       file: firefox.sym
   TestAtomics
     0BB44DDF8D4C7D6F71E67275E9E69F4E0
       file: TestAtomics.sym
   TestMacroForEach
     98B48C0DBB08971BAD20DFF32A96106A0
       file: TestMacroForEach.sym
   WriteArgument
     B0E0233EF4B856D4FC21F3AA6078CC150
       file: WriteArgument.sym
   TestSaturate
     AC76231F5EBDDF2E1961DA622CF52ADE0
       file: TestSaturate.sym
   libfake.so
     7683B9E3AD2E75CFC7BFF5865748FDBB0
       file: libfake.so.sym
   TestFastBernoulliTrial
     F460D019BF3657EB58DCF81ECF5550340
       file: TestFastBernoulliTrial.sym
   TestFunctionRef
     8ECA5B9EC7F89A2707DCC56075FDA4DA0
       file: TestFunctionRef.sym
   TestRandomNum
     8199C171D80F46A3C0EA8170B3508F460
       file: TestRandomNum.sym
   TestMacroArgs
     92A33DF728E861619B79AC3C388B5A3C0
       file: TestMacroArgs.sym
   libsmime3.so
     D73F41D4E47B9C408AABAF6C515C2AAF0
       file: libsmime3.so.sym
   TestAtomicBitfields
     8D6DD3503FCC2F33FCBF013B004DE0D40
       file: TestAtomicBitfields.sym
   TestBitSet
     B8DFFF68A7F63692DC132A43712804B50
       file: TestBitSet.sym
   DelegatedCredentialsServer
     A58F8BE8C51AE60C5A685FC596CB28380
       file: DelegatedCredentialsServer.sym
   TestVariant
     D404A95871A86FE94F5B1DBEE878CDD80
       file: TestVariant.sym
   TestIntegerPrintfMacros
     5C67A753330206A57A0DD775D78F48FE0
       file: TestIntegerPrintfMacros.sym
   TestPrintf
     6B02F863340BEB13B22E966D6808CFC00
       file: TestPrintf.sym
   libmozavcodec.so
     4AF8BB2D2CDEFFF40694C08F38938D4C0
       file: libmozavcodec.so.sym
   libmodules-test.so
     34C034D9F92B72E50BA2D2A22E483B930
       file: libmodules-test.so.sym
   ssltunnel
     925535F13203BA8C9CA0D18CEB0EDF480
       file: ssltunnel.sym
   TestCheckedInt
     A42DAD5C0333FB3CC3BE26241D65063C0
       file: TestCheckedInt.sym
   TestFunctionTypeTraits
     6A593CF3BF2707021F2C7CE3C65D1B070
       file: TestFunctionTypeTraits.sym
   TestBlockingProcess
     1E86C4B2788AE3A9B242A530946BC6550
       file: TestBlockingProcess.sym
   TestWeakPtr
     B44BF4BA4C6B10079ECECC69FF934CDC0
       file: TestWeakPtr.sym
   signmar
     3C02F652A31467D3CDC76FA521FC7BE70
       file: signmar.sym
   SanctionsTestServer
     EEC3F7A12C5DD3E04CE350DB87EA82C90
       file: SanctionsTestServer.sym
   TestXorShift128PlusRNG
     34C60F9CEC8923258613F48BE2CB50F50
       file: TestXorShift128PlusRNG.sym
   jsapi-tests
     9DBEDF04C418C5C3DB5650A67770A2690
       file: jsapi-tests.sym
   TestTemplateLib
     5452670B0BB0F42CE3412466AF90FFE60
       file: TestTemplateLib.sym
   TestWrappingOperations
     C088C86469FEAC7173652D3AA02650AD0
       file: TestWrappingOperations.sym
   libipcclientcerts.so
     305BE6643B42706C618487120A2C27EF0
       file: libipcclientcerts.so.sym
   libnssckbi.so
     E4E3247D280A25F9D98F7624569755C00
       file: libnssckbi.so.sym
   EncryptedClientHelloServer
     52165148E3FEBBCD53070EFA4B9E30080
       file: EncryptedClientHelloServer.sym
   GenerateOCSPResponse
     0DE302C5BB55A9CE07DDEEE9C790AD450
       file: GenerateOCSPResponse.sym
   libmozsqlite3.so
     26BE0D3EBAE3F8415C66B4716D60BED70
       file: libmozsqlite3.so.sym
   TestPoisonArea
     9371DF2A554DC559337F487247EB85E70
       file: TestPoisonArea.sym
   libplds4.so
     6AD6E1C2B789CCE58E4535FB929B66E00
       file: libplds4.so.sym
   TestCompactPair
     8353343C5618135ECD02A6887540C7000
       file: TestCompactPair.sym
   pk12util
     B5B304E79D5031B31D4669CD7D888B0A0
       file: pk12util.sym
   TestIntegerRange
     DEF6C1522AFF7C5EA445E0C66C9267780
       file: TestIntegerRange.sym
   shlibsign
     0D2994FA590D5FB1694A71C10030BBED0
       file: shlibsign.sym
   libssl3.so
     69DC43594FE8FD233EAED2C86F577E6A0
       file: libssl3.so.sym
   TestSplayTree
     B4AB95724F15218416676A0C181564AA0
       file: TestSplayTree.sym
   libjsctypes-test.so
     4A89DA4A07F75DE31F7BC86A6C0C2E7D0
       file: libjsctypes-test.so.sym
   ShowSSEConfig
     B44C56FBC9348AA18177B4CEC201009F0
       file: ShowSSEConfig.sym
   crashreporter
     BCDE01251CA268D6EA80BF49B04F9BDA0
       file: crashreporter.sym
   libmozwayland.so
     E3255EAE5C0EC22D9549F13B7E19A1D90
       file: libmozwayland.so.sym
   TestBaseProfiler
     C621581E697367224C3F97671923E7220
       file: TestBaseProfiler.sym
   libfakeopenh264.so
     F83A84D02F43B7607F454EBBD862D07C0
       file: libfakeopenh264.so.sym
   TestFloatingPoint
     1BE474C4E95D1E47F04C7CE1997BACCF0
       file: TestFloatingPoint.sym
   TestSegmentedVector
     729166B5540DF3FF8514F6E55B6306C10
       file: TestSegmentedVector.sym
   libfreebl3.so
     A14754F49A510C7E3D7B66A715B3D5E20
       file: libfreebl3.so.sym
   TestBloomFilter
     695186CEE3D063536060678D93C975260
       file: TestBloomFilter.sym
   SmokeDMD
     31A4498F637DDAEA701CEE3BE5856BB10
       file: SmokeDMD.sym
   TestCountZeroes
     AC7300A33E1A217B814BD6B575FC844F0
       file: TestCountZeroes.sym
   TestTypedEnum
     079459C35291F301D8DA2FE7D9DFD1700
       file: TestTypedEnum.sym
   TestScopeExit
     68620A47605029765A81EEDB353A82940
       file: TestScopeExit.sym
   TestCeilingFloor
     01EB17D3F957CAB16F805196E59959EF0
       file: TestCeilingFloor.sym
   TestUniquePtr
     8A8B163E88263F122B08A23E4DB2974D0
       file: TestUniquePtr.sym
   TestLinkedList
     E1D0C241CBB0447A89D68135D5F390900
       file: TestLinkedList.sym
   modutil
     7EF3358ED0E308CDCB668CD3A5DAD5FC0
       file: modutil.sym

My sneaking suspicion is that the debugids for the modules are somehow wrong (other possibility: corrupt symbol files) (other other possibility: symbol paths in rust-minidump are buggier than I expect).

Actually, libxul.so is a bit curious here:

   libxul.so
     61CD6124B55230FDE7165367012750660
       file: libxul.so.sym
     F86007E5B8EB0DD191CCD964267311440
       file: libxul.so.sym

Why are there two different versions of libxul in the local symbols?

Oh nice, the minidump gets uploaded. I can confirm it is looking for 61CD6124B55230FDE7165367012750660, so it seems to be working perfectly in my test case?

I noticed in the original log where willkg was running into trouble, there is no "Downloading symbols from: ..." line. This suggests to me that the configuration he was hitting is the evil one I discuss here.

Perhaps marionnette tests are failing to pass in a value for symbols_path when they invoke mozcrash.py?

(In reply to Aria Beingessner [:Gankra] from comment #11)

Why are there two different versions of libxul in the local symbols?

There always are two. One is the gtest libxul.

Yep, if you look at my crash:

https://treeherder.mozilla.org/logviewer?job_id=364047953&repo=try&lineNumber=2391

vs willkg's crash:

https://treeherder.mozilla.org/logviewer?job_id=361688385&repo=try&lineNumber=1371

and search for --symbols-path, then you see mine comes up with lots of results, but willkg's has none.

So marionette tests aren't getting any local symbols fed in at all, which means mozcrash.py has none to give to minidump-stackwalk.

So now the question is "why" and "how do we fix that"

I expect --symbols-path to originate in mozharness, like
https://searchfox.org/mozilla-central/rev/080e18fa4748456003164f58b0d925b8c3826a67/testing/mozharness/configs/unittests/linux_unittest.py#66

But I see no such provision in telemetry_client.py.

This is too many levels of inheritance and dependency injection for me to make sense of @-@

gbrown, are you experienced enough with these scripts to take it from here? (If not, is anyone?)

Flags: needinfo?(gbrown)

In case useful, desktop_unittest.py notes that symbols_path is supposed to be setup by the TestMixin._download_and_extract_symbols()

https://searchfox.org/mozilla-central/rev/080e18fa4748456003164f58b0d925b8c3826a67/testing/mozharness/scripts/desktop_unittest.py#564-568

and indeed we see some logic to this effect:

https://searchfox.org/mozilla-central/rev/080e18fa4748456003164f58b0d925b8c3826a67/testing/mozharness/mozharness/mozilla/testing/testbase.py#522-525

telemetry_client.py does also uses that mixin, but perhaps it's messing up the state the superclass needs..?

I'll see what I can do - having a look ...

Flags: needinfo?(gbrown)

Previously mozcrash would just create an empty temp dir and pass that in as a symbols_path.
The comment suggests this would be used as a --symbols-cache/--symbols-tmp, but that's
certainly not how the new minidump-stackwalk handles positional symbols-path arguments?

minidump-stackwalk is perfectly capable of automatically creating its own tempdirs,
so nothing actually needs to be done. Instead I emit a warning and remove the guards
against the value being missing.

Attached patch: doesn't actually fix the problem, but should help make it clear when it happens (and make it easier to tell if you fix it).

Pass the symbols_path from mozharness to the telemetry test harness,
to enable improved crash reports.

Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/17fcc18648c8
Set symbols_path in telemetry tests; r=releng-reviewers,jmaher

Backed out changeset 17fcc18648c8 (Bug 1746747) for causing telemetry test failures.
Backout link
Push with failures
Failure Log

Flags: needinfo?(a.beingessner)

It looks like symbols_path is not always set in the telemetry mozharness script. Now why would that be...

Flags: needinfo?(a.beingessner)
Backout by mlaza@mozilla.com:
https://hg.mozilla.org/mozilla-central/rev/25ebce40c654
Backed out changeset 17fcc18648c8 for causing telemetry test failures. CLOSED TREE IGNORE BAD COMMIT MESSAGES

leave-open for Gankra's patch.

Keywords: leave-open
Pushed by gbrown@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/6635ff053064
Set symbols_path in telemetry tests; r=releng-reviewers,jmaher
Pushed by abeingessner@mozilla.com:
https://hg.mozilla.org/integration/autoland/rev/38aba501f840
Provide accurate feedback about the about the absence of symbols_path. r=KrisWright

marking as fixed, but willkg is hoping to do some more testing

Status: NEW → RESOLVED
Closed: 2 years ago
Keywords: leave-open
Resolution: --- → FIXED

I verified that this works now.

Here's a build where I removed the flag to get symbols from symbols.mozilla.org and forced the telemetry-tests-client to crash:

https://treeherder.mozilla.org/jobs?repo=try&revision=71617fffef6e89c79ba8d98986027eb764e0571a&selectedTaskRun=EnuQnJyeRKCdvmI2KsTVfA.0

I see symbolicated stacks for Linux, macOS, and Windows.

Thank you!

You need to log in before you can comment on or make changes to this bug.

Attachment

General

Created:
Updated:
Size: