[mozcrash] use build symbols on disk where possible
Categories
(Testing :: Mozbase, defect)
Tracking
(Not tracked)
People
(Reporter: willkg, Assigned: Gankra)
References
Details
Attachments
(2 files)
I was working on bug #1594515 when I discovered that mozcrash runs minidump-stackwalk such that minidump-stackwalk fetches all the symbols from symbols.mozilla.org rather than using symbols from the symbols path.
https://bugzilla.mozilla.org/show_bug.cgi?id=1594515#c25
I claim this is not what we want. What we want is for mozcrash to use the symbols that were created with the build and use symbols.mozilla.org for system symbols. This will work in situations where symbols were not uploaded to symbols.mozilla.org and possibly speed up the tests a little (downloading the symbols from the build artifacts as a big zip vs. downloading individual symbols files from symbols.mozilla.org and wherever the symbols live (AWS S3 for now, but GCP GCS soon)).
My guess is that the whole "ondemand" thing isn't working right in some way. Either it's not downloading the build symbols at all, or it downloads them but doesn't unzip them, or it's putting them in a different place than it tells minidump-stackwalk about.
Reporter | ||
Comment 1•2 years ago
|
||
I'm game for debugging further and fixing it.
It'd be great if someone who knows more about or owns mozcrash could confirm this bug (you can look at my comments, try runs, and logs from bug #1594515) and give me the green light to fix it first.
Comment 2•2 years ago
|
||
The severity field is not set for this bug.
:gbrown, could you have a look please?
For more information, please visit auto_nag documentation.
Updated•2 years ago
|
Comment 3•2 years ago
|
||
:glandium - Recall that you added symbols.mozilla.org to the (automation) minidump-stackwalk command line, in bug 1735910. Your comment there says "Fallback to the symbols server" whereas this bug suggests the symbols server is being used preferentially. Can you clarify?
Comment 4•2 years ago
|
||
We switched to a new minidump-stackwalk in rust, that may have a different behavior.
Assignee | ||
Comment 5•2 years ago
|
||
At least on paper, the new implementation wires up the local symbols:
I can confirm the new implementation preferentially uses any local symbols it has over http. That said, I do not know if the new implementation and the old implementation disagree on the "format" of these symbols (or if they've just been broken forever).
I previously tried to make sense of where all the inputs from mozcrash.py come from, but the set of depenencies was too byzantine for me to make sense of. My gut is to add some debug printing to mozcrash.py to inspect the contents of the local symbols.
Assignee | ||
Comment 6•2 years ago
|
||
Checking the logs of a crash we have the line:
[task 2021-12-17T22:25:28.545Z] 22:25:28 INFO - mozcrash Copy/paste: /builds/worker/fetches/minidump_stackwalk/minidump_stackwalk --symbols-url=https://symbols.mozilla.org/ --human /builds/worker/workspace/build/tmpjug9jf6h.mozrunner/minidumps/28fdbde5-db71-475c-3dd3-7a5015a24132.dmp /tmp/tmpkzliztex
So we are indeed passing in something as local symbols -- /tmp/tmpkzliztex
. Gonna add some logging to inspect the structure of that.
Assignee | ||
Updated•2 years ago
|
Comment 7•2 years ago
|
||
Not sure if this is relevant, but....
aiui opt and debug tasks handle symbol downloads differently, depending on download_symbols:
https://searchfox.org/mozilla-central/rev/435a77f1a1aaf1a78d30a2aaa81c6158a2f83dba/testing/mozharness/mozharness/mozilla/testing/testbase.py#522
Debug tasks download and extract symbols at task startup, and opt only downloads and extracts them when there is a crash.
Here are some crashes:
https://treeherder.mozilla.org/jobs?repo=try&revision=6bfa2451fe6a60ae9041ba0f2e36784b96d02ba7
In the debug case I see:
[task 2022-01-03T18:14:34.900Z] 18:14:34 INFO - Downloading and extracting to /builds/worker/workspace/build/symbols these dirs * from https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/Et1p71iuTea6bB6ZBc_Mug/artifacts/public/build/target.crashreporter-symbols.zip
...
[task 2022-01-03T18:19:34.024Z] 18:19:34 INFO - mozcrash Copy/paste: /builds/worker/fetches/minidump_stackwalk/minidump_stackwalk --symbols-url=https://symbols.mozilla.org/ --human /tmp/tmpbxayigg0.mozrunner/minidumps/70e257c5-04bb-0e77-0356-e6315b8be504.dmp /builds/worker/workspace/build/symbols
Hopefully target.crashreporter-symbols.zip was extracted to /builds/worker/workspace/build/symbols.
In the opt case I see:
[task 2022-01-03T18:19:18.964Z] 18:19:18 INFO - mozcrash Downloading symbols from: https://firefox-ci-tc.services.mozilla.com/api/queue/v1/task/AA10PRfZQJybAkvAXeyWEg/artifacts/public/build/target.crashreporter-symbols.zip
[task 2022-01-03T18:19:23.863Z] 18:19:23 INFO - mozcrash Copy/paste: /builds/worker/fetches/minidump_stackwalk/minidump_stackwalk --symbols-url=https://symbols.mozilla.org/ --human /tmp/tmp_s12nra0.mozrunner/minidumps/6c78385a-04cb-23b6-0793-0d407ff55c17.dmp /tmp/tmp6jhmca8d
Hopefully target.crashreporter-symbols.zip was extracted to /tmp/tmp6jhmca8d.
Assignee | ||
Comment 8•2 years ago
|
||
This part of the code certainly seems dubious:
# If no symbols path has been set create a temporary folder to let the
# minidump stackwalk download the symbols.
if not self.symbols_path:
self.symbols_path = tempfile.mkdtemp()
self.remove_symbols = True
If there's no symbols it just makes an empty tempdir and passes that in. I think that code is trying to set --symbols-tmp/--symbols-cache but passing it as a symbols-path (which mdsw can create for itself if you don't supply them).
The other path in this function does correctly unzip and pass the unzipped dir as symbols-path. If that path is taken you get an info with "Downloading symbols from: ...". Not sure about the "Downloading and extracting to..." path, that seems to happen outside of mozcrash.py.
Still trying to get a crash in try with extra logging
Assignee | ||
Comment 9•2 years ago
|
||
At least for this crash, the local symbols seem perfectly cromulent: https://treeherder.mozilla.org/logviewer?job_id=364047955&repo=try&lineNumber=2413
Copy/paste: /builds/worker/fetches/minidump_stackwalk/minidump_stackwalk --symbols-url=https://symbols.mozilla.org/ --verbose=trace --human /tmp/tmp4ehgbhgm.mozrunner/minidumps/31729052-5e5e-8b20-6eff-8c335a60a875.dmp /tmp/tmpe4zc2ku2
Directory Spider:
/tmp/tmpe4zc2ku2
TestEndian
D3B0178D38D2032D1889DEDCCDDA5ED00
file: TestEndian.sym
BadCertAndPinningServer
EDDAF3A03CDDD2BAF00E9E1EBCBA1BD60
file: BadCertAndPinningServer.sym
TestCasting
C74F705AB09ACC85843EB4BA82EE4D770
file: TestCasting.sym
TestResult
2A32B5E0C0DF3B7A3CB88AE741B009220
file: TestResult.sym
TestSPSCQueue
1BBD5342B66310FFD64155EFA21FC1320
file: TestSPSCQueue.sym
updater
8694DC93A93A4B059DFFA446D0FF6D7D0
file: updater.sym
certutil
7A7ACC2255B1BCB6CA22F5633EC16BE70
file: certutil.sym
TestVector
E7925BED6C69B5F7A0AC9F0319FED0450
file: TestVector.sym
TestTypeTraits
E2ACE2A49F613A77407496F6FB3A9EC70
file: TestTypeTraits.sym
TestEnumSet
2E76757DD232F6EFA92FE7678D7F5C6F0
file: TestEnumSet.sym
TestEnumeratedArray
591BC98C869878505A3E42EACF36CB490
file: TestEnumeratedArray.sym
logalloc-replay
EA2048FD8FB9B8B3599DD5A23FEDE2FC0
file: logalloc-replay.sym
xpcshell
880E503B525B7DFBF75D754F09E5EF830
file: xpcshell.sym
libsoftokn3.so
56963F22D8EE32187D539D851121F95A0
file: libsoftokn3.so.sym
TestBufferList
E2E8DD0BFBE769856FFB03C2F41E80060
file: TestBufferList.sym
TestRange
28FF631E80CC7587585F689E75D006CA0
file: TestRange.sym
TestCountPopulation
6E5D77C766E5AEAAD135F1B5C18A77840
file: TestCountPopulation.sym
TestHashTable
5486884580B28B5677444B0C4A570ED00
file: TestHashTable.sym
TestJSONWriter
2C43ED64DB182DD227AA4E331ABE11DD0
file: TestJSONWriter.sym
TestRefPtr
F2C3CB9C0E6F8497C1C196BEB1605E260
file: TestRefPtr.sym
js
AE3727507BE769174B613236426EB23D0
file: js.sym
libnspr4.so
80082CFFCFA17587B5F9F7BA16D3C2660
file: libnspr4.so.sym
TestMaybe
A1DCFA82513FDF07186CB685BE0380610
file: TestMaybe.sym
TestQuickReturn
AF75F7944A131E7DC1A93C153E33AE630
file: TestQuickReturn.sym
TestTuple
ECF34ABB23321BB57FFD146CFEDDAC730
file: TestTuple.sym
OCSPStaplingServer
639277BB334DCE6718AC46FEADB6DD760
file: OCSPStaplingServer.sym
libplc4.so
CF55E15651D5AF1004F010B338DF45760
file: libplc4.so.sym
gdb-tests
787213784CE364C31712F9D9743520480
file: gdb-tests.sym
TestAlgorithm
A782ACEEAA9F47519CFE1ECAFC3B53AB0
file: TestAlgorithm.sym
TestRollingMean
D1B60AB935EEFCB4F0857D4012E043740
file: TestRollingMean.sym
TestSmallPointerArray
ABE7C0B6FCB8D32029B829C9B9E350110
file: TestSmallPointerArray.sym
firefox-bin
47089C6E4BD0E4699F66D1023E57EA6D0
file: firefox-bin.sym
TestMathAlgorithms
A86F85AC0022C2B8D11B82151CE1AAF40
file: TestMathAlgorithms.sym
pingsender
9A9F72805CAB57870B4E93E7585F63540
file: pingsender.sym
screentopng
1779F8652130D766EA8A1071067FBE390
file: screentopng.sym
TestMemoryPressureWatcherLinux
9AE8EFD68CCC37BD2CE47840DC7CBABF0
file: TestMemoryPressureWatcherLinux.sym
TestTextUtils
30AC34CC1A23C4196DB0254EA06BB93D0
file: TestTextUtils.sym
libnssutil3.so
FE018AFAB9B1E16F25BA86F57A82906C0
file: libnssutil3.so.sym
TestArguments
6ED970CE75D6D24D76D8633DA7A7D5600
file: TestArguments.sym
TestNotNull
FECF5ED5266CF3AFD2FC293CFB706F8E0
file: TestNotNull.sym
TestUnicodeArguments
F8A9835F3937FC2B9E118ED1CD0946280
file: TestUnicodeArguments.sym
TestBinarySearch
68076E5915D0711C4845F9B11A68C9190
file: TestBinarySearch.sym
liblgpllibs.so
9AFC395B4C85B792188289AB343ACFE50
file: liblgpllibs.so.sym
plugin-container
A81E2938C444833714027EBFD0D831150
file: plugin-container.sym
libfreeblpriv3.so
EAC43F4AECA9395DF2F32A73FBBF5B9C0
file: libfreeblpriv3.so.sym
TestUtf8
D555483A1252947C791B2F3B7069DE550
file: TestUtf8.sym
libmozavutil.so
F4361D966AC3AC49BDC6E70EAA268A980
file: libmozavutil.so.sym
libclearkey.so
C3968C5726D2A4F42E9175FA148CC0030
file: libclearkey.so.sym
TestEnumTypeTraits
B677404EDA92BD8CDA3445EBDCFC92790
file: TestEnumTypeTraits.sym
TestThreadSafeWeakPtr
AA6196E018084A799E2A06CC754898020
file: TestThreadSafeWeakPtr.sym
libmozsandbox.so
F28D515714C5AF18752869A8122280CC0
file: libmozsandbox.so.sym
minidump-analyzer
EE45ED6D53EA2E852193AB9B5D3DDCE40
file: minidump-analyzer.sym
libxul.so
61CD6124B55230FDE7165367012750660
file: libxul.so.sym
F86007E5B8EB0DD191CCD964267311440
file: libxul.so.sym
libmozgtk.so
291346C71689D358C6F5D68928AC3FFD0
file: libmozgtk.so.sym
TestNonDereferenceable
58770F76D35D00840ACC5F89EF562CD10
file: TestNonDereferenceable.sym
TestDoublyLinkedList
6C2C4E0E4B8F839AA7164296E34D84230
file: TestDoublyLinkedList.sym
TestArray
9A1B017EBC41E48464F605AC325F1BB30
file: TestArray.sym
libnss3.so
62EBD51AD7D1CF515C041D9A835C5DE90
file: libnss3.so.sym
TestArrayUtils
61E091E6AB00C71CAAEE55F1FAF742A40
file: TestArrayUtils.sym
TestSHA1
0554B7A0EC539742EE9BD22D324EB6540
file: TestSHA1.sym
TestPRIntN
70035B38A3E224BB20FD5CD51E8C3E300
file: TestPRIntN.sym
TestDefineEnum
E544DC6A94BB15B319E6B4FE64D10D010
file: TestDefineEnum.sym
rapl
83FE5A0D77C1F03C9ED3BD909DA675B30
file: rapl.sym
firefox
96806C35ED072D4F99C4FE082AE8E79C0
file: firefox.sym
TestAtomics
0BB44DDF8D4C7D6F71E67275E9E69F4E0
file: TestAtomics.sym
TestMacroForEach
98B48C0DBB08971BAD20DFF32A96106A0
file: TestMacroForEach.sym
WriteArgument
B0E0233EF4B856D4FC21F3AA6078CC150
file: WriteArgument.sym
TestSaturate
AC76231F5EBDDF2E1961DA622CF52ADE0
file: TestSaturate.sym
libfake.so
7683B9E3AD2E75CFC7BFF5865748FDBB0
file: libfake.so.sym
TestFastBernoulliTrial
F460D019BF3657EB58DCF81ECF5550340
file: TestFastBernoulliTrial.sym
TestFunctionRef
8ECA5B9EC7F89A2707DCC56075FDA4DA0
file: TestFunctionRef.sym
TestRandomNum
8199C171D80F46A3C0EA8170B3508F460
file: TestRandomNum.sym
TestMacroArgs
92A33DF728E861619B79AC3C388B5A3C0
file: TestMacroArgs.sym
libsmime3.so
D73F41D4E47B9C408AABAF6C515C2AAF0
file: libsmime3.so.sym
TestAtomicBitfields
8D6DD3503FCC2F33FCBF013B004DE0D40
file: TestAtomicBitfields.sym
TestBitSet
B8DFFF68A7F63692DC132A43712804B50
file: TestBitSet.sym
DelegatedCredentialsServer
A58F8BE8C51AE60C5A685FC596CB28380
file: DelegatedCredentialsServer.sym
TestVariant
D404A95871A86FE94F5B1DBEE878CDD80
file: TestVariant.sym
TestIntegerPrintfMacros
5C67A753330206A57A0DD775D78F48FE0
file: TestIntegerPrintfMacros.sym
TestPrintf
6B02F863340BEB13B22E966D6808CFC00
file: TestPrintf.sym
libmozavcodec.so
4AF8BB2D2CDEFFF40694C08F38938D4C0
file: libmozavcodec.so.sym
libmodules-test.so
34C034D9F92B72E50BA2D2A22E483B930
file: libmodules-test.so.sym
ssltunnel
925535F13203BA8C9CA0D18CEB0EDF480
file: ssltunnel.sym
TestCheckedInt
A42DAD5C0333FB3CC3BE26241D65063C0
file: TestCheckedInt.sym
TestFunctionTypeTraits
6A593CF3BF2707021F2C7CE3C65D1B070
file: TestFunctionTypeTraits.sym
TestBlockingProcess
1E86C4B2788AE3A9B242A530946BC6550
file: TestBlockingProcess.sym
TestWeakPtr
B44BF4BA4C6B10079ECECC69FF934CDC0
file: TestWeakPtr.sym
signmar
3C02F652A31467D3CDC76FA521FC7BE70
file: signmar.sym
SanctionsTestServer
EEC3F7A12C5DD3E04CE350DB87EA82C90
file: SanctionsTestServer.sym
TestXorShift128PlusRNG
34C60F9CEC8923258613F48BE2CB50F50
file: TestXorShift128PlusRNG.sym
jsapi-tests
9DBEDF04C418C5C3DB5650A67770A2690
file: jsapi-tests.sym
TestTemplateLib
5452670B0BB0F42CE3412466AF90FFE60
file: TestTemplateLib.sym
TestWrappingOperations
C088C86469FEAC7173652D3AA02650AD0
file: TestWrappingOperations.sym
libipcclientcerts.so
305BE6643B42706C618487120A2C27EF0
file: libipcclientcerts.so.sym
libnssckbi.so
E4E3247D280A25F9D98F7624569755C00
file: libnssckbi.so.sym
EncryptedClientHelloServer
52165148E3FEBBCD53070EFA4B9E30080
file: EncryptedClientHelloServer.sym
GenerateOCSPResponse
0DE302C5BB55A9CE07DDEEE9C790AD450
file: GenerateOCSPResponse.sym
libmozsqlite3.so
26BE0D3EBAE3F8415C66B4716D60BED70
file: libmozsqlite3.so.sym
TestPoisonArea
9371DF2A554DC559337F487247EB85E70
file: TestPoisonArea.sym
libplds4.so
6AD6E1C2B789CCE58E4535FB929B66E00
file: libplds4.so.sym
TestCompactPair
8353343C5618135ECD02A6887540C7000
file: TestCompactPair.sym
pk12util
B5B304E79D5031B31D4669CD7D888B0A0
file: pk12util.sym
TestIntegerRange
DEF6C1522AFF7C5EA445E0C66C9267780
file: TestIntegerRange.sym
shlibsign
0D2994FA590D5FB1694A71C10030BBED0
file: shlibsign.sym
libssl3.so
69DC43594FE8FD233EAED2C86F577E6A0
file: libssl3.so.sym
TestSplayTree
B4AB95724F15218416676A0C181564AA0
file: TestSplayTree.sym
libjsctypes-test.so
4A89DA4A07F75DE31F7BC86A6C0C2E7D0
file: libjsctypes-test.so.sym
ShowSSEConfig
B44C56FBC9348AA18177B4CEC201009F0
file: ShowSSEConfig.sym
crashreporter
BCDE01251CA268D6EA80BF49B04F9BDA0
file: crashreporter.sym
libmozwayland.so
E3255EAE5C0EC22D9549F13B7E19A1D90
file: libmozwayland.so.sym
TestBaseProfiler
C621581E697367224C3F97671923E7220
file: TestBaseProfiler.sym
libfakeopenh264.so
F83A84D02F43B7607F454EBBD862D07C0
file: libfakeopenh264.so.sym
TestFloatingPoint
1BE474C4E95D1E47F04C7CE1997BACCF0
file: TestFloatingPoint.sym
TestSegmentedVector
729166B5540DF3FF8514F6E55B6306C10
file: TestSegmentedVector.sym
libfreebl3.so
A14754F49A510C7E3D7B66A715B3D5E20
file: libfreebl3.so.sym
TestBloomFilter
695186CEE3D063536060678D93C975260
file: TestBloomFilter.sym
SmokeDMD
31A4498F637DDAEA701CEE3BE5856BB10
file: SmokeDMD.sym
TestCountZeroes
AC7300A33E1A217B814BD6B575FC844F0
file: TestCountZeroes.sym
TestTypedEnum
079459C35291F301D8DA2FE7D9DFD1700
file: TestTypedEnum.sym
TestScopeExit
68620A47605029765A81EEDB353A82940
file: TestScopeExit.sym
TestCeilingFloor
01EB17D3F957CAB16F805196E59959EF0
file: TestCeilingFloor.sym
TestUniquePtr
8A8B163E88263F122B08A23E4DB2974D0
file: TestUniquePtr.sym
TestLinkedList
E1D0C241CBB0447A89D68135D5F390900
file: TestLinkedList.sym
modutil
7EF3358ED0E308CDCB668CD3A5DAD5FC0
file: modutil.sym
Assignee | ||
Comment 10•2 years ago
|
||
My sneaking suspicion is that the debugids for the modules are somehow wrong (other possibility: corrupt symbol files) (other other possibility: symbol paths in rust-minidump are buggier than I expect).
Assignee | ||
Comment 11•2 years ago
|
||
Actually, libxul.so is a bit curious here:
libxul.so
61CD6124B55230FDE7165367012750660
file: libxul.so.sym
F86007E5B8EB0DD191CCD964267311440
file: libxul.so.sym
Why are there two different versions of libxul in the local symbols?
Assignee | ||
Comment 12•2 years ago
|
||
Oh nice, the minidump gets uploaded. I can confirm it is looking for 61CD6124B55230FDE7165367012750660, so it seems to be working perfectly in my test case?
I noticed in the original log where willkg was running into trouble, there is no "Downloading symbols from: ..." line. This suggests to me that the configuration he was hitting is the evil one I discuss here.
Perhaps marionnette tests are failing to pass in a value for symbols_path when they invoke mozcrash.py?
Comment 13•2 years ago
|
||
(In reply to Aria Beingessner [:Gankra] from comment #11)
Why are there two different versions of libxul in the local symbols?
There always are two. One is the gtest libxul.
Assignee | ||
Comment 14•2 years ago
|
||
Yep, if you look at my crash:
https://treeherder.mozilla.org/logviewer?job_id=364047953&repo=try&lineNumber=2391
vs willkg's crash:
https://treeherder.mozilla.org/logviewer?job_id=361688385&repo=try&lineNumber=1371
and search for --symbols-path
, then you see mine comes up with lots of results, but willkg's has none.
So marionette tests aren't getting any local symbols fed in at all, which means mozcrash.py has none to give to minidump-stackwalk.
So now the question is "why" and "how do we fix that"
Comment 15•2 years ago
|
||
I expect --symbols-path to originate in mozharness, like
https://searchfox.org/mozilla-central/rev/080e18fa4748456003164f58b0d925b8c3826a67/testing/mozharness/configs/unittests/linux_unittest.py#66
But I see no such provision in telemetry_client.py.
Comment 16•2 years ago
|
||
...actually, the option is available:
https://searchfox.org/mozilla-central/rev/080e18fa4748456003164f58b0d925b8c3826a67/testing/mozharness/scripts/telemetry/telemetry_client.py#69
...but is it used?
Assignee | ||
Comment 17•2 years ago
|
||
This is too many levels of inheritance and dependency injection for me to make sense of @-@
gbrown, are you experienced enough with these scripts to take it from here? (If not, is anyone?)
Assignee | ||
Comment 18•2 years ago
•
|
||
In case useful, desktop_unittest.py notes that symbols_path is supposed to be setup by the TestMixin._download_and_extract_symbols()
and indeed we see some logic to this effect:
telemetry_client.py does also uses that mixin, but perhaps it's messing up the state the superclass needs..?
Assignee | ||
Comment 20•2 years ago
|
||
Previously mozcrash would just create an empty temp dir and pass that in as a symbols_path.
The comment suggests this would be used as a --symbols-cache/--symbols-tmp, but that's
certainly not how the new minidump-stackwalk handles positional symbols-path arguments?
minidump-stackwalk is perfectly capable of automatically creating its own tempdirs,
so nothing actually needs to be done. Instead I emit a warning and remove the guards
against the value being missing.
Assignee | ||
Comment 21•2 years ago
|
||
Attached patch: doesn't actually fix the problem, but should help make it clear when it happens (and make it easier to tell if you fix it).
Comment 22•2 years ago
|
||
Pass the symbols_path from mozharness to the telemetry test harness,
to enable improved crash reports.
Comment 23•2 years ago
|
||
Pushed by gbrown@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/17fcc18648c8 Set symbols_path in telemetry tests; r=releng-reviewers,jmaher
Comment 24•2 years ago
|
||
Backed out changeset 17fcc18648c8 (Bug 1746747) for causing telemetry test failures.
Backout link
Push with failures
Failure Log
Comment 25•2 years ago
|
||
It looks like symbols_path is not always set in the telemetry mozharness script. Now why would that be...
Comment 26•2 years ago
|
||
No symbols for asan builds seems to be a thing: https://searchfox.org/mozilla-central/rev/3de56eb5f266f523340e739ae1b53258e0a95dfe/taskcluster/gecko_taskgraph/transforms/test/other.py#240-241
So I'll just guard against the None reference.
Comment 27•2 years ago
|
||
Backout by mlaza@mozilla.com: https://hg.mozilla.org/mozilla-central/rev/25ebce40c654 Backed out changeset 17fcc18648c8 for causing telemetry test failures. CLOSED TREE IGNORE BAD COMMIT MESSAGES
Comment 29•2 years ago
|
||
Pushed by gbrown@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/6635ff053064 Set symbols_path in telemetry tests; r=releng-reviewers,jmaher
Comment 30•2 years ago
|
||
bugherder |
Comment 31•2 years ago
|
||
Pushed by abeingessner@mozilla.com: https://hg.mozilla.org/integration/autoland/rev/38aba501f840 Provide accurate feedback about the about the absence of symbols_path. r=KrisWright
Comment 32•2 years ago
|
||
bugherder |
Assignee | ||
Comment 33•2 years ago
|
||
marking as fixed, but willkg is hoping to do some more testing
Reporter | ||
Comment 34•2 years ago
|
||
I verified that this works now.
Here's a build where I removed the flag to get symbols from symbols.mozilla.org and forced the telemetry-tests-client to crash:
I see symbolicated stacks for Linux, macOS, and Windows.
Thank you!
Description
•