Closed Bug 1080265 Opened 10 years ago Closed 6 years ago

[tracker] Buildbot->Taskcluster: feature parity and optimizations

Categories

(Taskcluster :: General, defect)

x86
macOS
defect
Not set
normal

Tracking

(Not tracked)

RESOLVED FIXED

People

(Reporter: jlal, Unassigned)

References

Details

(Keywords: meta)

      No description provided.
Depends on: 1077053
Depends on: 1085505
Depends on: 1085502
Depends on: 1085511
Depends on: 1085513
Depends on: 1085516
Adding flatfish builds here not mission critical but still important and another good show case for us.
Depends on: 910128
Depends on: 1085614
Depends on: 1085616
Depends on: 1085617
Depends on: 1085619
Depends on: 1085620
Depends on: 1085621
Depends on: 1085629
Depends on: 1085625
Depends on: 1086677
Depends on: 1086984
Depends on: 1087456
Depends on: 1088232
Summary: Gecko Try fully functional → Gecko + Taskcluster : Ready for sheriffs
Depends on: 1089934
Depends on: 1090404
Depends on: 1091850
Depends on: 1093486
Depends on: 1093830
Depends on: 1094431
Depends on: 1094728
Depends on: 1095480
Depends on: 1097971
Depends on: 1098035
Depends on: 1098406
Depends on: 1099217
Depends on: 1099235
Depends on: 1099238
Depends on: 1099239
Depends on: 1100730
Depends on: 1101178
Depends on: 1102010
Depends on: 1102226
Depends on: 1102238
Depends on: 1101998
Depends on: 1102247
Depends on: 1102314
Depends on: 1103031
Depends on: 1103136
Depends on: 1103149
Depends on: 1104007
Depends on: 1104012
Depends on: 1104013
Depends on: 1104015
Blocks: 1104504
No longer blocks: 1104504
Depends on: 1104504
Depends on: 1104842
Depends on: 1099259
Depends on: 1105182
Depends on: 1057882
Depends on: 1106203
Depends on: 1106226
Depends on: 1105439
Depends on: 1109365
Depends on: 1109890
Depends on: 1111983
No longer depends on: 1073295
Depends on: 1122774
No longer depends on: 1094431
Depends on: 1124340
Depends on: 1128414
Depends on: 1130106
Depends on: 1130696
No longer depends on: 1128414
Depends on: 1130758
Depends on: 1130759
Depends on: 1130763
Depends on: 1130764
Depends on: 1130772
Depends on: 1130773
Depends on: 1130778
Depends on: 1131154
Depends on: 1131157
Depends on: 1131160
Depends on: 1131165
Depends on: 1131166
Depends on: 1131206
Depends on: 1131450
Depends on: 1113281
No longer depends on: 1106203
Depends on: 1132562
Depends on: 1132565
Depends on: 1132573
Depends on: 1132574
Depends on: 1132583
Depends on: 1132676
Depends on: 1134056
No longer depends on: 1057882
No longer depends on: 1077053
No longer depends on: 1076687
Depends on: 1135849
Depends on: 1135928
Depends on: 1136028
Depends on: 1108154
Depends on: 1136738
Depends on: 1137820
Depends on: 1139048
Depends on: 1140468
Depends on: 1140587
Depends on: 1140647
Depends on: 1141250
Depends on: 1141448
Depends on: 1141876
Blocks: bb-to-tc
Depends on: 1143066
No longer blocks: bb-to-tc
Depends on: 1143723
Depends on: 1144506
Depends on: 1147864
Depends on: 1147867
Depends on: 1147958
Depends on: 1147963
Depends on: 1148847
Depends on: 1171026
Depends on: 1171809
Depends on: 1089303
Depends on: 1146188
Depends on: 1147528
Depends on: 1165759
Depends on: 1154669
Depends on: 1162675
Depends on: 1165760
Depends on: 1170701
No longer depends on: 1089303
No longer depends on: 1133120
No longer depends on: 1140647
Depends on: 1174263
I'm analyzing this bug to have a better understanding what are the current issues that sheriffs are facing with TC.
From looking at it, it seems that this bug tracks general transition issues + sheriff issues.

Sheriff issues (which we should consider as MAJOR):
###################################################
Bug 1147867 - Run failed with no usable log output to Treeherder

Bug 1147963 - tc-tools: Copying text from log viewer (Inspect Task) has very poor UX

Bug 1148847 - Remove pip errors from builder logs

Bug 1148941 - gecko, tc-vcs: Decision tasks should fail if tc-vcs fails to pull hg.m.o
* Patched becked out because missing redo module

Bug 1154669 - *CRITICAL* Taskcluster B2g Build fails with bustage
* Depends on Bug 1132574 - gecko: Use branch specific caches for builds

Bug 1165759 - *CRITICAL* Intermittent TC build failures due to failing to fetch
* Depends on Bug 1156326 - [garndt-q2-goal] Investigate/implement pushing/pulling docker images to s3
* Depends on Bug 1165760 - Jobs which fail fetching a Docker image need to improve output

Bug 1170701 - Taskcluster B2G-emulator test runs don't provide a crash stack for reftests

Bug 1171809 - docker-worker: Listen to pulse exchange to blow away named caches (proposal)

Releng issues:
##############
* Bug 1174263 - Update the clobberer tool to support clobbering Taskcluster-based builds

General migration issues (not sheriff specific):
################################################
* Bug 1130763 - gecko: Emulator ICS Mochitest 11 perma fail
* Bug 1130764 - gecko: Permafail emulator-ics mochitest 12
* Bug 1130773 - gecko: Permafail emulator-ics X(pcshell) fail due to what looks like b2g startup failure

Support issues:
###############
* bug 1108154 - Create try-extender
** The idea is to have a UI to help us extend which platforms are run on a try push
** This is similar to the trychooser UI, however, it would also allow you to select platforms and request them
** This can be easily be built with a self-serve for TC plus a UI
** This does not sound like a blocker but it will be more pressing as more platforms move
** In fact, we might be able to use the buildbot bridge to schedule buildbot jobs (bonus)
We can also look at this bug as tier-1 issues and/or requirements to fullfil the visibility needs:
https://wiki.mozilla.org/Sheriffing/Job_Visibility_Policy
Component: TaskCluster → General
Product: Testing → Taskcluster
selena: I would like to remove bug 1108154 from the list of dependencies since it is not related to keeping up to par with the buildbot setup (i.e. this is an enhancement that we don't have on the current buildbot setup).

I have my intern working on a tool like this but I don't believe it is a blocker to bringing TaskCluster up to par (bug 1185537).
Removed bug 1108154. ack through IRC.
No longer depends on: 1108154
Depends on: 1189830
Depends on: 1181742
No longer depends on: 1130763
No longer depends on: 1130764
No longer depends on: 1130773
No longer depends on: 1132574
No longer depends on: 1170701
Depends on: 1218537
Do we need this for TaskCluster? I don't think so but wanted to ask.

Jobs taken by individual hosts https://secure.pub.build.mozilla.org/builddata/reports/slave_health/slave.html?class=test&type=tst-emulator64-spot&name=tst-emulator64-spot-125
Assignee: nobody → sdeckelmann
Depends on: 1247090
Adding a bunch of bugs that need polishing from bug 1243024 since they don't specifically block Linux from becoming a tier1 platform but they're deficiencies of the TaskCluster set up or optimizations.
Summary: Gecko + Taskcluster : Ready for sheriffs → Buildbot->Taskcluster: feature parity and optimizations
blocking-b2g: 2.2r? → ---
Depends on: 1275774
In one of the email threads about Linux64 debug transition one of the developers raised the inability to "add new jobs" for L64 debug TC jobs.

martianwars is working on it in bug 1254325.

Adding it as a dependency.
Depends on: 1254325
Depends on: 1280570
Depends on: 1284911
No longer depends on: 1254325
Depends on: 1289823
Depends on: 1289824
I'd close this, but I see armen adding deps..
Assignee: sdeckelmann → armenzg
Depends on: 1300901
There's no need to have an assignee for this.
It's simple to track what features are left to reach parity.
Assignee: armenzg → nobody
Summary: Buildbot->Taskcluster: feature parity and optimizations → [tracker] Buildbot->Taskcluster: feature parity and optimizations
Ed, I sort of feel like this in your scope, then?  I suspect that the dependencies of this bug are a partial list of features that do not exist in TaskCluster and will, at some point, be required in order to port some functionality to TaskCluster.
Geoff, I'm not sure how using, or not using, MOZ_SIMPLE_NAMES is related to feature parity between Buildbot and TaskCluster..
Flags: needinfo?(gbrown)
No longer depends on: 1247090
(In reply to Dustin J. Mitchell [:dustin] from comment #13)
> Geoff, I'm not sure how using, or not using, MOZ_SIMPLE_NAMES is related to
> feature parity between Buildbot and TaskCluster..

My thinking was, buildbot tests can work with or without MOZ_SIMPLE_PACKAGE_NAME, whereas taskcluster tests can only work with MOZ_SIMPLE_PACKAGE_NAME. We're just discovering ways around that now. Still, practically, Linux and Android taskcluster build artifacts are currently generated with MOZ_SIMPLE_PACKAGE_NAME, so installers, etc are named differently (arguably in an inferior way) from those created by buildbot jobs.
Flags: needinfo?(gbrown)
No longer blocks: 1111495
No longer depends on: 992323
Depends on: 1379163
Depends on: 1379171
Depends on: 1388407
Status: NEW → RESOLVED
Closed: 6 years ago
Resolution: --- → FIXED
You need to log in before you can comment on or make changes to this bug.