Closed Bug 1138955 Opened 9 years ago Closed 9 years ago

Slow Builds and lagginess

Categories

(Infrastructure & Operations Graveyard :: CIDuty, task)

x86
All
task
Not set
normal

Tracking

(Not tracked)

RESOLVED WORKSFORME

People

(Reporter: cbook, Assigned: coop)

References

Details

06:46 < RyanVM> FWIW, we've been seeing  lagginess across all trees since yesterday's reconfig AFAIK
06:46 < RyanVM> i know philor mentioned it in this channel last night

06:54 < gerard-majax> Looks like it's ~2.5-3 times slower
06:54 < gerard-majax> build time: 20 mins at https://treeherder.mozilla.org/#/jobs?repo=try&revision=20030fe6da0d&exclusion_profile=false
06:54 < gerard-majax> built time: 48 mins at https://treeherder.mozilla.org/#/jobs?repo=try&revision=5ec2c29dd289&exclusion_profile=false

seems mostly windows builds it seems
At first glance, it seems like legitimate load. We had an higher-than-normal number of builds submitted this morning, and that leads to a corresponding proliferation of tests.

I'm still digging.
Here's a snapshot of the current pending test queue for linux:

ubuntu64-vm (1213)
  P4     8  mozilla-b2g34_v2_1
  P4    21  mozilla-b2g37_v2_2
  P4    54  mozilla-inbound
  P5    94  cedar
  P5   328  ash
  P5   708  try

Try has the lowest priority, and it's currently waiting behind a bunch of tests triggered by nightlies. I also suspect that it's losing out alphabetically to ash. 

We don't usually have this volume of try submissions at this time of day, and being blocked behind tests for nightly builds is expected.
Assignee: nobody → coop
Trees reopened at  2015-03-04T01:47:48
Status: NEW → RESOLVED
Closed: 9 years ago
Resolution: --- → WORKSFORME
Product: Release Engineering → Infrastructure & Operations
Product: Infrastructure & Operations → Infrastructure & Operations Graveyard
You need to log in before you can comment on or make changes to this bug.