Armen [:armenzg] Assignee
	Description • 12 years ago

foopy[39-45]
panda-0[082-169]

FTR we're aiming to move away from clientproxy.py if possible and just start buildbot.
If any imaging fails then buildbot should be shutdown gracefully.

	Armen [:armenzg] Assignee
	Comment 1 • 12 years ago

Here's my current distribution:
foopy39.p1.releng.scl1.mozilla.com - panda-00[82-94]
foopy40.p1.releng.scl1.mozilla.com - panda-0[095-107]
foopy41.p1.releng.scl1.mozilla.com - panda-0[108-120]
foopy42.p1.releng.scl1.mozilla.com - panda-0[121-133]
foopy43.p1.releng.scl1.mozilla.com - panda-0[134-146]
foopy44.p1.releng.scl1.mozilla.com - panda-0[147-159]
foopy45.p1.releng.scl1.mozilla.com - panda-0[160-169]

I will try tomorrow to see if I can make use of this version of devices.json that does not keep track of the relayhost and relayid:
http://hg.mozilla.org/users/armenzg_mozilla.com/tools/rev/8f6df37b8282

	Armen [:armenzg] Assignee
	Comment 3 • 12 years ago

I have one of the new pandas connected:
http://dev-master01.build.scl1.mozilla.com:8042/buildslaves/panda-0082

# as root
mkdir /builds/panda-0082
chown cltbld /builds/panda-0082
# as cltbld
wget -O/builds/panda-0082/buildbot.tac http://slavealloc.build.mozilla.org/gettac/panda-0082
/tools/buildbot-0.8.4-pre-moz2/bin/buildslave start /builds/panda-0082

I wanted to use this twistd command but it was failing [1]:
/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 /tools/buildbot-0.8.4-pre-moz2/bin/twistd --no_save --rundir=/builds/panda-0082 --pidfile=/builds/panda-0082/twistd.pid --python=/builds/panda-0082/buildbot.tac

I don't know why it would want to put the twistd.log under /builds rather than /builds/panda-0082.

[1]
[cltbld@foopy39 builds]$ /tools/buildbot-0.8.4-pre-moz2/bin/python2.7 /tools/buildbot-0.8.4-pre-moz2/bin/twistd --no_save --rundir=/builds/panda-0082 --pidfile=/builds/panda-0082/twistd.pid --python=/builds/panda-0082/buildbot.tac
Traceback (most recent call last):
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/application/app.py", line 631, in run
    runApp(config)
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/scripts/twistd.py", line 23, in runApp
    _SomeApplicationRunner(config).run()
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/application/app.py", line 374, in run
    self.application = self.createOrGetApplication()
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/application/app.py", line 439, in createOrGetApplication
    application = getApplication(self.config, passphrase)
--- <exception caught here> ---
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/application/app.py", line 450, in getApplication
    application = service.loadApplication(filename, style, passphrase)
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/application/service.py", line 400, in loadApplication
    application = sob.loadValueFromFile(filename, 'application', passphrase)
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/persisted/sob.py", line 210, in loadValueFromFile
    exec fileObj in d, d
  File "/builds/panda-0082/buildbot.tac", line 27, in <module>
    maxRotatedFiles=maxRotatedFiles)
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/python/logfile.py", line 49, in fromFullPath
    os.path.dirname(logPath), *args, **kwargs)
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/python/logfile.py", line 162, in __init__
    BaseLogFile.__init__(self, name, directory, defaultMode)
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/python/logfile.py", line 41, in __init__
    self._openFile()
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/python/logfile.py", line 167, in _openFile
    BaseLogFile._openFile(self)
  File "/tools/buildbot-0.8.4-pre-moz2/lib/python2.7/site-packages/twisted/python/logfile.py", line 76, in _openFile
    self._file = file(self.path, "w+", 1)
exceptions.IOError: [Errno 13] Permission denied: '/builds/twistd.log'

Failed to load application: [Errno 13] Permission denied: '/builds/twistd.log'

	Justin Wood (:Callek)
	Comment 4 • 12 years ago

(In reply to Armen Zambrano G. [:armenzg] from comment #3)
> I wanted to use this twistd command but it was failing [1]:
> /tools/buildbot-0.8.4-pre-moz2/bin/python2.7
> /tools/buildbot-0.8.4-pre-moz2/bin/twistd --no_save
> --rundir=/builds/panda-0082 --pidfile=/builds/panda-0082/twistd.pid
> --python=/builds/panda-0082/buildbot.tac
> 
> I don't know why it would want to put the twistd.log under /builds rather
> than /builds/panda-0082.

My guess was basedir in the .tac being /builds but currently its not, have you tried any of the following:

* having `pwd` be /builds/panda-0082 when you launch the twistd command
* adding a / to the basedir/rundir param/options
* doing a more permissive chmod on /builds/panda-0082 (rather than just chown to cltbld)
* doing a chgrp on /builds/panda-0082

	Armen [:armenzg] Assignee
	Comment 5 • 12 years ago

From inside of /builds/panda-0082 seems to do the job (thanks!):
/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 /tools/buildbot-0.8.4-pre-moz2/bin/twistd --no_save --rundir=/builds/panda-0082 --pidfile=/builds/panda-0082/twistd.pid --python=/builds/panda-0082/buildbot.tac

	John Hopkins (:jhopkins)
	Comment 6 • 12 years ago

Comment on attachment 686305 [details] [diff] [review]
attach staging pandas to armenzg's master

armenzg says: 133 is my test master #2 and poolid=16 is "preprod-tests"

Looks good to me.

	Armen [:armenzg] Assignee
	Comment 7 • 12 years ago

This does what I need.
jhopkins suggested to get a way to check that the bmm is up before starting buildbot.

	Armen [:armenzg] Assignee
	Comment 8 • 12 years ago

After lunch I will automate the creation of the /builds/panda-####.

	Armen [:armenzg] Assignee
	Comment 9 • 12 years ago

It is not complete yet but it shows what way I'm heading with.

	Armen [:armenzg] Assignee
	Comment 10 • 12 years ago

I used a bash script because make was not available on the machine.
We can store the bmm value as an environment variable if prefered (in ~/.bash_profile or so).

These are the steps to reproduce and below the output to show what gets created.

# as root
runuser -l cltbld -c "hg pull -u /builds/tools"
cd /builds/tools/buildfarm/mobile/
python setup_foopy.py

## Output to show what gets created
#####################################
[root@foopy39 mobile]# diff /builds/panda-0082/buildbot.tac /builds/panda-0083/buildbot.tac
14c14
< basedir = '/builds/panda-0082'
---
> basedir = '/builds/panda-0083'
16c16
< slavename = 'panda-0082'
---
> slavename = 'panda-0083'
[root@foopy39 mobile]# find /builds/panda-00*
/builds/panda-0082
/builds/panda-0082/make.sh
/builds/panda-0082/buildbot.tac
/builds/panda-0083
/builds/panda-0083/make.sh
/builds/panda-0083/buildbot.tac
/builds/panda-0084
/builds/panda-0084/make.sh
/builds/panda-0084/buildbot.tac
/builds/panda-0085
/builds/panda-0085/make.sh
/builds/panda-0085/buildbot.tac
/builds/panda-0086
/builds/panda-0086/make.sh
/builds/panda-0086/buildbot.tac
/builds/panda-0087
/builds/panda-0087/make.sh
/builds/panda-0087/buildbot.tac
/builds/panda-0088
/builds/panda-0088/make.sh
/builds/panda-0088/buildbot.tac
/builds/panda-0089
/builds/panda-0089/make.sh
/builds/panda-0089/buildbot.tac
/builds/panda-0090
/builds/panda-0090/make.sh
/builds/panda-0090/buildbot.tac
/builds/panda-0091
/builds/panda-0091/make.sh
/builds/panda-0091/buildbot.tac
/builds/panda-0092
/builds/panda-0092/make.sh
/builds/panda-0092/buildbot.tac
/builds/panda-0093
/builds/panda-0093/make.sh
/builds/panda-0093/buildbot.tac
/builds/panda-0094
/builds/panda-0094/make.sh
/builds/panda-0094/buildbot.tac
[root@foopy39 mobile]# cat /builds/bmm.hostname.txt 
mobile-imaging-001.p1.releng.scl1.mozilla.com

	Armen [:armenzg] Assignee
	Comment 11 • 12 years ago

I modified the sql statement a bit and landed it.
For every line I added this:
", basedir='/builds/panda-00##' " (e.g. panda-0082)

I can now start up all the buildbot instances of foopy39 ( I will skip the panda-00[82-89] since jhopkins holds them).

	Armen [:armenzg] Assignee
	Comment 12 • 12 years ago

This file deals with proper ownership of the files and directories.

The following output shows that from a known state we can get to a working state.
[root@foopy39 mobile]# rm -rf /builds/panda-00*
[root@foopy39 mobile]# python setup_foopy.py

[cltbld@foopy39 builds]$ for i in {90..94}; do cd /builds/panda-00$i; ./make.sh start; done
2012-11-30 08:26:08-0800 [-] Log opened.
2012-11-30 08:26:08-0800 [-] twistd 10.2.0 (/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 2.7.2) starting up.
2012-11-30 08:26:08-0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2012-11-30 08:26:08-0800 [-] Log opened.
2012-11-30 08:26:08-0800 [-] twistd 10.2.0 (/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 2.7.2) starting up.
2012-11-30 08:26:08-0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2012-11-30 08:26:08-0800 [-] Log opened.
2012-11-30 08:26:08-0800 [-] twistd 10.2.0 (/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 2.7.2) starting up.
2012-11-30 08:26:08-0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2012-11-30 08:26:08-0800 [-] Log opened.
2012-11-30 08:26:08-0800 [-] twistd 10.2.0 (/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 2.7.2) starting up.
2012-11-30 08:26:08-0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
2012-11-30 08:26:08-0800 [-] Log opened.
2012-11-30 08:26:08-0800 [-] twistd 10.2.0 (/tools/buildbot-0.8.4-pre-moz2/bin/python2.7 2.7.2) starting up.
2012-11-30 08:26:08-0800 [-] reactor class: twisted.internet.selectreactor.SelectReactor.
[cltbld@foopy39 panda-0094]$ for i in {90..94}; do cd /builds/panda-00$i; ./make.sh stop; done
buildslave process 23286 is dead
buildslave process 23293 is dead
buildslave process 23302 is dead
buildslave process 23309 is dead
buildslave process 23318 is dead

	Armen [:armenzg] Assignee
	Comment 13 • 12 years ago

I have setup foopies 40, 43, 44 & 45 like this:

# remove previous existing directories owned by root
rm -rf /builds/panda-0*
runuser -l cltbld -c "wget -q https://bugzilla.mozilla.org/attachment.cgi?id=687113"
# otherwise the files will be owned by root
runuser -l cltbld -c "cd /builds/tools && patch -p1 < ~/attachment.cgi?id=687113"
cd /builds/tools/buildfarm/mobile/
# odd - the patch says "new file mode 100755"
chmod 755 make.sh
runuser -l cltbld -c "hg pull -u /builds/tools"
python setup_foopy.py
su cltbld
for i in `ls /builds | grep panda-`; do cd /builds/$i; ./make.sh start; done

	Armen [:armenzg] Assignee
	Comment 14 • 12 years ago

jhopkins is going to try to image and run the cat command from his repo manually.
We have to figure out how to execute a cat command remotely (we're hoping Callek can lead us in the right direction).

armenzg will be polishing his buildbot-configs patches and BuildSlaves templates.

	Aki Sasaki (not active)
	Comment 15 • 12 years ago

Comment on attachment 687113 [details] [diff] [review]
setup_foopy.py, updated devices.json and make.sh

I'm not sure if we want a different target in make.sh to somehow mark the buildslave/device as bad (stop-and-email or stop-and-something-else) or if mozharness should do that and just run make.sh stop.

Other than that I'm mostly concerned about, "does this work?" and by your testing it looks like it does.

	Armen [:armenzg] Assignee
	Comment 16 • 12 years ago

Locally I'm running it with this:
MOZHARNESS_REPO = "http://hg.mozilla.org/users/armenzg_mozilla.com/mozharness"

This cannot land without mozpool and b2g_panda.py changes landing on mozharness first.

BTW I have made --mozpool-device not mandatory.

Unfortunately I had to use "print response" rather than "self.log(response)" since it said something about splitlines.

You can check results in:
http://dev-master01.build.scl1.mozilla.com:8042/builders/b2g_b2g_panda%20cedar%20opt%20test%20gaia-mochitest

	Armen [:armenzg] Assignee
	Comment 17 • 12 years ago

I know that the naming does not follow the example of panda_android (arch_os) but whatever.

	Armen [:armenzg] Assignee
	Comment 19 • 12 years ago

Interestingly enough my scripts shows some weirdness about pings:
http://mobile-imaging-001.p1.releng.scl1.mozilla.com/ui/log.html?device=panda-0100

I don't know when the board is done rebooting and meanwhile there is inconsistency with pings.
http://mobile-imaging-001.p1.releng.scl1.mozilla.com/ui/log.html?device=panda-0100

FTR, if anyone wants to trigger more jobs while I'm away you can do the following:
* land a change to the b2gpanda branch on my user repo
* rebuild the previous job OR force build with "script_repo_revision":"b2gpanda"

	Armen [:armenzg] Assignee
	Comment 20 • 12 years ago

FTR my patches only add the test builders but there are no sendchanges to trigger them:
https://tbpl.mozilla.org/php/getParsedLog.php?id=17492455&tree=Firefox&full=1

I will be working on that side of it on Monday.

All the patches can land even though without the mozharness patches the manually triggered jobs would fail (until we land our changes on mozharness).

	Aki Sasaki (not active)
	Comment 21 • 12 years ago

Comment on attachment 687245 [details] [diff] [review]
BuildSlaves Templates in buildbot-configs

>         'mock': 'pass'
>+        'b2g_panda': 'pass'

You'll need a comma after the mock entry in mozilla/BuildSlaves.py.template.

Putting a comma after both lines may help avoid this in the future.

	Aki Sasaki (not active)
	Comment 22 • 12 years ago

(In reply to Armen Zambrano G. [:armenzg] from comment #16)
> Unfortunately I had to use "print response" rather than "self.log(response)"
> since it said something about splitlines.

Try

    self.info(str(response))

to both make the log level INFO, and ensure that response is in string format.

	Aki Sasaki (not active)
	Comment 23 • 12 years ago

Comment on attachment 687239 [details] [diff] [review]
buildbot-config changes to add b2g dummy job

+PLATFORMS['b2g_panda']['slave_platforms'] = ['b2g_panda']
+PLATFORMS['b2g_panda']['env_name'] = None 

Nit: Could you remove the trailing whitespace here?

+BRANCHES['cedar']['platforms']['b2g_panda']['b2g_panda']['opt_unittest_suites'] = [
+    ('gaia-mochitest', {'suite': 'gaia-mochitest',
+                        'mozharness_repo': MOZHARNESS_REPO, 

Also trailing whitespace here.

	Aki Sasaki (not active)
	Comment 24 • 12 years ago

Armen, I got this failure when running test-masters.sh:

   test_stag_not_in_prod ...                                            [FAIL]

===============================================================================
[FAIL]
Traceback (most recent call last):
  File "test/test_slave_allocation.py", line 33, in test_stag_not_in_prod
    'declared as staging-only:\n%s' % '\n'.join(sorted(common_slaves))
  File "/Users/asasaki/wrk/virtualenv/bb08/lib/python2.6/site-packages/Twisted-12.1.0-py2.6-macosx-10.6-x86_64.egg/twisted/trial/unittest.py", line 270, in assertEqual
    % (msg, pformat(first), pformat(second)))
twisted.trial.unittest.FailTest: Staging-only slaves should not be declared as production and vice versa. However, the following production slaves declared as staging-only:


I think we might be adding production slaves to staging automatically.
Just wanted to highlight this since preprod will start whining if you land as-is.

	Justin Wood (:Callek)
	Comment 25 • 12 years ago

Comment on attachment 687247 [details] [diff] [review]
BuildSlaves-tests.py.erb - puppet changes for the masters

Review of attachment 687247 [details] [diff] [review]:
-----------------------------------------------------------------

::: modules/buildmaster/templates/BuildSlaves-tests.py.erb
@@ +21,5 @@
>      'tegra_android-armv6': '<%=scope.lookupvar('secrets::tegra_android_tests_password')%>',
>      'tegra_android-noion': '<%=scope.lookupvar('secrets::tegra_android_tests_password')%>',
>      'fedora-b2g': '<%=scope.lookupvar('secrets::linux_tests_password')%>',
>      'panda_android': '<%=scope.lookupvar('secrets::panda_android_tests_password')%>',
> +    'b2g_panda': '<%=scope.lookupvar('secrets::panda_android_tests_password')%>',

please can we use a new secret here? I don't like the idea of reusing "panda_*android*...." for b2g pandas.

	Justin Wood (:Callek)
	Comment 26 • 12 years ago

(In reply to Armen Zambrano G. [:armenzg] from comment #10)
> I used a bash script because make was not available on the machine.

Not needed to worry about, but its simple to add make to Linux foopies if we want :-)

> We can store the bmm value as an environment variable if prefered (in
> ~/.bash_profile or so).

IMO its far harder to change (for the system) the bmm value if we store it in .bash_profile since foopies don't reboot often, but easier to discover in running jobs (since it will show in buildbot output) end-result I'm indifferent on that

(In reply to Armen Zambrano G. [:armenzg] from comment #14)
> jhopkins is going to try to image and run the cat command from his repo
> manually.
> We have to figure out how to execute a cat command remotely (we're hoping
> Callek can lead us in the right direction).

cat of the remote devices filesystem, or do a cat of a file on the foopy remotely? from wher and why? (I'm not sure what you're trying to achieve, so advising is hard)


(In reply to Aki Sasaki [:aki] from comment #15)
> Comment on attachment 687113 [details] [diff] [review]
>
> I'm not sure if we want a different target in make.sh to somehow mark the
> buildslave/device as bad (stop-and-email or stop-and-something-else) or if
> mozharness should do that and just run make.sh stop.

IMHO mozharness marking the device bad in whatever way is accurate is probably best, bmm itself in some way should probably be responsible for the notification sendout somehow (as in, *IT* can try to recover in the way[s] we teach it, if it fails then it notifies us, mozharness/foopies should just be responsible in telling bmm/lifeguard/whatever that the device can't swim) - The "lifeguard" can then call the "ambulance" (us/relops/dcops) if it needs help.

	Justin Wood (:Callek)
	Comment 27 • 12 years ago

Comment on attachment 687113 [details] [diff] [review]
setup_foopy.py, updated devices.json and make.sh

Review of attachment 687113 [details] [diff] [review]:
-----------------------------------------------------------------

All the below is nits/questions, and overall this looks great! I marked it r- since I need the questions answered to give it a proper final review (and want them before we land), so when you answer them please re-request or flag me on a new patch, I'll grab it right away.

::: buildfarm/mobile/make.sh
@@ +1,1 @@
> +#!/bin/bash

I'm not a huge fan of the naming of this file (I'd like something that makes sense at a glance, and makes us know what it is used for easily. |make.sh| feels so generic, and almost makes me feel I should type "make" to run it (even with the .sh extension). -- Sadly I have no better suggestions atm, so unless you can think of one, I won't block/bikeshed on it, but I just wanted to voice my sadness at it. ;-)

@@ +4,5 @@
> +DEVICE=`basename $PWD`
> +opt="$1"
> +
> +if [ "$opt" = "buildslave" ]; then
> +    if [ "$2" = "start" ]; then

Nit: rather than sprinkle $2 throughout this file, why not do a test if $2 exists (if not exit 1) and set a var like $buildslave_cmd for it, or something

@@ +16,5 @@
> +    rm buildbot.tac
> +    wget -q -O/builds/$DEVICE/buildbot.tac http://slavealloc.build.mozilla.org/gettac/$DEVICE
> +elif [ "$opt" = "start" ]; then
> +    /tools/buildbot-0.8.4-pre-moz2/bin/python2.7 \
> +    /tools/buildbot-0.8.4-pre-moz2/bin/twistd --no_save \

Nit I'd love these paths to be constants at the top of this .sh script, such as $BUILDBOT_PATH $BB_PYTHON $BB_TWISTD etc. Makes it easier if we ever do change the versions/details.

::: buildfarm/mobile/setup_foopy.py
@@ +1,1 @@
> +import urllib2

..is the idea that this file is to be used for all panda foopies, all b2g foopies, or *all* foopies [eventually, not necessarily at landing].  If *all* this is fine, if b2g only, perhaps rename to setup_b2g_foopy.py or something?  [I'm open to arguments why that is a bad idea]

@@ +9,5 @@
> +toolsdir = "/builds/tools"
> +path_to_bmm_host = "/builds/bmm.hostname.txt"
> +# cltbld's uid and gid
> +const_uid=500
> +const_gid=500

Do we really need numberic hardcoded uid/gid and can't use strings here or something? I don't like these types of buried hardcodes, waaay too easy to bite us later.

@@ +38,5 @@
> +match=re.search(".*.p([0-9]*)\..*", hostname)
> +vlan=int(match.group(1))
> +if not os.path.exists(path_to_bmm_host):
> +    local_file = open(path_to_bmm_host, 'wb')
> +    local_file.write("mobile-imaging-0%02i.p%i.releng.scl1.mozilla.com" % (vlan, vlan))

is this `mobile-imaging-0[VLAN].p[VLAN]` naming guaranteed to hold, or is there a chance we could have a .p23 with a mobile-imaging-010 for example.

	Armen [:armenzg] Assignee
	Comment 28 • 12 years ago

Where is the secrets file so I can add that extra key? You can PM me if preferred.

	Justin Wood (:Callek)
	Comment 29 • 12 years ago

Comment on attachment 687745 [details] [diff] [review]
BuildSlaves-tests.py.erb - puppet changes for the masters

Review of attachment 687745 [details] [diff] [review]:
-----------------------------------------------------------------

Its buried on old-puppet that we use for the buildbot masters. https://wiki.mozilla.org/ReleaseEngineering/Puppet/Usage#Masters if you need more help in finding this secrets file we can go through it on irc.

	Armen [:armenzg] Assignee
	Comment 30 • 12 years ago

I have addressed all of your comments.

I would like setup_foopy.py to be used as the way of setting up every foopy.
For now, we are only doing it with the foopies for b2g devices but I'm not stopping anyone for trying it for the other foopies.

For now, we only have 10 mobile imaging servers because we're using 10 racks (one per rack).
The naming as of now is mobile-imaging-0[1-10].p[1-10].releng.scl1.mozilla.com

	Justin Wood (:Callek)
	Comment 31 • 12 years ago

Comment on attachment 687775 [details] [diff] [review]
setup_foopy.py, updated devices.json and manage_buildslaves.sh

Review of attachment 687775 [details] [diff] [review]:
-----------------------------------------------------------------

r+ with 1 real issue, and 2 nits

::: buildfarm/mobile/manage_buildslave.sh
@@ +1,3 @@
> +#!/bin/bash
> +PWD=`pwd`
> +DEVICE=`basename $PWD`

nit (non-blocking): we should probably sanity-check $DEVICE with some form of "startswith panda- or tegra-" check. To prevent running this with a wrong cwd. Since we use $DEVICE pretty indiscriminately below.

@@ +3,5 @@
> +DEVICE=`basename $PWD`
> +opt="$1"
> +BB_PATH=/tools/buildbot-0.8.4-pre-moz2/bin/buildslave
> +BB_PYTHON=/tools/buildbot-0.8.4-pre-moz2/bin/python2.7
> +BB_TWISTD=/tools/buildbot-0.8.4-pre-moz2

that's not the twistd binary :-P

@@ +20,5 @@
> +    echo "We will run with the twistd command instead of calling buildslave"
> +    ${BB_PYTHON} ${BB_TWISTD} --no_save \
> +    --rundir=/builds/$DEVICE \
> +    --pidfile=/builds/$DEVICE/twistd.pid \
> +    --python=/builds/$DEVICE/buildbot.tac

bikeshed-nit, indent the carried-on lines slightly

	Armen [:armenzg] Assignee
	Comment 32 • 12 years ago

Comment on attachment 687775 [details] [diff] [review]
setup_foopy.py, updated devices.json and manage_buildslaves.sh

I fixed the 2 problems.

	Armen [:armenzg] Assignee
	Comment 33 • 12 years ago

Comment on attachment 687245 [details] [diff] [review]
BuildSlaves Templates in buildbot-configs

Checked in with fixes in comment 21.

	Armen [:armenzg] Assignee
	Comment 34 • 12 years ago

Comment on attachment 687745 [details] [diff] [review]
BuildSlaves-tests.py.erb - puppet changes for the masters

checked-in and modified secrets.pp.

Now looking at [root@buildbot-master19 ~]# tail -F /var/log/messages

	Armen [:armenzg] Assignee
	Comment 35 • 12 years ago

Comment on attachment 687239 [details] [diff] [review]
buildbot-config changes to add b2g dummy job

I can finally run test-masters locally and have fixed the issue.
This has been landed.

Tomorrow I will be working on the sendchanges from the panda builds.

	Armen [:armenzg] Assignee
	Comment 36 • 12 years ago

I will leave this bug open to setup the foopies with "manage_buildslave.sh" rather than "make.sh". I want all of it to be consistent. Not a high priority though.

	Rail Aliiev [:rail]
	Comment 37 • 12 years ago

In production

	Armen [:armenzg] Assignee
	Comment 38 • 12 years ago

I setup again the current foopies:
# ssh as cltbld
cd /builds/tools
hg pull -u
hg id
su -
rm -rf /builds/panda-*
cd /builds/tools/buildfarm/mobile
python setup_foopy.py
exit
for i in `ls | grep panda`; do /builds/manage_buildslave.sh start $i; done

	Armen [:armenzg] Assignee
	Comment 39 • 12 years ago

An improvement that we landed is this:
http://hg.mozilla.org/build/tools/rev/6bf3e47a84d9
which got reviewed and landed in bug 802317:
https://bug802317.bugzilla.mozilla.org/attachment.cgi?id=688923