mozilla :: #taskcluster

9 Aug 2017
02:58nthomasis git installed on gecko-1-b-win2012 workers ?
03:08nthomasi/o on i-0e918473dc36892c8 seems very slow, ~1 hr to get a gecko working dir
10:10xidornwhen a field is keyed by another value, what would happen if there are multiple regular expressions match a given value?
10:10xidornthe document in taskcluster/docs/transforms.rst seems to only mention that they would be used as regular expression, but doesn't mention if there are multiple matches, which one would win
10:11xidornwould the first match win, or the last match win?
12:30xidornanyone has answer for ^
13:35hassandustin: jonasfj https://queue.taskcluster.net/v1/provisioners/aws-provisioner-v1/worker-types :)
14:40dustinxidorn: exact matches are preferred, after which it is an error if multiple regexes match
14:41dustinwe don't try to reverse-engineer the regexes to detect collisions in advance, but if a value appears with >1 matches, an exception is raised
14:44dustinjhford: thanks for the merge! Are you looking at that in staging or should I?
14:47dustinjhford: what's the papertrail system for the staging provisioner? "aws-provisioner-sta" doesn't find it..
14:52dustin"provisioner" seems to crash papertrail
15:13jhforddustin: that's because all workers have the provisioner id in their name
15:13dustinyeah, that's what I guessed
15:13jhfordi haven't bothered to get it hooked up to papertrail I think... I just use heroku logs -t --app provisioner-staging --ps web
15:14dustinah
15:14jhfordit's really there only for targetted testing, so I didn't see the value in doing PT
15:14jhfordthe unit tests all pass, and that was the most likely thing to catch a bug in that code
15:15jhfordso I'm not too concerned but still want to double check it. I was planning on doing it tomorrow
15:15dustinoh, cool - I thought those only did API stuff
15:15dustinok
15:15jhfordthey do
15:15jhfordbut this isn't something which should have much affect on the backend that doesn't have tests
15:15dustinI suspect as long as this doesn't crash provisioning for workerTypes without AZ info, we're good to go to prod, and I can try it out there
15:16dustinisn't the launchSpec generation in the backend?
15:16jhfordit is, but it's generated in the API to test that it's a valid WT definition
15:16jhfordso all that checking code is unit tested
15:16dustinah, ok
15:17jhfordso, all the codepaths which are likely to be a problem are tested through API testing, and the rest shouldn't care because once we've got a launchspec, the backend doesn't care any longer
15:17dustinyeah
15:17dustinsounds good
15:17* jhford gets ready to bike homne
15:17dustinhttps://bugzilla.mozilla.org/show_bug.cgi?id=1388481 will be a bit more complicated - thoughts on that?
15:17firebotBug 1388481 NEW, nobody@mozilla.org Require security group IDs instead of names in worker type definitions
15:27aobreja|builddutywcosta:Hi, so I added you on CC for Bug 1376807 - Autologin is failing on the mac machines , tomorrow between 10-11 CEST ( 17 hours from now) I will do the changes and we must check all test OS X machine to be sure that are taking jobs and we don't have pendings, is this time fine for you?
15:27firebothttps://bugzil.la/1376807 NEW, jwatkins@mozilla.com Autologin is failing on the mac machines
15:28wcostaaobreja|buildduty: yep :)
15:29wcostagarndt: any trick in your gloves to quickly check machines are taking jobs?
15:29aobreja|builddutywcosta:great^^ ,I will update the bug and if anything happen in the meantime I will let you know on iirc or on bug
15:30aobreja|builddutythank you :-)
15:30garndtyup
15:31garndtwcosta: aobreja|buildduty so this is happening tomorrow, not now, right?
15:31aobreja|builddutyyep
15:31garndtok, I'll get the script over to wcosta and can make sure he can run it. Let me run it now to get a baseline of things that might not be working
15:32* aobreja|buildduty is also curious of garndt trick
16:18garndtaobreja|afk: wcosta I'll be sending you both an email about this shortly
16:24gerard-majaxgarndt, dustin, hell
16:24gerard-majaxo
16:25gerard-majaxgarndt, dustin, can you help me? I need all those tasks to be cancelled
16:25garndtheh, I thought for a second you were going to yell at us
16:25garndtall of what tasks?
16:25dustinme too
16:25gerard-majaxyes, all of them :)
16:25gerard-majaxthanks!
16:25garndtheh
16:33gerard-majaxgarndt, they are still running, is there a bug?
16:33garndtgerard-majax: I am not sure what tasks you are speaking of, which is why I asked about what tasks
16:34garndtI'm not sure what the problem is
16:34dustingerard-majax: cancelling all tasks would probably make gecko developers unhappy
16:34dustinalso there are probably >>10,000 of them so it would take a while
16:34gerard-majaxwaaat
16:34gerard-majaxyou don't read in my keyboard?
16:34gerard-majaxhttps://tools.taskcluster.net/groups/GgFFps7WSf228AUk3TMQHg
16:35gerard-majaxlooks like the link failed to get sent :)
16:35garndthehe
16:35gerard-majaxmy bad
16:35garndt you want them canceled? they seem to be running
16:35garndtand one of them just finished
16:35gerard-majaxyep
16:35gerard-majaxbut I know the code is wrong
16:36gerard-majaxit's going to fail in 2hrs+
16:36garndtok
16:36gerard-majaxI see exceptions, thanks!
16:36garndtdone
16:36garndtno problem!
16:36garndtso the workers will know in < 20 minutes that they were canceled and will stop running them
16:36gerard-majaxperfect
16:36gerard-majaxthey should be finished (and hopefully green) by the time I reach home then :)
16:37gerard-majaxthat should be my last round of Tensorflow changes to have OSX builds :)
16:37garndtsweet
16:37gerard-majaxI got DeepSpeech builds working as well
16:37gerard-majaxsee: https://tools.taskcluster.net/groups/EVc_GWqAQ-OEEgyQRDfHcg
16:37gerard-majaxall thanks to you
16:38gerard-majaxincluding pmoore|PTO-back-aug21 that I pinged a lot :)
16:38garndtspeaking of tensor, I came across this yesterday that I want to check out https://www.blog.google/topics/google-cloud/google-cloud-offer-tpus-machine-learning/
16:38gerard-majaxcheck out?
16:39garndtwell, read the blog post first
16:39garndtand learn more about TPUs
16:40gerard-majaxwell I know about TPUs
16:40gerard-majaxbut to my knowledge you cannot buy them
16:40garndtwell, you can use them in google cloud :)
16:41gerard-majaxI&#39;ll loose network soon, tunnels :/
16:41garndtoh, I guess it&#39;s still in the alpha phase right now for google cloud
16:41gerard-majaxwe already have hardware (TITAN X GPUs) in the Berlin office
16:41garndtgerard-majax: if you have any deep learning book/blog suggestions, send them my way :)
16:41gerard-majaxbrb
16:50gerard-majaxgarndt, so, we would be able to schedule tasks on Google Cloud ?
16:51garndtnot now
16:52garndtbut who knows what the future holds
16:52garndtwell...
16:52gerard-majaxhehe
16:52garndtyou can always stand up workers whereever you want, as you&#39;ve seen with the machines you ahve. So if you had workers running in google cloud, you can schedule tasks for them, no problem
16:52gerard-majaxregarding book/blogs suggestions, I dont really have good ones
16:52garndtnow if you want to dynamically provision them, that&#39;s a different story, but still not difficult
16:53gerard-majaxgarndt, if you are interested, there&#39;s a machine learning journal club being held each wednesday at 18:00 CEST
16:53garndthrm, is there information about that somewhere?
16:54gerard-majaxthat&#39;s a good question
16:54gerard-majaxwhat kind of informations would you be looking at?
16:54gerard-majaxs/at/for
16:56garndtwell, I mean, what is this club, who hosts it, how do I participate in it
16:57gerard-majaxoh
16:57gerard-majaxthat&#39;s just a bunch of people sitting around on vidyo and discussing machine learning
16:58gerard-majaxround-robin selection of one people to present a paper related to machine learning
16:58gerard-majaxit&#39;s in the deepspeech vidyo room :)
17:03garndtoh cool!
17:03garndtI know almost 0 about machine learning, but it&#39;s on my goals to start learning
17:04gerard-majaxwell, it might be a hard step
17:05gerard-majaxgarndt, I still learn a lot
17:05gerard-majaxgarndt, but I spent most of my work in the past months on non-ML stuff :)
17:05garndtthen we can build taskcluster&#39;s neural net
17:05gerard-majaxthen we can take over the world
17:05gerard-majaxsounds like a plan.
17:28dustinwell, use a machine for that and you&#39;ll be done
18:27akitwo tabs with taskgraphs of 3000+tasks + treeherder seem to hang nightly
18:31akii wonder if we can lazy load the tasks in a task group, kind of like how gmaps doesn&#39;t load the entire world&#39;s tiles, just the ones in the area you&#39;re looking at
18:32* aki waves hands vaguely
18:33dustinsounds like a good PR for you to make :)
18:33dustinI think it only loads the taskgraph, not fetching each individual task
18:34akii do have the use case where i&#39;m following a single task, and it&#39;s keeping track of the taskgraph
18:40akii filed https://github.com/taskcluster/taskcluster-tools/issues/271 at least
18:42dustinyeah, I think wanting to look at a single task is the common case
18:51hassanted: your wish has been granted. the task creator gives you a feedback now. thanks for pointing that out, it was a regression actually
18:52dustingood fix :)
18:56tedhassan: thanks! i thought i remembered carping about that in the past and someone (jonas?) fixing it
19:10gerard-majaxgarndt, dustin, probably not expected, ENOSPC while uploading https://tools.taskcluster.net/groups/fTu9PY09TgWZC4vVDfn_4w/tasks/fTu9PY09TgWZC4vVDfn_4w/runs/0/logs/public%2Flogs%2Flive_backing.log#L36125
19:11dustinthat&#39;s weird
19:11dustinwhy is writing an artifact using disk space
19:12gerard-majaxuploading, actually
19:19dustinyeah, that&#39;s what I mean
19:43dustinsomeone remind me where I find a version of zstd new enough to read image files?
19:43dustinI feel like we should put that in the error message from ./mach taskcluster-load-image (which hangs..)
19:46tomprinceFedora 25 has it.
19:47dustinbut not Ubuntu :(
19:53dustinI used interactive mode instead :)
19:57jonasfjdustin: I always end up building zstd from source..
19:57dustinheh
19:57dustinI guess &quot;std&quot; stands for something other than &quot;standard&quot; then :)
20:22Aryxhi, can anybody check if machine i-0482629eca683e78d is doing fine? thanks. there are two failing builds on autoland with &quot;Cannot find the host C compiler&quot;
20:25dustinI can kill it
20:25dustinlink to a task?
20:26dustinah, found it
20:26dustinkilled it dead
20:31Aryxthank you
10 Aug 2017
No messages
   
Last message: 8 days and 55 minutes ago