mozilla :: #taskcluster

15 Mar 2017
00:01jonasfjbstack: yeah, just make jungle2 or come up with a better name :)
00:01bstackcool
00:01bstackjust wanted to make sure that makes sense
00:02jonasfjbstack: you can just add the "taskclusterauth" azure account to AZURE_ACCOUNTS
00:02jonasfjbstack: it's not the end of the world if auth hands out write credentials to someone who gets the right scope
00:03bstackhmm
00:03jonasfjbstack: 1) because that scope is likely '*', 2) because tc-auth HMAC signs everything in the table so write access isn't enough
00:03bstackI was thinking we would want to be careful with read-write and that account
00:03jonasfjhence, no code change necessary...
00:03bstackand just never-ever give it out
00:03bstackyeah, that would definitely work.
00:03jonasfjhmm,,
00:04bstackI'm just wary of making that something we can give out
00:04jonasfjyeah, I mean (1) consider (2), we seems to be a fair well covered
00:04bstackit seems like something that would show up in a security related postmortem in a couple years :p
00:04jonasfjyeah, but on thinking about it are we really worried considering 1 and 2
00:04bstackgiven 2... no
00:04bstackok, I'll just add it
00:04bstackwith the secondary key, right?
00:04jonasfjyeah
00:04jonasfjwait..
00:05jonasfjsee man
00:05jonasfjmana
00:06jonasfjThe primary shared access key for a storage account should be stored in the config for auth.taskcluster.net.
00:06jonasfjIf a service needs shared access key for an azure storage account, it should be issued the secondary shared access key.
00:06jonasfjbstack: ^
00:06bstackah. just had it opposite of mana
00:07jonasfjyeah, but one for each use-case
00:07jonasfjit's nice to be consistent, so that rotating won't be so painful...
00:07jonasfjI'm sure we're bound to leak the all at some point :)
00:07bstackI like it when you convince me to do less work!
00:07bstackthat would be an unfun leak
00:08bstackbut actually not that bad if we've encrypted things correctly
00:08bstackif we leaked heroku config, that would be quite bad
00:16jonasfjbstack: oh, I just saw azure-entities uses fast-azure-storage 0.3.7, and we're at 1.0.1, we're missing all the fixes from outreachy! yay, hopefully fewer bugs...
00:16bstackhah, oh
00:16bstackyeah, we should upgrade for sure
00:17jonasfjdone new version out... I hope, assuming travis passes...
00:20bstackhehe
07:35Tomcat|sheriffdutyjhford: good morning
07:42Tomcat|sheriffdutyjhford: pmoore|away ping
08:32pmooreTomcat|sheriffduty: morning!
08:32Tomcat|sheriffdutyhi pmoore
08:32Tomcat|sheriffdutythere was a it seems a taskcluster probelm this morning
08:33Tomcat|sheriffdutypmoore: like https://treeherder.mozilla.org/logviewer.html#?job_id=83918705&repo=autoland&lineNumber=1124
08:34Tomcat|sheriffdutywith checking out a pip thing
08:34Tomcat|sheriffdutybut this recovered
08:35pmooreah good
08:36pmooresome mysteries are best left not understood ;-)
08:38Tomcat|sheriffdutypmoore: for such things wouldn't i be useful to have our own mirror
08:38Tomcat|sheriffdutyjust saying in general
08:38Tomcat|sheriffdutywhen pypi has problems
08:41pmooreTomcat|sheriffduty: I thought we did mirror stuff, but maybe some things have slipped through the net.. This would be a good question for releng.
08:42Tomcat|sheriffdutyshall i file a bug like package x was fetched by pypi and should use our own mirror ?
08:50pmooreTomcat|sheriffduty: that's a great idea, perfect. I think in the past maybe we even blocked access to the canonical pypi to encourage this, but I might be misremembering.
08:51pmooredustin/catlee will know
08:53Tomcat|sheriffdutyok
10:55Tomcat|sheriffdutypmoore: btw filed bug 1347483
10:55Tomcat|sheriffdutynow
10:55firebothttps://bugzil.la/1347483 NEW, nobody@mozilla.org blessings package should be downloaded from internal pypi and not from external one
10:56pmooreTomcat|sheriffduty++
10:57Tomcat|sheriffdutypmoore: i found a old but where blessings was uploaded, so we maybe just need to upload the latest version
10:58pmooreTomcat|sheriffduty: that might be it. but also in general, it would be good if we could monitor what we are downloading externally, even if we don't block it, just to know when we are pulling in external content
10:59Tomcat|sheriffdutyyeah
10:59pmooreTomcat|sheriffduty: that also is a security concern, as it offers a way for unvetted content to make it into the browser
11:00pmoorei'm sure we've already tackled this problem before though, it might just be that we need to revisit, in case changes have occurred which allow external content to be brought in again
11:01pmooreor maybe this is stuff during tests rather than builds (i didn't delve into the logs/links)
11:01pmoorein any case, we'd want to be resilient to external services being unavailable, i guess
11:02Tomcat|sheriffdutyyeah
11:03Tomcat|sheriffdutyautoland looked bad for a moment this morning
11:03Tomcat|sheriffdutywhen this happened with the pypi site
11:03Tomcat|sheriffdutyand so we would be more independet from this kind of bustage
11:04Tomcat|sheriffdutypmoore: filed bug 1347488 as more general one
11:04firebothttps://bugzil.la/1347488 NEW, nobody@mozilla.org We should audit what we download from external sites
11:04Tomcat|sheriffdutyfor the general audit where we touch external downloads
11:09Standard8theres also https://bugzilla.mozilla.org/show_bug.cgi?id=1302773 I believe
11:09firebotBug 1302773 NEW, nobody@mozilla.org install tox in the lint image
13:10kmoirhow can you tell the task ids associated with a previously run task from .cron.yml? For instance, with https://tools.taskcluster.net/hooks/#project-releng/cron-task-mozilla-central I can only see the taskid + log of the last run but would like to look at the logs for several previous runs of the cron jobs
14:33mjfAnything special I should do when Im finished with a one-click loaner?
14:40dustinno, it will go away after you logout
14:40dustinkmoir: I don't think they're indexed, so probably no good way to find those
14:40mjfdustin: thank you!
14:40dustinthe decision tasks they triggered should be in treeherder tho
14:41kmoirdustin: yes, I didn't see the decision tasks in th, but will look again
14:41kmoirthanks
14:42kmoirI guess I could change the time in tree and trigger manually
15:00garndthrm, seems like having a history of previous runs of a hook wouldn't be bad. not sure if it's possible or ideal in the long run though.
15:09dustinI think that indexing them is the right approach there
15:09dustinwhich would of course require parameterization :)
15:11garndthrm, but indexing only works for successful tasks, what if the task that the hook ran was not successful?
15:21dustinwell, (a) I think we should fix that about indexing, by adding a task.indexPaths to the payload
15:21dustinand (b) if it's failing, I think the most recent would have failed, which is accessible as an attribute of the hook
15:22dustinI don't want to create an ever-growing body of history in the hooks table
15:22dustinas a computer scientist, "1" seems like the only important number between 0 and infinity
15:22* dustin preaching to the jensens in the audience here
15:25halfwiw, GitHub keeps records of all hook firing. feels like audit trail
15:25catleedid anyone notice that the nightly decision tasks failed?
15:26catleehttps://tools.taskcluster.net/task-inspector/#IDN7ru9GStOdDwMqAFnF8g/
15:26dustinhal: similar only in name :)
15:26dustincatlee: yeah, we fixed it
15:26catleedustin: great, thanks
15:26catleeis there a bug?
15:26catleeprobably the cause of https://bugzilla.mozilla.org/show_bug.cgi?id=1347566
15:26firebotBug 1347566 NEW, nobody@mozilla.org update.xml empty for fr nightly Linux 64, build not updating while there are newer builds on the ftp
15:26garndtcatlee: https://bugzilla.mozilla.org/show_bug.cgi?id=1347569
15:27firebotBug 1347569 NEW, dustin@mozilla.com Decision Task for Nightly Desktop + Android failed with KeyError: u'NS7jKig_R8-1F_7DWSTQ-Q'
15:27dustinoh, Callek didn't land it - I just did
15:27dustinTomcat|sheriffduty: once that's landed on central, should be good to retrigger
15:27catleeare we retriggering?
15:27catleenm
15:27dustin:)
15:29Callekdustin: sorry, I don't have a habit of force-landing reviews unless the reviewer is known-away or doesn't have commit privs ;-)
15:30dustinno worries I don't know why I assumed you did
15:30Callek(I suppose I could do that from now on, when I have no review nits... unless there is a known reason to hold off)
15:30Callek(something to think about)
17:27bastienHello guys, i created a PR for taskcluster-notify to add an optional email template
17:27bastienhttps://github.com/taskcluster/taskcluster-notify/pull/10
17:27bastienI cannot test it without AWS credentials, right ?
17:27bastienTravis CI also complains about the lack of credentials
17:36bstackbastien: yeah, I want to make it so it can be tested without creds at some point. I just ran the tests locally and it just has a little bug and then we can ship this
17:36bstackhttps://github.com/taskcluster/taskcluster-notify/pull/10#pullrequestreview-27145532
17:36bstackit looks really great, ty :)
17:37bastiengood catch !
17:37bastieni could not test my code, so i expect the css part to work
17:38bastieni edited manually a taskcluster email to see the changes i needed, and transferred them to the css file... but i don't know it will look good :/
17:40bastieni pushed a fix
17:40bstackyeah. the nature of how the email bits work make it a bit hard to test
17:40bstackif you want to do more work on it at some point, I'll get you all of the creds you need
17:41bstackand it is easy to push a new version if you want to make tweaks to your css/etc no worries there
17:41bastienok, thanks for the clarification
17:46bastienand thanks for the merge, that was fast !
17:51bstacktc services are nice like that :)
17:51bstackbastien: ok, it should be deployed now. give it a shot and let me know if it works
18:05bastienok bstack i'll keep you posted, i just did the PR on mozilla-releng/services, but as Rok is AFK for a few days, i don't really know when it will reach the hook in production
18:05bstackah, ok
18:05bstackno rush from my end
20:04wcostarwood: ping
20:10rwoodwcosta: hello!
20:11wcostarwood: can you give a direction how to run the narrowed tests in local machine?
20:14rwoodwcosta: if you first make the changes in my patch: https://hg.mozilla.org/try/rev/befe8a08b8e8e0bfa3597336512b6df2e10279c21
20:14wcostarwood: "revision not found"
20:14rwoodwcosta: one sec
20:15rwoodwcosta: https://hg.mozilla.org/try/rev/befe8a08b8e8e0bfa3597336512b6df2e10279c2
20:15rwoodthat's better
20:15rwoodthen to run locally (one sec)...
20:17jmaherwcosta: do you know if the osx native tc worker will support custom tasks? (this is how the new feature for retrigger a job with different options works)
20:17rwood./mach talos-test --suite g2
20:17wcostajmaher: what do you mean by custom tasks?
20:18wcostarwood: but how I specify the correct firefo binary?
20:18rwoodwcosta: the mach command is for when running in your mozilla-inbound or central with your local build done
20:19wcostarwood: ah, did you test the tc build locally?
20:19jmaherwcosta: not sure, wlach can you expand on what a custom task is?
20:19rwoodwcosta: yes I pulled it down but couldn't repro the regresion locally
20:20wlachwcosta: I mean executing a task with an arbitrary payload (i.e. set of shell commands, etc)
20:21wcostahrm, well, it can execute but not with more than one command (this has been debated a lot internally)
20:22wcostarwood: ok, thanks, but this helps, I guess I can do with mozharness
20:22wlachwcosta: what do you mean by not more than one command?
20:23wcostawlach: yes, there is no agreement in the team to make payload having more than one command. What users have been doing is concatenate the command with &&
20:24wcostawlach: but tc native engine support a context field, which can be a link to a shell script to download
20:24wlachwcosta: ok, but you could have the command execute a shell script which can in turn do anything, correct? much like we do with test-ubuntu.sh already
20:24wlachwcosta: ok right
20:24wcostawlach: yep
20:24rwoodwcosta: you can run the talos damp test on a browser *other* than from your local build but that is a bit more tricky, you need to run a local webserver, install the talos damp addon, then visit the test-url your webserver makes availble. I tried that but couldn't repro the regression
20:25wlachjmaher: ok I think we should be able to make it work w/o too much trouble :)
20:25jmaherwlach: nice!
20:25wlachwcosta: sounds awesome
20:25jmaherthanks for confirming wlach wcosta :)
20:25wlachI guess we'll be using the native worker to run talos as well?
20:25wcostawlach: yep
20:25wlachthat opens the door to all sorts of fun and exciting custom talos retriggering
20:26jmaher:)
23:32whimboojhford: hi. are you still around
23:32whimboo?
16 Mar 2017
No messages
   
Last message: 127 days and 9 hours ago