mozilla :: #taskcluster

18 May 2017
08:54franziskuspmoore: ping
08:55pmoorefranziskus: hi
08:55franziskuspmoore: hey, I was wondering if you could tell us how to get 32 bit windows for NSS. we already have windows 2012 x64
08:57pmoorefranziskus: sure. for gecko, we build 32 bit on the 64 bit machines - another option is we set up a new worker type that is natively 32 bit
08:57franziskuspmoore: ah ok, so it's really only the other toolchain? that should be easy enough
08:59pmoorefranziskus: yeah, see
08:59pmoorethat is what we currently have on the workers
09:01franziskuscool, so it might just work. I'll give it a try
09:01pmoorefranziskus: good luck! also, if you want any changes, this is where the definition lives, so you can make changes there via a PR if you like
09:03pmoorefranziskus: the workers are a clean windows 2012 r2 base image from amazon, with this powershell script run against them - that is all there is to it
09:03pmoorefranziskus: if you like, you can even base off a windows 2016 base image if you fancy trying something new! i see there are base images in EC2 now
09:05franziskuspmoore: heh, well I actually want to get 32-bit builds with minimal effort :)
10:44whimboogrenade|rust-training, pmoore: do we turn of windows defender / security essentions on our TC workers?
11:21pmoorewhimboo: yes
11:23pmoorewhimboo: e.g.
11:24pmoorewhimboo: erm - well
11:25pmoorewhimboo: we disable firewall, i don't think we install windows defender / security essentions - or is that something built into windows?
11:27pmoorewhimboo: i saw this pic from nthomas in #releng, although i suspect this is in buildbot:
11:27pmooreah yes, from the machine name (y-2008-spot-001) we can see this is buildbot
11:28pmooremaybe same issue in TC?
11:28pmoorewhy do you ask? :)
11:28pmoorenot sure if "windows live essentials 2011" is related to "security essentions" ?
11:28pmoorewhat is "security essentions" ?
11:29jhfordif it's spelled that way, probably a virus?
11:29jhford(not trying to be a smartass)
11:29pmooreor maybe whimboo mistyped? :)
11:30jhfordi've seen a lot of viruses which have typos like that
11:30jhfordbut it's probably a typo
11:40whimboopmoore: live essentials should be something different
11:40whimbooi talk about the internal virus scanner from MS
11:40whimboothing is it takes a lot of cpu load while running the tests
11:40whimbooand as such causes test failures due to timeouts
11:41whimboojust want to know if we are in parity
12:01pmoorein parity with buildbot?
12:02pmooreare you seeing the virus scanner running on both win7 and win10?
12:02pmoorewhimboo: ^
12:03whimboopmoore: i said what we do. simply want to know what we do for TC workrrs
12:04pmooreah, in your jenkins stuff?
12:04pmoorei forgot about the mozci / mozmillci (forget which it is) stuff you have
12:05pmoorewhimboo: this is what we do on win10:
12:05pmoorethat is the complete list of steps
12:05pmooreand this is for win7:
12:05pmooreif it isn't in there, we don't do it :)
12:09whimboopmoore: i don't see anything regarding the securtiy essentials
12:09whimbooso keep in mind that this will slow down those workers
12:09whimboonot sure what we did for older windows versions on buildbot
12:09whimbooor still do
12:10pmoorewhimboo: can you raise a bug for this?
12:10pmooreand maybe CC me and grenade|rust-training
12:10pmoorethen we can look into it
12:10whimboonowadays its called windows defender
12:11whimboopmoore: which component_
12:12whimbooaws provisioner?
12:13pmooremaybe Infrastructure & Operations : RelOps
12:14pmooreit doesn't matter too much, can also be Release Engineering : Platform Support or even Taskcluster : Integration
12:15jhfordwhimboo: if it's something happening inside the instance, aws-provisioner isn'
12:15jhfordt the right place :)
12:16whimbooi will use integration
12:16jhfordsounds great :)
12:21whimboopmoore: bug 1365909
12:21firebot NEW, Investigate influence of Windows Defender and maybe turn it of for Windows workers
13:58dustinpmoore: what is going on in this screenshot?
13:58dustinit looks like it's some kind of desktop-changing view?
14:00firebotBug 1364534 NEW, toolkit/components/passwordmgr/test/test_prompt_async.html fails on windows 10
14:01pmooredustin: beats me
14:02pmooredustin: win10 feature that didn't exist on previous versions?
14:02pmooremaybe tests need tweaking for win10?
14:03pmoorewindows' answer to os x mission control?
14:14pmooredustin: i don't think it is related to the handling of desktops by generic worker, since iirc i only generate desktops when generating task users - when running as current user, i use the current user's existing desktop
14:15pmooredustin: note i'll be pushing out a new release when testing is complete, that simplifies the handling of users, desktops, login sessions etc - i'll be sending out an email with details when it is ready
14:15pmooreexpect something in the next couple of days
14:22garndt!t-rex explicit priority in task definitions has landed on autoland and will merge around. Shouldn't be too much fuss, hopefully no complains about missing scopes
14:24pmooregarndt: we should be able to audit clients and roles for existing scopes in the old format, right?
14:25pmooreor using the Scope Inspector maybe?
14:25pmoorewas there any email sent around about the change to how priority scopes?
14:26* pmoore should read his email more frequently :D
14:26pmoore*to how priority scopes work*
14:27garndtI already wrote like 5 line script to find the roles that have the legacy scops
14:27pmooreah nice - should we do the same for clients?
14:27pmoore(in case they aren't using roles)
14:27garndtit's not urgent to migrate, those that have the legacy scopes will continue to operate the same as before really
14:28garndtwe need to audit everywhere
14:28garndtthere was an RFC that talked about priority scopes, and a queue PR with the technical details
14:29pmooreah nice - i guess there was an email then, as that is part of our new RFC process :)
14:29* pmoore will clear his email backlog next week :D
14:30garndtI believe jonas emailed about it, or at least he should have :)
14:30pmoorecommand-A; delete
14:30pmoore.... which should also help me with disk space on my machine
14:30garndtah yea,
14:30garndtI am very close to archiving everything in my inbox
15:07jhfordjonasfj: your voice is really choppy, i suspect it's that your mic boom is further from your mouth than it should be
15:08jonasfjjhford: hmm, it could also be me who is choppy... :)
15:17* jmaher wonders if you could intentionally talk choppy and mix that with voip to get silence? ;)
15:29philordoes taskcluster have something that will prevent it from infinitely retrying a job?
15:38jonasfjbut you can click retry indifinitely
15:40davehuntbstack: re: yesterday's discussion on GitHub projects running against Firefox CI.. I'm a bit lost where to get started.. should I dive into TaskCluster, or get something running locally using mach first?
15:40bstackI would start with mach I think
15:40bstackthe taskgraph stuff shouldn't be too hard
15:41davehuntokay, are you a good person to talk to about mach, or should I be talking to someone else? :)
15:46garndtI'm actually having quite a bit of audio issues all around, vidyo is teh suck this morning
15:50bstackdavehunt: probably not me :/
15:51bstackI'm not sure who though. dustin will at least be able to point you in a better direction ^
15:52dustinI can answer questions
15:52dustinI don't think "mach" is the specific topic (it's just a way of running scripts)
15:54davehunthey dustin! so what I initially want to do is build an add-on and run Firefox tests (mochi, talos, etc) with that add-on installed
15:55davehuntultimately this would be run in a Try build, with the add-on being a GitHub project overlayed into the hg repo in a subdirectory
15:55davehuntthe 'build an add-on' will differ between projects, and in some cases would be a case of mixin some content from the project and build Firefox
16:24dustindavehunt: sorry, done with meeting now
16:24davehuntdustin: np, I have a meeting shortly, perhaps I could schedule some time to chat tomorrow?
16:25dustinI'm going to PyCon until next week (
16:25dustinleaving shortly
16:25dustinmaybe right after your meeting?
16:25davehuntah, okay.. I have to leave after my meeting..
16:26davehuntwe can probably continue on irc through my meeting
16:26davehuntdid what I say above make sense?
16:27dustinso are these always going to pop out as a browser installer?
16:28davehunthmm.. not sure I understand the question.. the projects ultimately will be merged into m-c as features.. screenshots, activitystream, are examples
16:29davehuntwe want to run the existing Firefox CI tests with these projects integrated, to see if they cause regressions
16:32dustinI guess my question is, would you ever run a test (mochitest, for example) where the test first downloads a "vanilla" browser installer, then downloads/installs the add-on into it, then runs the test
16:32dustinor will the tests always download a single installer that comes with the add-on already installed
16:34davehuntdustin: I'm expecting that Firefox would be built, perhaps unless it's not needed, in which case if there's a way to skip that and fetch a build from somewhere that's ideal
16:34dustinbut how does the add-on get into that fetched copy?
16:34davehuntI don't think anything would *need* to be downloaded, though that might be an optimisation
16:35dustinwell, let me tell you what I'm envisioning and you can see if it makes sense :)
16:35dustinright now we have "build" tests that compile firefox, and output an installer
16:35dustins/tests/tasks/g sorry
16:35dustinand "test" tasks that run a chunk of a test suite and take an installer as input
16:35davehuntso the idea is that a pull request is opened in a GitHub project for a (system) add-on.. then we'll create a patch for try that puts the state of the repo at that commit into a subdirectory.. then on Try we want to run all the tests with that (system) add-on installed
16:37dustinso maybe that particular add-on becomes a task in a new "addon-build" task kind, which either builds firefox from scratch with the add-on, or depends on a "build" task and repackages it to contain the add-on
16:37dustinand then the same kind of "test" tasks depend on those "addon-build" tasks they still take an installer as input, it just happens to be an installer that includes the designated add-on
16:37davehuntdustin: that sounds neat
16:38dustinthis is only happening in try? or will they eventually land in-tree?
16:40davehuntinitially only planning for pull requests and try, but a natural evolution may be for landing changes to overlayed to autoland
16:41davehuntand even then, most likely would remain one-way
16:42mcoteI have thoughts on this :)
16:42dustinI bet :)
16:42mcotewas part of my thinking on "conduit"
16:42mcotewhich is slightly distracted with the phabricator move, but the principles remain
16:42mcotecommit tracking throughout the engineering "pipelie"
16:43dustindavehunt: so to get started, I'd suggest looking at the "build" kind, and adding a new "build" that includes an add-on
16:43dustinyou've seen
16:44davehuntdustin: yep, and I got confused pretty quickly :P
16:44dustinit's probably good to try to understand how it works for the single-repo case first :)
16:44dustinevery time we've mixed a second repo, we've done it differently (e.g., graphcs, servo)
16:44dustinit's a super-complex problem, so the solutions are always pretty complex too
16:45davehuntyeah, gps and glob have helped me to understand the servo/stylo parts
16:47dustinbtw the suggestion would basically amount to a new set of platforms like "win64-activitystream/opt"
16:48dustinI don't think that's a good long-term approach, but it's probably a good place to start understanding this stuff
16:48davehuntdustin: when are you back next week? maybe I could dive into the docs and catch up when you're back
16:49dustinlater we can pretty easily change that from 'build with platform win64-activitystream/opt' to 'addon-build with platform win64/opt and addon activitystream'
16:49davehuntdustin: is there a TaskCluster basics doc I could/should read before reading the gecko rtd? It appears to go straight into details.
16:50dustinbut of course there's lots more detail available than you need, so the trick is getting you the details you need
16:50dustinothers in this channel can probably help there, too
16:51davehuntwhat key things should I focus on? build tasks, platforms, any others?
16:52davehuntdustin: also, are there things I can experiment with locally?
16:53dustinyou can create tasks with the task creator
16:53dustinI would start with the TC tutorial, just to figure out basically what a task is
16:54dustinthen dig into the in-tree taskgraph generation, which is a big ball of wax which, at the end of the day, just creates some tasks.
16:55davehuntokay, so if I'm understanding correctly, we'd use TaskCluster to create a custom build or repackage a build with an add-on installed.. then the test tasks work as normal, just against this customised build? so there's no need to tweak/duplicate the test tasks?
16:58dustinwell, there's no reason to copy/paste them
16:58dustinthe task-graph generation stuff will generate additional test tasks
16:59davehuntokay.. thanks! enjoy PyCon.. I'll try to get educated while you're there :)
19:30froydnjI'm getting lots of "TaskclusterRestFailure: Run 0 was already claimed by another worker" on my Mac try jobs that known? bug 1308266 is what treeherder suggests, but that doesn't look quite right...
19:30firebot NEW, Intermittent "errorCode: RequestConflict" bustage due to "TaskclusterRestFailure: Run 0 on task XXX
19:31froydnjlooks like it might be trying to upload packages
19:34dustinrail: ^^?
19:39railfroydnj: do you have the url?
19:40froydnjrail: the three mac retriggers
19:47railI think I found the issue
19:49garndtKWierso: would you be able to uplift this patch for me to m-c? is there something I need to flag on the bug?
19:49firebotBug 1364266 NEW, Task priority to be defined based on project branch
19:49KWiersogarndt: can it wait a few hours for my normal merge?
19:50garndtno problem!
19:50railfroydnj: uses the same taskId for all reruns and it fails to attach artifacts to an already resolved task without rescheduling it
19:51froydnjrail: ah, that would explain the situation, yes
19:51railwe should fix that code
19:53froydnjis that new-ish code?
19:53* rail blames it
19:54rail says since 45
19:55froydnjmaybe I just never retrigger build jobs?
19:59railheh, maybe
22:06ochameaudustin: thanks, I got POST to work with your tips
22:20ochameaubstack: hi
22:20bstackwhat's up?
22:20ochameauwould you have a minute to take a look at ?
22:20ochameauI think I got a pretty decent patch now
22:20bstackyeah, definitely
22:20ochameauand I'm ready to provide one for pull request comment if we can agree on that first one
22:26bstackochameau: I just have a couple nits and questions on the review but it looks great :)
22:28ochameaubstack: cool, thanks for the immediate feedback!
22:28bstackno problem
22:30garndtI was looking at that some time back
22:32gpsprompted from IRL discussion because i got confused seeing a "->>'foo'" snippet in a code review and was like WTF
22:32gps"that's not valid SQL!"
22:36garndtbstack: did you work on the "add new jobs" stuff in TH?
22:36bstackwhat's up with it?
22:37garndtnothing's wrong, i was talking with catlee in another channel about this situation we're in where we can't backfill BBB jobs
22:37garndtand there was a question about how that works
22:37garndtI believe it reads something from the decision task to know which jobs *could* have been scheduled, and then based on that action tasks are submitted to add theM?
22:37garndtI don't know much about how it's actually implemented
22:37garndtother than magic and unicorns
22:37bstackit's all in-tree
22:38bstackbut yeah, that's the general idea
22:38bstackit just reads full-taskgraph and goes from there
22:38bstackyou mean the part where it shows you the big list of possible jobs?
22:38bstackI didn't do that bit
22:38bstackalthough I expect it works similarly
22:38bstackbut with something to get bb jobs as well
22:38garndtok, so somehow that `mach taskgraph action` is called with the label of the job to schedule, and things are just done automatically?
22:39garndtand that action task is scheduled by TH just like backfilling as well?
22:41bstackat that level, yes
22:42bstackall of that should get ported to action.json at some point, btw
22:42garndtso the situation we're in is that we had to turn off SETA for BBB tasks...whcih right now are all of OS X and causing very large backlogs because we do not have SETA. We thought maybe we could push the migration to be quicker, but we identified some operational areas we need to work on first before migrating...and now I'm trying to see how we can decrease
22:42garndtthis backlog
22:44bstackwell, somebody could figure out how to make bbb work with action tasks
22:44bstackI don't know if anybody has any good ideas there though
22:47garndtyea, the option I have is pretty terrible
22:49bstackochameau: there's some magic deferredAuth stuff we can do to make the branch scopes stuff work if we want it to
22:50bstackwhich seems like maybe the correct approach here?
22:50garndtbasically report BBB tasks to TH as tier 3 and setup an exclusion profile to hide them. Then backfill will determine that it was really a BBB task that scheduled that BB job, find the job id for that BBB task, and submit an action task for that job id
22:50bstackI'm happy to make a PR into your PR with the code if so
22:50garndtand that is just kind of crappy
22:51bstackif bb can report the name of the tc job that triggered it to th then there's an easier solution
22:51bstackwhich is to just make the th stuff pick the tc job name out of the th api
22:52bstackso currently bb reports the job status to th?
22:53bstackdoes it know the name of the tc task that triggered it?
22:54bstackor is that all elided by bbb
22:54garndtI do not believe so, but that's a qustion for those that understand BBB, which is what schedules the jobs in BB
22:54garndtbut in TH, we know of the task ID that scheduled a particular bb job
22:54garndt"reason": "Created by BBB for task PRl2UKS4ToCuHkZTrddLAw",
22:54ochameaubstack: yes, I'm note sure I'll be able to write that without any existing example
22:54garndtoh...if that reason is supplied like that, then yes, BBB/BB must be knowing of the task id some how
22:55bstackochameau: those two lines show how we do deferred auth in the queue for example
22:55bstackif you want to poke at it for a bit and see if you can make it work
22:56bstackand if not I can try to get around to it tomorrow if you ping me?
22:56bstackgarndt: but bbb doesn't write to th, does it?
22:56garndtnot that I'm aware of
22:56bstackso it's likely that bb doesn't actually know the name of the task
22:56bstackI bet that a bbb person can answer this sort of thing quickly though
22:57garndtcamd: do you know how TH is aware of BB jobs? like what does TH the jobs it should display for a resultset? Is that the build4h magic?
22:57bstackto me that seems like the most straightforward way of fixing this
22:57garndtbstack: name of the task as in
22:57camdgarndt: yeah, it polls builds4hr
22:57bstackthe name as in whatever the key is in taskgraph.json
22:58camdgarndt: we load the whole thing, then skip anything we already know about
22:58garndtok, BBB will have no knowledge of that
22:58garndtcamd: thanks sir!
22:58camdgarndt: you bet! :)
22:58bstackgarndt: or even just a list of "BB job name" -> "tc task name" for all bbb jobs would do the trick
22:59garndtI'm not sure any task is aware of the label it had in the taskgraph
22:59bstackth can consult with that before submitting it to the proper place
23:00bstackseems like name is the label from the taskgraph
23:00bstack for instance
23:00bstackso from the tc side we're find
23:01garndtthis is probably because I don't know backfilling well enough, but I am not sure of the connection of selecting a BB jobs on TH, and clicking backfil to knowing the BBB task name and backfilling that
23:02bstackso when we ask to backfill, we inherently know the bb builder name
23:03bstack"bb builder name" -> "tc task name" is the mapping we want
23:03bstackwhere is the bbb code? I'll look and see how it works
23:03bstackthere must be some sort of mapping
23:04bstackok, reading it now
23:04bstackone sec
23:04garndtso the problem is I think there is no more "builder name" since the schedulers for those were turned off
23:05bstackgarndt: ah
23:06bstackyeah, that was the problem we diagnosed on monday or whatever
23:06bstackbut builder name exists, it just points to nowhere
23:06bstackif this really is costing us $$$$$$$ why not turn those schedulers back on?
23:07bstackat least for the time being
23:07garndtit's a fixed hardware pool
23:07garndtturnign schedulers back on I believe means double scheduling as long as we're using BB B as well
23:07bstackhow long are we thinking we'll be on bbb
23:08bstackbut also the plan I'm drawing up does not involve any of the issues with the turned off schedulers, afaict
23:08bstacksince it wouldn't use mozci at all
23:11garndtthe originally plan was to be migrating off of BBB as early as Monday but we have discovered today that there are some operational tools sheriffs and others need around these hardware tools that they already have for that needs to be worked on before we make any large migration
23:12bstackso... months? :p
23:12garndtI sure as heck hope not
23:12garndtmaybe weeks? kmoi.r has someone looking at the dashboard for some of this, then work needs to be done to hook it up to my DB and report stats to it
23:13garndtbasically they hve a dashboard that lets people see what machines are in the ppol, how long they've been up, when was the last time they claimed a job, what's the last 10 jobs they ran, etc
23:13bstackyour db is suddenly production when it was just a hacked together thing as of a week ago. Is that going to be ok?
23:13garndtand from there a person can click a button to reboot the machine (which calls another api that talks to the PDU that it's physically plugged into and toggles the power ont hat outlet)
23:13garndtlet me rephrase, not my db specifically, they will use my app as inspiration of how you can listen to task events adn stuff them in a DB to query
23:14bstackok, so this will take a while to get running
23:14bstackI'm pretty sure we can just end-around the whole thing with that mapping I mentioned
23:14bstackshould I try to make that work now rather than my other stuff?
23:15garndtI think spending a little time looking into it would be worthwhile if it's possible to do it. I'm not sure if I follow exactly how you think it's possible, but I'm certainly happy if it is
23:16garndtanother optoin is to revert it all pre-BBB and let it run like before until we're ready again
23:16bstacklet's talk about it more first then
23:17bstackso the issue right now is that TH doesn't know how to trigger the tc jobs
23:17bstacksince it just looks like a bb job
23:18bstackand therefore pulse_actions tries to handle it and can't find the build that the tests are supposed to run on
23:18bstacksince the builder that used to do it doesn't exist anymore
23:19bstackI think we can fix that by checking in treeherder to see if this bb job is actually a tc task that goes through bbb
23:19garndtI think so, but that's a grey area for me that I do not understand as much. If we want to understand this more and we are not sure what happens in that nebulous, we need to consult some others about it
23:20garndtright, you can easily find that out by the "reason" field of the job in looks like BB reports that it was created by BBB and gives the task ID
23:20bstackthen you can find the task definition from that
23:20bstackit is just a matter of resubmitting that job or something along those lines
23:20garndtthis is backfilling, not add new job
23:20bstackwe don't even need to look there, we can just go to the taskgraph directly
23:20bstackyep, it still works
23:20garndtand the bakfilling requires a treeherder job id
23:21bstackit doesn't
23:21garndtand there is no treeherer job id for a BBB task
23:21bstackit doesn't really require one
23:21bstackwhat it uses that for is to find the taskgraph artifact
23:21bstackwe can shuffle it around so that the taskgraph is found in th
23:22bstack*** anything is possible in zombocom ***
23:22bstackI dunno, it's all a lot of work probably
23:22bstackanother option is to make mozci not try to find the build it was based on
23:22bstackor at least not fail when it can't find it
23:27bstackgarndt: another thing is that if we're this hazy on how it all works we should maybe not do the work? maybe just advise whoever does?
23:29garndtyea, it needs to involve others that understands it more, but I think most are in EST and have left for the day. Complicating it is the fact that we have all had our hands in this backfilling situation in one way or another and I dont' think there is one true person to go to about it. It's going to require some collaboration to solve
23:31garndtI believe those that know more about some of the aspects of this were the same ones helping us troubleshoot it the other day and we're hitting some dead ends
23:32bstackhonestly we know more about this than anyone afaict
23:32garndtand if we're less hazy than others on how this work, who do we really advise? :)
23:33bstackpersonally I think we should do the work
23:33bstackI just don't think we're that hazy
23:33bstackunless this is a situation with unknown unknowns
23:34garndtoh, then I think I misunderstood what you were saying earlier, sorry, long day
23:34garndtyea, I think it's something we could have a hand in trying to figure out, which is why I'm trying to explore option and dig in
23:34garndtI also like to know more about this anyways...we can share the pain
23:36ochameaubstack: so deferAuth: true and req.satisfies is easy. but figuring out the branch of a given sha, that's not possible via GitHub API.
23:37ochameaubstack: so I don't know exactly how to determine the branch we should assert
23:40bstackochameau: we can ask github which branch the commit is on maybe?
23:40bstacknot sure, haven't thought about it a lot yet :/
23:41ochameauafaik, this is not something the github api offers
23:42bstackgarndt: is everything in the task graph?
23:43bstackor are there jobs in th that are still just straight bb?
23:43garndtwell the BBB tasks that scheduled this is in the task graph
23:43bstackochameau: sorry, I'm a bit occupied with this bbb stuff at the moment :/ I'm really excited for that feature though, happy to talk soon.
23:43garndtwhich then ends up scheduling a BB job that displays in TH
23:44ochameauok. I'll post my findings to the PR. may be dustin will have some suggestion as well
23:44bstackgarndt: yeah, but they all go through bbb at this point, right?
23:45garndtall of the OS X tasks do
23:46bstackjonasfj brings up that if everything is bbb, we can turn off buildbot writing results at all
23:47bstackand then we can go back and make tc write bbb results
23:47bstackand then this all Just Works
23:47jonasfjalso then treeherder wouldn't be doing BB ingestion anymore..
23:48garndtoh right, you two are having your thursday hack day eh?
23:48garndtok, so the BBB task will get resolved when the build/test is done....what about artifacts that the buildbot job produced?
23:49bstackgarndt: do you want to vidyo?
23:49garndtand not everything is BBB, it is for OS X, but BB is still used for many other things that do not involve BBB
23:49bstackwe're sitting in front of one of the fancy new lightup vidyo things now
23:49bstackit hurts our eyes to look at it
23:50garndtlightup vidyo? that sounds scary
23:50garndtit's almost 7pm, I need to head off to feed the family in a minute
23:50bstackok, sounds good
23:50bstackI think we can do the quick hacky thing pretty quickly
23:50bstackI'll be a bit sad to stop other work, but if it is necessary that is fine
23:51garndtwe have two options....get SETA working against for OS X, or rolling back the buildbot and BBB changes needed for OS X.....i'm not opposed to that, I just need to understand more what's involved there and the impact
23:52garndtif we don't have a clear idea of what to do here to get things scheduled, or it would take too much to context switch, I understand. We can discuss with others some options tomorrow before work is done.
19 May 2017
No messages
Last message: 6 days and 7 hours ago