mozilla :: #rust-infra

21 Apr 2017
00:07aidanhswhoa I did not know appveyor cuts off the beginning of logs if they get too long
00:07aidanhsbit scary
00:10acrichtoaidanhs: yeah :(
00:10acrichtoinstead of travis "kill the build" appveyor just gives you the last 4MB
00:10acrichtowhich arguably should be the most interesting megabytes
00:11acrichtoshould turn off trace logging on beta...
00:11aidanhsjust means backporting #41209
00:12acrichtoyeah shouldn't be too bad
00:13aidanhsseems like only 2.3MB for #41427, which is odd
00:13aidanhswant me to port that PR to beta? or is it basically done now?
00:14aidanhs(given the title of 41427 is 'final backports to beta'
00:14aidanhs)
00:26acrichtoaidanhs: oh feel free
00:26sarnoldI've got a self-inflicted problem that makes the play.rust-lang.org site more annoying to use than it could be
00:26acrichto"final" in the sense of "probably last"
00:27sarnoldI use firefox with noscript, and the url shortener used with the sandbox causes most of my attempts to load pages to fire the noscript warning bar at the top "NoScript filtered a potential cross-site scripting (XSS) attempt from [chrome:]. Technical details have been logged to the Console."
00:31projektirsarnold: you're not the only one with that problem
00:33sarnoldprojektir: have you heard anything that allows it to work more pleasantly but still break most of the internet? :)
00:33sarnoldI've never had success adding whitelist entries to noscript's dialog box
00:34sarnoldI can't reasonably ask anyone else to spend time on this but I thought I'd mention it in passing, just in case the next time someone works on this and has an idea how to avoid it, ...
00:34sarnold.. might be willing to fiddle with something while working on it. :)
00:37projektirI'm not sure, really, since most code runners don't seem to cause that problem...
03:11aturonedunham: still planning on starting that thread re: access rights?
03:12aturonedunham: the most important thing is just to kick the discussion off
12:46simulTest
12:46simulacrumOkay, I should be able to attend today's meeting nearly in full via phone, at least
13:05frewsxcvI'll probably be attending via mobile too
13:56shepwoo it's time to ping all the people
13:58shepI feel like I should have special text for PRs opened by acrichto and other core team members: "Welcome to the Rust project" is not as amusing as it could be
13:58acrichtoit's ok highfive doesn't discriminate
13:58acrichtoneither should we!
14:12shepacrichto: to clarify, I want to use even more newbie text for you
14:13shep"Welcome! Rust is a project aimed at ...." "You can build Rust via ...."
14:13acrichtohehe
14:13frewsxcvis there an open issue tracking these spurious 'could not parse git object' errors?
14:13frewsxcvhttps://travis-ci.org/rust-lang/rust/jobs/224338090
14:13frewsxcvi think those are just spurious network errors though
14:14acrichtofrewsxcv: oh those aren't spurious
14:14acrichtothose are expected
14:14acrichtowe do a shallow clone of git repos
14:14shepfrewsxcv: I haven't seen these specifically, but in other projects its because of a force push
14:14acrichtoexponentially increasing the depth until we find the commit
14:14acrichtothe failure there is the download from crates.io
14:16frewsxcvoh gotcha
14:52shepacrichto: may i borrow your experienced eyes? https://github.com/rust-lang/rust/pull/41024
14:52shepthat seems like it's in a no-mans-zone of who should do somethign next
14:52acrichtohm, lemme look
14:54acrichtoshep: maybe broaden to cc rust-lang/compiler?
14:54acrichtoif no one's willing to help get it across and it's not suitable to merge
14:54acrichtothen there's not much that can be done :(
14:54acrichtobut better to broaden before closing
14:56shepacrichto: one more - any objections to me `r=you` on https://github.com/rust-lang/rust/pull/40113?
14:56acrichtoshep: yeah I need to take another look at that
14:56acrichtofeel free to comment saying such
14:57shepok, then I won't r if you wanna look-see
14:58shepConsider that your ping ;-)
15:02acrichtothanks!
18:24rillianacrichto: just the currect state of affairs. :)
18:24acrichtorillian: the rust-lang/rust repo has a cargo submodule
18:24acrichtoand that's the one released
18:28rillianacrichto: ok, that seems straightforward. And to bump the cargo version one makes a pr with the submodule hash updated?
18:29acrichtorillian: indeed!
18:29acrichtowe do it every so often
18:29acrichtoand should probably do it more often...
18:32rillianacrichto: and the cargo version in the manifest comes from the crate metadata instead of from git, which is why there are no channel branches in the cargo repo?
18:32acrichtorillian: correct yeah
18:32rillianI guess there's some signing process to get the release tags into the cargo repo based on what's in the rust tree?
18:32acrichtothe cargo repo has branches for the rust versions
18:32acrichtobut that's it
18:33acrichtoyeah so travis builds all our artifacts
18:33acrichtoand uploads them on every commit
18:33acrichtoand a separate process comes through every night to promote those to releases
18:33acrichtosigning them in the process
18:33rillianok, that all makes sense now. Thanks!
19:20TimNNJust FYI, I likely won't be able to participate in tonights meeting
19:39aturonTimNN: no worries, thanks for the heads up
21:12frewsxcvMeeting in twenty minutes?
21:13aturonyes
21:14aturonfrewsxcv: i've made a google drive folder, linked now in the topic
21:14aturoni'll talk a bit more about that when we meet
21:18frewsxcvsounds good
21:27ericktfyi I'm a bit swamped so I might not be paying much attention to the meeting
21:28aturonerickt: np, we'll be sure to talk all about security policies today
21:30aturonfrewsxcv: lol @ the pic you added
21:30aturonwe should also get the oiling gears logo in there
21:30frewsxcvYou can thank Carol for that picture
21:30aturon(which makes me want to establish logos for all the subteams)
21:30aturonding ding ding, it's that time!
21:30aturonwave if you're here
21:30aturono/
21:30acrichtoo/
21:31* frewsxcv waves
21:31misdreavuso/
21:31* nagisa is here
21:31aidanhso/
21:31acrichto(TimNN said he may be out today)
21:31aturoni believe we expect simulacrum to be late and/or on phone
21:31aturoncarols10cents: shep: you coming today?
21:32aturonok, while we're waiting: i've now set up a google drive folder for us here: https://drive.google.com/drive/folders/0By65ti_gFrw5bWcxVFJJVEdrN1k?usp=sharing
21:32aturonit's currently open to anyone with the link; we'll probably want to secure it at some point
21:32aturona couple notes about what's there
21:33aturonfirst, the document we'd been working on has now been split into several
21:33aturonso if you're looking for the PR process, it's called "Stale PR process"
21:33aturoni tried to resolve everybody's notes before moving it over
21:33aturonalso, there's a Meeting notes document, which will have an agenda for each meeting
21:33aturonyou can use this to jot down stuff you'd like to cover at the next meeting
21:33aturon(top of the doc should always be the most recent)
21:34aturonand you can see today's agenda there
21:34aturoneverybody finding their way around?
21:34acrichtoI feel like we're leaving the stone ages
21:34aidanhsyup
21:34acrichtoand it's only 2017
21:34aturonlol
21:34aturonok cool
21:35aturonso i think we'll go ahead and dive in, and carols10cents/shep can join later if they're around
21:35aidanhsyou can tell stories of the bad old days when infra was passed down by word of mouth
21:35ericktaturon: noooo
21:35aturonfirst off, i wanted to touch base on the stale PR process a week in
21:35aturoni've added a graph tab to the PR tracking sheet
21:35aturonso you can see how we're doing over time
21:35frewsxcvheh
21:36aturonso far the waiting for review count is not too bad, and getting a bit better over time
21:36aturonas a reviewer on the receiving end, i've already found this process helpful
21:36aturonbut, a couple of questions:
21:37aturon- Are people generally happy with the process/guidelines? Any questions you've run into or things you want to change?
21:37aturonsimul: howdy!
21:37simulo/
21:38frewsxcvI'm pretty happy with the process
21:38aturonsimul: not sure if you can see this on your phone, but we're going through docs in https://drive.google.com/open?id=0B_I79nrlHhQwcnJGY293NHltY28
21:38acrichtoyeah it seems like it's working well
21:38acrichtoalthough shep/carol have been active as well and may have thoughts
21:38simulWrong link?
21:38aturonOK, and i think it's worked pretty well for people to log issues as comments on the google drive doc
21:38aturonsimul: https://goo.gl/AhFakN
21:39aturonso yeah, if you run into issues, please continue leaving comments and we'll keep honing
21:39simulWill do.
21:39aturonbut i feel like with the revisions we made, the labels are now quite clear, and the per-day workload is pretty low
21:39aidanhsI just want to take a sec to say thanks to whoever initially fleshed out that doc and to anyone who's contributed to it, by the time I got to it it was incredibly easy to follow
21:39acrichtoyeah it took me ~15 min the first time
21:39aturonok so next question on this topic:
21:40aturonhow do we want to set up the rotation going forward?
21:40aturonone option would be to literally have people assigned by day of the week
21:40aturonwhich is nice because we don't have to constantly establish new schedules etc
21:40aturonand we have enough people that i think it can work
21:40aturonthoughts?
21:40acrichtoI'm game for that
21:40simulI think the amount of time makes that fine
21:41acrichtowe could have 2 per day
21:41acrichtoand whoever gets there first wins the race
21:41aturonheh
21:41acrichtoand that'd account for vacations as well
21:41aturonso i'm assuming we want to continue not assigning on weekend?
21:41aturonor maybe some people would prefer signing up there?
21:41aturonthe rotation is at the top of the doc
21:41* frewsxcv doesn't have any strong opinions or great ideas about rotations
21:42aturondoes anybody prefer to take saturday?
21:42simulI can't edit right now, but I can do either or both weekend days
21:42aturonOK
21:42* carols10cents is here now oops
21:42aturonhi!
21:43aturoncarols10cents: have some backscroll
21:43simulOr really any day if it's just a 15-20 minutes
21:43carols10centsyuuuup working on it :)
21:43aidanhsno particular thoughts. I guess for holidays people can just note on IRC, ideally before the weekly meeting preceeding the slot
21:43aturonyeah im' not too worried about that, since we only miss one day/week
21:44aturonalright, so let's go with the roster as it is for now
21:44aturonsunday will be our day of rest :)
21:44aturonOK i think that's it for stale PRs, feel like we've got solid momentum there now
21:44acrichtoagreed
21:44aturonnext up is spurious failures
21:45aturonso one thing i realized is that we should set up @rust-lang/infra as a team
21:45aturonthen potentially we can just make homu ping that in the failure comment
21:45aturonrather than the hacky nagbot setup
21:45aturonthat way we also all see what happens *after* the failure
21:45carols10centsoooooo
21:45acrichtoit runs a risk of cc'ing everyone on too many things
21:45aturoni'm not sure what's involved in changing our current homu instance, but i suspect that'd be really easy to do
21:45acrichtobut otherwise seems fine
21:46aturonacrichto: well so it'd be cc'ing no more than today
21:46aturonthat is, right now we're all getting email on every bors failure
21:46nagisapinging a team does not subscribe all the team members I think
21:46acrichtonah b/c once you're cc'd
21:46acrichtoyou keep getting emails
21:46acrichtowhereas today you get no emails after the failure
21:46acrichto(unless you comment)
21:46aturonnagisa: i believe it does subscribe
21:46carols10centsi mean... i'm already ccing myself because i keep forgetting to unsubscribe after triaging
21:46nagisa'kay then
21:46acrichtothat's also true
21:46simulI might've missed something, but I have not seen any non GitHub emails....
21:46acrichtoaturon: let's try this
21:47acrichtosimul: oh they're just normal github emails, just forwarded from a different account
21:47aturonsimul: the email would look like a normal github email
21:47aidanhsless keen on auto-subscribe to every single PR that ever fails bors, but I guess we can see
21:47acrichtosimul: have you seen build failures in your inbox?
21:47aturondefinitely open to leaving the status quo
21:47frewsxcvI see the failures, but not the retry comments
21:47simulYeah, but I subscribe to all rust-lang/rust so..
21:47carols10cents:-O
21:48aturondoes anyone feel like they'd prefer to be subscribed (so they see retries)?
21:48acrichtoI'm ok w/ clicking through to figure that out
21:48shepsimul: https://res.cloudinary.com/teepublic/image/private/s--6YSXCdJe--/b_rgb:fffffe,t_Heather%20Preview/c_limit,f_jpg,h_630,q_90,w_630/v1446240311/production/designs/280470_2.jpg
21:48aturonalternatively we could start logging retries out of band
21:48* simul doesn't care because is subscribed anyway
21:48aidanhsif I have some way of distinguishing it I can just set up an email filter which would be fine
21:48aturonyou wouldn't, it'd ping you through the normal github process
21:49aturonOK, i think let's just leave it with the nagbot forwarding for now
21:49shepacrichto: after being cced, you can still click through to github and unsubscribe, no?
21:49aturonso my impression so far is that y'all have been awesome at doing retries, tying back to issues, and generally keeping an eye on bors
21:49simulOhh... I now understand why I saw duplicate emails from GitHub
21:49acrichtoshep: correct
21:50aturonmy impression is also that (1) holy cow we spurious failures are happening a lot and (2) bors is also failing in some other weird ways
21:50aturone.g. there have been cases recently where homu thought it was testing two PRs at once
21:50aturonthings like that
21:50acrichtoaturon: since unsubscribing from rust-lang/rust I feel like mypersonal perspective on retries is skewed now
21:50aturonacrichto: i.e. they look high freq?
21:50acrichtoI don't feel like we've had a lot of spurious failures, but I know I'm wrong
21:50acrichtonah just I feel like I've gotten fewer failure notifications in my inbox
21:51acrichtowonder if others feel the same
21:51aturonhm but you should have all of them still
21:51frewsxcvSpurious failures are definitely better than they used to be, but they still happen
21:51simulYeah, it feels like I have seen prs get rolled up and then homu doesn't realize they are merged
21:51acrichtoaturon: it could just be less surrounding email so it feels less
21:51frewsxcvUsed to be like 75%, now it's like 40% of tries
21:51aturonwell, so of course *any* spurious failures are too many :)
21:51acrichtoaturon: I've also been traveling so many things are merged before I get to see them
21:51acrichtoso it may no be representative
21:51aturonacrichto: success!!!
21:51acrichtonot be*
21:52frewsxcv(totally unmeasured percentages)
21:52acrichtolol :+1:
21:52aturonok so, what i'm wondering is,
21:52aturonnow that we've got a good discipline for at least retrying etc, what's the next step?
21:52aturonsome thoughts:
21:52aturon- we could try to start accumulating some data on how often the various issues are being hit
21:52aturon(for triage purposes)
21:52aturon- we could pick a couple of issues we anecdotally think are common, and try to make progress
21:53aturon- we could work on the homu situation
21:53aturonIIRC there are various forks floating around
21:53aturonsimilarly with highfive
21:53acrichtoI would personally like most to see statistical analysis of retries
21:53acrichtoin terms of graphs over time broken down by issue-at-fault
21:53acrichtothat'd immediately tell us what's highest priority
21:53acrichtobe it ar.exe or homu
21:53frewsxcvThat'd be pretty cool
21:54aturonok, so staying in our mode of "make it work manually first",
21:54aturonwe could do this pretty easily i think by making another sheet
21:54aturonand basically whenever you retry, you log an entry
21:54acrichtoyeah that's what I was thinking
21:54aturonOK, seems like a good step regardless
21:54acrichtoalternatively we could update nagbot to forward to a new email on all emails that contains "@bors: retry"
21:54aturon(also reduces the need to actually be CC'ed)
21:55acrichtomanual seems like the best first step
21:55aturonagreed
21:55aturonOK, i will take as an action item to set up that spreadsheet
21:55aturonwith that in mind, are there other steps we should take right now?
21:55aturonacrichto: are you actively working on any spurious failures?
21:56acrichtoI believe I fixed the ar.exe failure (there's a PR for it)
21:56acrichtoothers, no
21:56aturonooo nice
21:56aturonok
21:56acrichtoopenssl on osx I feel is the next highest priority
21:56aturonissue #?
21:57simulacrumOkay at computer now :)
21:57carols10centshttps://github.com/rust-lang/rust/issues/40417
21:57acrichto^
21:57aturonty
21:57aidanhsI've been looking a little at network stuff just because I'm familiar, but we've not seen it recently. unsure if that's because my partial fixes (now rolled back) were helping...
21:57aturonanybody who's not acrichto want to look at https://github.com/rust-lang/rust/issues/40417 ?
21:57aturonaidanhs: ok, i'll pencil you in as "network failure person" :)
21:58aturonwe also have the option of surfacing these to the broader community
21:58simulacrumaturon: I have macOS, so I can try, but last I heard we weren't able to reproduce on non-Travis
21:58aturon"This week in Rust infra: oh god please help us fix #40417"
21:58aturonyeah generally the hardest problem with these is repro
21:59aturonwe'll touch on permissions issues in a bit, i'd like for everybody here to have roughly the same ability to debug these as acrichto does currently
21:59frewsxcvThat's the most common spurious i see fwiw
21:59aidanhsyeah first step for the osx one is probably just to set up a reduced travis thing that you trigger a run of every hour or so
21:59aidanhsdoesn't even require you to have osx
21:59aturonOK, we need somebody to "take point"
22:00aidanhsI can ignore the network stuff and give it a go
22:00simulacrumI don't really know Travis well enough and can't dedicate enough time to it at this point unfortunately (perf.rlo is focus for me)
22:00aturonsimulacrum: no problem!
22:00aidanhscould use someone familiar with osx to ping if I need help though
22:00aturonaidanhs: alright great!
22:00aturonyeah so in general pinging acrichto for help working through these issues is fine
22:01aturon(but please fill out the FAQ document when you end up asking questions)
22:01aidanhsok
22:01aturoncan also try on #rust-internals in general
22:01aturonok aidanhs, thanks much for stepping up!
22:01aidanhslol initially it's probably going to be things like "why does osx have /home and /Users"
22:01aidanhsbut will keep it in mind
22:01aturonzomg ok maybe don't ping acrichto ;-)
22:02aidanhsyeah, SO to the rescue
22:02nagisaor "who let apple to write an OS?"
22:02nagisaright?
22:02aturonbut ok, i think a rough goal of always having a "top" spurious failure we're trying to resolve, and a point person for it, is good
22:02aturoneach week we can check in on our progress, triage with the stats, etc
22:02aturonany other thoughts on spurious failures?
22:03nagisanothing an appropriate number of retries shouldn't be able to fix
22:03aturonok cool, onto the next topic
22:03aturonso one thing we need to work on soon is going through our permissions set up
22:03aturongenerally expanding the set of people who have access to pieces of infra, documenting how to access, etc
22:04aturonedunham was going to do some research and start a thread,
22:04aturonbut it seems like maybe that stalled out
22:04aturonacrichto: as i understand it, part of the problem right now is the bastion
22:04aturoni.e., i think we have a single chokepoint for accessing most of our infra
22:04acrichtohm well so this is sort of a big topic
22:04aturonand access to that gives you access to everything
22:04acrichtoand depends on the piece of infra
22:04acrichtodo you have a particular one in mind or just the general problem?
22:04aturonacrichto: understood; i'm not sure how best to break it into incremental pieces
22:05aturonwell, let me turn it around for a sec
22:05carols10centsas i understand it erickt wants to do a survey and audit of absolutely everything
22:05acrichtowe can definitely figure out how to turn it into incremental pieces
22:05aturonfor people who have been trying to do infra work, where have you gotten blocked by permissions?
22:05acrichtoit'll take some work on our end but it's not impossible
22:05aturoncarols10cents: yes we should do that, not sure if it's the right first step, but might be
22:05simulacrumI've been ~blocked since I can't deploy to perf.rlo, but I worked around by getting free heroku app up
22:05frewsxcvRestarting homu
22:05aturonsimulacrum: i know you ran into some permissions issues
22:05frewsxcvHaven't needed it recently though
22:05* carols10cents likes volunteering people for things when they're not here
22:06aturonwhat about travis/appveyor?
22:06* erickt perks up
22:06aidanhsspeaking of "rust has lots of infra", worth moving https://docs.google.com/document/d/1ZxrAhkFPP8884eTr09Q8rYh16M1brWw5qn9Xmyn7014/edit into the folder?
22:06carols10centsoh crap, busted
22:06ericktI have been summoned
22:06aturonlol
22:06aturonaidanhs: yes let's get that into the folder
22:07aturonfor travis/appveyor, are there perms that would be useful for people?
22:07aturon(momentarily keeping the focus on the PR queue)
22:07ericktWhat's wrong with using a bastion? Too many people need access to it to get to other services?
22:07shepI'm ready to enter into the FAQ; what's "bastion"
22:08shepother than the general English word
22:08ericktGenerally speaking choke points are convenient for security auditing
22:08simulacrumNothing for travis/appveyor. I think we can restart PR builds (not bors, just the individual)
22:08aturonacrichto: ^ :)
22:08aturonwhat about r+ rights etc? is that all set up for everybody?
22:08simulacrumretry yes
22:08simulacrumnagisa was wanting r+, I think?
22:08ericktshep: it's a machine sitting on the boundary between the public Internet and internal services
22:08acrichtoshep: bastion == a computer you have to ssh through to get to the rest of the network
22:08aturonsimulacrum: taht's set
22:09acrichtoshep: all of our computers are configured to only accept incoming ssh connections from the bastion
22:09aturonerickt: i'm going on what edunham said when we talked about this, where she was saying the bastion was currently preventing us from giving finer-grained permissions
22:09aturonit's possible i misunderstood though
22:09acrichtonah I don't think that'll be much of a problem
22:09aturonit sounds though like we're in pretty good shape for the core PR queue stuff
22:10aturonother than restarting homu
22:10ericktI can see that since if everyone is routed through the bastion
22:10shepDoes it happen to have a hostname called "bastion" (i.e. do you always say "log in to bastion" or "to the bastion")
22:10acrichtoshep: it has an IP (instance on ec2), no hostname
22:10aturonbut we call it "the bastion"
22:10aidanhsI don't have retry but there are enough people watching PRs that if I explain a spurious failure someone will get to it in 5min
22:10ericktaturon What we can do though is to populate instances with ssh keys and only put keys on the machines people have permission to get to
22:11aturonacrichto: what do we need to do to get everybody retry etc?
22:11acrichtoaturon: just configuring homu
22:11acrichtoI thought aidanhs had retry
22:11acrichtoI'm checking
22:11aturonok while he's doing that:
22:12aturonso in general, if you run into something where you need permissions you don't have,
22:12aturonplease log it in the agenda for the next meeting
22:12aturonfor the time being, i think we can grow this organically
22:12simulacrumAgenda is in the etherpad?
22:12simulacrumhttps://public.etherpad-mozilla.org/p/rust-infra?
22:12aturonsimulacrum: no, it's in the Meeting notes doc in the google drive folder
22:12simulacrumAh
22:12simulacrumMakes logical sense :)
22:12aturon(everything from here on will live in that folder)
22:12ericktWe could build a simple admin service that acts as our proxy to cut down on access
22:13ericktIf all you want is a button to restart things
22:13aturonso like right now i think only acrichto can redeploy rust-central-station
22:13aturonbut that seems ok for the time being
22:13aturonso we'll organically log things that people run into,
22:13ericktAre these instances running in heroku or aws?
22:13acrichtocorrect (only I can deploy)
22:13aturonand then separately at some point a few of us can sit down and exhaustively list all infra,
22:13aturonand figure out a more comprehensive story
22:14aturondoes that make sense?
22:14ericktacrichto: please shelter inside until we can get some redundant keys from you
22:14acrichtoaturon: that sounds ok to me
22:14aturonok cool
22:14aturonso, more broadly as we start to look beyond the PR queue,
22:14aturonare there things that are blocking any of you from doing infra work you'd like to do?
22:15simulacrumperf.rlo deploy permissions for me, but nothing urgent (the free heroku app I have up now should be fine, though obv now ideal
22:15simulacrumnot ideal*
22:15aidanhsI would like more hours in the day please
22:15frewsxcvNot me personally
22:15aturonsimulacrum: yep, got that noted down
22:15shepditto as simulacrum s/perf.rlo/playground/
22:15aturonaidanhs: above my paygrade
22:16aturonshep: noted
22:16acrichtosimulacrum: FWIW I don't know who runs perf.rlo
22:16acrichtoshep: I can help you get access to that
22:16acrichtoping me after this
22:16shepdeploy + "spin up new instances", as we talked about lo many months ago
22:16aturonacrichto: please log info in the FAQ or elsewhere as you do that
22:16acrichtothat make take a lot longer :)
22:16acrichtoaturon: hm...
22:16aturonin general whenever we take infra steps, it'd be good to document them
22:16acrichtowe're still in sorta ad-hoc mode there
22:16simulacrumacrichto: IIRC, there's two machines (server under brson's desk and an EC2 instance(?))
22:16acrichtobut I'll do it anyway
22:16aturonyes ad hoc is fine
22:17acrichtosimulacrum: oh yeah was just an FYI, I know nothing about perf :)
22:17simulacrumedunham and nrc, possibly nmatsakis have access aiui
22:17aturonright now just writing down the status quo is helpful
22:17aturone.g. if i want to add a new reviewer, at the moment i have to ping you/brson to even know how
22:17aturonpermissions aside
22:17carols10centslol it's not real production infra unless there's a server under someone's desk
22:17* erickt sobs
22:17aturonacrichto: i figure to start, we should just organically and aggressively log stuff as it comes up
22:18aturonand then once it's accumulated we can start getting more organized, push to the forge, etc
22:18aturonother blockers in general?
22:18simulacrumNot that I can think of
22:19acrichtoaturon: sgtm
22:19aturonOK and again, if things come up, please log them down in the agenda for the next meeting
22:19aturonso to close out, a quick look forward
22:19aturoni think we're pretty close to having a good routine around both sides of the PR queue
22:20aturonso starting around next week, we should think about how to expand out from that
22:20aturonif you have time to think about what your interests are, or how you'd like to see Rust infra evolve, that'd be good to bring to next week's meeting
22:20aturon(and you can log your thoughts, again, in the agenda :)
22:20aturonexamples include:
22:21aturon- basic docs on what our infra even is, who can access it, etc
22:21aturon- monitoring infrastructure
22:21aturon- highfive and homu improvements
22:21aturon(just thinking about some of my personal wishlist)
22:22aturonotherwise, i think that's pretty much a wrap for today
22:22aturonto recap, i'll set up a spreadsheet for spurious failure tracking and send mail about it
22:22aturonaidanhs will take point on 40417, and we'll check in on that again next week
22:22aturonsg?
22:22simulacrumReal quick, just going to put this out there: please ping me if you have any ideas regarding perf.rlo improvements
22:22nagisaIMO were being a bit overzealous with pings on the PRs
22:23aturonnagisa: for reviewers, authors, or both?
22:23frewsxcvGreat work with the perf stuff :)
22:23nagisathere were cases I think were we pinged people on PRs twice over 2 weeks without giving them any time to come back
22:23aturonnagisa: have you looked through the procedure?
22:23nagisaI havent, Im just saying what I see on my mail client
22:23aidanhs...we've only had the procedure for one week, right?
22:23carols10centsi dont think we've been doing this for 2 weeks,
22:23carols10cents:)
22:23nagisaI guess then it overlaps with pings from before the procedure
22:23nagisaor something
22:24* nagisa shrugs
22:24aturoni could believe we'll need to tweak some of the day counts
22:24aturonoh, that reminds me!
22:24shepI think i commented on 2x things that carols10cents had commented on 4 days ago
22:24shepwhere the last non-ping was > 1 week ago
22:24aturonsomebody added some columns to the spreadsheet about IRC pings
22:24aidanhsoh yeah, they confused me
22:24aturonbut i don't think it's been incorporated into the actual process
22:25acrichtowhat's that column for?
22:25aturonthe process currently says to log on github that you did an IRC ping
22:25aturonacrichto: that's my question :)
22:25aturonit looks like the intent was to track people who needed to be pinged on IRC
22:25acrichtooh ok
22:25aturoni'm guessing whoever added that column doesn't happen to be here right now
22:26aturonhas anybody had to do an IRC ping so far?
22:26shepghost spreadsheets
22:26shepI've done a handful
22:26aturonshep: how did you feel about that process?
22:26simulacrumI've been feeling hesitant when I feel like I should (but this was before most of the current reworking)
22:27shepoverall it was fine. My personal issue is that I don't always idle on IRC so if someone responds, there's a chance I've dropped
22:27simulacrumProbably just me though :)
22:27shepbut if they respond, theoretically they can get to the issue to update it
22:27aturoni wonder if it'd help to provide some standard wording
22:27shepeven if it's just to say "i can't get to this"
22:27aturonand yes, direct them to the PR to respond
22:27aturon(together with leaving a comment there so that they don't keep getting pinged)
22:27simulacrumStandard wording and possibly centralized directions (PR response + maybe a comment here?)
22:27shep> so that they don't keep getting pinged
22:28shepI thought part of the idea was to piing them everyday
22:28aturonthe current process is not so aggressive
22:28aturonto keep things manageable
22:28aturonit's worth spelling this out
22:28aturonso, let's take S-waiting-on-review
22:29aturonyou're only supposed to look at items with >3 days of inactivity
22:29aturonand you're supposed to leave a comment (which counts as activity) when you ping on IRC
22:29aturonso basically, after 3 days stale, *one* IRC+github ping to reviewer
22:29aturonafter 6 days stale, *one* cc of the subteam
22:30aturoni wonder how we can improve the wording to make that more clear
22:30shepbut if we don&#39;t check with <3 days, waiting for review might be inaccurate
22:30aturonyes, but no activity for a labeled item has an action under 3 days
22:30aturonso it&#39;ll always get looked at and re-filed within the appropriate window
22:31aturon(this was part of the revamp -- to make it possible to not have to check *everything* every day)
22:31aturonis this making sense?
22:32shepnot to me :-)
22:32aturonOK, lemme be more concrete
22:32shepIf we are ok with the labels not being accurate for a few days, then I understand
22:32aturonah, yes, we are --
22:32aturonand i&#39;m saying it&#39;s fine because the mislabeling won&#39;t actually impact our actions
22:32aturonsince regardless of the label, we&#39;ll see it after 3 days of activity
22:32aturonand at that point relabel if needed
22:33shepk
22:33aturonthis is not designed for 100% accuracy
22:33aturonjust a fuzzy backstop :)
22:33aturonalright cool
22:33aturonwe&#39;re over time at this point
22:33aturonthanks so much everybody! i love how quickly this is coming together
22:34acrichtothanks again aturon for sheperding!
22:34simulacrumthanks aturon!
22:34misdreavus<3
22:34shep\o/
22:35frewsxcvWoo
22:35carols10cents!~*~!~*~!
22:38misdreavusi got &quot;lucky&quot; today and had a morning migraine shift my work schedule back a couple hours, so lurking for the whole meeting wasn&#39;t a problem :P
22:39misdreavusas far as &quot;future interests&quot;, i&#39;m personally interested in understanding the release process, especially w.r.t. the shift to travix/appveyor
22:39misdreavus**travis
22:40acrichtomisdreavus: it&#39;s my dream to write a blog post detailing this
22:40acrichto&quot;From commit to release&quot;
22:40acrichtodetails everything from our PR process to bors to homu to travis to appveyor to rust-central-station to the train model to backports to the actual release
22:40simulacrumacrichto: That would be amazing :)
22:40aturonomg yes please
22:41misdreavusditto, that sounds delightful
22:44frewsxcvIsn&#39;t that what the forge is?
22:45aturonah i assume acrichto had in mind a kind of higher-level post giving a world-facing overview of some of the ways rust has innovated here
22:45aturonbut yes both that and the full details should ultimately land on the forge
22:45acrichtolol of course I had that in mind
22:45acrichtoI&#39;m like 100% sure brson had that in mind
22:45aturonacrichto: you know i&#39;ve got your back :)
23:05ManishearthFWIW, while I do want to be more involved in infra, I don&#39;t think I&#39;ll be able to get involvd again till after rustfest. jetlag, work, and talk prep take precedence :)
23:09aturonManishearth: no worries, we&#39;ve got a ton of helping hands here
23:09aturonManishearth: just let me know if at some point you don&#39;t want your email on the list etc
23:10Manishearthno im fine with that :)
23:24frewsxcvManishearth: what&#39;s your talk on?
23:28Manishearth&quot;Open&quot; as in &quot;Welcoming&quot;: Facilitating Contributions
23:39simulacrumJust wanted to note that there may be some instances where I do stale PR checks on Sunday instead of Saturday
23:43frewsxcvManishearth: sounds like a good talk :)
23:43frewsxcvcool, mac builds aren&#39;t starting https://travis-ci.org/rust-lang/rust/builds/224481255
23:44frewsxcvrestart it?
23:45arielbywhere&#39;s the spreadsheet for spurious failures?
23:45arielbyfrewsxcv: we had that failure mode before
23:45arielbyand restarting fixed it
23:51frewsxcvrunning &#39;synchronize&#39;
22 Apr 2017
No messages
   
Last message: 1 day and 14 hours ago