mozilla :: #conduit

12 Sep 2017
14:28circleci-botSuccess: purelogiq's build (#77; push) in mozilla-services/mozphab (pull/29) -- https://circleci.com/gh/mozilla-services/mozphab/77?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
14:30* imadueme o/
14:44imaduemeckolos: fwiw that ^ build updated the contribute.json as well as added your fix to the Phabricator CSP policy which you found when deploying it (i.e. also including cloudfront in the default-src). It's all in the PR.
14:44imaduemeAlso, I've noticed that Lando-API-dev currently has the wrong phabricator api key environment variable set. Whenever you get a chance I can give you a new one to use. Thank you 100%
14:47zalunirc standup?
14:47dklyeah
14:48zalunI'm working on stacked revisions - reorganizing the way patches are requested and then saved and adding (mostly testing) timestamps in objects
14:48ckolosimadueme: it's using the only key I have; please do supply a new one if needed.
14:49davidwalshYesterday I reviewed DKL's PRs, met with imadueme, and created a PR for optional bug requirements
14:49davidwalshToday I will again review DKL's PRs and meet with imadueme
14:51davidwalshckolos: Can you look at the phab logs to ensure they are receiving our doorkeeper daemon log messages?
14:52ckolosdavidwalsh: sure, if you give me an idea of what to look for
14:53davidwalshckolos: This is an example format: https://gist.github.com/darkwing/83ed6d65cdc404222b7e0bb37ec499d2
14:53ckolosokay, give me a bit
14:54davidwalshckolos: I'm using PHP's error_log() function, which you said LGTM and you mentioned when to a specific process, so I just want to make sure we're on the same page
14:54davidwalshThanks!
15:03imaduemestandup: I'll be on my lando cards and working with marti_ if he has any questions. davidwalsh, lets me around the same time as yesterday if that works for you
15:07dklstandup: updating demo env with latest workflow changes
16:13davidwalshdkl: When I pull your BMO branch down into the machine, oddly enough, BMO gives me 500s
16:14davidwalshdkl: I also get permission issues when trying to look at any /var/log dirs
16:28davidwalshYet when I pull it down with vagrant everything works...OK then
16:34imaduemezalun: I'm going to land D44 (since it is approved and very simple) - to test my hypothesis as to why Phabricator isn't auto closing revisions, thanks
16:45dkldavidwalsh: for the logs do 'sudo tail ...'
16:46davidwalshSo it's not something you've done, updating master to current master makes the 500s happen too
16:46dkldavidwalsh: the 500 error is probably the missing perl lib added in recent updates. I need to do a rebuild
16:46dklrebuild of docker-bmo image
16:46davidwalshdkl: I completely rebuilt my machine this morning
16:46davidwalshOh
16:46dklwill do that now
16:46dklin the meantime you can do 'sudo /usr/local/bin/cpanm Algorithm::BloomFilter'
16:46dklwhich should fix it
16:46davidwalsh"Can't locate object method "params" via package "Bugzilla" at Bugzilla/Sentry.pm line 156."
16:47davidwalshThat...seems bad
16:47davidwalshdkl: omfg that worked
16:55imaduemeckolos/bobm: it ded :'( https://phabricator.services.mozilla.com/
16:55ckolosawesome
16:55ckolosworking on that now actually
16:56imaduemeah, thanks. Sorry for the troubles
16:56ckolosno worries at all, thanks for the heads up
16:56ckolosyou beat my monitoring ;)
16:57* imadueme ;)
16:57imaduemelast seen normal time was like 1253 est so not too bad
16:58ckolosrunning now
16:59imaduemeyou're running or phabricator is? haha. still ded :'(
17:03davidwalshFFS dkl is going to kill me
17:04davidwalshSo dkl: Do we want to squash and merge or rebase and merge? There's a bunch of stuff in your BMO commit
17:04dklsquash
17:05dklfor bmo stuff i normally do squash and merge
17:05davidwalshdkl: Baller work, the BMO patch is in
17:05davidwalshLooking at doorkeeper now
17:06davidwalshYeah, phab still dead for me
17:10dklnew docker-bmo image pushing. you will need to pull down the new one
17:13davidwalshdkl: Yessss, your doorkeeper stuff works too
17:15davidwalshdkl: One thing I did notice: If we change the bug number, the old BMO bug page still shows the attachment (and it's correctly added to the new bug)
17:15davidwalshdkl: Should we add a new card for obsoleting during a revision bug change?
17:18davidwalshckolos: Just a heads up that we're still down /me runs
17:19ckoloswhat?
17:19ckolosfrog
17:19ckolossec
17:19* ckolos finds a hole in monitoring
17:20ckolosdavidwalsh: try again please
17:20davidwalshYay, back
17:20ckolosyeah, dunno wtf.
17:20ckolossorry about that
17:33dkldavidwalsh: yeah we should do that for sure
17:49mcoteckolos: what went down out of curiosity?
17:50ckolosI needed to restart the journal and there's a dependency cycle between the docker process and the journal. When the journal was restarted, the container went down and didn't come back up.
17:50mcoteaha
19:08mcotestill getting a retweet or two of my post every day
19:08mcotethis resonated more than I expected
19:13davidwalshWe're up to 53 revisions in phab now; nice
19:37davidwalshimadueme: I can meet whenever you're ready
19:38imaduemedavidwalsh: Now works fine, I'll hop in your room
19:39ckolosI'
19:39davidwalshk
19:39ckolosdavidwalsh: I'm looking for your logging stuff and not seeing it in the daemon logs
19:40ckolosI am seeing, however, an Aphront Error
19:40davidwalshckolos: https://github.com/mozilla-services/phabricator-extensions/blob/master/logging/MozLogger.php#L15
19:41ckolosoh duh, (for me) that's still not live.
19:41davidwalshckolos: Oops, this: https://github.com/mozilla-services/phabricator-extensions/blob/master/logging/MozLogger.php#L18
19:41davidwalshckolos: OK, just making sure
19:42ckoloshere's the Aphront stack trace, or what there is of it. https://ckolos.pastebin.mozilla.org/9032254
19:55chartjesHey conduit folks -- this might be the wrong place to ask but where could I find details about if there is a BugZilla API I could use to automatically create and then delete users for some automated tests?
19:55mcotechartjes: http://bmo.readthedocs.io/en/latest/api/index.html
19:56mcotenot *quite* everything is documented there, but nearly
19:56mcoteI imagine everything you would need anyway
19:56mcotechartjes: dylan and dkl can answer specific questions here as well
19:57chartjesmcote: thanks
20:00mcotedkl: wish I could fix vidyo's resolution issue. I wonder what is different about your and my setups
20:17mcoteimadueme: so it's known that we can't run lando-ui outside of docker at all?
20:17imaduememcote: Where did you hear that?
20:17mcoteFileNotFoundError: [Errno 2] No such file or directory: '/app/version.json'
20:17mcotemy terminal ;)
20:18mcoteI installed a few apt packages, and then pip installed the common and dev requirements, ran setup.py develop, and then lando-ui-dev
20:18mcotewhich errored out
20:18mcotealso python setup.py test gives a similar error
20:19mcoteI don't mind trying to fix it if it should work with in a virtualenv without docker
20:19imaduemeAh I see, docker-compose helps set all the environment variables lando-ui depends on. So if you don't use docker you'll need to set all of those env vars in the shell. You can see the list in the docker-compose.yml file, there are a few as well which are left emtpy that you need to set mcote
20:19davidwalshmcote: Could just create a bogus app/version.json, no?
20:20imaduemeWith that said, I highly recommend you use docker-compose mark :)
20:20mcoteI just want to see what the barrier to contributing is :)
20:20mcotelike if I just want to do some javascript fixes or such, is it possible I could run it locally without the whole docker thing
20:20imaduemeIt is much easier to use docker that to not use docker, 100% guarantee or your money back.
20:21mcoteheh
20:21mcoteeasier, sure
20:21imaduemeWhen i write the getting started docs, I'll expect new contributors to install docker :). All the invoke commands (that run tests and so on) use docker too
20:22mcotewhat would be the exact reason, out of curiosity?
20:22mcotelike we have a relatively simple app here. would be nice if someone could do small changes without the whole docker thing
20:22imaduemeThe team decided early on to use docker on all the projects even in dev - all of our projects use docker :)
20:22mcoteuse docker, yes
20:22mcoteI didn't think we said we'd *require* docker
20:22mcotethere's a difference :)
20:23mcotelike maybe you need docker for a full experience
20:23mcoteI guess I'm curious how common it is now for apps to absolutely require docker for development
20:29imaduememcote: docker helps everyone get the exact same environment, run the app in exactly the same way, and gain access to the same invoke commands for testing, building, and so on. That's what docker does for this and all our projects. By no means is it required, but, not using it means you take on the burden of figuring out how to set the app up for your
20:29imaduemespecific environment.
20:29imaduemeA contributor that wants to make a small change simply updates the docker-compose.yml file and runs docker-compose up --build :), far simplier that the apt and pip commands you are doing
20:29mcotetrue
20:30imaduemeI will have to get on the getting started docs as I know contributors is one of our OKRs
20:30imaduememcote: fwiw this isn't my decision, the old conduit monorepo had the same thing going on, even our react app :)
20:30mcoteI may start those docs :)
20:30* imadueme o/
20:30mcotes/may/will/
20:31mcoteif I can stumble through :)
20:31mcotehm too bad the default docker for windows setup seems to require Windows 10 Professional
20:32imaduememcote: no vm? I use my vm like it is just a normal program. The linux subshell just isn't there for me yet
20:32mcoteheh more requirements for contributors...
20:33mcoteso now they need to install a VM to contribute? :\
20:33imaduemeNo, but they need a linuxy shell
20:34mcotewell there aren't too many options there. although I guess admittedly requiring a VM isn't much different from requiring the linux subshell
20:34mcotesubsystem
20:34ckolosyeah, but that requires turning on dev mode
20:34imaduemeNext thing you know they'll need a computer to contribute! C'mon guys!
20:35mcoteI *have* worked with a number of volunteer contributors over the years; I'm not just trying to make things complicated
20:36imaduemefair :)
20:36* ckolos doesn't think requiring docker is too much of an ask, but doesn't carry much weight in this discussion
20:37mcoteI'm not really trying to argue that it's too much to ask
20:37mcoteI am more wondering how difficult it would be to support local environment
20:39ckolosno snark intended, but doesn't that in itself imply support issues wrt general platform suppport?
20:39ckolosi.e. which platforms are reasonable for development of the product?
20:39imaduemeThere's nothing preventing local environment - really is it just installing the packages and setting the environment variables. You can see the Dockerfile-dev for what happens. The main downside is invoke commands won't work, but, you can just look at the command and run the same thing that it runs in the docker container mcote
20:39mcotehm
20:40mcoteckolos: true, but I don't *think* lando-ui has really complex requirements
20:40ckolostouche
20:40mcoteckolos: all the web apps I've contributed to are pretty system-independent
20:40mcotewell most anyway
20:41ckolosgiven the backend requirement, if you wanna make changes to lando-ui and they are somehow affecting the communication between ui and api, you're going to need docker or something to test
20:42ckolosother than that though, you're completely correct, I think.
20:42ckoloscss or js changes etc etc
20:43mcotewell, ideally our tests would mock out stuff
20:43mcoteI'm not sure what docker gives us in terms of interaction with the api server that we couldn't do locally
20:43ckolosthe ability to run the api server without needing the infrastructure
20:44mcoteright, and one goal we had in this project is not to require all the infrastructure to be stood up to do development
20:44mcoteqv the mozreview environment which was a source of serious development friction
20:45mcotewe may *benefit* from being able to stand it all up, but it certainly shouldn't be a requirement
20:45ckolosI'm not familiar at all with any of those issues, but could it be said that they stemmed from the complex requirements of the app?
20:45ckolosi.e. you need a db, and a repo, and a pony, and a wizard hat, etc, etc,
20:46ckolosvs. install docker, grab this file. go nuts.
20:46mcotethat's exactly my point. as the system gets more complicated, the setup, dockerized or not, gets increasingly complicated and difficult and time consuming
20:46mcoteeventually you end up spending tonnes of time refreshing your setup, running tests, etc.
20:47mcoteagain, we went this way with mozreview and it ended badly
20:47ckolossure thing
20:47mcotethe microservices approach, which we are vaguely trying to follow, suggests heavy use of mocking along with strong and backwards-compatible API contracts
20:48mcoteyou should be able to work on a service in isolation with relative confidence that it will not break the whole system
20:48imaduememcote: I think comparing 'lando-ui' requiring a service called 'lando-api' to mozreview is heartless :(.
20:48imaduemeThe amount of more work to stub every interaction with lando-api would be too great - we already stub known test cases, but, we cannot stub every interaction that can happen with the UI. Lando UI is basically a screenshot + error handling without lando-api.
20:48mcoteI dunno
20:48mcoteand I think words like "heartless" aren't particularly helpful here heh
20:49mcoteif we don't try to do this, then we have to always deploy lando-ui and lando-api together
20:49mcotethat's not ideal
20:49mcoteagain, I'm not making this stuff up; read the Building Microservices book
20:49mcoteit's very enlightening
20:50mcoteand no, you probably can't stub every interaction... but does that mean we have automated tests for the lando-ui/api combination that verifies every single interaction?
20:50smacleodimadueme: "pip install betamax"
20:50imaduemeNot to mention that by actually using lando api we can test with real data, real revisions, and so on.
20:50imaduemeI retract my heartless statement, apologies :sweat_smile:
20:50imaduemeWe don't have to deploy the two together. Lando UI depends on the API, one way. The api can update itself, release with some new feature and later the UI can release and take advantage of it.
20:50smacleodmcote: docker is required for our supported development
20:51smacleodmcote: if you want I'll write up the reasons in our tech stack doc, but that won't happen this week
20:51mcoteheh yes I am starting to realize that. but my greater concern is now the apparent lack of isolation between the ui and api in terms of development
20:51smacleod(key word, "supported", there is no reason you can't develop without it, we just don't want to write a million environment setup docs, and test with everything)
20:51mcotesure
20:52* smacleod goes back to his cave
20:52mcoteI will concede defeat on the docker thing. I don't care *that* much :)
20:54imaduememcote/smacleod: Lando UI 100% needs Lando API to function and develop at a fast pace. We can run betamax every single time we want to try something new. If the API updates then we have to remake all the affected betamax stuff. I'm really not trying to sugar coat it, you can't locally develop Lando UI without Lando API also running, it's just the truth :).
20:54imaduemeOn the other hand the API doesn't even know the UI exists so the microservice principle isn't completely broken.
20:54imaduemecant***
20:54mcoteI still disagree with that assertion
20:55mcoteanyway will continue this argument later :) gotta run for now
20:55imadueme(lando ui currently needs lando api at least), ok see ya!
20:55mcoteimadueme: please read that book at some point. :)
20:55mcoteit has a section on UIs specifically
20:55smacleodimadueme: I'm referring to "test with real data"
20:55smacleodinteractive development, sure
20:56mcoteyes of course
20:56imaduemeI'm arguing for interactive development, not testing
20:56mcotebut you said you can't mock out everything
20:56imaduemeTests will all be mocked out (and currently ar)
20:56mcoteso how do we test?
20:56mcotehaha
20:56imaduemeWe have canned responses of all lando api responses which the tests need :)
20:57imaduemeMaybe there has been a misunderstanding
20:57smacleodthere's no reason you couldn't have a canned responding server for interactive development
20:57imaduemeIt would literally be easier to just start the actual lando api server
20:57smacleodwhether or not that *should* be done is a separate argument. But the argument that everything can't be mocked is incorrect
20:58smacleodimadueme: possibly
20:58smacleodbut then, what about the services it talks to?
20:58smacleoddo you mock those? now you're mocking several services instead of one
20:58smacleodor if you start all of them, now you're running phabricator
20:58smacleodetc
20:59smacleodit's not a simple problem either way
20:59* smacleod isn't arguing either way right now, just brain dumping
21:05imaduemeI retract what I said about the ui 100% needing the api to run or develop.
21:05imaduemeMy main point is that the cost-benefit for mocking lando-ui -> lando-api doesn't seem to be worth it (too much to mock, little cost to run a lightweight api server).
21:05imaduemeOn the other hand mocking lando-api -> phabricator or lando-api -> bmo would be worth it (medium-high work, but also high cost of running those heavy services).
21:05imaduemeHope this makes sense to everyone :)
21:06smacleodimadueme: that sounds like a pretty good argument to me. Out of curiosity, do we have a lando api mode for mocked BMO / phab as of today?
21:06imaduemelando api doesn't talk to bmo, it does talk to phabricator but we use the hosted dev instance so there is no need to run it locally.
21:07imadueme(you can of course run phab locally and use it)
21:07smacleodah okay, so you're just pointing it at phabricator-dev.allizom?
21:07imaduemeexactly
21:08smacleodWhat about transplant?
21:08smacleodoh wait, I guess we have that mock transplant app right?
21:08imaduemewe created a mock service at http://fake-transplant.herokuapp.com/, yup
21:22imaduememcote: Just realized that you could just point your local lando-ui to the hosted lando-api dev instance. That would also make it easy for new contributors to get up an running on a single project :)
22:24marti_ping imadueme
22:33imaduemeI stepped out marti_ but I can answer your question when i get back!
22:34marti_how to run the project landoui
22:34marti_i am tryng to run python setup.py develop
22:34marti_and then lando-ui-dev
22:35marti_but It doesn't have from mozlogging import MozLogFormatter
22:37marti_mozlogging is in common.txt
22:37marti_i don't know what is the error
22:46marti_I had to install sudo pip install 'mozlogging'
22:47marti_but then it has an another error: AttributeError: 'module' object has no attribute 'getLogger'
22:50marti_I will back in 2 hours I am going to home in about 30 min
23:50mcoteimadueme: yeah okay you could persuade me of that (lando-ui/-api lightweight enough to not bother with too much isolation for dev)
23:50mcotelooks like marti_ was trying to run it locally though heh
23:50mcoteadmittedly we have no instructions so can't fault him
13 Sep 2017
No messages
   
Last message: 9 days and 10 hours ago