mozilla :: #balrog

11 Aug 2017
14:14aksareenbhearsum: replied to comment on PR
14:22bhearsumaksareen: i wonder if it would help to move the process_rule_form checks after the null/None checks
14:23aksareenbhearsum: found another thing. commenting on PR in 2 min
14:23bhearsumokay
14:23bhearsummy main goal here is improving the readability of this code, the specific suggestions i've made are just ideas about doing that
14:26aksareenbhearsum: commented on the PR
14:28bhearsumaksareen: can you sum it up, please? it sounds like you have an idea of how to simplify or otherwise make it more readable?
14:29cloudops-ansiblebalrog-web #148: building mozilla/balrog:master-28af89fcc56c7ed88669ef4c33df53161ef2ba76
14:30aksareenbhearsum: all the empty/blank strings will be converted to None values due to this line https://github.com/mozilla/balrog/blob/master/auslib/web/admin/views/rules.py#L36 . So we need to only check for Null values in mapping and fallbackMapping. Also I have added comments over the compound conditional checks so as to improve readability and understanding of the code in this commit :
14:30aksareenhttps://github.com/mozilla/balrog/pull/368/commits/6d8fb34b5a447fee2cc1d3da38285d5f6f491597
14:41cloudops-ansiblebalrog-web #148: mozilla/balrog:master-28af89fcc56c7ed88669ef4c33df53161ef2ba76 deployed to stage /cc relud bhearsum
14:44cloudops-ansiblebalrog-admin #178: master-28af89fcc56c7ed88669ef4c33df53161ef2ba76 deployed to stage /cc relud bhearsum
14:50aksareenbhearsum: should I make any changes ?
15:03bhearsumaksareen: i'm going to have another look shortly
16:02aksareenbhearsum: updated the PR
16:25ex-devbhearsum, I saw your comment on bugzilla(1381509), probably you can to close https://bugzilla.mozilla.org/show_bug.cgi?id=1387063 too
16:50bhearsumex-dev: i'll probably tweak it to talk about openapi 3.0
16:50bhearsumit's a good point though
17:15bhearsumaksareen: have you verified that none of the update bugs i found earlier are present in the latest version of the PR?
17:20aksareenbhearsum: yes. none of them occur
17:27bhearsumok, cool
17:28bhearsumthe code looks a lot better now - i'll do a final pass review
17:44bhearsumaksareen: alright, i think we're all set now!
17:47aksareenbhearsum: alright. So this wraps up the gsoc project. sorry I couldn't figure out the uwsgi+UI issue.
17:49ex-devPR #368 was merged!!! Very good aksareen !
17:49aksareenthanks ex-dev. well I'm off to sleep, got a flight to catch tomorrow.
17:49bhearsumthanks aksareen - it was a monster of a project
17:50bhearsumlots of twists and turns, and unexpected problems - you did well in handling them
17:51aksareenbhearsum: thanks for helping me out along the way. I'll be wrapping up the documentation + blog part over the next few weeks. Let me know if any code change needs to be done from my end
17:51bhearsumwill do
17:51bhearsumthis should go to prod on monday, so we should know soon if there's any issues we haven't caught
17:52aksareencool. I'll try and come online I get off early from work else you can comment on bugzilla link and I'll check it later.
17:52bhearsumsure
17:52bhearsumsafe travels
17:52cloudops-ansiblebalrog-web #149: building mozilla/balrog:master-e99b26ba54acb807ed699a63fa681447187cbcb8
17:55aksareenthanks bhearsum :D
18:04cloudops-ansiblebalrog-admin #179: master-e99b26ba54acb807ed699a63fa681447187cbcb8 deployed to stage /cc relud bhearsum
18:05cloudops-ansiblebalrog-web #149: mozilla/balrog:master-e99b26ba54acb807ed699a63fa681447187cbcb8 deployed to stage /cc relud bhearsum
18:09bhearsumrelud: if you have time today, do you think we could try simulating the next schema upgrade (the one that drops the should-be-totally-unused-now column) in stage?
18:12reludbhearsum: sure, what should i deploy to where and when?
18:13bhearsumrelud: i was hoping we could downgrade the stage code to 6d1c9d181c05c8acf521f6d045595fc4759c12a5, import the current prod db dump, and upgrade stage to the latest code again
18:14bhearsumi just need to double check order of operations for code/schema upgrade for the last part
18:15reludbhearsum: so how about i start by importing the prod db, and deploy 6d1c9d181c05c8acf521f6d045595fc4759c12a5 to admin and web, then when you are ready I deploy latest to admin only, then when you are ready I deploy latest to web. does that work?
18:16bhearsumdoes "deploy latest to admin" imply the database schema upgrade too? or just the code?
18:16reluddeploying admin runs database migration
18:16bhearsumah, we should upgrade web code first in that case
18:17bhearsumotherwise that sounds perfect
18:40bhearsumjust let me know when everything is downgraded
19:22bhearsumrelud: ^
19:52cloudops-ansiblebalrog-web #150: building mozilla/balrog:master-6d1c9d181c05c8acf521f6d045595fc4759c12a5
19:52reludbhearsum: okay, i imported the prod db, when balrog admin and web are deployed to stage, everything is downgraded ^
19:57bhearsumi see the db changes \o/
19:57bhearsumi'll have a look once the deploy finishes
20:02cloudops-ansiblebalrog-web #150: mozilla/balrog:master-6d1c9d181c05c8acf521f6d045595fc4759c12a5 deployed to stage /cc relud bhearsum
20:03cloudops-ansiblebalrog-admin #180: master-6d1c9d181c05c8acf521f6d045595fc4759c12a5 deployed to stage /cc relud bhearsum
20:07bhearsumrelud: ok, all good so far - we can upgrade web
20:15cloudops-ansiblebalrog-web #151: building mozilla/balrog:master-e99b26ba54acb807ed699a63fa681447187cbcb8
20:25cloudops-ansiblebalrog-web #151: mozilla/balrog:master-e99b26ba54acb807ed699a63fa681447187cbcb8 deployed to stage /cc relud bhearsum
20:26reludbhearsum: ^
20:27bhearsumlooking
20:27bhearsumyay, it's not 500'ing
20:27bhearsumwe can do admin now
20:27reludwoot :D
20:27cloudops-ansiblebalrog-admin #181: building master-e99b26ba54acb807ed699a63fa681447187cbcb8
20:37cloudops-ansiblebalrog-admin #181: admin in stage failed /cc relud
20:39reludbhearsum: https://irccloud.mozilla.com/pastebin/f0TQq5IW/
20:41bhearsumhmmm
20:41reludalso: admin deploys to stage now rebuild the stage db from the current prod dump
20:42bhearsumhm, when you're rebuilding the stage db are you just running the sql from the dump? or creating a whole new db?
20:42reludrunning the sql from the dump
20:42reludit drops and recreates tables
20:42reludthat's also what i did manually to import the prod db before that change got merged 5 minutes ago.
20:43bhearsumi don't think it does that for all tables, unfortunately
20:44bhearsumso we seem to be in a weird state where we're halfway between the two most recent versions of the db
20:45bhearsumi think we need to drop the db entirely, recreate it by hand at a specific version, and then import the dump
20:54reludbhearsum: so I drop the db, run update-db using the current prod container version on an empty db, import the dump?
20:57bhearsumrelud: we need to create it as the current prod version, so it's slightly different. drop the db, create an empty db, then run "python /app/scripts/manage-db.py -d $DBURI --version 27 create"
20:57bhearsumi'll do a quick sanity check with old db + new code
20:57bhearsumthen we import the dump
20:57bhearsumerr, sorry - switch those two
20:58bhearsumi'll sanity check after importing
20:58bhearsumthen we upgrade stage admin for the final check
21:02reludbhearsum: so i dropped the db, then created it, then ran your create command, then imported the prod dump
21:02* bhearsum looks
21:03reluddo i need to do this every time, or is it normally okay to just import the prod dump?
21:06bhearsumuntil https://bugzilla.mozilla.org/show_bug.cgi?id=1376331 is fixed, we'll need that procedure, except the --version arg needs to be current prod db version
21:07bhearsumack - need one more line run: 'insert into permissions (username, permission, data_version) values ("bhearsum@mozilla.com", "admin", 1);'
21:09bhearsumsorry about all this - obviously the process of recreating the db is a little busted right now
21:22bhearsumrelud: we can finish this up monday if you prefer, i realize i've taken up a lot of your time today
21:22reludbhearsum: no worries, my day goes much later than yours
21:22reludand i just added your user
21:22reludmissed that there was a command waiting for me
21:22bhearsumnp
21:23bhearsumokay, everything looks cool now - and AFAICT the database should upgrade correctly this time
21:23bhearsumready to try the admin deploy again
21:28reludbhearsum: okay, admin is updated
21:28reludmigration succeeded this time
21:30bhearsumhm, getting a 502 from https://balrog-admin.stage.mozaws.net/api/rules, and https://balrog-admin.stage.mozaws.net/__version__ shows the old version still
21:31reludsorry, missed a step. done.
21:31bhearsumnp
21:31bhearsumok, cool
21:32bhearsumnothing is blowing up now, and the db appears to be in the expected state
21:33bhearsumi think we can say the db migration will go fine in prod at this point
21:33bhearsumthank you a ton for working through this.
21:34reludsweet :D
21:34reludand np
21:34reludenjoy your weekend. imma break the admin for 5 min
21:34bhearsumhaha, alright
21:34bhearsumciao
21:41cloudops-ansiblebalrog-admin #182: master-e99b26ba54acb807ed699a63fa681447187cbcb8 deployed to stage /cc relud bhearsum
12 Aug 2017
No messages
   
Last message: 11 days and 23 hours ago