mozilla :: #screenshots

11 Aug 2017
02:32GitHub[screenshots] mozilla-pontoon pushed 1 new commit to master: https://git.io/v7yzk
02:32GitHubscreenshots/master 98a75c2 YFdyh000: Pontoon: Update Chinese (zh-CN) localization of Firefox Screenshots...
02:45cloudops-ansiblescreenshots-dev build #830: building mozilla/screenshots:latest
02:48cloudops-ansiblescreenshots-dev build #830: mozilla/screenshots:latest deployed to Dev
03:31GitHub[screenshots] mozilla-pontoon pushed 1 new commit to master: https://git.io/v7y28
03:31GitHubscreenshots/master 98605b7 Luna Jernberg: Pontoon: Update Swedish (sv-SE) localization of Firefox Screenshots...
03:44cloudops-ansiblescreenshots-dev build #831: building mozilla/screenshots:latest
03:47cloudops-ansiblescreenshots-dev build #831: mozilla/screenshots:latest deployed to Dev
04:31GitHub[screenshots] mozilla-pontoon pushed 1 new commit to master: https://git.io/v7ywe
04:31GitHubscreenshots/master 2217866 Adeel: Pontoon: Update Urdu (ur) localization of Firefox Screenshots...
04:31thunderanyone here work on the testpilot website? I'm trying to figure out how to make the install button
04:32thunderchecking user agent to display an install firefox vs addon button, etc
04:32thunderI got to here, but I don't understand what's happening:
04:32thunderhttps://github.com/mozilla/testpilot/blob/f9ed59f287ef3511d340fde3bdbb6f5ff0bbcd07/frontend/src/app/lib/addon.js#L17
04:45cloudops-ansiblescreenshots-dev build #832: building mozilla/screenshots:latest
04:48cloudops-ansiblescreenshots-dev build #832: mozilla/screenshots:latest deployed to Dev
08:10GitHub[screenshots] mozilla-pontoon pushed 1 new commit to master: https://git.io/v7yy4
08:10GitHubscreenshots/master 2b60163 Hvar Henriksen: Pontoon: Update Norwegian Bokml (nb-NO) localization of Firefox Screenshots...
08:23cloudops-ansiblescreenshots-dev build #833: building mozilla/screenshots:latest
08:26cloudops-ansiblescreenshots-dev build #833: mozilla/screenshots:latest deployed to Dev
10:22GitHub[screenshots] johngruen created spritify-buttons (+1 new commit): https://git.io/v7yxW
10:22GitHubscreenshots/spritify-buttons c32bf22 John Gruen: correct and spritify icon buttons
10:23GitHub[screenshots] johngruen opened pull request #3319: correct and spritify icon buttons (master...spritify-buttons) https://git.io/v7yxR
10:24GitHub[screenshots] johngruen force-pushed spritify-buttons from c32bf22 to 4f08f2f: https://git.io/v7yxr
10:24GitHubscreenshots/spritify-buttons 4f08f2f John Gruen: correct and spritify icon buttons
10:27GitHub[screenshots] johngruen commented on issue #3114: I'm gonna wontfix this for the sake of a minor victory https://git.io/v7ypJ
10:27GitHub[screenshots] johngruen closed issue #3114: Make homepage product tour headers bold https://git.io/vQ9v7
10:29GitHub[screenshots] johngruen closed issue #3137: Update full page save button to match selection save button https://git.io/vQbR5
10:29GitHub[screenshots] johngruen closed issue #3147: The loading spinner is not animated when the image is creating (dev) https://git.io/vQjBG
10:31GitHub[screenshots] johngruen commented on issue #3319: Fixes #2886 https://git.io/v7yp8
10:32GitHub[screenshots] johngruen commented on issue #3319: @ianb this hides favicons in CSS, i can take them out of the view too, but i'd rather figure out how to get them back. https://git.io/v7ypE
10:32circleci-botFailed: johngruen's build (#3027; push) in mozilla-services/screenshots (spritify-buttons) -- https://circleci.com/gh/mozilla-services/screenshots/3027?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
11:10GitHub[screenshots] mozilla-pontoon pushed 1 new commit to master: https://git.io/v7Sec
11:10GitHubscreenshots/master 511dcd2 Alfredos-Panagiotis Damkalis: Pontoon: Update Greek (el) localization of Firefox Screenshots...
11:23cloudops-ansiblescreenshots-dev build #834: building mozilla/screenshots:latest
11:27cloudops-ansiblescreenshots-dev build #834: mozilla/screenshots:latest deployed to Dev
11:31GitHub[screenshots] mozilla-pontoon pushed 1 new commit to master: https://git.io/v7Sf4
11:31GitHubscreenshots/master 8291fb3 Matja Horvat: Pontoon: Update Slovenian (sl) localization of Firefox Screenshots...
11:44cloudops-ansiblescreenshots-dev build #835: building mozilla/screenshots:latest
11:47cloudops-ansiblescreenshots-dev build #835: mozilla/screenshots:latest deployed to Dev
12:22GitHub[screenshots] jvehent opened issue #3320: Implement upper limit for number of shots in a single account https://git.io/v7Skl
12:24GitHub[screenshots] jvehent created zap-baseline-2 (+1 new commit): https://git.io/v7Skz
12:24GitHubscreenshots/zap-baseline-2 ddbef85 Julien Vehent [:ulfr]: Update ZAP command in circleci
12:25GitHub[screenshots] jvehent opened pull request #3321: [Do Not Merge] Update ZAP command in circleci (master...zap-baseline-2) https://git.io/v7Skg
12:32circleci-botFailed: jvehent's build (#3030; push) in mozilla-services/screenshots (zap-baseline-2) -- https://circleci.com/gh/mozilla-services/screenshots/3030?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
13:03GitHub[screenshots] jvehent force-pushed zap-baseline-2 from ddbef85 to 51ee9da: https://git.io/v7SqU
13:03GitHubscreenshots/zap-baseline-2 51ee9da Julien Vehent [:ulfr]: Update ZAP command in circleci
13:17circleci-botFailed: jvehent's build (#3031; ssh by jvehent) in mozilla-services/screenshots (zap-baseline-2) -- https://circleci.com/gh/mozilla-services/screenshots/3031?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
13:30GitHub[screenshots] jvehent force-pushed zap-baseline-2 from 51ee9da to 6570a44: https://git.io/v7SqU
13:30GitHubscreenshots/zap-baseline-2 6570a44 Julien Vehent [:ulfr]: Update ZAP command in circleci
13:39circleci-botFailed: jvehent's build (#3034; push) in mozilla-services/screenshots (zap-baseline-2) -- https://circleci.com/gh/mozilla-services/screenshots/3034?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
13:44GitHub[screenshots] jvehent force-pushed zap-baseline-2 from 6570a44 to 2724360: https://git.io/v7SqU
13:44GitHubscreenshots/zap-baseline-2 2724360 Julien Vehent [:ulfr]: Update ZAP command in circleci
13:45GitHub[screenshots] Zitronella opened issue #3322: Shot has no expire date on some websites https://git.io/v7SsY
13:49circleci-botFailed: jvehent's build (#3033; ssh by jvehent) in mozilla-services/screenshots (zap-baseline-2) -- https://circleci.com/gh/mozilla-services/screenshots/3033?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
13:49GitHub[screenshots] Zitronella commented on issue #3322: when created the gif i forgot to reload https://www.twodns.de/ but this is not from relevance. The wrong behavior is the same. https://git.io/v7Ssd
13:57GitHub[screenshots] jvehent commented on issue #3321: The ZAP scan now runs correctly and returns the following warnings that we should address before merging this PR.... https://git.io/v7SGK
14:02GitHub[screenshots] psiinon commented on issue #3321: Kind of depends on what the details are ;)... https://git.io/v7SZ8
14:12GitHub[screenshots] jvehent commented on issue #3321: # ZAP Scanning Report... https://git.io/v7SnK
14:13GitHub[screenshots] jvehent commented on issue #3321: # ZAP Scanning Report... https://git.io/v7SnK
14:14GitHub[screenshots] jvehent commented on issue #3321: # ZAP Scanning Report... https://git.io/v7Snb
14:16GitHub[screenshots] psiinon commented on issue #3321: Note that we can whitelist google-analytics and other 3rd party sites we want to use via baseline configs https://git.io/v7Sce
15:31jgruen_6a68 ianbicking stand up?
15:50_6a68ianbicking: https://bugzilla.mozilla.org/show_bug.cgi?id=1386333#c8
15:50firebotBug 1386333 FIXED, jhirsch@mozilla.com Remove Screenshots rollout pref
16:15_6a68slow motion car crash emoji
16:19clouserw_6a68: oh yeah, actually, I don't get reception at my new place, so, probably won't look
16:19_6a68haha, that's perfect
16:19ianbickingclouserw: you moving?
16:20clouserwianbicking: yeah! in a couple weeks
16:20clouserwoh, people just got here. /me afk
16:20ianbickingphew, if you showed up with a new background without announcement, might freak me out
16:23_6a68jgruen | ianbicking: hey, we didn't discuss the fact that our fix for the broken button issue will break the shutdown crash guard: https://bugzilla.mozilla.org/show_bug.cgi?id=1386333#c17
16:23firebotBug 1386333 FIXED, jhirsch@mozilla.com Remove Screenshots rollout pref
16:24_6a68I needinfo'd kris on the right way to fix that, haven't heard back yet, and didn't bring this up in the mtg :-\
16:25ianbicking_6a68: which broken button issue? Ive lost track of all these details
16:26_6a68ianbicking: lol, tell me about it. this one https://github.com/mozilla-services/screenshots/issues/3257
16:26_6a68this was where startup got double-invoked because of shield pref flip timing
16:27_6a68it turns out that synchronizing shutdown to the startup chain will delay the shutdown timer that kris is relying on to avoid the process crash
16:27ianbickingah
16:27_6a68(that's the comment 17 in the BMO bug above)
16:27ianbicking_6a68: well, maybe unconditional shutdown is fine?
16:27ianbickingthat was never clear though
16:27_6a68maybe in that case, but what about when the addon is upgraded?
16:28_6a68could an update be applied early enough to cause a race? if so, what happens?
16:28ianbicking_6a68: unconditional shutdown with the right reason?
16:28_6a68ah yeah, maybe. if APP_SHUTDOWN then just go for it?
16:28ianbickingi.e., on app shutdown, but not for other reasons
16:28_6a68I guess that adds another patch to get reviewed, landed, and uplifted
16:28ianbickingI have no confidence in anything, but maybe?
16:28_6a68which throws off the schedule
16:29_6a68haha, that's gold
16:29_6a68sums up my entire experience with the firefox side of this project
16:31jgruenianbicking: clouserw i think we don;t want to request uplift for beta 55 for https://bugzilla.mozilla.org/show_bug.cgi?id=1388805
16:31firebotBug 1388805 FIXED, ianb@mozilla.com Update Screenshots to version 10.11.0
16:31jgrueni think we are going to use balrog instead
16:31jgruenright?
16:31ianbickingjgruen: eh? I already requested uplift, but I could revert that of course
16:31_6a68jgruen: I think we need to make sure it's ok to use balrog instead
16:32_6a68there was some pushback in wil's screenshots maintenance doc about not having stuff in the tree
16:32ianbickingjgruen: if we really want to just practice, we could make a 10.12 with no changes
16:32_6a68I need to change the shutdown code to prevent crashes though, so we need a 10.12 anyway
16:35jgruenwait can you explain this to me
16:35jgrueni need to write an IIS
16:36_6a68jgruen: sure, have a look at the scrollback starting at :23
16:36ianbickingjgruen: _6a68 has a theory about how to fix this issue, per John-Galts suggestion, and so we could make a 10.12 that includes that
16:38_6a68that BMO comment, and the fact that we can accidentally break something way deep in the stack, is rather unsettling tbh
16:39jgruenwhich BMO comment?
16:39_6a68jgruen: https://bugzilla.mozilla.org/show_bug.cgi?id=1386333#c17
16:39firebotBug 1386333 FIXED, jhirsch@mozilla.com Remove Screenshots rollout pref
16:40_6a68jgruen: 10.11 prevents a broken button by waiting for startup to complete before calling shutdown, but it turns out we have to shutdown immediately if firefox is exiting, otherwise the "shutdown blocker" (timer that kills the content process) won't run in time
16:40_6a68so I need to change the shutdown code and call it 10.12
16:40ianbicking10.10 made the startup change, right?
16:40_6a68but the very fact that this step is required...is a wee bit terrifying?
16:40jgruenwhat does that mean in an actual build of firefox?
16:40jgruenlike how would i see this error
16:40_6a68jgruen: I...have no idea?
16:40_6a68but if we don't land the fix, talos will continue to crash, I think
16:41_6a68I didn't realize how little I understand shrinkwrapped desktop software until just recently
16:41ianbickingthose ts_paint fixes, I guess? I forget exactly how to run those locally, and even then...
16:41_6a68the errors only occur on windows builds
16:41_6a68I think?
16:41_6a68I don't even know at this point
16:41ianbickingoh yeah
16:42_6a68I'm curious if starting firefox, then closing it right away, will break stuff for end users
16:42_6a68but if a content process crash is NBD, maybe none of this matters
16:46GitHub[screenshots] 6a68 opened issue #3323: On Firefox shutdown, immediately shut down webextension https://git.io/v7SMo
16:47GitHub[screenshots] 6a68 created 3323-addon-shutdown (+1 new commit): https://git.io/v7SM9
16:47GitHubscreenshots/3323-addon-shutdown 483b28b Jared Hirsch: Fix #3323, immediately exit when Firefox is exiting
16:49jgruen_6a68: 6:48 PM /me RyanVM feels like the best path forward is going to be trying to reproduce locally so it can be caught in a debugger
16:49jgruenRyanVM suggests running Talos tests against that patch locally
16:51_6a68jgruen: wait, what? like, we should be the ones to bisect m-c to find the breaking change?
16:51_6a68I mean, I guess I could try to figure out how to do that on my windows laptop
16:52jgruenTIL that is a thing
16:52_6a68IIRC though, Talos config on the server is generally not reproducible locally
16:52_6a68so it may not be worth doing
16:52_6a68we could try kicking off N jobs at different points in the timeline, and bisect it correspondence chess style
16:53_6a68"which of the 10 jobs failed, which didnt, now bisect 10 more steps in there"
16:53_6a68RyanVM: o hai
16:53RyanVMhey
16:53jgruenRyanVM: i think we have questions about your run talos locally suggestion
16:54_6a68RyanVM: jgruen was just saying we should try to bisect locally, but I'm not sure Talos results are the same locally vs on Try
16:54RyanVMI wouldn't trust the numbers
16:54RyanVMbut for "does it crash?" I don't see an issue?
16:54_6a68oh, okay
16:54_6a68so we'd need to bisect between which two revs?
16:54* RyanVM would suggest a windows dev try building Beta locally with the patch applied and see what happens when running talos other
16:55RyanVMnot for bisection, just trying to catch the crash in a debugger
16:55_6a68ohhhh
16:55_6a68hoo boy, I guess I'd need to get visual studio configured?
16:55_6a68this is sounding like a multi-day thing
16:55RyanVMI'm assuming it'll hit locally, which could be totally wrong too if it is indeed a timing issue :\
16:55_6a68heh
16:56circleci-botFailed: 6a68's build (#3036; push) in mozilla-services/screenshots (3323-addon-shutdown) -- https://circleci.com/gh/mozilla-services/screenshots/3036?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
16:56RyanVMgetting up and running with a build environment shouldn't be too bad if you already have a windows machine handy
16:56_6a68I do, and I think I have the right visual studio version installed
16:56_6a68RyanVM: so you're suggesting we take beta with that patch applied, and see if we can trigger a shutdown crash locally?
16:56RyanVMyes
16:57_6a68OK, I'll give it a go
16:57RyanVMI can keep trying to bisect on Try, but as of now, I'm not entirely sure what we'll find if anything, so I don't want people waiting on something from me
16:57_6a68Yeah, and QA hasn't observed any actual crashes
16:58_6a68so it may not be worth it, since kris is going to bump the shutdown blocker timer down from 1 second to something shorter
16:58RyanVMright now, the only real data points we have are that Beta was nearly 100% reproducible in CI with that patch applied
16:58RyanVMNightly57 simulated as Beta doesn't reproduce
16:58_6a68RyanVM: ok, and the crash only happened in that one Talos test, where sessionrestore didn't fire, is that right?
16:58RyanVMbut I haven't managed to get any form of Nightly to reproduce either
16:58* _6a68 is having trouble keeping track of all the weirdness
16:59RyanVMall the crashes were in ts_paint
16:59_6a68ok
16:59_6a68RyanVM: do you happen to know anything about attaching a debugger to a Talos run?
17:00_6a68or maybe do you know who to ask?
17:00RyanVMI would have suggested jmaher, but I don't think he's around
17:00RyanVMmconley might, actually
17:01RyanVMhrm, I think he's PTO too
17:01RyanVMmaybe try asking in #fx-team or #developers
17:03_6a68cool, I'll see if I can figure it out, ask around if not :-)
17:03_6a68RyanVM: thanks again for all your help trying to pin down this issue
17:08_6a68ianbicking: any idea what this error means? https://circleci.com/gh/mozilla-services/screenshots/3036?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
17:09_6a68" Error: Failed to install add-on: Error: Add-on screenshots@mozilla.org is not compatible with application version. add-on minVersion: 51.0a1. add-on maxVersion: *."
17:09_6a68isn't circle supposed to download the latest nightly?
17:09ianbicking_6a68: yeah, thats weird; rebuild?
17:10_6a68ok
17:12GitHub[screenshots] ianb pushed 1 new commit to spritify-buttons: https://git.io/v7SS0
17:12GitHubscreenshots/spritify-buttons 7130bcd Ian Bicking: Fix sass-lint error (leading 0)
17:16jgruenianbicking: _6a68 can one of you follow up about getting that SA to flip a user pref?
17:16_6a68jgruen: yeah, I got it
17:17jgruencool, thx
17:17_6a68np
17:17RyanVM_6a68: OK, I think I can work around the bustage some of my recent Try pushes are hitting
17:17RyanVMbasically, the 57-as-Beta builds got broken for a span of about 4 days due to some Photon refactoring
17:19circleci-botFailed: 6a68's build (#3037; retry by 6a68) in mozilla-services/screenshots (3323-addon-shutdown) -- https://circleci.com/gh/mozilla-services/screenshots/3037?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
17:20_6a68RyanVM: oh really?
17:20_6a68that's great news
17:20circleci-botFailed: ianb's build (#3038; push) in mozilla-services/screenshots (spritify-buttons) -- https://circleci.com/gh/mozilla-services/screenshots/3038?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
17:20_6a68do I not need to try to debug that crash further, then?
17:20RyanVM_6a68: no, keep going
17:21_6a68or were there just multiple broken things
17:21_6a68ok, lol
17:21RyanVMthis just get me maybe bisecting on Try again
17:21ianbicking_6a68: huh, I guess rebuilding didnt help. If you look two sections up from the failure, clearly the firefox version is 57.0a1, so the error doesnt seem right at all
17:21_6a68I tell you, web stuff is awfully different from client software
17:21ianbickingthank god someone makes browsers so everyone doesnt have to make client software
17:21_6a68lol
17:22ianbicking_6a68: so, the tests also fail for me locally on master today, yay?
17:23_6a68huh
17:23ianbickingIll poke a bit
17:24ianbickingpretty big layout change landed today...
17:25RyanVM_6a68: OK, so https://treeherder.mozilla.org/#/jobs?repo=try&revision=353f6195b1afe5360ba915d85f72baabfe5c7d31 will hopefully reproduce the crashes. If so, that gets Try bisection back on track at least.
17:25RyanVMthat's the first merge to m-c that includes the pref change
17:25_6a68sweet.
17:25RyanVMif *that* hits talos crashes, then we can bisect what fixed it (since we know m-c tip is green anyway)
17:25_6a68nice
17:26RyanVMwon't promise a root cause even then (if it's timing related, any random change could have changed things), but can't hurt
17:26ianbicking_6a68: oh, webextension-only landed today!
17:26_6a68nice
17:27ianbickingsomeday well be only a webextension
17:27_6a68ianbicking: does that mean we need the addon signed for circle-ci to be able to test it?
17:27ianbickingsomeday
17:27_6a68haha right
17:27ianbickingI think there might be a pref
17:29GitHub[screenshots] ianb created fix-tests-again (+1 new commit): https://git.io/v7SHW
17:29GitHubscreenshots/fix-tests-again 1c319a4 Ian Bicking: Fix tests, enable legacy extensions via pref during tests
17:29GitHub[screenshots] ianb opened pull request #3324: Fix tests, enable legacy extensions via pref during tests (master...fix-tests-again) https://git.io/v7SHR
17:31GitHub[screenshots] ianb commented on issue #3319: I'm fine leaving in the favicon code, we'll get them back before too terribly long (probably after some add-on updates to save favicons with the shot). https://git.io/v7SHX
17:31GitHub[screenshots] ianb closed pull request #3319: correct and spritify icon buttons (master...spritify-buttons) https://git.io/v7yxR
17:31GitHub[screenshots] ianb deleted spritify-buttons at 7130bcd: https://git.io/v7SHD
17:38GitHub[screenshots] ianb opened issue #3325: Create access/authorization tests https://git.io/v7SQ9
17:40circleci-botFailed: ianb's build (#3040; push) in mozilla-services/screenshots (master) -- https://circleci.com/gh/mozilla-services/screenshots/3040?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
17:46GitHub[screenshots] 6a68 closed pull request #3324: Fix tests, enable legacy extensions via pref during tests (master...fix-tests-again) https://git.io/v7SHR
17:46GitHub[screenshots] 6a68 deleted fix-tests-again at 1c319a4: https://git.io/v7S7X
17:50jgruenianbicking: how about that sprite sheet...talk about perf wins on the server!
17:50ianbickingjgruen: admittedly I merged that on faith, but yeah!
17:51jgruenianbicking: trust me, it's hella awesome
17:52GitHub[screenshots] ianb commented on issue #3318: We could add something to db.js to log all the queries. The results would go to mozlog, so we'd want to aggregate them from there somehow https://git.io/v7S5X
17:54GitHub[screenshots] ianb opened issue #3326: Create production XPI with stage release https://git.io/v7Sdv
17:57_6a68ianbicking: have you looked at my shots perf at all? are there any easy wins on the front-end? CDN maybe?
17:59cloudops-ansiblescreenshots-dev build #836: building mozilla/screenshots:latest
17:59circleci-botFixed: 6a68's build (#3041; push) in mozilla-services/screenshots (master) -- https://circleci.com/gh/mozilla-services/screenshots/3041?utm_campaign=chatroom-integration&utm_medium=referral&utm_source=irc
18:01_6a68rhelmer: hey, who owns the one-off-system-add-ons repo on github? I'm curious if an xpi is required or not
18:02_6a68I need to make an update to the screenshots enable xpi
18:02rhelmer_6a68: there are a set of folks who can review... we don't tend to land XPIs in the github repo, just attach them to bugs to get signed and shipped etc.
18:02cloudops-ansiblescreenshots-dev build #836: mozilla/screenshots:latest deployed to Dev
18:02rhelmercory owned it before so I guess it's rdalal now? :)
18:03_6a68rhelmer: cool, I'll just open a PR then
18:03rhelmerwfm
18:04jgruenianbicking _6a68 https://telemetry.mozilla.org/crashes/
18:05jgruenif you click on any of the 56 cells beta crashes go nuts
18:05jgruenthat's the day 56 went beta
18:05jgrueni wonder if we're falling victim to the larger beta thing
18:05jgruenor, worse, causing it
18:05_6a68jgruen: Ryan-VM was just saying that he had to bisect around 4 days of breakage due to Photon refactorings
18:06_6a68I think we've been backed out of beta, so it should be easy to tell if we were the problem
18:06jgruenall of screenshots?
18:06RyanVMwe never shipped a 56 beta w/ the pref flip in it
18:06jgruenRyanVM: right
18:09_6a68right, the pref flip was backed out
18:09_6a68and the shield studies all ended, we can track down the dates there
18:10ianbicking_6a68: I havent looked super close, mostly I was focused on the slow server times which I wasnt able to reproduce
18:10ianbickingrelud: would it be possible to copy the prod database to stage? Were seeing production performance problems that we cant replicate, which Im guessing are due to the database size on prod
18:10_6a68ianbicking: I strongly suggest we not use prod data for testing
18:10_6a68there are all kinds of privacy concerns there
18:11ianbicking_6a68: I think stage is similarly secured?
18:11ianbicking_6a68: we could try to load up dev too
18:11reludianbicking: stage is not similarly secured
18:11reludand dev is definitely not
18:11ianbickingwell then no
18:12ianbickingwe could use load_test_exercise to load up dev
18:12ianbickingprobably it is close enough to prod for testing performance?
18:12_6a68yeah, reproducible tools sound good
18:13ianbickingI could try harder to reproduce it locally, but I dunno
18:13_6a68wow, prod is really slow!
18:13* _6a68 generates a waterfall, just to see
18:14ianbickingrelud: could you give us a count(*) of all the tables on prod, to give us a sense of scale?
18:15jgruen__6a68 trying to compose a release drivers email
18:15jgruendo you have any sense of a patch that would resolve the ts_paint crashes, or nah
18:17_6a68jgruen: hmm
18:17_6a68jgruen: oh, yes, one sec
18:18jgruenianbicking: wrt perf, you think most of the hit is coming from fetching from the DB?
18:18jgruendev/stage do feel a lot faster to me
18:18ianbickingjgruen: I think so, given that its only a problem on prod
18:19ianbickingand its not related very much to how many shots you have
18:19jgruenno, i wonder if pushing the deferred fetch/loader stuff to prod would make it at least feel better
18:19ianbickingit should, sure
18:19ianbickingthats why we did it!
18:19jgruenit feels a ton better on stage
18:20ianbickingthis is the query:
18:20ianbickinghttps://irccloud.mozilla.com/pastebin/elhdMEzE/
18:21jgrueni wonder if we could drop users who have no shots stored anymore
18:21ianbickingmaybe theres no key on data.deviceid
18:21ianbickings/key/index
18:21_6a68yeah, if it's a database issue, there has to be a full scan in there somewhere
18:21ianbickingwell that would do it
18:21_6a68haha yup
18:22_6a68where is expire_time set?
18:22_6a68you don't select it from data
18:22ianbicking_6a68: creation or if you edit the expire time
18:22_6a68cool
18:22_6a68I meant, I don't see you defining expire_time in your SELECT statement
18:22ianbickingI guess we dont show expire time on My Shots, so it never ended up in there
18:22ianbicking_6a68: oh, data.expire_time
18:23_6a68yeah
18:23ianbickingrelud: we have a stage release thats ready to go, maybe we can take that to prod? Then ill add this index and make a release just for that
18:24_6a68ianbicking: I haven't looked through the schema, is it a missing index for reals?
18:24_6a68that's like, the best news ever
18:24reludianbicking: okay, right now? is it important this go out on a friday? i won't be near a laptop during the day this weekend.
18:24ianbickingrelud: we could wait until monday, yeah
18:25reludokay, i'll set a reminder to deploy this monday
18:25ianbickingI could also roll a new release now, get it to stage, and we take that to prod monday
18:25jgruen_6a68: wrt my earlier question
18:26_6a68aklsdjnkladjfnsldsf
18:26_6a68yes, please, go ahead
18:26jgruendo you have even a rough concept of what a patch to resolve the ts_paint crash would look like?
18:26_6a68yes
18:26jgruendocumented or mental?
18:27jgruentrying to capture SotA for release-drivers
18:27_6a68jgruen: give me 10 more seconds
18:27jgruenkk
18:27jgruenthanks, sorry for being a pain in theass
18:27jgruenor the ass
18:27GitHub[screenshots] 6a68 opened pull request #3327: Fix #3323, immediately exit when Firefox is exiting (master...3323-addon-shutdown) https://git.io/v7SAx
18:28_6a68jgruen: oh no, your question is very reasonable
18:29jgruenoof, i wish our circleCI wasn't failing everything
18:29ianbickingit didnt start passing again?
18:29jgruenmakes it hard to link to a patch in an email that goes to VPs
18:29jgruenianbicking: https://github.com/mozilla-services/screenshots/pull/3327
18:29jgruenwomp womp
18:29ianbicking_6a68: can you rebase https://github.com/mozilla-services/screenshots/pull/3327 off master?
18:29_6a68jgruen: ok, let me try to provide enough context here: 10.10 was a fix for https://github.com/mozilla-services/screenshots/issues/3257
18:30_6a68but here, kris mentions that waiting to shutdown until startup is done is actually really bad for these talos test crashes https://bugzilla.mozilla.org/show_bug.cgi?id=1386333#c17
18:30firebotBug 1386333 FIXED, jhirsch@mozilla.com Remove Screenshots rollout pref
18:31_6a68the PR I just opened, https://github.com/mozilla-services/screenshots/pull/3327, fixes this by immediately shutting down if the reason is APP_SHUTDOWN (other reasons would be, say, that you're shutting down in order to upgrade, and we do still want to synchronize the start/stop flow)
18:31jgruenreally bad for === causes?
18:32_6a68if we export this to FF, and kris's reasoning is correct, we should see a normal talos crash rate
18:32jgruenkk, i guess can we tag 10.12 and merge it into nightly?
18:32_6a68on the other hand, Ryan-VM found that the crashes stopped happening somewhere between 56 and 57, and he's bisecting Try to figure out why
18:33jgruenright
18:33_6a68I think that's a complete account of events, but I'm not sure anymore
18:33RyanVM_6a68: if you want help running any speculative fixes through Try, LMK
18:33jgrueni think i have everything need to know, _6a68 what's the fastest way to do talos tests against 10.12 in beta?
18:33_6a68RyanVM: does regular Try syntax work? How many retries would I want to use?
18:34RyanVMI've been using |try: -b o -p win32,win64 -u none -t other-e10s[Windows 7,Windows 10] --rebuild-talos 5|
18:34_6a68jgruen: for me to export that patch on top of beta, then push to Try manually, then we wait like half a day for Talos to run
18:34_6a68RyanVM: perfect, I'll use that syntax
18:34RyanVMdon't forget to re-include the pref flip patch too
18:34GitHub[screenshots] ianb created add_deviceid_index (+1 new commit): https://git.io/v7SxK
18:34GitHubscreenshots/add_deviceid_index afca1b3 Ian Bicking: Add an index on data.deviceid to speed up My Shots...
18:34RyanVMlink me to the run as well so I can make sure you don't end up with broken windows builds (long story - trust me)
18:34GitHub[screenshots] ianb opened pull request #3328: Add an index on data.deviceid to speed up My Shots (master...add_deviceid_index) https://git.io/v7SxP
18:34_6a68RyanVM: ha! ok, I'll do the export now, one sec
18:35_6a68jgruen: do you have enough information? anything still unclear?
18:35jgruen_6a68: i think i am square
18:35GitHub[screenshots] ianb pushed 1 new commit to 3323-addon-shutdown: https://git.io/v7SxN
18:35GitHubscreenshots/3323-addon-shutdown a729aed Ian Bicking: Merge branch 'master' into 3323-addon-shutdown
18:35_6a68jgruen: also we're landing code to update the system addon that flips prefs (for balrog). I opened a bug, it's https://bugzilla.mozilla.org/show_bug.cgi?id=1389584
18:35firebotBug 1389584 NEW, nobody@mozilla.org Update pref-flipping system addon for Balrog-driven rollout
18:35_6a68there's code written, I just need to find a reviewer
18:36_6a68rhelmer: hey, would you be comfortable reviewing https://github.com/mozilla/one-off-system-add-ons/pull/53?
18:36rhelmersure
18:36_6a68thanks!
18:36ianbickingjgruen: tests are re-running on that PR, should be green when they are done
18:43_6a68rhelmer: thanks!
18:46GitHub[screenshots] ianb force-pushed 3323-addon-shutdown from a729aed to 916c353: https://git.io/v7Shm
18:46GitHubscreenshots/3323-addon-shutdown 916c353 Jared Hirsch: Fix #3323, immediately exit when Firefox is exiting
18:49jgruenianbicking _6a68 email sent
18:49jgruenthat sucked to write
18:49jgrueni am gonna go eat dinner now :)
18:50_6a68nice, have a good one
18:50jgruenill prob be back later
18:50jgruendepending on responses
18:51ianbickingjgruen: if you must! But you neednt, it is what it is
18:51_6a68yeah, wil's out too. seems like a good time to go afk
19:05GitHub[screenshots] 6a68 pushed 1 new commit to master: https://git.io/v79e2
19:05GitHubscreenshots/master 219392c Ian Bicking: Add an index on data.deviceid to speed up My Shots (#3328)...
19:05GitHub[screenshots] 6a68 deleted add_deviceid_index at afca1b3: https://git.io/v79ea
19:20cloudops-ansiblescreenshots-dev build #837: building mozilla/screenshots:latest
19:23cloudops-ansiblescreenshots-dev build #837: mozilla/screenshots:latest deployed to Dev
22:20_6a68ianbicking: could i get a review of https://github.com/mozilla-services/screenshots/pull/3327?
22:20_6a68or better, John-Galt, would you mind taking a look at https://github.com/mozilla-services/screenshots/pull/3327? 5-line diff
22:23John-Galt_6a68: lgtm
22:30GitHub[screenshots] ianb closed issue #3323: On Firefox shutdown, immediately shut down webextension https://git.io/v7SMo
22:30GitHub[screenshots] ianb deleted 3323-addon-shutdown at 916c353: https://git.io/v794c
22:31GitHub[screenshots] mozilla-pontoon pushed 1 new commit to master: https://git.io/v794B
22:31GitHubscreenshots/master c4cd4f7 Georgianizator: Pontoon: Update Georgian (ka) localization of Firefox Screenshots...
22:42cloudops-ansiblescreenshots-dev build #838: building mozilla/screenshots:latest
22:45cloudops-ansiblescreenshots-dev build #838: mozilla/screenshots:latest deployed to Dev
22:45cloudops-ansiblescreenshots-dev build #839: building mozilla/screenshots:latest
22:48cloudops-ansiblescreenshots-dev build #839: mozilla/screenshots:latest deployed to Dev
23:44GitHub[screenshots] 6a68 commented on issue #3327: got R+ from kmag in irc: http://logs.glob.uno/?c=screenshots#c20946... https://git.io/v79zy
23:45GitHub[screenshots] 6a68 commented on issue #3327: oh cool https://git.io/v79zS
12 Aug 2017
No messages
   
Last message: 5 days and 22 hours ago