mozilla :: #fx-team

21 Apr 2017
00:09MossopNightly performance seems terrible lately. Random hangs in webpages going on and I'm too uneducated to read profiles to figure out why :(
02:09AsaMossop: I'm see ing the same thing
02:09Asadisabled my two extensions and still seeing it
09:36pastAsa, Mossop: I've been seen this too, but all profiles I've looked at point to bug 1331680 which is well known. Reducing content processes to 1 actually improves startup time for me
09:36firebot NEW, Consider not doing sync IPC for document.cookie getter/setter
09:36pastI'd be interested to look at any profiles you may have
10:22timdreamdolske: re: emojione: where did you saw that? It's still CC-by-4.0
10:34mikedeboertimdream: that's for the v3 release:
10:49timdreammikedeboer: mikedeboer: ah i see
16:19Mossoppast: I have a profile with a massive event processing delay on the tab thread. How do I pass that over to you?
16:20MossopMy rough look suggests most of the time is in cycle collection
16:21pastMossop: there should be a Share button on top
16:23pastthanks, I'll take a look after this meeting
16:48MossopThe problem with having the profiler be a web page is that when your browser is constantly hanging for multiple seconds it becomes really frustrating to work with :(
16:51florianMossop: click the "Share" button as soon as you can, and then look at the profile from another machine (that's what I usually do)
16:55MossopAsa: Do you use Google Inbox?
17:37AsaMossop: nope. is that your slowdown?
17:37MossopAsa: I thought it was for a moment but apparently not. What add-ons do you use?
17:39AsaI disabled my add-ons yesterday. still experiencing the slowdown. seems nightly is using some of my cpu all the time and that's causing pauses and glitches in scrolling.
17:46MossopAsa: For me the slowness has gone away after a restart so it might be something that builds up over time. Don't know if you saw that
17:49AsaI have only restarted to apply updates so didn't pay close attention. I'll try that and monitor more closely
18:08ehsanaswan, bsmedberg, hello
18:09ehsanaswan, (I just pinged bsmedberg on a different channel to ask him about our discussion earlier today)
18:09ehsanbsmedberg, so basically the summary is that the plan on the bug there is to have an info bar that directs people to a page that tries to explain that we have disabled non-MPC extensions on Nightly in order to collect perf telemetry data
18:09aswanokay is there another channel i should join?
18:10ehsan(no here is fine)
18:10ehsanand tries to either sway them to using beta instead to keep their add-ons working, or set a hidden pref through about:config to make the add-ons work again on Nightly
18:11ehsanand there will be a scalar probe submitted in the telemetry ping indicating whether the user has any non-MPC extension
18:12ehsanbsmedberg, now if we decide to do this, there is a chance that a significant portion of users with these add-ons end up setting that pref and us in the end not being able to get useful telemetry data from this effort
18:12ehsananother risk aspect is swaying Nightly users to beta and never getting them back to the Nigthly channel
18:12bsmedbergjust so I'm clear: are we also disabling shims as part of this?
18:12ehsanbut all of these are "soft" concerns and there is no way to know for sure which way things will move before we do this
18:13bsmedbergbecause as I understand nightly config, e10s is on for everyone (regardless of MPC setting)
18:13ehsanbsmedberg, no the shims will be enabled and will still be used if someone enabled the pref allowing a non-MPC add-on to use them
18:13ehsanaswan, (please correct me if I'm wrong there^)
18:13aswanthat's correct
18:13bsmedbergthat's the part I still have concerns about
18:14ehsanafaik the plan for the shims is to remove them on or after 57
18:14aswani know we don't want to drag this out but we could add that telemetry bit now and start getting some infromation about how many users this potentially affects
18:15aswanie how many (or what fraction of) nightly users have extensions that are not MPC
18:15ehsanaswan, are we worried about the number of users being affected on nightly or about the rage factor issues?
18:15ehsan(since the number of nightly users in general is fairly small...)
18:16aswani'm not sure i understand the difference. rage factor in the addons community is already pretty high, its hard to see it getting a lot worse
18:17aswani don't feel strongly either way, but others feel strongly that we shouldn't irreversibly turn them off without some notice
18:17* andym is one of those
18:18ehsanaswan, well the difference is that the rage factor is very hard to measure through telemetry :)
18:18bsmedbergMy goal is to use nightly as a reliable testing mechanism for 57/quantum.
18:19bsmedbergWhat that means in practice is I need as much population with the following characteristics:
18:19bsmedberg1) no shims ever
18:19bsmedberg2) no ghost windows
18:19bsmedbergso realistically what I want is most of nightly to be webextensions-only starting ASAP
18:20bsmedbergI'm happy to have an escape valve as long as ~nobody uses it
18:20ehsanaswan, andym: my point is, we need to be straightforward about what informs our decision, if we're only worried about the rage factor, counting the number of users affected by changes isn't useful
18:21ehsanbsmedberg, I think my goals are similar to yours, but andym and aswan's are different (not enraging users earlier than we have to) and I understand those goals also
18:21andymits multiple broken promises and the different goals for nightly
18:21ehsansadly I don't know where the line in the sand must be drawn :(
18:21ehsanand who should draw it
18:22andymwe can push this out and see how many people reverse it
18:22aswani see. i'm personally not that worried about the rage factor, mostly that reasonable users don't get a really terrible experience
18:23andymif enough do, its a sign that people need those add-ons
18:23aswanalso, i havent' spent enough time hands on with telemetry to know how realistic this is but if we have the bit that indicates whether there are any non-mpc extensions, those users can be excluded from relevant perf statistics
18:24aswanthat's probably naive though and rewriting queries and dashboards and such takes non-zero effort
18:25dolskesorry, what's "MPC"?
18:26aswanmultiprocess compatible
18:26evilpieoh the compact theme is really nice
18:26aswansomething that isn't mpc uses shims which end up adding a bunch of noise to perf related telemetry
18:26bsmedbergmy other concern is that we should remove the shim code ASAP (at the latest, beginning of 57 cycle)
18:28ehsanaswan, it's *really* difficult to filter out telemetry data from the clients that have these types of extensions in practice fwiw
18:28bsmedbergwell... what I want to do is rely on the .isWebextension thing
18:28bsmedbergso MPC itself doesn't matter as much as disabling all addons which aren't webextensions
18:29ehsansure, if we go down the route of disabling all non-webextension add-ons immediately
18:29ehsanbut that's not what's being discussed right now
18:29ehsan(that's more severe that what _is_ being discussed)
18:30andymdisabling all non-WebExtensions on nightly is a no go
18:30ehsanwell then
18:30bsmedbergI don't understand.
18:31bsmedbergOr: that will be the config once we get to 57, right? So we're mainly talking about nightly 55-56?
18:31andymwe are landing the changes to limit add-ons to WebExtensions only in 55, behind a pref flagged off, we'll flip it on first week of 57 nightly
18:32aswanto be pedantic, we don't even honor the pref at all starting early in 57
18:32aswanand that rides the trains
18:33bsmedbergok, so what would it take to flip that pref on in nightly sooner, with the same UI being discussed here so users can opt back in for a few releases?
18:33aswanthere's a bunch of work to do to disable non-webextensions that won't be ready for some time (unless you want to break all system addons at the same time!)
18:34andymand the majority of add-ons used on nightly are not web-extensions yet
18:34aswanbsmedberg: i think the biggest single thing is we have a method for signing extensions that get an exemption (ie system addons, testpilot, etc)
18:34andymeg adblock plus and ublock origin
18:34andymno adblocking for you on nightly!
18:34aswanthe browser side requires a chunk of work but so does the actual signing infrastructure
18:34ehsanisn't ublock origin a webextension already?
18:34andymthe last APIs firefox needed landed in 54
18:35andymand chatting to the author, it will probably flip to be a webextension once 54 comes out
18:35andymadblock plus will likely be a bit slower
18:37bsmedbergCan you explain why signing is a blocker or involved? I'm talking about the addons that people have today in nightly. Only activate them if 1) system addon or 2) webextension. Is that harder than the MPC=true bits already in this bug?
18:37aswanbsmedberg: on the technical side that would break things like test pilot extensions
18:38aswanyou'd also get enormous pushback from ux, product, etc
18:39bsmedbergdoes "product" mean jeff? Because I don't believe you.
18:39aswani was thiknng of kev
18:39bsmedbergAnd also I'm totally ok with breaking test pilot in nightly.
18:39ehsando we have a way to identify test pilot based extensions?
18:39ehsanbsmedberg, really???
18:39aswanehsan: nothing simple
18:39bsmedbergWe need to take more risks in order to measure and ship quantum.
18:39andymwe are implementing a new signing infrastructure which will allow us to do just that
18:40andymi think there's a bunch of us here with quite different ideas and perspectives
18:40ehsanbsmedberg, let's take a step back for a sec
18:40andymwe might need to bounce this up to someone else and talk this through
18:41ehsanbsmedberg, the specific things that are hurting the measurement needs that I know of are CPOWs and the e10s shims
18:41andymalso, what's a "ghost window" and is it as cool as it sounds?
18:42bsmedbergehsan, I'm seeing ghost windows->CC pauses and chrome jank from addons as well, that I'd like to get rid of.
18:42* dolske takes a step back, swings his partner, do-si-do.
18:42ehsanbsmedberg, I understand that one way to deal with that is to just disable anything non-WebExtension on Nightly (and believe me I wish we could just do that so that I could move on to worry about other things) but we can also pick smaller hammers here, can't we?
18:42ehsanbsmedberg, ah hrm
18:42ehsanyeah window leaks are a good point...
18:42* ehsan thinks
18:43aswanthere are problably other things we're overlooking, gecko profiler for instance
18:43ehsangecko profiler is using add-on sdk right now
18:43ehsanmstange, ping
18:43ehsanmstange, is the gecko profiler gonna be on add-on sdk for much longer/
18:43mstangehopefully not
18:44ehsando you have an ETA?
18:44mstangedthayer has an experimental webextensions experiment add-on, and the patch to add a WebExtensions API to firefox is under review
18:44firebotBug 1329108 NEW, [meta] Clean up the nsIProfiler API
18:44mstangewrong bug
18:44mstangethis one
18:44firebotBug 1326572 NEW, Provide an API for nsIProfiler
18:44mstangemaybe in a few weeks?
18:44ehsanok thanks
18:45ehsanbsmedberg, we cannot kill gecko profiler, no way
18:45bsmedbergyes that's true
18:45ehsanwhat a mess :(
18:46aswanif we put the webextensions thing aside for a moment, we can ship the non-mpc changes relatively quickly
18:47* bsmedberg just wants to turn shims off
18:47bsmedbergas the first/quick change
18:48andymturning shims off is basically making the non-mpc changes non-reversible
18:48bsmedbergit's a pref
18:48bsmedbergI think...
18:49andymand i wouldn't be terribly upset if we did that
18:50aswanthere is a pref that just turns them off but then extensions may break silently
18:50aswanwe have changes that disables extensions that use them so it is less mysterious to anybody affected
18:50andymyeah, breaking silently would just be confusing for everyone
18:53kevAswan/bsmedberg - will read this thread again when I'm not on a phone, but breaking in nightly I think is fine/expected, breaking elsewhere not fine. But again, that's a quick scan so will re-read when I'm at my desk.
18:54ehsanandym, aswan, do we know what add-ons that will break, and are we ok with breaking those?
18:54ehsanfor example, from my limited understanding, I'm fairly certain that it should break things like ABP
18:55ehsan(and other add-ons using the content policy API, since that's one of the things using these shims)
18:55andymwe have no concerns about the non-MPC add-ons on nightly
18:55ehsanok cool
18:55andymsure some people will have some wierd add-on, but thats the long tail
18:55aswanabp is multiprocess compatible
18:56ehsanaswan, oh ok, good to know
18:56andymthe big ones are MPC true
18:56aswanall the big guys are either webextensions or mpc
18:56ehsanis there still going to be some kind of UI that tells people what happened?
18:56ehsanand are we concerned about that UI driving users away from the Nightly channel etc?
18:56ehsanbsmedberg, ^
18:56aswanjust the startup notification
18:57ehsan(just dotting the i's and crossing the t's...)
18:57aswanwe don't have to drive them away from nightly, we will direct folks to a page that explains what happened and how to re-enable
18:57aswanup to us if we include "use another channel" as one of the ways to re-enable
18:57ehsanaswan, well but there will be no way out of it, right?
18:57aswanor just document the pref
18:58ehsanoh there is still a pref
18:58ehsanyes I missed that part
18:58aswanwell its up to us to decide, if we leave it settable by a pref then the page can say "here's the pref to flip but please consider leaving it"
18:58aswani mean, any affected extension is going away soon enough anyway
18:59ehsanhonestly in terms of long term risk, I am more afraid of losing Nightly users to Beta
18:59ehsanso I'd prefer to message to them about the pref
18:59ehsanthan tell them to run Beta and risk having them never come back to Nightly
19:00aswanyeah, suggesting other channels was a not-well-thought-out idea, we can just scrap that
19:00ehsan(even at the risk of this making our measurement story sad)
19:00ehsanok, cool
19:00ehsanbsmedberg, what do you think?
19:06bsmedbergok short-term
19:06bsmedbergI still want to get us to a world where pretty quickly nightly only runs webext by default. But let's solve at least the profiler first ;-)
19:08aswanif we're going to do that we should announce it asap
19:12ehsanyeah, and plan for it
19:12* bsmedberg will follow up about that
19:12ehsanfor example I would expect the people who are running the test pilot experiments care very much whether their add-ons break on nightly
19:13ehsanbsmedberg, I'd appreciate if you can help keep track of this
19:13* ehsan has zero bandwidth to deal with this :(
19:13ehsanaswan, andym: thanks so much guys for bearing with me so far! :)
19:14aswanfwiw, existing bug 1336576 is about doing that work for 57. adding stuff to pull (parts of it) forward can be added there too
19:14firebot NEW, Limit add-ons loaded into Firefox
19:14ehsansounds good
19:14ehsanaswan, do you still think we should double check with billm?
19:14aswanno problem, thank you!
19:14aswani thought i saw him appear here a few minutes ago
19:14ehsanbillm, oh hi
19:15aswanwhat we've talked about so far seems uncontroversial, there's still the cpows for devtools, for the sdk etc where he is the resident expert
19:19ehsanaswan, yeah I'm planning to talk to him about CPOWs in more detail, but I first need to educate myself a bit more about the code side of things...
19:19* ehsan has been meaning to do that for ~3 weeks now...
19:19ehsanpretty high on my todo list :)
19:19aswanheh. the short-term path for shims looks clear to me
19:19ehsanok sounds good then
19:19ehsanlet's go ahead with this
19:36kevso, re-reading, wearing a product hat, what had been discussed was disabling non-MPC addons and MPC addons that included shims, so that shims wouldn't affect the population at large. that that's still reasonable to me, as is turning off shims by default provided there's some kind of messaging and a way to re-enable for the people who would like to.
19:37kevI'd also like to understand the impact to nightly of those addons, we should be able to get a reasonable estimation, and I'll talk to ben. I haven't seen any impact assessments in terms of how this is breaking our measurements or by how much, but agree that the sample size is small enough that it's likely true that we do see skew
19:38billmehsan: sorry, was at lunch
19:38kevand just to make sure I understand completely, we're doing this because we want to be able to see effects of changes as we land them on nightly and minimize the skew from add-ons that will likely throw a monkey wrench into the measurements we use to track effect
19:40makEnn: sorry, mozreview is really bogus and it lost your r+ on my patch :(
19:40kevI'm not at all crazy about turning everything off without understanding how that will affect people, and how many people it involves, which is something we should look into, so I'll ask ben for some help
19:40makEnn: I think it has problems with the "+" in your nickname
19:41makEnn: oh, nevermind, by manual editing it picked up the r+ again...
19:41kevaswan / bsmedberg / ehsan - let me know if I'm missing something
19:42kev^^ there, I mean
19:43kev(basically no pushback if people have a means to revert, and +1 on tracking how many do so we can adjust, because understand and agree being able to see perf improvements as it gets landed is kinda important :) )
20:10ehsankev, I think this is mostly accurate
20:10ehsanbillm, no worries
20:11ehsanbillm, in order to save you some time, we are discussing going ahead with turning the non-MPC add-ons off on nightly
20:11ehsanbillm: and then figuring out when and how we can proceed to turn off non-WebExtension add-ons
20:12ehsanbillm, thinking about issuing such as user rage, test pilot, gecko profiler and whatnot
20:12ehsanit's still not clear whether we'll do the latter and if so when and in what form
20:12ehsanbillm, just wanted to double check with you that you're ok with this?
20:13billmehsan: wait, it's not clear whether we'll turn off non-webextensions? I thought that was definitely happening for 57.
20:14ehsanbillm, yeah I meant doing it *before* 57
20:14billmoh, ok
20:14ehsan(sorry to scare you!)
20:14billmyeah, I don't really care what we do
20:15billmehsan: well, I guess I do care. most extensions people use right now are not webextensions but they are MPC=true. so we should wait to turn off non-webextensions until that changes somewhat.
20:16ehsanbillm, good point. we should get some data about that
20:16ehsanbillm, I think bsmedberg is doing that...
20:16billmehsan: well, until the extensions I use are webextensions, I don't need data :-)
20:16aswanthe telemetry to distinguish webextensions only just landed in nightly
20:16ehsanbillm, how does one tell if their own extensions are webextensions anyway??
20:17ehsanaswan, sweet
20:17aswanehsan: one waits for bug 1354682 :(
20:17firebot NEW, Flag a legacy add-on with "legacy" in Add-ons Manager
20:17aswanor if you're determined, you can find the xpi in your profile (in the extensions/ directory)
20:17billmehsan: I guess you can look at the xpi for an install.rdf file
20:17aswanif it contains install.rdf, its not a webextesnion
20:17ehsanaswan, waiting it is :)
20:18* ehsan is mostly mildly curious
20:18aswanspeaking of being mildly curious, what are these "ghost windows" that smedberg mentioned earlier?
20:19ehsanaswan, cases where an add-on has a leak and keeps a window alive
20:19ehsanwe refer to these as ghost windows in about:memory
20:20billmI think ghost windows typically are not caused by add-ons
20:20billmnuking should ensure that
20:20John-GaltNot necessarily an add-on... We do it ourselves sometimes, too :(
20:20billmit's usually bugs in our code
20:21John-Galtbillm: Some add-ons still cause them when they do things like store windows in nsIPropertyBags and such.
20:21ehsanyeah could be out code too
20:21billmok, that's true
20:24ehsanaswan, and this is the nature of the problem, things that are a mix of stuff caused by add-ons and our code, fwiw
20:45SolidSnakest3fan: ping
20:46SolidSnakeHi, I was wondering if you thought bug 1353367 was incorrectly marked WONTFIX. It seems to be me its still valid because this bug could still be lurking and rear its ugly head in a update to 8.0
20:46firebot WONTFIX, Open Tabs deleted on update to Firefox iOS v7.0
20:47kbrosnanyou want #mobile
20:47SolidSnakewell st3fan's lead of the iOS product, no? :P
20:47SolidSnakethat's why I brought it to his attention :)
20:48st3fanSolidSnake: no we will make sure that it will not happen in an update to 8.0
20:48kbrosnanSolidSnake: sure but this is more of a desktop firefox channel
20:48SolidSnakest3fan: alright, thanks. I didn't see any follow up bug to track this work so I was worried
20:49SolidSnakekbrosnan: understood. I only usually ask desktop questions here :)
21:39st3fanSolidSnake: qa will be on the upgrade path, and we have some plans for code changes to make things more robust
22:33SolidSnakest3fan: awesome thanks!
22 Apr 2017
No messages
Last message: 119 days and 3 hours ago