mozilla :: #perf

18 Apr 2017
13:56igoldanrwood: good morning!
13:56igoldanrwood: what branch should I use when doing a change on Talos?
13:58rwoodigoldan: hello, you can clone mozilla-central, or mozilla-inbound
13:58igoldanrwood: great, thanks!
13:58rwood:)
14:00rwoodigoldan: whichever one you clone, I would recommend doing a Firefox build first and then ensuring you can run the existing talos test(s) locally i..e with ./mach talos-test
14:00igoldanrwood: I see
14:00igoldanrwood: will do that
14:01rwoodigoldan: cool, after you clone do ./mach bootstrap, then ./mach build
14:02rwoodI'm assuming you're running on osx or linux
14:02igoldanrwood: actually, I'm running on windows
14:03rwoodigoldan: 64 bit?
14:03igoldanrwood: yes
14:03rwoodigoldan: ok good, so I've never built on windows before, but here's the info: https://developer.mozilla.org/en-US/docs/Mozilla/Developer_guide/Build_Instructions/Windows_Prerequisites
14:03rwoodhopefully that page is up to date
14:05igoldanrwood: this page looks familiar
14:05igoldanrwood: I followed its steps when I did my 1st Firefox build
14:05igoldanrwood: to make sure my environment is correctly setup
14:06rwoodigoldan: ah so you've already built Firefox on your machine, awesome then you already have central cloned so you're all set
14:06rwoodyou should be able to run talos locally then, and it will run against that build
14:06igoldanrwood: that's right
14:06igoldanrwood: it should suffice
14:07igoldanrwood: but I'll 2x check after I finish the bug
14:07igoldanrwood: with a new rebuild
14:08rwoodigoldan: if you're just changing talos no need to rebuild Firefox, you just need a build to run the talos tests on/point talos at
14:08rwoodbut good to update now and then anyway
17:21_6a68Hey all! We've just landed Screenshots (a new user-facing feature / first embedded webextension system addon) in Nightly, and I'm working on measuring its impact on performance
17:22_6a68There does not seem to be a canonical list of measurements, so I'm working on building one
17:22_6a68Based on conversations in #teamaddons this morning, it seems like the memory consumption after startup is a good thing to measure, with and without the feature
17:23_6a68From the about:startup docs, it seems like the Talos measurements can be simulated by restarting locally a few times, then looking at the `firstPaint` times: https://mdn.io/Measuring_add-on_startup_performance
17:24_6a68I'm curious what else would be a good baseline set of measurements, and what kind of statistical certainty we'd want around the generated numbers
17:24_6a68I was just going to ping ehsan about this, but looks like he's just left the channel
17:24_6a68Input welcome :-)
17:30wlach_6a68: normal procedure with something large like this would be to run all the tests 6 times on a version of firefox with the change and compare the results against a base revision without it
17:31wlach_6a68: it's sometimes hard to anticipate exactly what a new feature will affect
17:31_6a68wlach: hi! which tests exactly? and would it be acceptable to run on a developer machine, or would we want to run them on a low-spec windows machine?
17:31_6a68or...is it better to push to try with some specific incantation, so that the tests run on a range of platforms?
17:31wlach_6a68: I would start by running this on all the platforms in automation
17:32wlach_6a68: something like ./mach try -b o -p all -u none -t all --rebuild-talos 5 should do the trick
17:33_6a68wlach: fantastic! thanks very much. and what numbers specifically would I be looking at? Or would I just be looking for regressions in the existing tests?
17:33wlachrunning try builds on a small subset of tests is cheap compared to the cost of backing stuff out if it has problems so don't worry too much about resource consumption :)
17:33wlach_6a68: just use perfherder compare and look for tests that regress with a high confidence
17:33wlachhttps://treeherder.mozilla.org/perf.html#/comparechooser
17:33_6a68whoa, this is new to me
17:33_6a68thanks, I'll give it a try
19 Apr 2017
No messages
   
Last message: 93 days and 13 hours ago