mozilla :: #ateam

11 Sep 2017
09:16whimbooato, jgraham: hello. I got a PR for mozrunner for you: https://github.com/jgraham/rust_mozrunner/pull/14. Thanks
13:11RyanVMdylan: you around? :)
13:11mikelingarmenzg: ping
13:12armenzgmikeling: hey!
13:12mikelingarmenzg: hey! Does this pr looks good to you? https://github.com/mozilla/mozilla_ci_tools/pull/513
13:13mikelingAnd are there any issue I can work on about SETA or ? :)
13:13armenzgmikeling: I have to change mozci and pulse_actions to say that we're not doing any further development
13:13armenzgI shut it off recently
13:14armenzgmikeling: there's a SETA bug that you could work on
13:14mikelingshut if off?
13:14armenzglet me file it
13:14armenzghow have you been?
13:15dylanRyanVM: what's up?
13:15dylanneed anything?
13:15RyanVMdylan: just checking in, glad to see you online :)
13:16mikelingI just finished my GSoC, so I want to come back and look for something I can work on :)
13:17armenzgmikeling: cool; what was your GSoC?
13:18mikelingarmenzg: refactor a Machine Learning toolbox with C++11 feature
13:22armenzgmikeling: nice!
13:23armenzgmikeling: this will then be *very* easy https://bugzilla.mozilla.org/show_bug.cgi?id=1398771
13:23buggbotBug 1398771: Treeherder: SETA, normal, nobody, NEW , When a new platform is found do not follow defaults
13:24mikelingarmenzg: got it, thanks ;)
13:38jgrahamjmaher|afk: I'm pretty sure that "Windows builds are slow" is not going to magically get better since we are presumably already spending effort on it
13:39jgrahamI"m not sure what the argument for running non-e10s tests only on Windows is; for -bc tests it probably makes sense, but for platform tests the main non-e10s shipping configuration is Android, not Windows
13:41jmaher|afkjgraham: windows is our #1 target for users, it is silly to treat linux as our #1 platform and continue to ignore windows
13:42whimboojgraham: thanks a lot! Mind also releasing a new version to crates.io?
13:42jmaher|afkjgraham: right now nobody is willing to fix the #1 intermittent failure which happens to be on non-e10s, without developer support for non-e10s it seems foolish to add more platforms
13:44jgrahamjmaher|afk: Windows is not out #1 platform for non-e10s
13:44jgrahamIt's not a platform for non-e10s at all
13:45jgrahamIt only makes sense to run non-e10s tests on Windows where they aren&#39;t actually testing anything to do with e10s but the tests don&#39;t work with e10s because <bad reasons>
13:46jgrahamWhich I think is true for -bc tests, but not really true for ServiceWorker or similar
13:47jmaher|afkjgraham: what is our #1 platform for non-e10s?
13:47jgrahamAndroid
13:47jmaher|afkjgraham: and we have full coverage of tests on android
13:47jmaher|afkwhich run non-e10s and have for years
13:47jgrahamjmaher|afk: I am going to go out on a limb and say we don&#39;t want to encourage people to run more try jobs on android
13:48jgrahamSo the question is &quot;which platform makes most sense to test non-e10s on, given that it isn&#39;t android&quot;
13:48jgrahamAnd Linux is closest in terms of OS and best in terms of general features and speed
13:49jgrahamSo it seems pretty clear to me that it makes sense
13:49jmaher|afkjgraham: are developers specifically working on fixing android only issues? It could be I don&#39;t understand the issues here, I do know that we get fairly decent non-e10s coverage (including on jsdcov tests which run on linux on every m-c push)
13:50jgrahamjmaher|afk: Any non-e10s issue in platform code is an android-only issue in current (shipping, supported) configurations
13:50jgrahamafaik
13:51* jmaher|afk boarding a plane, back in a few hours
13:51RyanVMwe don&#39;t have full Android test voerage
13:51RyanVMw-p-t immediately comes to mind
13:51RyanVMcoverage*
14:04whimboojgraham, ato: could one of you release a new mozrunner crate? I would like to finish the two bugs which I would like to see in geckodriver 0.19 too
14:05atowhimboo: I dont have access to that.
14:05whimbooah k
14:05whimboowe should get access
14:06whimboogiven that james will be on pto for a while
14:06whimboobest would be anyway to move it to the mozilla org
14:07jgrahamI can do the release
14:07jgrahamI don&#39;t mind moving it ofc, but you will need AutomatedTester probably
14:07whimboothanks
14:08whimboojgraham: does he have access?
14:08whimbooif yes it wouldn&#39;t be that urgent
14:11whimbooato: https://bugzilla.mozilla.org/show_bug.cgi?id=1397007#c10
14:11buggbotBug 1397007: geckodriver, normal, nobody, NEW , Intermittent /webdriver/tests/minimize_window.py | minimize_window.py::test_payload
14:11whimboolets make this a topic for this weeks chit chat?
14:11whimbooI could explain it. it&#39;s actually easy to at least run marionette tests
14:12whimbooi assume wdspec tests wouldn&#39;t be that different
14:14jgrahamwhimboo: published 0.5.0
14:14whimboojgraham: thanks
14:15jgrahamato: For that bug, if you think the problem is timing, it seems reasonable to put in a delay
14:17jgrahamwell executeScript(&quot;var start = Date.now(); while(Date.now() - start() < 1000) { if (document.hidden) {return true} }; return false}&quot;)
14:17jgrahamLike we will probably still get some intermittents, but fewer
14:18jgraham(one click loaners are pretty easy now, since wpt has some support for running in that environment)
14:22atowhimboo: As I said, I dont think a loaner would be useful.
14:22atojgraham: The problem is indeed timing. DOM properties propagate slowly and are not guaranteed to be synchronous.
14:23atojgraham: Ive found ChromeWindow.windowState to be more reliable, sizemodechange somewhat less so, and then DOM properties.
14:26atoWe have a similar horrible hack for maximising the window: https://searchfox.org/mozilla-central/source/testing/marionette/driver.js#3057-3071
14:26atoThis times out after 2s if the window size doesnt change.
14:30whimbooato: why not useful?
14:30whimbooits perfect to replicate the issue
14:30atowhimboo: Because I know what the problem is.
14:30whimbooand to get it fixed
14:30atoReproducing it isnt the big problem.
14:30whimbooanyway i have to head out for a while.
14:31whimboobbl
14:31atoIf you want reproduction in this case, you want it at scale, which try is great for.
16:12armenzgwhat am I doing wrong in this?
16:12armenzghttps://irccloud.mozilla.com/pastebin/v9iTX3Hg/
16:12armenzgit might be dead obvious
16:21armenzgjanx: ping (can you please unblock us? https://github.com/mozilla-releng/services/pull/595)
16:21janxarmenzg: hello! on it
16:22armenzghello monsieur!
16:22armenzglong time!
16:23janxarmenzg: done! also, bonjour :) it&#39;s been a long time indeed
16:24armenzgjanx: will I see you at Texas?
16:24armenzgyeeha!
16:24janxarmenzg: very probably :)
16:24armenzgcool; same here
16:25janxarmenzg: looking forward to it, although I&#39;m a bit sad about the change of venue (we were planning to go diving in Cancun, and also it was a non-US venue for once)
16:25janxbut it&#39;s hard to complain about paid travel :)
16:25armenzgjanx: fair enough
16:26armenzgit would have been hard to work :P
20:29jmaherRyanVM: reftest-gpu on mozilla-central is a single chunk for opt/pgo/debug/nightly
20:29RyanVMyes, that&#39;s why I filed the bug in the first place?
20:30jmaherI don&#39;t understand then?
20:30RyanVM*every* other Win7 TC reftest job is 8 chunks
20:30RyanVMeven my Try push pretty clearly shows that
20:31jmaherok
20:31jmaherwhy not debug as 8 chunks? that is a 90 minute run vs 30 for opt
20:31RyanVMBBB
20:31RyanVMcan I do that without having to do the &quot;add buildernames&quot; dance?
20:32RyanVM(I linked to the bug in question that deals with win7 debug issues in that suite too)
20:33jmahersince rg runs in vm, why is it not in tc?
20:33jmahersorry for not knowing the full picture
20:33RyanVM&quot;Lots of hackery for debug thanks to scheduling via BBB still. Will be much simpler once bug 1398284 gets sorted out.&quot;
20:33buggbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1398284 RelOps, normal, rthijssen, NEW , Debug reftest e10s tests break Windows 7 GPU instances in Taskcluster
20:33jmaheroh, I see
20:34jmaherhave we looked at running reftest-gpu/debug in 8 chunks to see if it fails in TC? maybe it is an issue with the long running job
20:35RyanVMthe issue was related to bug 1366288 IIRC
20:35buggbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1366288 Canvas: WebGL, normal, rthijssen, NEW , win10 vm in AWS fails webgl tests with: WebGL2 context because of blacklist entry: FEATURE_FAILURE_UNKNOWN_DEVICE_VENDOR
20:35RyanVMi.e. not a runtime issue
20:35RyanVMhttps://bugzilla.mozilla.org/show_bug.cgi?id=1356240#c4
20:35buggbotBug 1356240: Task Configuration, normal, nobody, NEW , Win7 reftest-gpu-e10s not being scheduled on Taskcluster
20:36jmaherso many bugs, so little time
20:37RyanVMonce everything&#39;s moved over to TC, it would be nice to spend some time playing around with # of chunks to get something more optimal from a runtime & overhead standpoint. I&#39;d be happy to play around with it myself, but it&#39;s not really practical to do yet IMO. For now I just want parity.
20:51RyanVMjmaher: on a different note - Android doesn&#39;t run w-p-t yet, which is pretty significant missing piece of coverage parity IMO
20:54tedRyanVM: seems like we ought to be able to take the mochitest timings input and make it generate balanced chunks with a target runtime nowadays
20:55tedi guess we could ideally just do that while generating the task graph
20:55RyanVMted: this is reftest, but yes, auto-chunking would be sweet
20:55RyanVMaki actually did that for l10n repacks now (based on # of locales)
20:56jmaherRyanVM: yeah, I was under the impression the concern on dev.platform this morning was slow windows build times, not lack of coverage
20:57tedRyanVM: nice
20:57RyanVMI think there may be some validity to the argument that linux32 is the closest equivalent to Android where there&#39;s missing coverage
20:57jmaherRyanVM: I don&#39;t know why we haven&#39;t prioritized wpt for android; to me that indicates a lack of commitment to android which puts some doubt in how much we should put into android
20:58jmaherRyanVM: well, to be honest I am sort of frustrated as there are hundreds of non-e10s only failures and nobody has lifted a finger for weeks to look at them, I am reluctant to consider adding more; it seems sort of pointless- yeah, things are broken and we have other priorities :(
20:59RyanVMchasing oranges is indeed an exercise in frustration
21:01jmaherthere are higher failure rates on non-e10s, if I had confidence we would treat our tests as &#39;oh this is pretty broken (i.e. failing 100 times/day), we need to fix our browser&#39;, then I would see there is value in adding more tests to an obscure config
21:02jmaheranyway, that is besides the point, I would like to know why we haven&#39;t added wpt to android if it is the only tests that cover large blocks of code that runs on android
21:46tedjmaher: seems reasonable to push back on those two points
21:46teda) if we need wpt coverage on Android, we should do that, not rely on other things as a proxy
21:47tedb) if there are tons of non-e10s failures and nobody is fixing them, you should not spend time enabling more non-e10s tests until there are people invested in fixing that.
21:50jmaherted: I am somewhat flexible, but would prefer a bit more reason than it makes my life easier, we have done a lot of work to overload our capacity thanks to running stylo in duplication where that wasn&#39;t really the case when we turned off non-e10s- I want to be realistic that a request to run these tests will only serve to starve our resource pool
21:52tedmy argument is not &quot;it makes Joel&#39;s life easier&quot; it&#39;s &quot;if nobody cares enough about the results to make sure they pass then the tests don&#39;t have as much value as people claim&quot;
21:52tedhow many times have we seen in the past people claiming you can&#39;t turn off some test suite even though it fails all the time, or a perf test that has constant regressions that nobody dealt with?
21:54tedI don&#39;t think it&#39;s a valuable use of your time to get a test suite green if nobody is invested enough to help.
21:55RyanVMpreach!
21:55jmaherheh
21:56jmaherok, offline for me to check into room and find dinner- see y&#39;all online tomorrow
12 Sep 2017
No messages
   
Last message: 8 days and 17 hours ago