mozilla :: #automation

10 Jan 2017
09:03whimboostandups: totally messed up my commit series with histedit late yesterday. Thankfully all good again.
09:03standupsOk, submitted #40994 for https://www.standu.ps/user/whimboo/
10:06travis-cimozilla/mozmill-ci#1020 (master - c941166 : Henrik Skupin): The build passed.
10:06travis-ciChange view : https://github.com/mozilla/mozmill-ci/compare/a68a944c944dc83624f24c728f7288d678f4945c...c9411664d5b536af15372d9c84c1fa42c5c9fae2
10:06travis-ciBuild details : https://travis-ci.org/mozilla/mozmill-ci/builds/190555780
10:41whimboodavehunt: can you see that geckodriver sets focusmanager.testmode to true?
10:43davehuntwhimboo: yes, https://github.com/mozilla/geckodriver/blob/master/src/prefs.rs#L150-L152
10:44whimbook, i will add this to the bug too
11:07whimboodavehunt: fyi. i have a patch for bug 1311350 now
11:07firebothttps://bugzil.la/1311350 ASSIGNED, hskupin@gmail.com close() and closeChromeWindow() do not wait until the underlying window has disappeared
11:09davehuntwhimboo: sweet!
11:10whimbooi will have to see what I have to do for geckodriver
11:10davehuntwhimboo: I'm hoping to merge b4hand's patch later today, then will rebase my notifications one for another round of reviews
11:10whimboosounds good
11:10whimboodavehunt: are there any other needs beside the focs
11:10whimboofocus issue we need for our demos?
11:11davehuntI don't think so
11:11davehuntThe focus one is not a blocker anyway
11:11whimbooyeah
11:11davehuntI'd like to have a user guide in place, and API docs
11:11davehuntb4hand has started working on that
11:11whimboowe should give him a hand
11:11davehuntIndeed
11:11whimbooso we definitely have all that by mid of January
11:12davehuntWould also be good to get AppVeyor running so we have confidence we're not breaking Windows support
11:12whimbooall new methods added then should come with API docs included
11:12davehuntwhimboo: I totally agree
11:16whimboodavehunt: we should also start with our slides to get in a rough structure at least which we can fine tune step by step
11:16whimboodid we agree on an engine yet?
11:17davehuntengine? oh for the slides.. I put a suggestion in the etherpad
11:22AutomatedTesterbbiab, off to shower
11:25whimboodavehunt: that one sounds good
11:50davehuntbbl lunch
12:02whimboodavehunt: shall i put it into my github repo as where the other slides are already locate?
13:20davehuntwhimboo: sounds good to me
13:20davehuntI have a different talk to give two days before, so I'll also be working on that
13:47whimboodavehunt: ah, that's why Amsterdam?
14:11davehuntwhimboo: yep
14:19whimboodavehunt: hm, looks like the latest github emails I got today don't contain the fragment url again :(
14:25davehunt:/
15:48b4handwhimboo: hey henrik. Do you have any experience with appveyor? I just want to know if it always as flakey as it has been for me :/
15:49davehuntb4hand: have you been able to get any tests to pass? when I last tried it was consistently failing
15:49whimboowe had some issues in the past
15:50whimboowhat is the problem
15:50davehuntbut I fixed an issue in rust_mozrunner that should help with Selenium
15:50davehuntit also will require you to install geckodriver, and I don't know if the chocolatey package for geckodriver is up to date
15:51davehuntoh, it has :) https://chocolatey.org/packages/selenium-gecko-driver
15:51davehuntare you installing that b4hand?
15:53b4handYes I am
15:53b4handIt just always fails
15:54b4handSeems like it's either really slow or the browser never opens
15:54b4handdavehunt: ^
15:55davehuntb4hand: got an example?
15:55davehunt(sorry, I know you've shared one before)
15:56b4handdavehunt: Let me know if you can't access it
15:56b4handhttps://ci.appveyor.com/project/jrbenny35/foxpuppet-ocdgs/build/job/los26m5ue896bebi
15:56davehuntI can see it
15:57davehuntb4hand: some of your debug looks familiar
16:00davehuntb4hand: do you have an example with a passing test?
16:00davehuntlooks like it's timing out waiting for the new window to open
16:04b4handdavehunt: https://ci.appveyor.com/project/jrbenny35/foxpuppet-ocdgs/build/84-add_appveyor/job/x7t9gku62afi8oqt
16:04b4handThis one is interesting because there is a handle that is named '1'
16:04b4handBut one of the tests do pass
16:05davehuntthat is interesting, we have a rogue window!
16:06davehuntI wonder if we have a crash, or similar
16:08davehuntperhaps whimboo has an idea
16:10b4handI've tried everything heh, I don't have any more ideas!
16:11davehuntb4hand: you can always add print statements, and could try adding some time.sleep statements to see if it's timing related
16:13b4handI extended the wait for 60 seconds, and it will still fail
16:15whimboodavehunt, b4hand regarding handle 1 see bug 1315611
16:15firebothttps://bugzil.la/1315611 NEW, nobody@mozilla.org Ignore GFX sanity check window during startup of Firefox
16:15whimbooyou will have to ignore that
16:16whimbooand really only work with browser windows
16:16whimboosomething I mentioned in my last review comments already
16:16davehuntokay, so appveyor may need to be after we deal with non-browser windows
16:16whimboowindow[0] won't work in those cases
16:16whimboowe need the first window of type browserwindow
16:17davehuntyeah
16:17whimboopuppeteer is hanlding that
16:17whimboothe window with hanlde 1 is only open for a split of a second
16:17davehuntwe can take that next then
16:19b4handAH okay cool
16:19b4handI'll stop messing with it then
16:24whimbooyes
16:32whimboodavehunt: question. when a test in selenium crashes firefox we do not see any stacktrace. is it something selenium users are usually not interested in?
16:32whimbooif we would flip the fx ui tests to selenium we would need it
16:32whimboois it selenium then which would need a mozcrash like feature?
16:32davehuntthe selenium community would very much like that information, I'm sure
16:33davehuntwouldn't it need to surface through geckodriver somehow?
16:34whimboowe would have to set some env variables to not submit, and that we can process it locally
16:35whimbooi think that is part of geckodriver maybe?
16:35whimboobut not sure about the actual processing of the crash
16:35whimboowhere this code has to live
16:42davehuntwhimboo: that could go into Selenium, or perhaps the test harness could use something like mozcrash directly, if Selenium/GeckoDriver passes back what's needed
16:44whimboodavehunt: has seleniumn any other abstraction for different browsers beside the driver ones?
16:44davehuntwhat do you mean?
16:44whimboolike checking for crashes... it would be different for each browser
16:45davehuntI don't know if there's anything in Selenium for handling crashes atm
16:45davehuntAutomatedTester would know if it's been discussed for the spec
16:46whimbook, he might read that... so lets wiat :)
19:58AutomatedTestercrashes are not cared about by the spec
19:58AutomatedTestercrashes are not cared about by selenium
19:59AutomatedTesterif a crash happens normally it returns a WebDriverException and people go...doh...
19:59AutomatedTesterwhimboo: ^
21:21whimbooAutomatedTester: so is there anything we could do at least for our testing or lets say a Mozilla specific extension? if we want to follow your idea of using Selenium for our tests we would need proper crash checks
21:49AutomatedTesterwhimboo: yes, we can add our own point if we feel it should be done in rust. There is nothing stopping us from using our current python when we hit a point and we get disconnected from GeckoDriver
22:14whimbooAutomatedTester: so where would be the injection point exactly?
22:15whimboostandups: happy that my debugging session for bug 1223277 was successful. I'm getting closer to understand all the promise and generator code in Marionette server.
22:15standupsOk, submitted #41051 for https://www.standu.ps/user/whimboo/
22:15firebothttps://bugzil.la/1223277 ASSIGNED, hskupin@gmail.com Clicking an element that closes the current window causes the session to hang
11 Jan 2017
No messages
   
Last message: 6 days and 19 hours ago