mozilla :: #ateam

7 Aug 2017
07:19Tomcat|sheriffdutywhimboo: good morning
07:19Tomcat|sheriffdutywhimboo: do you know if https://treeherder.mozilla.org/logviewer.html#?job_id=121332843&repo=autoland is expected or a new failure ?
07:24whimbooTomcat|sheriffduty: hi. hm, haven't seen it before
07:24whimboois it permanent?
07:25Tomcat|sheriffdutywhimboo: seems so https://treeherder.mozilla.org/#/jobs?repo=autoland&bugfiler&noautoclassify&filter-searchStr=t%20firefox%20functional%20tests%20(local)%20executed%20by%20taskcluster%20with%20e10s%20test-macosx64%2Fopt-firefox-ui-functional-local-e10s%20tc-fxfn-l-e10s(en-us)&selectedJob=121332843
07:26whimbooTomcat|sheriffduty: download of requests fails
07:26whimboo22:44:42 INFO - http://pypi.pvt.build.mozilla.org/pub uses an insecure transport scheme (http). Consider using https if pypi.pvt.build.mozilla.org has it available
07:26whimbooso a releng issue
07:27whimbooseems to happen for all packages we are trying to download from our internal mirror
07:27whimboomaybe releng knows something
07:33Tomcat|sheriffdutywhimboo: ok will ask buildduty
07:34whimbooTomcat|sheriffduty: k, let me know what it is. thanks
07:37Tomcat|sheriffdutywill do thanks whimboo
10:20whimbooouch.... bug 1388012
10:20bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1388012 Integration, major, nobody, NEW , Build jobs on at least Windows indicate debug=False in mozinfo.json for debug builds
10:20whimbooTomcat|sheriffduty: ^
10:20whimbooi could expect that we see some more failures as expected on that platform
10:21whimboomeans tests which should be skipped for those builds are getting run
10:21Tomcat|sheriffdutyok
10:22Tomcat|sheriffdutygrenade: hi ! can you take a look at this bug from whimboo
10:23whimbooi put him on ni? already
10:39emorleydavehunt: hi! hope you are well :-) I see you've filed bug 1388018 for mozfile -- treeherder needs something similar for mozlog, so I was thinking of filing another, unless you were already in the middle of doing that? (It just makes more sense for Treeherder's "switch to Python 3" buyg to depend on a mozlog specific bug than the overall mozbase bug, since
10:39emorleyit doesn't use any other packages from it)
10:39bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1388018 Mozbase, normal, nobody, NEW , [mozfile] Add support for Python 3
10:40davehuntemorley: mozfile is just the start, it's mozlog that I want too, but I believe the mozlog tests depend on mozfile
10:40emorleyah right
10:40davehuntemorley: https://dxr.mozilla.org/mozilla-central/source/testing/mozbase/mozlog/setup.py#21
10:40davehuntfeel free to clone though, they can be worked on at the same time for sure
10:41davehuntonce we have Python 3 support in mach python-test and a python3 task running in TaskCluster, it shouldn't be too hard to add support for any mozbase package
10:45davehuntemorley: thanks :)
10:49tedpresumably the python3-test task will just run all of the tests that have opted-in in their manifests
10:50davehuntted: that's the idea, yeah
10:50davehuntso initially it'll be a noop, unless there's already something that supports py3
11:00emorleyincrementally doing it is definitely the best way forwards
11:01tedi think we've got a solid plan, just need to get some pieces implemented to actually stand it up
11:02emorleyalso there may be some benefit to just running eg python3 flake8 against the packages even if the package's tests themselves aren't passing
11:02emorleysince flake8 will detect the low hanging fruit of things like iteritems() etc
11:04tedthat is a nice idea
11:04tedwe could duplicate the existing lint job to run a python3 lint check
11:10emorleyI seem to remember there are even some specific tools to identify what to change when converting to Python 3
11:14emorleyah linked from https://docs.python.org/3/howto/pyporting.html
11:23tedyeah, there's plenty of that
11:23tedi keep saying we should look into using modernize: http://python-modernize.readthedocs.io/en/latest/index.html
11:23tedwhich is a wrapper around 2to3 that uses six to make code work on python 2 or 3
11:24davehuntted: yeah, I had some good results with modernize whilst experimenting
11:24tedcool!
11:37emorleythe guide above mentions both modernize and futurize, though seems to suggest the latter for cleaner code
12:34whimbooato: hi. do you know which platformVersion WindowsXP returns?
12:34whimboothat in the Marioentte caps
12:44whimbooi think checking <6 should indicate that
12:48tedemorley: oh interesting, i hadn&#39;t heard of futurize
13:53davehuntted: python-test calls out to the python from the virtualenv, swapping out another python interpreter means certain dependencies aren&#39;t present such as mozunit
13:53davehuntdo you know where the virtual environment is set up?
13:54davehuntI suspect I can&#39;t switch the interpreter for the virtual environment without having an effect on the Firefox build, but perhaps I can create a new virtualenv for the tests?
13:56tedyeah, maybe
13:57tedhttps://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozbuild/base.py#197
13:57tedand https://dxr.mozilla.org/mozilla-central/source/python/mozbuild/mozbuild/virtualenv.py
13:57davehuntthanks
13:57* ted gym, bbiaf
14:23atowhimboo: Not off the top of my head.
14:34davehuntgps: in order to get the mach python-tests to use an alternate python interpreter, do I need a new virtualenvmanager?
14:34davehuntjust swapping in another python path means I don&#39;t have the required packages such as mozunit
15:03whimbooato: i think i got it in my patch :)
15:03whimboohm, interesting change of the Nightly icon
15:03whimbooI think I have to get used to it
15:04atowhimboo: Yeah, I r+ed it on the presumption that it was correct (-:
15:11whimbooato: for safety I wills till run a try build but have to fix my local networking issues first. pages take ages to load since about 2h
16:47gpsdavehunt: yes, we&#39;d need to create a separate virtualenv for python 3
16:48davehuntgps: is that something that should be possible currently? create a new virtualenv manager, and call ensure + activate?
16:48davehuntI know there&#39;s a version check, which I&#39;d need to modify
16:49davehuntgps: also, any objections to me filing a bug to vendor in six?
16:49gpsno objections to vendoring six. i&#39;m quite surprised it isn&#39;t already vendored
16:49jgrahamWell it is, but only transitively
16:50davehuntoh? I did look for it
16:50jmaherdavehunt: we recently did py3+py2.7 for the new tp6 talos test, that was sorted out in ~May
16:51davehuntjmaher: are those run via mach? could you point me to some bugs?
16:56jmaherdavehunt: let me find the bug
16:57jmaherdavehunt: the tracking bug is https://bugzilla.mozilla.org/showdependencytree.cgi?id=1357177&hide_resolved=0
16:57jmaherdavehunt: https://bugzilla.mozilla.org/show_bug.cgi?id=1361462 and https://bugzilla.mozilla.org/show_bug.cgi?id=1361732 might be good
16:57bugbotBug 1361462: General, normal, armenzg, RESOLVED FIXED, Mozharness support for Python3 binaries
16:57bugbotBug 1361732: Mozharness, normal, gbrown, RESOLVED FIXED, Mozharness support for Python3 venv creation
16:57davehuntjmaher: awesome, thanks!
17:04davehuntjmaher: how/does mozharness interact with mach? do they independently manage virtual environments?
17:06jmaherdavehunt: I really don&#39;t know
17:06jmaherI believe all of our CI based tests launch a mozharness script
17:06jmaherbut then mach in some cases loads mozharness
17:22whimbooAutomatedTester: do the python bindings for Selenium support the addon installation?
17:32whimbooato: so I get a content crash of the first tab when starting firefox via geckodriver
17:32whimbooit&#39;s constantly happening. will investigate that now
17:32whimboomaybe that&#39;s something what other people are seeing
17:33whimboosadly we leave Firefox open
17:39whimboodoh. its the same as bug 1379906
17:39bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1379906 Marionette, critical, haftandilian, RESOLVED DUPLICATE, Assertion and crash during startup when running Marionette tests due do security.sandbox.content.level=3
17:45davehuntwoop! I have |mach python-test --py3 testing/mozbase/mozinfo/tests| passing locally!
17:48jgrahamdavehunt: Nice
17:53whimboojgraham: mind pointing me to the code in geckodriver which is the main loop? I have a hard time finding it
17:53whimboothere is run() in main.rs but it does only seem to start the browser
17:58jgrahamwhimboo: https://github.com/mozilla/webdriver-rust/blob/master/src/server.rs#L240
18:02whimboojgraham: so its a thread we spawn. do we ever join it? or is that automatically done in the Builder class?
18:03whimbooi asusme that handling the binary is also done in webdriver then?
18:03whimboooh, we have start_browser() in geckodriver
18:04jgrahamwhimboo: I feel like there&#39;s an underlying question here I don&#39;t know
18:04jgrahamWhat problem are you trying to solve?
18:05whimboojgraham: i&#39;m looking for a place where I can check if the application is still running when we are about to exit geckodriver
18:05whimbooso we can force kill it
18:06whimbooright now we leave Firefox running in some situations
18:06whimboowhich is bad
18:06jgrahamwhimboo: Whichc situations?
18:07whimboojgraham: eg in a case like bug 1379906
18:07bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1379906 Marionette, critical, haftandilian, RESOLVED DUPLICATE, Assertion and crash during startup when running Marionette tests due do security.sandbox.content.level=3
18:07whimboobut just for geckodriver
18:07whimbooi have also seen it on bug 1387403
18:07bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1387403 geckodriver, major, nobody, NEW , Application is not getting closed for exceptions in capability parsing
18:07whimboowhen we fail for some capabilities
18:08whimbooso there is no session yet, and delete_session will not be called as I would assume
18:09jgrahamwhimboo: I see
18:11atowhimboo: Yes, just check the source: https://github.com/SeleniumHQ/selenium/blob/master/py/selenium/webdriver/firefox/webdriver.py#L231-L253
18:11whimbooi will try to find it myself
18:12whimbooato: interesting because it gave me unknown command
18:12whimbooi will check again once i solved the other problem
18:12jgrahamwhimboo: Looks like we have a delete_session property on errors that should get set if the new session command fails. Possibly that isn&#39;t happening
18:13whimboowe fail with creating the connection already
18:13atowhimboo: Well Google is using it in their infra, so I assume it works.
18:14atowhimboo: Can you check your open issue in https://reviewboard.mozilla.org/r/165654/#issue-summary again?
18:15jgrahamwhimboo: http://searchfox.org/mozilla-central/source/testing/geckodriver/src/marionette.rs#410 perhaps
18:17jgrahamOr in the caller there
18:18bholleyjmaher|bbiab: I don&#39;t see non-sequential-stylo tp6 tests anymore: https://treeherder.mozilla.org/perf.html#/graphs?timerange=604800&series=%5Bmozilla-central,fd75e33f207c37d5b2af9f8bbe6ce2e2a8d626ee,1,1%5D&series=%5Bmozilla-central,8ca3961a8c89fcc110eb10f6a839dd510a958c4c,1,1%5D&series=%5Bmozilla-central,292b1aa99795847a0f5e90268383dcfd5c87c60d,1,1%5D
18:21jmaherbholley: let me look
18:22jmaherbholley: is non sequential threads=1 ?
18:22bholleyjmaher: to be clear, I&#39;m saying the _only_ thing I see (stylo or not) for amazon pgo tests are the stylo 1thread results
18:22bholleyjmaher: I don&#39;t see either regular stylo or non-stylo
18:23bholleyjmaher: see the graph I linked to above
18:23bholleyjmaher: the results cut off on august 4th
18:23jmaherbholley: oh, I believe that is when we switched over to tp6
18:23jmaherbut that could be wrong
18:23bholleyjmaher: I still don&#39;t see it searching for tp6
18:24bholleyjmaher: basically, I go to perfherder, select m-c and windows 10, and then filter by &quot;amazon pgo&quot;
18:24jmaheryeah, something seems broken
18:24bholleyjmaher: under that I see three tests, two of them quantum_pageload and the other tp6
18:25jmaheryeah, it looks like it isn&#39;t running
18:25bholleyshucks
18:25jmaherI can figure that out
18:25jmaherthanks for letting me know bholley
18:26bholleyjmaher: thanks! - the sooner the better, really trying to keep track of the various perf improvements flying in, and a lot landed over the weekend :-)
18:28whimbooato: done
18:28atoCheers
18:28whimboojgraham: thanks. i&#39;m already in that method. lets see which cases i find
18:29whimboojgraham: what I dislike is that we always append to geckodriver.log
18:30whimbooand do not clear it at the beginning
18:30whimbooforces me to reset the file each time :/
18:30atoI dont think geckodriver produces a log file at all.
18:30atoIt just prints to stdout.
18:30whimboono
18:30whimbooit creates geckdriver.log
18:30atoYou must be talking about the Selenium Python client, which opens the file in &quot;a+&quot; mode.
18:30whimbooah
18:31whimbooso there is the code i have to blame :/
18:31atoFWIW thats the normal way to produce log files.
18:32ato&quot;tail -f&quot; is great for monitoring logs.
18:33davehuntjgraham: where is six being vendored in?
18:33whimbooato: the amount of output doesnt make it easy. is there a way to grep
18:35atowhimboo: With tail?
18:35atotail -f FILE | grep --line-buffered PATTERN
18:37whimbooato: oh, i was using grep with &#39;**&#39; instead of &#39;\*\*&#39;
18:37whimboonow its better! thanks
18:38atoHah, yes (-:
18:38atoI have to go now, but will be back tomorrow!
18:38whimboohappy evenign!
18:53jgrahamdavehunt: http://searchfox.org/mozilla-central/search?&case=true&path=six.py
18:56davehuntjgraham: thanks!
18:58whimboojgraham: is there a reason why no struct/enum definition in webdriver-rust is deriving Debug?
18:58whimbooi would really like be able to print some debug output
18:58whimbooif its wanted I can add it and open a PR
18:58jgrahamwhimboo: No reason
18:59whimbook, so I will do
8 Aug 2017
No messages
   
Last message: 15 days and 2 hours ago