mozilla :: #ateam

15 May 2017
10:42AutomatedTesterjgraham: how do I specify that a test is an expected fail for wdspec?
10:58atoAutomatedTester: In the metadata file in testing/web-platform/meta
10:58atoAutomatedTester: Create a .ini file with the same path and name in that directory, as the test under testing/web-platform/tests.
10:59AutomatedTesterok
10:59atogps: Will you have time to land my changesets today?
10:59AutomatedTesterhmmm
11:03jgrahamAutomatedTester: Or |mach wpt path/to/test --log-raw wpt_raw.log; mach wpt-update wpt-raw.log|
11:16AutomatedTesterok
11:25AutomatedTesterdid we move everything to be wrapped in value :{} for webdriver?
11:26AutomatedTesterwe did... thats not why my test is failing... hmmmm
11:29whimbooAutomatedTester: hello. so how are we going forward with the current window handle issue in Marionette?
11:30AutomatedTesterwhimboo: the race condition?
11:30whimbooshall i disable all of the affected tests for os x and linux
11:30whimbooyes
11:30whimbooits causing lots of failures
11:30whimboo#4 of OF for last week
11:32AutomatedTesterwhimboo: yea, disable it and then work on the issue, I am still working on the other thing (which isnt appropriate for a public channel)
11:32whimbooAutomatedTester: ++
11:37atoAutomatedTester: Yeah, we had a bit of breakage because of the value-wrapping change, but I think thats mostly been addressed now. (-:
11:46AutomatedTesterato: "mostly"
11:46atoAutomatedTester: OK, I dont know of any cases where we return the wrong thing.
11:47AutomatedTesterato: in my case we arent returning anything
11:47atoMaybe I can help, but you have to explain what youre trying to do.
11:48AutomatedTesterI am looking for code
11:48AutomatedTesterplease hold the line caller
11:50AutomatedTesterato: we dont appear to return details in SetWIndowRect
11:54AutomatedTesterI will investigate in a minute, lunch
11:54atoI can reproduce that here:
11:54ato% curl -d '{"x": 20, "y": 40}' http://localhost:4444/session/f52954c1-b580-418f-ab19-189b8bfc698f/window/rect
11:54ato{"value": {}}
11:54whimbooato: whats the best way to use skipif in unit tests to only run tests on windows? sys.platform?
11:54whimbooi not sure what to check it agianst
11:54ato1494849254943 Marionette TRACE 0 <- [1,2,null,{&quot;x&quot;:16,&quot;y&quot;:36,&quot;width&quot;:1280,&quot;height&quot;:1040}]
11:54ato1494849254957 geckodriver::marionette TRACE <- [1,2,null,{&quot;x&quot;:16,&quot;y&quot;:36,&quot;width&quot;:1280,&quot;height&quot;:1040}]
11:54ato1494849254958 webdriver::server DEBUG 200 OK {&quot;value&quot;: {}}
11:54whimboogiven mingw and other shells
11:55AutomatedTesterato: marionette returns so its geckodriver bug
11:55atoAutomatedTester: Marionette returns the correct thing, but geckodriver isnt passing on.
11:55AutomatedTesterato: I will look more after lunch
11:56atowhimboo: skip-if = os == &quot;win&quot;
11:56whimboothats the in the manifest
11:56whimbooin dont want to disable all restart/quit tests
11:58atoDid you mean to ask about unittest decorators?
11:58atoI think the safest is to look at the session capabilities for the platformName.
11:58atoWindows is represented as &quot;windows_nt&quot;.
11:59whimbooalso when the tests are run via mingw?
12:00atoI dont know XPCOM internals, but if its not the same it should be considered a bug in Gecko.
12:00atoWe get it from Services.sysinfo.getProperty(&quot;name&quot;).
12:01whimboowe dont have our own magic skipIf
12:01whimbooor runIf
12:01atoDefine magic.
12:01whimbooall is tight to a specific decorator
12:01whimbooas defined in decorators
12:02whimbooso i h ave to use the unittest.skipIf or unless method
12:04atoYou can provide a new decorator like skip_if_mobile or perform an if-condition calling unittest.skip().
12:04whimbooi want the latter, but then i need a value to compare agianst
12:04whimboobut maybe i have to spin up my windows vm
12:04whimbooand run some checks
12:04atoif self.marionette.session_capabilities[&quot;platformName&quot;] == &quot;windows_nt&quot;: unittest.skip()
12:05whimboohm, what also works?
12:05whimbooso no decorator necessary
12:06atoAutomatedTester: https://github.com/mozilla/geckodriver/blob/master/src/marionette.rs#L640
12:06atoAutomatedTester: This needs to be moved down together with GetWindowRect.
12:06atoAutomatedTester: But please dont submit any PRs on geckodriver until it lands in central.
12:06AutomatedTesterato: yup, just found that
12:07AutomatedTesterato:
12:07AutomatedTesterato: as long as I can tag it as expected fail its fine :)
12:07atoYes, looks like expected!
12:07atoI have all the import patches ready, but I dont have permission to push them.
12:07AutomatedTesterahh cool
12:07atoSo waiting for Canada to awake.
12:07AutomatedTesterCanada?
12:08atoOr I guess hes in MTV.
12:08AutomatedTesterGreg? SF
12:08atoThat one then (-:
12:08atohttps://hg.mozilla.org/users/atolfsen_mozilla.com/gecko/
12:58AutomatedTesterI have 8 failing tests for Window Rect in WPT
12:59atoI cant wait to get geckodriver into central so it will be easier to write them.
12:59AutomatedTesteryup
13:06AutomatedTesterjgraham: I tried that command you said about expected fails and its not worked
13:07AutomatedTesterjgraham: I did
13:07AutomatedTester./mach web-platform-tests testing/web-platform/tests/webdriver/window_rect.py --webdriver-binary=/Users/dburns/development/geckodriver/wires/target/release/geckodriver --log-raw wpt_raw.log; mach wpt-update wpt-raw.log
13:08AutomatedTesterand then I ran it again and its still has fail
13:08AutomatedTesterUnexpected results: 8 (FAIL: 8)
13:09AutomatedTesternevermind, PEBKAC error
13:14AutomatedTesterjgraham: now that I got update running its failing
13:14AutomatedTesterIOError: [Errno 2] No such file or directory: &#39;/Users/dburns/development/mozilla-central/wpt-raw.log&#39;
13:14AutomatedTesterdburns in ~/development/mozilla-central pwd
13:14AutomatedTester/Users/dburns/development/mozilla-central
13:14AutomatedTesterdburns in ~/development/mozilla-central ls wpt_raw.log
13:15AutomatedTesterwpt_raw.log
13:15AutomatedTesterhttps://pastebin.mozilla.org/9021732
13:21AutomatedTesterFirst batch, now to fix bugs before updating this further https://reviewboard.mozilla.org/r/139270/diff/1#index_header
13:23atoNice.
13:26maja_zfjgraham: do you have admin rights in the w3c/web-platform-tests repo? https://bugzilla.mozilla.org/show_bug.cgi?id=1364564#c2
13:26bugbotBug 1364564: web-platform-tests, normal, mjzffr, NEW , Prototype a service that pushes wpt github PR to try server
13:28AutomatedTestermaja_zf: if he doesnt we can ask for specific information (but I suspect he does)
13:29maja_zfi suspect the same
13:33jgrahamI does
13:36jgrahammaja_zf: Done
13:36maja_zfcooool
13:38jgrahammaja_zf: Will try to come to a meeting tomorrow, but it&#39;s not 100% certain
13:39maja_zfYep, sounds good. Will catch you up on Thursday if you don&#39;t make it.
13:40jgrahamThanks
13:51atowhimboo: it&#39;s actually not easy to understand.
13:52atowhimboo: I think this is the understatement of the year.
13:57whimbooato: hm?
13:58atoIm quoting you from https://bugzilla.mozilla.org/show_bug.cgi?id=1363368.
13:58whimbooato: how did you hear that I just said this about Rust
13:58bugbotBug 1363368: Marionette, normal, hskupin, ASSIGNED , Registering a browser doesn&#39;t wait for contentBrowser to be ready
14:12davehuntekyle: if you&#39;re around, I could do with some help constructing a query against ActiveData
14:12ekyledavehunt: yes, I am around
14:12davehuntekyle: great, let me share the one that&#39;s working, and tell you what I&#39;m trying to add
14:12ekyledavehunt: ok
14:13davehuntekyle: okay, so https://activedata.allizom.org/tools/query.html#query_id=neFcsVgk shows me the 90th percentile for duration of all tests
14:14davehuntekyle: what I&#39;d like to also know is the run.job_name, but when I add that as an additional group, the aggregation of duration seems to be adversely affected
14:16davehuntthe longest duration for my original query is 47.157000, but I don&#39;t know what job that&#39;s for
14:17ekyledavehunt: ok, give me some time to look at it, and I will get back to you
14:17davehuntby also grouping on run.job_name the longest duration jumps to 852.7504, which I suspect is an aggregation
14:17davehuntekyle: great, thanks :_
14:17davehunt:)
14:24ekyledavehunt: how is it adversely affected?
14:25davehuntekyle: by also grouping on run.job_name the longest duration jumps to 852.7504, which I suspect is an aggregation
14:26davehuntperhaps I can demonstrate it with a couple of queries where I limit the job name
14:26ekyledavehunt: ok, still looking. I too am limiting to job name
14:28ekyledavehunt: it may be legit: there is more then one job_name with that test
14:28davehuntekyle: I was just coming to the same conclusion..
14:28ekyleadd some time limits to your query eg {&quot;gt&quot;:{&quot;run.stats.start_time&quot;:{&quot;date&quot;:&quot;today-week&quot;}}}
14:28davehuntI wasn&#39;t trusting the data because it&#39;s such a long duration :/
14:29ekyleto diagnose problems like this, I limit myself to a single set of properties, and then look at the raw data: eg:
14:29ekylehttps://activedata.allizom.org/tools/query.html#query_id=sEYd2gf5
14:29ekyledavehunt: ^
14:31ekyledavehunt: of course that example does not have the extremes you see because it is only the past week; you may want longer (or shorter) history
14:32davehuntekyle: right, that&#39;s helpful.. thanks! :)
14:44AutomatedTesterbrb
14:56davehuntekyle: limiting by date (last 4 weeks) and Jenkins URL (to filter out adhoc/staging jobs) is giving me some useful results, thanks for the tips
14:56davehuntekyle: https://activedata.allizom.org/tools/query.html#query_id=iaNSJjzY
14:56ekyledavehunt: np! glad to be of help!
14:57ekyledavehunt: but you link does not have any filter
14:58davehuntekyle: it has a where with the jenkins URL and start time
14:58davehuntperhaps filter was the wrong word
14:59ekyledavehunt: please send me the link again
14:59davehuntekyle: https://activedata.allizom.org/tools/query.html#query_id=iaNSJjzY
15:41tracyjmaher: ping
15:41jmahertracy: hi
15:42tracyjmaher: Hello Joel. Wondering if you can confirm if some talos tests are knwon broken or not.
15:42tracyhttps://wiki.mozilla.org/Electrolysis/Multi_Release_Criteria#Misc._broken.3F
15:43tracyI added those to that wiki page of e10s-multi release criteria for discussion.
15:43* jmaher looks
15:43tracythanks
15:45jmahertracy: so xres is bimodal with a real value and a missing value- but we could easilyl track a change in that since the 2 modes are consistent
15:47jmahertracy: the 3 tp5n main_* tests; those have a lot of &#39;random&#39; outliers, maybe 5% of the data points; it does make it hard on try to compare those
15:49tracyjmaher: alright, XRes is ok then. Normal and Startup netio have mostly 0 values.
15:49jmaheryeah
15:49tracyis that expected?
15:49jmaherthere was a network fix ~6 months ago that put those almost at 0 all the time
15:50tracyjmaher: So nothing to see here. :) thank you.
15:50marcoekyle: jmaher: is there a bug regarding converting the JS coverage file names to actual paths?
15:53jmahertracy: thanks for doing this
15:53jmahermarco: there is...let me hunt for it
15:55jmaherhmm, not easy to find
15:58jmahermarco: found it: https://bugzilla.mozilla.org/show_bug.cgi?id=1224691
15:58bugbotBug 1224691: Code Coverage, normal, cmanchester, NEW , Rewrite JS coverage lcov output based on what the preprocessor did
16:22marcojmaher: thanks!
16:43marcochmanchester: are you still going to work on that bug?
16:46jmahermarco: I think gmierz will be working on that next week
16:46marcojmaher: cool, thanks
16:47chmanchestermarco: I don&#39;t plan to, but I&#39;d be happy to get that re-based or whatever is necessary to get it landed
16:47chmanchestermarco: I think the only thing remaining was finding an appropriate reviewer
16:59marcochmanchester: perhaps mshal (who reviewed bug 1349640)?
16:59bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1349640 Code Coverage, normal, cmanchester, RESOLVED FIXED, Figure out what to do with header files in build directory that are symbolic links to files in the source directory
17:04marcochmanchester: was https://people-mozilla.org/~cmanchester/jscov/ a full report? really few lines compared to the C/C++ code
17:05chmanchestermarco: perhaps, it&#39;s not build system code exactly, someone closer to the code coverage project might be a better candidate at this point
17:05chmanchestermarco: I&#39;m pretty sure that was a single directory
17:06chmanchestermarco: at the time I found a number of issues with the lcov output including many missing lines
17:06marcochmanchester: I found https://bugzilla.mozilla.org/show_bug.cgi?id=1364940
17:06bugbotBug 1364940: JavaScript Engine, normal, nobody, NEW , JSVM code coverage generates SF records that span multiple lines
17:06marcowith data: URLs
17:07marcoanyway, I can help reviewing
17:09chmanchestermarco: bug 1227735 and bug 1229277 were the two I was thinking of
17:09bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1227735 Code Coverage, normal, nobody, NEW , Incorrect lcov output for xbl sources
17:09bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1229277 JavaScript Engine, normal, nobody, NEW , LCOV output emits 0 coverage for some files
17:10chmanchestermarco: if you can take a look at bug 1224691 and verify that&#39;s how we&#39;d like to do that (I can imagine you might prefer to do that work somewhere else in the pipeline), I&#39;ll get those rebased and request review from you
17:10bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1224691 Code Coverage, normal, cmanchester, NEW , Rewrite JS coverage lcov output based on what the preprocessor did
17:11marcochmanchester: ok, I&#39;ll take a look soon
17:34jgrahamato: Did you see the dev.platfrom email about removing Task.jsm?
17:35atojgraham: Yes, I forwarded it to tools-marionette@ some time ago.
17:35jgrahamato: Oh :)
17:36jgrahamThat presumably doesn&#39;t include mationette, but there&#39;s a patch
17:36jgrahams/patch/script for generating a patch/
17:36atoOh I skipped that part.
17:39maja_zf|afkbbl
17:39atoThe change doesnt seem too hard to make manually, actually.
17:39atoIts basically s/Task.spawn/(async )()/
17:40atoIf you can forgive me a terrible regexp.
17:44jgrahamirc magic regexp syntax
17:49bkellyanyone seen this cool error before? WindowsError: [Error 206] The filename or extension is too long: &#39;y:\\hg-shared\\8ba995b74e18334ab3707f27e9eb8f4e37ba3d29\\.hg\\data/mobile/android/tests/browser/chrome/tp5/amazon.com/www.amazon.com/Kindle-Wireless-Reader-Wifi-Graphite/dp/B002Y27P3M/%5C%22http%3A/g-ecx.images-amazon.com/images/G/01/kindle/shasta/photos&#39;
17:49bkelly[log]
17:56jmaherbkelly: yeah, the guid for hg-shared is too long, there are path limits- is that in automation?
17:57bkellyjmaher: btw, I&#39;m hoping bug 1364276 and bug 1364277 help reduce some service worker orange a bit
17:57bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1364276 DOM: Service Workers, normal, bkelly, ASSIGNED , dom/test/mochitest/fetch/fetch_test_framework.js races service worker state change
17:57bugbotBug https://bugzilla.mozilla.org/show_bug.cgi?id=1364277 DOM: Service Workers, normal, bkelly, ASSIGNED , dom/workers/tests/serviceworkers/test_controller.html races service worker state
17:59jmaherbkelly: that would be great to reduce some intermittents
18:30atomaja_zf|afk: Sorry, I have one more review to ask of you: https://reviewboard.mozilla.org/r/139348/diff/2#index_header
18:30atomaja_zf|afk: (Fixing the failing Mn test to not rely on a hardcoded value.)
20:26AutomatedTesterRyanVM: what is the status of WIn10, I appear to have picked it somehow on tc and getting a weird try failure
20:26RyanVMwin10 on TC is mostly in testing still with a few jobs running as Tier 2 in production now
20:27AutomatedTesterok
20:27AutomatedTesterI have added jobs for the other windows to see if it will pass
20:27RyanVMyou can check m-c as well and see if the jobs are running there
20:27RyanVMand ignore if they aren&#39;t
20:27AutomatedTesterI looked at m-i
20:28AutomatedTesterI didnt see any win10 there
20:28RyanVMhttps://treeherder.mozilla.org/#/jobs?repo=mozilla-central&filter-searchStr=win%20x64%20tc
20:33AutomatedTesterRyanVM: its totally unrelated code that is failing... like I can&#39;t nor can ato see how it is related or failing
20:40AutomatedTesterRyanVM: how do I backfill on treeherder, i thought I knew but I can&#39;t seem to figure it
20:42philorStep 1. You avoid wanting to backfill a Taskcluster job, since that&#39;s broken, that&#39;s the reason trees were closed on Friday, and was fixed not by making backfilling work but by shutting of SETA so you wouldn&#39;t want to backfill
20:43AutomatedTestergot it
20:43AutomatedTesterphilor: lucky me I dont want TC :D
20:43philorselect the job, black bar toward the bottom, click the expressively-named &quot;...&quot; menu, Backfill
20:45philoreither it works, eventually, possibly showing you that they are running eventually, possibly not showing until you refresh the whole page, possibly not showing until they finish, or perhaps you don&#39;t have permission, or maybe you need to log out of treeherder and back in, or it&#39;s broken for what you want, too
20:46AutomatedTesterphilor: I can&#39;t seem to get the pinboard to appear. I did &quot;add jobs&quot;, selected a few and then did &quot;Trigger New Jobs&quot;
20:46jmaherAutomatedTester: the add new jobs worked for me earlier, but not in the last 2 hours
20:46AutomatedTesterjmaher: ah, ok, it might be a real issue then
20:46AutomatedTesterI am trying to add windows jobs
20:47jmaheryeah, I had trouble with that
20:47jmaherwindows is 100% outside of taskcluster
20:47AutomatedTesterI have a weird win10 issue which is TC
20:47jmaherit worked great 4 hours ago, so something odd happened, not sure if it is treeherder, pulse_actions/mozci, or something else
20:47jmaherok, win10 is 100% tc; although that could still be treeherder or pulse/mozci
20:48AutomatedTesteryup
20:49jmaherthe tc backfilling problem was specifically tc jobs that used BBB
20:50jmaherthat is osx* and linux64 talos*
20:56AutomatedTesterhttps://treeherder.mozilla.org/#/jobs?repo=try&revision=6b043f3d869d
20:56AutomatedTesterjmaher|afk: my try push now has Mn jobs appearing on windows
16 May 2017
No messages
   
Last message: 69 days ago