mozilla :: #ateam

13 Feb 2017
04:02xidornwhat is the source of treeherder's failure summary tab? is that from the errorsummary or the raw log?
07:35KWierso|afkxidorn: parsed raw logs for buildbot jobs, some submitted job artifact for taskcluster jobs, iirc
10:24AutomatedTesterjgraham: since you're working in this area, can you review please
10:26jgrahamAutomatedTester: Yep, I'll do that today
10:26AutomatedTesterthank you!
10:48Ms2gerAutomatedTester, who deals with mochitest nowadays?
11:07jgrahamMs2ger: hah
11:07jgrahamMs2ger: ahal is your best best I think
11:11xidornKWierso|afk: thanks
12:47wlach|afkKWierso|afk: xidorn: it's actually the raw log for taskcluster too
14:53whimbooato: can I admit that I still hate about:blank?
14:53atowhimboo: No.
14:54atowhimboo: Let me teach you about the joys of about:blank:
14:54atoI guess the first sentence, none the less, of that page illustrates it: about:blank is probably the hardest Web page to load.
14:55atoIt might make sense to abandon all hope of web progress events for about:blank and just load it.
14:55atoWe might have to reflect this in the spec.
14:56whimbooato: wow. i will read through
14:56atoNote that its fairly old.
14:56whimbooato: i feel we should not hvae re-enabled about:blank for new tabs :(
14:56atoBut we know some of the things still apply.
14:56whimbooi have cases where I do get no events at all
14:56whimboonor progress events
14:57atoI think we have two choices: Either to use the default new tab page, which causes plugin containers to launch on Windows, or to ignore web progress events for about:blank altogether.
14:58whimboowell, in my case its loading a xpi when about:blank is open
14:58whimbooit works well for all pages except about:blank before
14:58atoPlease explain?
14:59whimbooif about:blank is open and I trigger a get for an XPI no events are fired
14:59whimbooas far as I can see now
15:00whimbooi got test_navigation to pass meanwhile
15:00whimbooalso for framesets
15:00whimbooand added three more failing cases from before
15:00atoBut if youre on and navigate to XPI, it all works fine?
15:00whimboocan also be about:home
15:01atoSo simply being on about:blank causes the web progress listener to be confused and not send events until the navigation to a new protocol is completed?
15:02whimbooits only about:blank. other about pages work
15:02whimbooi can dig more into it, or we can have workarounds for now
15:03whimboomaybe I simply miss something
15:03atoIm fine with workarounds for now. The situation isnt worse now than it was before, presumably.
15:03whimbooato: its already well better for my local testing
15:03atoBut it worries me that a new tab lands on about:blank, which we cant support very well.
15:04atoIf the current document is about:blank, can you navigate to https:// and http://?
15:04whimbooato: we can revert the patch which we landed lately
15:04atoWell, is about:home any better?
15:04whimboobug 1333736
15:04bugbotBug Marionette, normal, hskupin, RESOLVED FIXED, Disable usage of new tab page now that loading about:blank twice is no longer an issue
15:05atoIm considering whether we should configure Firefox to use data:, as the new tab page.
15:05atoWe only use about:blank because it doesnt involve network requests and drawing.
15:06atoAre data: documents safe to sue?
15:06whimboohm interesting. i actually do not see the install notification
15:06atoI.e. can you do data: http://*/addon.xpi?
15:06whimboolet me further check if that might even be a different issue
15:06atoRight, .xpis could be special.
15:07atos/could be/are/
15:07atoBut I would seriously suggest seeing if setting the new tab page to "data:," would help.
15:08atoThen we could file bugs for the web progress listener and about:*.
15:09whimbooato: i will check with various urls today
15:10whimbooato: i will also have to check if we can do the page load strategy thing with my current code
15:10whimbooi hope so
15:29whimbooAutomatedTester: btw. for bug 1078237 you didnt run any test for android
15:29bugbotBug Marionette, normal, nobody, NEW , Intermittent TestSwitchFrame.test_should_be_able_to_carry_on_working_if_the_frame_is_deleted_from_under_us | AssertionError: 0 != 1
15:29whimboomaybe trigger those before landing the patch
15:29AutomatedTesterwhimboo: why?
15:29whimbooif there is a permanent failure it will be disabled again
15:30AutomatedTesterwhimboo: the patch is only enabling windows... how would this affect android?
15:30whimboooh wait
15:30whimbooyeah dismiss it
16:41davehuntekyle: can the bucket activedata pulls from be public? would that work?
16:41ekyleyes, that would work
16:43davehuntekyle: great, that's what we're thinking
16:44davehuntekyle: as it's going to be public through ActiveData anyway, right?
16:47ekyledavehunt: yes, that is right. It would be nice to figure how to do this securely, but that's not needed here
17:28sewardjjmaher: ping
17:28jmahersewardj: pong
17:28jmahersewardj: I suspect you are going to ask me about valgrind cron job and where are the actual valgrind tests? :)
17:28sewardjjmaher: hi!
17:29jmahersewardj: how was your weekend?
17:29sewardjjmaher: I was just thinking up a way to enquire
17:29sewardjjmaher: busy .. but went to a good cake shop in Berlin
17:29jmahersewardj: funny, about 10 minutes ago I looked at the cron job for code coverage which landed a couple days ago and it doesn't have tests running either, so I have 2 reasons to investigate this
17:30jmaheroh, cake shop!
17:30jmahernot php-cake I assume
17:31sewardjjmaher: if you're ever in Berlin (we have a nice office :-) I recommend it:
17:31sewardjjmaher: ok well that would be cool (re the code coverage)
17:32jmahersewardj: more reasons for me to got to Berlin now
17:32sewardjjmaher: it would be nice to have the automated runs since at the moment I am falling over definedness regressions only on an ad-hoc basis
17:33jmaheryes, we have the same problem with code coverage, let me go ask dustin right now
17:34sewardjjmaher: cool
18:25davehuntahal: could you suggest a try syntax for my mozlog patch?
19:01ahaldavehunt: I'm not sure if you can run the marionette-harness tests with try syntax
19:01ahalbut I think you're right.. there's not much using pytest-mozlog in mozilla-central
19:08ahaldavehunt: are there any mozlog tests for the pytest-mozlog plugin?
19:08ahalif not, then there's not much point in running those either
19:31tedahal: it would definitely be nice if our python unittests actually used structured logging
19:31tedAFAIK the mozunit logger adaptor does not
19:31ahalted: yeah. I'd like to get mozunit to use pytest, and then we could use davehunt's pytest-mozlog plugin to stream structured logs out of it
19:32ahalI think that might actually be a relatively trivial change
19:32ahalsince pytest should *just work* with unittest based tests
19:33ahal(not saying I'm planning to do it.. just that we should :p)
19:33ahalI guess I should at least file a bug
19:37atoGreat. Now I know a single inconsiderate test that doesnt clean up its state can fail 800 tests.
19:37atoPlease dont ask me how long it took to find that out.
19:38atoahal: On that note, it would also be nice if we didnt have to write unittests at all with Marionette. (-:
19:39ahalato: you mean, use pytest for the Mn task?
19:39atoahal: Yes.
19:39ahalagreed.. but that's definitely not something I'm volunteering for :p
19:39atoahal: No, of course not.
19:39atoahal: I remember thinking some time ago that it might be a decently-sized intern project.
19:40ahalactually, I almost did it by accident the other day
19:40atoahal: But then, we also want to get rid of a lot of Mn tests in favour of Wd/WPT tests.
19:40ahalI confused the main manifest with the harness test manifest
19:40ahaland pytest was failing
19:40atoahal: But since Mn is used by developers internally for other things, it would be nice of us to provide something slightly less fugly than unittest to them.
19:40ahaland I started debugging, but realized I was running the wrong tests
19:41ahalato: yeah, I think using pytest is win-win, especially since they can still use unittest if they really want to
19:42atoYes, and it is _so_ much nicer to work against on an internals level.
19:42ahaland _so_ much nicer to figure out what failed in the test from the log
19:43atoOh yes.
19:43atoWith the work I did on integrating pytest with wptrunner for the WebDriver WPT tests, I found pytest lends itself well to the functional OOP tests we do.
19:44atoAlthough fixtures have a wiff of magic about them that I dont particularly like, they do make tests more composable.
19:44ahalI love fixtures.. they are a bit confusing at first, but once you understand what's happening, you can write tests with practically 0 boilerplate
19:44ahaland no inheritance
19:45atoYes, absolutely.
19:45atoThe thing I dont like is how theyre implemented. They appear to come out of nowhere with injected code at runtime, etc.
19:45KWiersoato: ping
19:46ahalyeah, it makes reading pytests when you aren't familiar with them quite confusing
19:48atoKWierso: ?
19:48KWiersoato: any idea what would be needed to fix ?
19:49atoKWierso: Yes. We can lower the expected number that I picked out of the air.
19:49atoKWierso: 150 is an arbitrary number I thought was low enough for even the slowest environment.
19:50atoKWierso: Looks like I was too optimistic (-:
19:50KWiersoif you can put up a patch, I can land it
19:50atoKWierso: We wont be sacrificing quality by committing a new number, but I can go via code review youd like.
19:52KWiersoato: it's just bumping right?
19:53atoKWierso: Yes. You can s/150/15/ or something.
19:54atoKWierso: Thats it. I can prepare a .patch if you want.
19:54KWiersoato: go for it
19:55atoRight, this is on autoland. Would you like a follow-up to that patch, or a modification of it so you can back this patch out?
19:58KWiersoa followup's fine
20:03atoKWierso: ty!
20:14atowhimboo, maja_zf|afk: Sorry if you were asked to review something you already have on, but the mozreview/Bugzilla r? state was broken when I unset them last time.
20:14bugbotBug 1333014: Marionette, normal, ato, ASSIGNED , Return element click intercepted error when clicking obscured element
20:14whimbooato: np
20:14whimboowe all know about the weirdness involved with re-reviews
20:15whimbooato: btw. re get()... I assume such an exception message is a bit too long?
20:15whimbooanything you would prefer?
20:15whimbooonly "Failed to navigate to 'about:doesnotexist'" ?
20:16atowhimboo: I think thats absolutely fine.
20:16whimbooto have the full message?
20:16whimbooit indeed helps to identify the problem
20:17atoYes, and it follows Unix tradition by appending the more specific error message after a colon.
20:17whimbooato: its bad that wptserve doesnt correctly set the HTTP response code for 404 pages
20:17atoAlso this should not happen, so if it does we should be specific.
21:10wlachAutomatedTester: whimboo: does geckodriver support getting the browser log? I'm getting an error from my script
21:10AutomatedTesterwlach: nope
21:11wlachAutomatedTester: :( is there any alternative if I want to do that?
21:11whimboowlach: what do you need?
21:11wlachwhimboo: I want to see if there are any messages in the browser console, and fail if so
21:12wlachah, hmm
21:13AutomatedTesterwlach: when we do we will have the necessary infra to do that
21:13bugbotBug 1250290: Marionette, normal, dburns, NEW , Make performance timeline data available via WebDriver
21:14wlachAutomatedTester: if not console.log output, can I at least check for errors when loading the page?
21:15AutomatedTesterwlach: um, tbqh... not sure
21:17wlachAutomatedTester: google suggests not
21:18AutomatedTesterwlach: there must be a way since RUM tools seem to show them up
21:18whimboowlach: maybe by calling some Gecko API via execute_script
21:35mcotedoes the autoland branch get merged directly into m-c or does it go into m-i?
21:36mcoteoh nm seems direct
21:36ahalyep, direct
21:36mcotedifferent question then: how can I get a complete (linear as possible) history of m-c? like if I wanted to know how many commits were landing per day?
21:36mcotethe merges seem to complicate things
21:37ahalmcote: you mean you want to count commits, but skip merge commits?
21:38mcoteit seems that a simple "hg log" won't even get the full commit list including merge commits
21:38* mcote sounds like a total n00b he realizes :)
21:38ahalyou can use the "not merge()" revset
21:38ahale.g, hg log -f -r "not merge()"
21:38ahal(after updating to wherever you want to start from)
21:39ahaloh hm, that doesn't seem to work
21:40mcotedoesn't give me the changesets that were merged in
21:41mcoteglad to see this isn't a trivial thing heh
21:42ahalok, I think passing in the -r was overriding the -f.. so try `hg log -r "ancestors(.) and not merge()"`
21:44mcoteheh that's certainly making hg think
21:44mcotehm nothing
21:45ahalmcote: so, that is basically just the same as hg log -f, but skipping merge commits
21:45ahalwhat exactly are you trying to find?
21:45mcotethe total number of commits to m-c every day, from any branch
21:45mcotelike the sum of autoland + m-i + whatever + direct commits to m-c
21:46ahalmcote: so, can you wait for those commits to merge to m-c before counting them?
21:46mcotebasically I want to calculate num_commits(autoland) / num_commits(mozilla-central)
21:46ahalsince they all end up on m-c, you should be able to only count them there
21:46mcoteyeah that's fine
21:46mcoteyes exactly
21:46mcotethat's what I'd prefer :)
21:46ahalso, why is that revset not working?
21:46ahal(it seems to work for me)
21:46mcotedunno :) it just returns an empty set
21:46mcotewell I was trying to limit by one day
21:46mcotemaybe that's an issue
21:47mcotelemme try last 7 days
21:47ahalthere's a date revset too, see
21:47ahal(or hg help revsets)
21:48ahalyou'll probably want something like `hg log -r "date(blah blah blah) and not merge()"
21:48mcoteyeah I was trying -d
21:48mcotebut maybe that's not the right way to do that
21:48mcoteokay lemme try "date(-7) and ancestors(.) and not merge()"
21:48mcoteto get the last week's worth
21:48ahal(I don't know how to use the date revset off the top of my head)
21:49mcotesupposedly -X means last X days
21:49mcoteif I'm reading that correctly
21:49ahalyou can probably skip the ancestors thing if you're using date
21:49ahalespecially if you don't have local commits in your local repo
21:49ahalyou could throw in "and public()" to be safe
21:49mcoteahh here we go
21:50mcoteyeah date(-7) and no ancestors(.) seems to work
21:52mcotehg log -r "date(-7) and not merge()" seems to give me 721 commits
21:52mcotethat seems low, no?
21:52mcoteespecially since that same commit on an autoland clone returns 761 commits...
21:53KWiersomcote: I'd guess we average ~200 commits per weekday, less on weekends, so that almost averages out
21:54mcotebut I guess m-c is merged back into autoland?
21:54ahalyeah, that's probably it
21:55KWiersothe three trunk branches should be roughly equivalent once merges happen
21:55mcotecan I figure out what was committed directly to autoland?
21:55mcoteI can figure that out from autoland logs probably
21:55mcotebut it would be easier from hg
21:56KWiersothat sounds like a question your should hunt down gps to ask
21:56ahalthere's probably a way to do not an ancestor of any merge with revsets
21:56mcotewho is not here :\
21:56mcotesmacleod: how're your revset skillz? :)
21:57ahalsmacleod might be another good person to ask.. he came up with a crazy revset for me once
21:58KWiersothere was also the graphics branch merge last friday, which was 408 changesets on its own
21:58smacleodmcote: sooo, what are you trying to do?
21:58KWiersodunno if that would throw off your count
21:58mcotesmacleod: trying to get a list of all commits directly to autoland
21:58mcotesmacleod: i.e. not those merged in from m-c/m-i
22:00KWiersomcote: fun wrench to throw in: not all of the merges from integration branches happen via merge commits
22:00mcotesmacleod: hm is there an easier way to query that?
22:01jesupping - treeherder/try question
22:01KWiersojesup: hi
22:01jesupI pushed a try with -u mochitest-media,mochitest-media-e10s
22:01jesupI see e10s mda tests
22:01jesupI see non-e10s mda tests on mac and windows
22:01jesupI see none on linux
22:02jesupI added jobs, and selected linux tc-M mda tests, and triggered. Still no non-e10s mda tests
22:02jesupKWierso: hi! any idea what's up?
22:02smacleodmcote: if this doesn't cover what you need, we can figure other things out
22:03jesupDid we kill non-e10s linux tests??
22:03* jesup still needs to be able to trigger them
22:03ahaljesup: not that I know of.. but the busted A task is why the Add New Jobs thing didn't work
22:04ahalarmenzg would normally be the person to ask about that, but he's pto until the end of March
22:05KWiersojesup: looks like I got them triggered
22:06KWiersono clue why they didn't get scheduled normally
22:09KWiersojesup: though it looks like you're still missing the android mda1 chunk
22:09mcotesmacleod: hm, if I'm reading this correctly, that looks like 1690 changesetes, which seems too high
22:09mcotefor the last week I mean
22:10jesupKWierso: didn't do mda1, just 2 and 3 (why doesn't mochitest-media hit media-1,2 and 3 by default?
22:12jesupKWierso: thanks. Why didn't my trigger start them??
22:12jesupaha, the busted A task.
22:12KWiersothe action task failed to clone mozilla-unified, by the looks of things
22:12jesupAre other Try tests running non-e10s linux jobs?
22:15KWiersocommonality is they all just run "mochitests"?
22:15KWiersounsure why that'd make a difference
22:15KWiersojesup: probably work filing a bug for... trychooser?
22:16mcotesmacleod: I think that includes merges
22:16mcoteunless KWierso has become an extremely prolific developer :)
22:17KWiersoyou don't know
22:17mcote454 commits in one push; surprised it didn't overload review board ;0
22:17mcoteer ;)
22:18smacleodmcote: yes, just filter them, that's each push
22:19smacleodcount if push != merge else ignore
22:23mcotebut how can I tell if it's a merge?
22:24mcotethe individual changesets in this one merge push I'm looking at have only one parent
22:25mcotei guess I could exclude the sheriffs
22:25mcoteassuming only they ever do merges, and only merges
22:27AutomatedTestermcote: sheriffs do checkin-needed
22:27AutomatedTestermcote: how are you looking at the pushes?
22:27mcotethe pushlog
22:28mcoteit is amusingly difficult to calculate the portion of commits going into m-c that come from autoland
22:28AutomatedTesterone sec please caller
22:28mcotei could corroborate merge commits from hg log with the push log I guess
22:30AutomatedTestermcote: gets the pushes and then calls the other link I shared
22:30mcoteah okay, so if I'm reading that correctly, in each merge push there should be a commit with the description containing "merge to"
22:31KWiersoagain, with the caveat that some merges don't actually create merge commits
22:31mcoteand I suppose the false positive rate is low :)
22:31mcoteah heh
22:31mcoteokay the other way I do it is the autoland logs
22:31mcoteexcept they aren't easily accessible
22:33mcoteokay guess this might work
22:34mcotewill try hacking something together later
22:34mcoteAutomatedTester: thanks!
22:34AutomatedTestermcote: anytime
22:34mcoteKWierso: how often does that happen, do you think?
22:34mcote(merges without merge commits)
22:36KWiersomcote: haven't figured out exactly what causes it (something about multiple heads), but if mozilla-central hasn't had anything land on it before you try to merge autoland back to it a second time, it will say there's nothing to merge, so you just do an | hg up -r foo && hg push -r . central | and those autoland revisions up to rev foo get landed to m-c
22:36KWiersowithout a merge commit
22:36KWiersoso probably once or twice a day, given that the periodic update commits land every morning to m-c
22:36KWiersoand we do around two sets of merges each weekday
22:37KWiersogps's endgame is to eliminate merge configs by killing mozilla-inbound and using a fancy new tool to graft csets from autoland onto m-c
22:38KWiersoso you can wait for the heatdeath of the universe and your problem goes away :)
22:44mcote|afkKWierso: so I'm more interested in filtering out merges *to* autoland
22:45KWiersomcote|afk: hrm, I'd imaging it's much harder to do a mergeless "merge" TO autoland...
22:45KWiersoimagine, even
22:46KWierso(so, all of the merge pushes TO autoland should start with a commit like "merge m-c to autoland a=merge")
22:46KWiersowith "all" and "should" being in scare quotes
22:47KWiersounsure how the new stylo sync stuff would show up :)
22:53davehunthey ahal, hg push try is complaining about remote heads.. is it safe to force that?
22:54davehuntI seem to have lost most of my hg knowledge, but hoping it will come back to me
22:54KWiersodavehunt: sounds right to me
23:00davehuntKWierso: thanks
23:02KWiersodavehunt: though I'd try | hg push -r . try | first and see if the complaints go away
23:03davehuntyeah, I tried that first
23:03KWiersoor enable the push-to-try hg extension and use | ./mach try | instead
14 Feb 2017
No messages
Last message: 8 days and 39 minutes ago