mozilla :: #releng

9 Aug 2017
00:06nthomasoremj: are you still around ?
00:07nthomasn/m, I see you got 1388567 already, thank you
00:08oremjyep, running it now
01:09akiglandium: do you know why mach artifact toolchain would be failing on date-branch post-merge? https://treeherder.mozilla.org/#/jobs?repo=date&selectedJob=121839970
01:10akiName or service not known, seems like
01:11akisame workerType as on m-c though
01:30glandiumaki: it doesn't make sense. It only uses http://relengapi if RELENGAPI_PORT is set, which it should only be when the proxy is there
01:31akihm, i don't see that in either log (busted date or working m-c)
01:32glandiumaki: you don't see what?
01:32akiRELENGAPI_PORT in the logs
01:32glandiumit's presumably set at the worker level
01:32glandiumor whatever sets the releng proxy in place
01:34glandiumone thing is sure: the Nr job is using the relengapi proxy on m-c too
01:34glandiumso the question becomes why can't your worker resolve it on date
01:34akiyeah. if that's the only problem, we're probably fine merging to m-c; i'm worried about hidden bustage though
01:41akiah, i don't see TOOLTOOL_CACHE set
01:46akii think i found it
07:06KWierso|afkanyone around able to see if the #buildduty alerts are going to be a problem? some active directory server running out of memory?
07:09aki-awayaiui mdc1 isn't live yet
07:19aselagea|builddutythose will be downtimed
07:55bogdan_marisHello, I'm running automated update tests for 56.0b1-build5 on beta-cdntest and I see that Linux and Mac machines are down, only windows machines are available, is this a known issue? Should I run them manually?
07:55bogdan_marisHere is a link from treeherder: https://treeherder.mozilla.org/#/jobs?repo=mozilla-beta&revision=da6760885a24&group_state=expanded&filter-tier=3&filter-searchStr=Fxup-beta-cdntest(
07:56bogdan_marisNote that we also ran them manually for 56.0b1-build3 as well but only for mac builds
08:23aselagea|builddutyjlorenzo: were those tests supposed to run on that push?
08:24* jlorenzo takes a look
08:24aselagea|builddutyit's true that we had some backlog on OS X last night, but all the jobs from the above mentioned commit are finished by now
08:25jlorenzotests have to be run against da6760885a24e03e13f3b566f319fc255dbb4027, which matches that push
08:26jlorenzoI don't know anything about the Linux/Mac machines involved, though
08:28aselagea|builddutyI don't think the machines are the issue in this case
08:29aselagea|builddutyif those tests were supposed to run, I wonder why they didn't
08:34jlorenzobogdan_maris: do you know how the tests are scheduled?
08:36* jlorenzo sees http://mm-ci-production.qa.scl3.mozilla.com:8080/job/ondemand_update/78822/ are scheduled on windows only
08:38jlorenzoit seems build #78802 to build #78853 only target windows
09:02aselagea|builddutyso it looks that is expected
09:02aselagea|builddutyjlorenzo: thanks for looking into this
12:02ex-devHi guys!
12:02ex-devI have plan to work on a bug next weekend.
12:02ex-devAnyone had trouble to run a project of mozilla/services via ./please ?
12:02ex-devI got coreutils build errors...
12:05bogdan_marisjlorenzo: Sorry for the late reply I was afk. Hmm, based on the .ini file I loaded It should have scheduled updates on Mac and Linux as well
12:05bogdan_marishttps://wiki.mozilla.org/QA/Desktop_Firefox/Releases/Configs/Fx56b1#Win_64_builds_config_file Here are the config files I used
12:08jlorenzoex-dev: hey! garbas may be able to help you
12:09garbasex-dev: are you using latest master? which os?
12:09jlorenzobogdan_maris: np! I see linux and mac is indeed in the ini file. Do you find a linux/mac job under the latest jobs http://mm-ci-production.qa.scl3.mozilla.com:8080/job/ondemand_update/ ?
12:13bogdan_marisjlorenzo: I can't seem to find one... only windows
12:14jlorenzobogdan_maris: I see. Do you have the logs of what created the Jenkins jobs?
12:15bogdan_marisjlorenzo: Not that familiar with Jenkins tbh so I don't know where I could get them
12:15bogdan_marisI see though that for some reason there was a problem with the .ini that contained mac/linux - http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/1130/
12:16jlorenzobogdan_maris: nice finding! that jobs has a stacktrace
12:16jlorenzo00:41:52 mozdownload.errors.NotFoundError: Specified build has not been found: https://archive.mozilla.org/pub/firefox/tinderbox-builds/mozilla-beta-macosx64/mozilla-beta-macosx64/
12:17* jlorenzo wonders whether that's related to the Taskcluster Migration
12:19bogdan_marisIt's interesting that yesterday I ran the same tests but for different builds and locales for ESR branch, and all was good, all builds (regarding of platforms) ran without problems
12:20jlorenzoESR hasn't been migrated to taskcluster, so the theory still holds
12:21* jlorenzo tries to find where test_packages.json ususally is
12:25jlorenzobogdan_maris: do you have the script called "trigger.py"?
12:31bogdan_marisjlorenzo: I don't think I do, I just load those config files and hit the build option in jenkins
12:32jlorenzobogdan_maris: crap. I think that script will have to be updated. Would you have a copy of `trigger.py`, whimboo? It's the one used in http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/
12:33whimboojlorenzo: why an update?
12:34whimboojlorenzo: that is a build issue
12:34whimboofiles are not at the place mozdownloads expects them
12:34jlorenzowhimboo: that's probably because we migrated mac and windows builds to Taskcluster
12:34whimboofor os x you have to manually check those broken pre builds
12:34whimboono
12:35whimboosee bug 1381774 for details
12:35firebothttps://bugzil.la/1381774 WONTFIX, nobody@mozilla.org Build log files for Tinderbox builds on OS X ending up in tinderbox-builds/mozilla-beta-macosx64/
12:36whimbooshould only happen for 55beta builds
12:36whimbooas source
12:38* jlorenzo doesn't know enough about OSX cross-compiled builds
12:40jlorenzowhimboo: I'm sorry, I don't think I have enough context. b10 failed, but 55.0rc2 (which is like "b14") passed http://mm-ci-production.qa.scl3.mozilla.com:8080/job/trigger-ondemand/1108/console. What changed between bug 1381774 and that Travis job?
12:41jlorenzos/Travis/Jenkins
12:42whimboojlorenzo: talk to nthomas|away in case you have questions. he has the oversight
12:42whimboomaybe 55.0rc2 failed due to something else
12:42jlorenzoI don't see rc2 failing
12:43whimboooh passing
12:43whimbooyes. so its an intermittent issue
12:44whimbooonly happens sometimes that the files are not put at the right location
12:47jlorenzookay, I'll ask nthomas|a.way once he's online. cc bogdan_maris
12:52bogdan_marisThanks jlorenzo, until then I just started to ran them one by one manually
12:53bogdan_marisThis should take a while but it would be nice to get to the bottom of this for future testing
12:54garbasex-dev: which issue were you looking to work on?
12:54garbasex-dev: if you need any help just ping me directly
12:59jlorenzobogdan_maris: agreed
13:08travis-cibuild-tools#1911 (master - b059168 : ffxbld): The build passed. (https://travis-ci.org/mozilla/build-tools/builds/262654501)
14:19philorhttps://public-artifacts.taskcluster.net/RBUoLXiDTAu7xvvQy2CYIg/0/public/test_info/mozilla-test-fail-screenshot_637gui.png
14:22Callekphilor: :::jaw-drop::: -- thats "interesting"
14:23hwinecreator's update?
14:25philorand, ugh, are we still running them at that impossibly small resolution?
14:26philorfrom a look at what we are and aren't running on win10, guess that's a yes
14:29philorguess crashtest (and maybe talos, but not my problem) is the only thing where it's a bit worrisome that we don't test accelerated graphics
14:43Callekhwine: theres a known issue with creators update where we cant create the fake monitor attached to the gpu for our testing and screen resolution changing
14:43Callekso could be related for the resolution
14:47jmaherCallek: I still have a pending review for you on the mochitests for win10 :)
14:47Callekjmaher: ooo I thought I flipped that via mobile last night after I ate.
14:47Callekjmaher: spoiler: r+
14:48jmaherok
14:48jmaherI will wait for the official work, then land it
14:48jmaheras a note, browser-chrome has 3 bugs, one should be fixed today, and the other two we will see where they get this week- things are moving
14:49jmaheralso reftest should run with a few (<5) tests to fix
14:49Callek\o/
14:49Callekjmaher: I think I have the buildbot patch needed for clipboard to run ok on w8 too
14:50jmaherCallek: great
14:50Callekjmaher: we should probably uplift the in-tree patch for that to beta though, since aiui we want the jetpack tests for clipboard running there, so its fine as is.
14:52jmaheryeah, we should uplift much to beta :)
15:08bhearsumAPI/backend folks: the OpenAPI 3.0 spec was released recently - https://www.openapis.org/blog/2017/07/26/the-oai-announces-the-openapi-specification-3-0-0
15:11ex-devgarbas, Sorry by delay, I went to a meeting.
15:13ex-devgarbas, I use Ubuntu Gnome 16, I have tested on opensuse 42 too
15:14ex-devgarbas, I plan learn elm-lang, and work in Bug 1386307
15:14firebothttps://bugzil.la/1386307 NEW, nobody@mozilla.org Expose Nightly updates status to public website
15:19* garbas looks at 1386307
15:38garbasex-dev: i think we can handel this without elm, since it is just a simple status page for nightly
15:40garbasjlorenzo: should 1386307 be only done for nightly or we can extend this &quot;status&quot; page for other channels?
15:42garbasi assume the rule you mentioned could be extended also for other channels, but i&#39;m not that familiar with existing balrog rules
15:46jlorenzogarbas: I think Nightly is a good start. We should be able to extend it to other channels. However the rules are more complex there. So we&#39;ll require a different logic
15:47garbasok i see, &quot;plan&quot; for others but only do nightly
15:49Callekkmoir: to be explicit https://bugzilla.mozilla.org/show_bug.cgi?id=1387831#c15 what say you?
15:50firebotBug 1387831 NEW, bugspam.Callek@gmail.com mochitest clipboard isn&#39;t running on windows anymore
15:50jlorenzogarbas: yup, that plan sounds good :)
16:01travis-cibuild-puppet#1742 (master - b5290ab : Jake Watkins): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262728307)
16:05travis-cibuild-puppet#1743 (production - 75042dd : Jake Watkins): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262730095)
16:13ex-devgarbas, Ok, I just ask because, other frontend projects use elm too. There is any restriction to use JQuery, or is preferable use pure JS?
16:52travis-cibuild-puppet#1744 (master - c2b4559 : Aki Sasaki): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262749521)
16:56garbasex-dev: we can also just handlel everything in backend (python)
16:57travis-cibuild-puppet#1745 (production - 0f74aca : Aki Sasaki): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262751259)
17:29travis-cibuild-puppet#1746 (master - f1c1df6 : Jake Watkins): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262762005)
17:37travis-cibuild-puppet#1747 (production - 58e2bf0 : Jake Watkins): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262764618)
18:01travis-cibuild-buildbot-configs#2803 (production - f0db8c4 : Justin Wood): The build was broken. (https://travis-ci.org/mozilla-releng/build-buildbot-configs/builds/262772928)
18:37travis-cibuild-puppet#1748 (master - ba19e46 : Jake Watkins): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262786411)
18:46travis-cibuild-puppet#1749 (production - 2279631 : Jake Watkins): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262789321)
18:53Callekwhops, I broke buildbot-configs, my latest fix should fix it.
18:55Callekbah I&#39;m messing up here today
18:55* Callek fixes that right
18:57travis-cibuild-buildbot-configs#2805 (production - 755d012 : Justin Wood): The build was fixed. (https://travis-ci.org/mozilla-releng/build-buildbot-configs/builds/262793277)
18:57travis-cibuild-buildbot-configs#2806 (master - 86d62fc : Justin Wood): The build was fixed. (https://travis-ci.org/mozilla-releng/build-buildbot-configs/builds/262793293)
18:59* Callek should take a breath :(
19:01travis-cibuild-buildbot-configs#2808 (production - 2b5ba9e : Justin Wood): The build was broken. (https://travis-ci.org/mozilla-releng/build-buildbot-configs/builds/262794659)
19:04travis-cibuild-buildbot-configs#2809 (master - d61f4d7 : Justin Wood): The build was broken. (https://travis-ci.org/mozilla-releng/build-buildbot-configs/builds/262795971)
19:06travis-cibuild-buildbot-configs#2810 (production - 8cb5589 : Justin Wood): The build is still failing. (https://travis-ci.org/mozilla-releng/build-buildbot-configs/builds/262795979)
19:08Callek:/
19:08* Callek gets ready to back out :(
19:12Callekmaybe, ... one more attempt to get this right.
19:12* Callek whistles the song of apparently testing in production.
19:14bytesizedMy try builds keep failing because they cannot find ccache: Is this expected?
19:14bytesizedhttps://treeherder.mozilla.org/#/jobs?repo=try&revision=03fcbe303640
19:14garbasbytesized: i was just about to ask the same thing :)
19:15bytesizedgarbas: having the same problem?
19:17garbasyes, https://treeherder.mozilla.org/#/jobs?repo=try&revision=c033c79acdb4
19:17travis-cibuild-buildbot-configs#2812 (master - b75fd3f : Justin Wood): The build was fixed. (https://travis-ci.org/mozilla-releng/build-buildbot-configs/builds/262800081)
19:17travis-cibuild-buildbot-configs#2811 (production - b2fd998 : Justin Wood): The build was fixed. (https://travis-ci.org/mozilla-releng/build-buildbot-configs/builds/262800027)
19:22garbasbytesized: errors look identical. i wonder if we can get more info on this from sheriffs
19:22KWiersogarbas/bytesized: buildbot windows is known broken
19:22KWiersojust look at the taskcluster windows builds
19:22KWiersounless you&#39;re trying to do something with mozilla-release or esr
19:23garbasKWierso: I&#39;m trying to point to new url of tooltool and i was hoping to test on try
19:24KWiersogarbas: buildbot-based windows builds are no longer used on trunk or beta, so them being broken is fine
19:24Callekgarbas: BB windows will serve you no value if your patch is against mozilla-central (or a recent beta)
19:24Callekgarbas: that is, right now BB windows is the windows &#39;B&#39; tc windows is the &quot;tc(B)&quot;
19:25* garbas is lost in the try letters :)
19:25garbasthis is my try request: -b o -p all -u all -t chromez-e10s,g1-e10s,svgr-e10s,tp5o-e10s
19:26garbasdo i need to change it somehow?
19:26garbasor i just need to &quot;wait&quot; :)
19:26Callekgarbas: for what you want you can&#39;t easily get around windows support, unless you want to do essentially `./mach try fuzzy` .. `select all` which may be overkill ;-)
19:26Callekerr I probably phrased that bad
19:27Callekgarbas: your push is probably pretty good on try syntax, just ignore those broken windows &#39;B&#39; jobs due to sccache
19:27garbasCallek: tnx. is there a bug i can point to for this?
19:27garbasi suppose this one https://bugzilla.mozilla.org/show_bug.cgi?id=1318172
19:28firebotBug 1318172 INCOMPLETE, nobody@mozilla.org Intermittent TC Windows build mozmake.EXE: *** [configure] Error 1 meaning &quot;cannot get attributes fo
19:28Callekgarbas: 1385053
19:31garbastnx
20:06markcoOpen question. I am trying to edit some retrigger some task from: https://treeherder.mozilla.org/#/jobs?repo=try&revision=f212c73abcbba7aef6ff870228f021ceb16da83e&selectedJob=122050585 , but when I try to inspect the task all I get is small window stating backfill. Is there a way to work around that?
20:10KWiersomarkco: talos is still on buildbot, iirc
20:10KWiersono task to inspect
20:10markcoHmm ok. ty
20:31travis-cibuild-buildbot-configs#2807 (master - 542ed3b : Justin Wood): The build was broken. (https://travis-ci.org/mozilla-releng/build-buildbot-configs/builds/262794630)
21:25travis-cibuild-tools#1912 (master - ce726f9 : ffxbld): The build passed. (https://travis-ci.org/mozilla/build-tools/builds/262842116)
22:05travis-cibuild-puppet#1750 (master - 7c03925 : Jake Watkins): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262855500)
22:13travis-cibuild-puppet#1751 (production - cc03675 : Jake Watkins): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262857832)
23:23travis-cibuild-puppet#1752 (master - ffddcaf : Jake Watkins): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262878185)
23:31travis-cibuild-puppet#1753 (production - 7f5ee49 : Jake Watkins): The build passed. (https://travis-ci.org/mozilla/build-puppet/builds/262879997)
23:53hwinebtang: this might be of interest: https://blog.mozilla.org/blog/2016/10/28/our-role-in-protecting-the-internet-with-your-help/
10 Aug 2017
No messages
   
Last message: 12 days and 5 hours ago