mozilla :: #build

9 Aug 2017
13:49Alex_GaynorMmm,so ocasionally while running ./mach build on macOS, the build hangs after "Packaging mozscreenshost@...". Sampling both the mach processes, they're spending 100% of their time in select(), but there's no child processes running. Is this a known bug?
13:58Alex_Gaynorlldb seems to indicate that it's specifically in a multiprocessing.Connection.poll call.
14:01Alex_Gaynorkilling it with control-c gives me the following traceback(s)
14:01Alex_Gaynorhttps://www.irccloud.com/pastebin/59kilprD/
15:04tedAlex_Gaynor: hm, interesting
15:04tedAlex_Gaynor: maybe it's losing track of something?
15:05Alex_Gaynorted: seems like a reasonable bet, I don't know enough about the mach code to guess what it's losing track of or why though.
15:05Alex_GaynorI gather this isn't a known bug then?
15:05tednot that i've heard of, certainly
15:05tedAlex_Gaynor: what version of macOS?
15:05Alex_Gaynorlatest, 10.12.6
16:09tedgps: https://source-highlight.herokuapp.com/https://gecko-generated-sources.s3.amazonaws.com/318c1e8eb64b13914272e71f5c084728437cbe739235e89199ff606667d7f6a80e6e65e1da4a226a24737b4c03cdba307fc820c1a0222e929eb4906aba073e48/dom/bindings/AudioNodeBinding.cpp#l-309
16:09tedmade a tiny flask app with pygments
16:10tedseems to work better than any of the JS source highlighting libs i tried
16:16padenothave you tried the one github replaced pygments with ?
16:22tedpadenot: no, i hadn't heard of that
16:22padenotthey used to use pygments, because it rocks, but github is a rails app, and calling python from ruby is not ideal
16:22tedpadenot: i'm just trying to get something useful stood up here so i can make links to generated source files from crash reports work
16:23padenotso they ended up writing something in js, that they are also using for atom, their editor
16:23tedinteresting
16:23padenothttps://github.com/atom/highlights
16:23padenotI've heard good thing, haven't tried it myself
16:23padenotbut pygments sure is amazing
16:23tedseems to do a reasonable job here
16:24tedand really as long as it can handle C/C++/Rust it should be fine
16:33tedchmanchester: is there any reason we don't stick the MOZ_AUTOMATION_BUILD_SYMBOLS=0 line in mozconfig.artifact instead of repeating it in every mozconfig? https://dxr.mozilla.org/mozilla-central/source/browser/config/mozconfigs/linux32/artifact
16:34tedoh, wait
16:34tedbecause mozconfig.automation gets included in the middle there, i guess?
16:44gpsAlex_Gaynor: i need to fix the resourec monitor bug this is related to
16:44gpsit's been filed like 5 different ways
16:45Alex_Gaynorgps: hehe, what bug #? I'll go follow all 5 of them :-)
16:45gpsit's one of those paper cuts that is easy to sweep under the bug
16:45gpsbug 1384062 is the most recent one with activity
16:45firebothttps://bugzil.la/1384062 UNCONFIRMED, nobody@mozilla.org AssertionError: assert self._running when shutting down resource monitor
16:47Alex_Gaynorthanks
16:49gpsi suspect it is a simple-ish bug
17:45Callek_cloud9gps: so, why do you consider symlinks "dangerous" in hg, especially if the files involved are near-guaranteed to only ever be used on linux?
17:45Callek_cloud9gps: my desire for discussion lies around the intent to prevent them from being checked in and the symlinked linux files I added in testing/mozharness/configs/...
17:46* Callek_cloud9 has edited hg symlinks from windows before too, since they expose as a flat txt file with the filename-pointed-at in the expanded repo, (but file metadata is still symlink) so if I edit the file I can change the symlink target :-)
18:59dmajorfroydnj: I'm assuming it goes without saying that you shouldn't flip lto if you're working on stylo perf? (or is it really just code size and not any runtime benefit?)
19:05froydnjdmajor: IIRC, there was a small-to-nonexistent performance win with LTO on Windows and Linux, and a very significant win on Mac
20:36RyanVMfroydnj: on the bright side, as long as people don't accidentally fold that LTO patch into another one they're working on, they won't be able to push it
20:36RyanVMwe have commit hooks that'll barf on it not having a bug # in the commit message
20:36RyanVMif you wanted extra insurance, throw some try syntax in there while you're at it :P
20:40tedgps: bleh, so i had this entire patch series for the "upload generated source files" basically done and then realized that it's going to wind up recalculating the hashes for every file every time we dump symbols
20:40tedi guess it's probably not the end of the world since 99% of them are only going to be used in libxul anyway?
20:41gpsi don't follow what the problem is
20:42tedgps: i&#39;m storing them in s3 as <sha-512 hash>/<objdir-relative-path>
20:42tedso i wrote the patch to calculate the hashes as it uploads them
20:43tedbut then during symbol dumping, we also need to calculate the hash so we can rewrite the FILE lines so socorro can generate URLs
20:44tedthis is probably not worth optimizing though, since most of the files are probably either tiny or only in libxul, so we&#39;d probably only have to actually hash them once during symbol dumping
20:45gpsdepends how much data. SHA-512 is only a few hundred MB/s.
20:46gpsbut even with hashing a few GB, I suspect we could find those lost seconds elsewhere with less effort
20:48AutomatedTesterfroydnj: your --enable-optimize email just made my day :D
20:48AutomatedTesterI loved the Q&A
20:48gpsperfect example of the &quot;why we can&#39;t have nice things&quot; that you deal with maintaining a build system
21:08froydnjAutomatedTester: I&#39;m glad you enjoyed it! :)
21:14gpsglandium: i assume you have plans to get gtk3 and rust ported to the new toolchains tasks and tooltool manifests for builds will be removed? i&#39;m asking because artifact builds are still downloading these from tooltool and i&#39;m wondering if it is worth eliminating the tooltool manifest for that task
21:15glandiumgps: they probably don&#39;t need them
21:15glandiumgps: rillian said he&#39;d look into rust
21:16glandiumand for gtk3, I&#39;m thinking about moving to debian-based docker images
21:16gps\o/
21:17glandiumgps: do a try push with the TOOLTOOL_MANIFEST removed? although that might make it use a tooltool manifest defined in mozharness, I&#39;m not sure if the relevant one is gone from there. I removed a bunch but had to leave some
21:19gpsi may try that
21:20gpsi should know the answer to this, but do we use the artifacts produced by artifacts builds at all? i think the answer is &quot;yes&quot; because try tasks can substitute in the artifact build task
21:21gpsthe thought that spurred this question is how much work would it be to remove mozharness from artifact build tasks :)
21:24glandiums/from artifact build tasks// ;)
21:24gpsyou are saying change the build configuration for the build task to do an artifact build? if so, yes :)
21:31gpsi&#39;ve also wanted to upload custom artifacts so we don&#39;t need to post-process files as part of doing an artifact build
21:31gpsdo that processing as part of the build and then artifact builds just slurp data to disk
22:44glandiumyay bug 1350646 \o/
22:44firebothttps://bugzil.la/1350646 NEW, kmaglione+bmo@mozilla.com Remove UI, extension, and child_process modules
22:45glandiumgah, there&#39;s still a lot in addon-sdk/ :(
22:45glandiumhttps://hg.mozilla.org/integration/mozilla-inbound/file/c3108aebee35/addon-sdk/moz.build
22:47glandiumah, but all the xpis are gone... so this rule should not be used anymore: https://hg.mozilla.org/integration/mozilla-inbound/file/c3108aebee35/addon-sdk/Makefile.in#l11 \o/
22:54gpso rly?
22:54gpsthat rule was killing us on windows
22:54glandiums/on windows//
22:55glandiumgps: I emailed kris so that he looks into cleaning up that makefile
22:55RyanVMbug 1388288 is sadmaking :(
22:56firebothttps://bugzil.la/1388288 FIXED, cpearce@mozilla.com Make playback/webaudio code in dom/media build in non-Unified mode
23:00RyanVMoh nvm, misread that bug :)
23:01glandiumRyanVM: it&#39;s still sadmaking, but we knew unified builds would quickly break building without it
23:01RyanVMyup
23:02* RyanVM wasn&#39;t a fan of killing those either
23:10froydnjRyanVM: we see that Servo is now importing code from more sophisticated browser engines ;)
23:12froydnjone downside of building non-LTO is that cargo produces zero output from mach--at least in emacs, not sure about from a terminal
23:12froydnjlooks like things have totally hung
23:12glandiumfroydnj: that happens with lto too
23:13froydnjglandium: well, yeah, but I haven&#39;t gotten *any* output about packages being compiled atm
23:13froydnjand I normally get &quot;Compiling X...&quot; and so forth
23:13glandiumfroydnj: that doesn&#39;t sound right
23:13glandiumlike, it can&#39;t really be related
23:14froydnjglandium: weird buffering issues somewhere along the way?
23:16RyanVMwould we ever consider bringing back non-unified builds?
23:18froydnjdoubtful
23:18froydnjmaybe as a one-off on automation?
10 Aug 2017
No messages
   
Last message: 13 days and 22 hours ago