mozilla :: #rust-infra

20 May 2017
01:06dtolnayi would like to help with the upcoming deluge of `log` PRs
01:06dtolnaywould it be possible to get the right permission on rust-lang-nursery/log?
01:08simulacrumcc aturon, acrichto
01:08simulacrumMaybe steveklabnik?
01:08simulacrumNot sure who has perms
01:09acrichtodoing so
01:09dtolnaythx
01:09acrichtodtolnay: done
01:12acrichtodtolnay: libs members in general are owners of the nursery
01:12acrichtojust made you so
01:12acrichtojust avoid the "delete all repos" button
01:12dtolnay\o/
12:37simulacrumFiled https://github.com/rust-lang/rust/issues/42118 and https://github.com/rust-lang/rust/issues/42117
12:37simulacrumcrates.io and git spurious failures
13:02PhrohdohHas anyone built a crates.io/registry to pull straight from disk on the host machine instead of S3?
13:03simulacrumNot that I know of, but it's possible.
13:03simulacrumWell, I assume it's possible, I don't actually know
13:03PhrohdohI don't see any reason it wouldn't be. You just send back an application/octet-stream
14:03carols10cents(the answer is yes, it's been done, sort of https://github.com/rust-lang/crates.io/issues/734#issuecomment-302874950)
14:03carols10centsalso https://gmjosack.github.io/posts/dissecting-cratesio-minimum-mirror/
14:13Diggseyhas something changed in how components are build and packaged on nightly?
14:14Diggseythe `manifest.in` files contain backslashes for those built on windows
14:14Diggseywhich makes them uninstallable elsewhere
14:14Diggseythis didn't use to be the case
14:17Diggseyah, I guess it's the new rust version of rust-installer
14:23Diggseyhttps://github.com/rust-lang/rust-installer/issues/65
15:40acrichtoDiggsey: ah yeah that was the main difference
15:40acrichtoDiggsey: if it's causing problems though we should fix that!
15:44acrichtosimulacrum: those crosstool-ng failures are worrisome
15:44acrichtothey all got repeated 3 times
15:44acrichtoe.g. not spurious
15:44simulacrumNetwork-related
15:44acrichtosomeone's removed something from somewhere
15:44simulacrumWell, I'm guessing they're fine since we have been landing PRs though
15:44acrichtooh ok
15:44acrichtomaybe someone's server was down for a bit
15:44simulacrumI just saw one merge
15:56simulacrumacrichto: So what would it take to make libstd depend on backtrace-rs?
15:56simulacrum(if that's even possible)
16:01acrichtosimulacrum: we'd need to make backtrace-rs compile with no_std, but that's about it
16:01acrichtoor provide a mode to compile with no_std
16:01simulacrumYeah
16:01acrichtowhich shouldn't be so hard modulo the platform sycnhronization primitives
16:01acrichtoI almost had it working awhile ago
16:01simulacrumRight, that's what I thought when I looked at it this morning
16:01acrichtotbh we should just make it unsafe
16:01acrichtoe.g. "needs external synchronization"
16:01acrichtothen it'd be super easy I think
16:01acrichtoand libstd would do it it iself
16:02simulacrumYeah, that'd work
16:02simulacrumunsafe on no_std
16:15simulacrumacrichto: Could we depend on alloc? It'd be nice (we have to?) to be able to use CString inside backtrace. Also, this feels like the wrong place for discussion but I don't know where
16:17acrichtosimulacrum: oh nah discussing here is ok
16:17acrichtosimulacrum: I'd prefer to not depend on alloc
16:17acrichtofor that we could just make a stack-local abstraction
16:17acrichtowhich works for symbols up to N bytes long
16:17simulacrumhm, yeah, and .. panic! if too long?
16:17acrichto(I forget precisely where this was applicable)
16:18acrichtosomething like that yeah seems fine
16:18acrichtoor tbh
16:18acrichtowe could use a global buffer
16:18acrichtowhich could be like a kilobyte large
16:18acrichtohence the "requires external synchronization"
16:18simulacrumJust static mut BUF: [u8; 1024] = [0; 1024]? I guess that'd work
16:19acrichtoyeah
16:27acrichtosimulacrum: so the errors coming out of appveyor look like segfaults
16:27acrichtoand this may be related to https://github.com/rust-lang/rust/pull/41967
16:27simulacrumYou mean rustc failing?
16:27acrichtoyeah
16:27simulacrumHm, yeah
16:27acrichtorustc doesn't just fail w/ no output
16:27acrichtounless it segfaults
16:27simulacrumwhen running the compile-fail test suite when a should-fail test panics.
16:27acrichtoand cargo's not doing a good job and is hiding tha tinfo by default
16:28simulacrumIt seems like the opt-level=2 was only for should_fail tests though
16:28simulacrumUnless that expanded over time
16:28acrichtothat's the only emprical evidence
16:28acrichtodoesnt' mean there's more
16:28simulacrumTrue. I can put up a PR to revert?
16:29acrichtohm
16:29acrichtohave you been tracking these failures?
16:29acrichtoI'd want to confirm that 100% of them came after that PR
16:30simulacrumhmm maybe
16:30simulacrumnot the last one I think I forgot
16:30acrichtook I'll take a look
16:31acrichtook seems so so far
16:31acrichtosimulacrum: ok want to send a revert PR?
16:31simulacrumSure
16:35simulacrumacrichto: https://github.com/rust-lang/rust/pull/42123
16:38acrichtosimulacrum: thanks
17:04simulacrumacrichto: So backtrace::lock uses Once, Mutex, etc. I can get those through https://docs.rs/spin/0.4.5/spin/ which is what lazy_static uses
17:04simulacrumShould we?
17:05acrichtooh lord definitely not
17:05acrichtolol
17:05acrichtospinlocking is like 100% the wrong thing the do on contention here
17:05simulacrumSo just do "nothing" I guess?
17:05acrichtoyes
17:05acrichtogenerating a backtrace takes on the order of milliseconds
17:05acrichtowhich means contenting threads waste tons of cpu just spinnning
17:06acrichtohm lazy_static shouldn't be using this either ...
17:06simulacrumhttps://github.com/rust-lang-nursery/lazy-static.rs/blob/master/src/core_lazy.rs
17:06acrichtoyeah...
17:08simulacrumacrichto: Build scripts always have access to std, right? Like that won't be a problem
17:08acrichtosimulacrum: yes
17:09simulacrumacrichto: And there's a lot of std's c_void and similar, is it fine to replace those with libc's equivalents? Or at least I hope equivalents?
17:09acrichtoyeah that's fine
17:21simulacrumacrichto: I get the feeling that making backtrace no_std without support from alloc or equivalent will be practically quite hard due to things like OsString::from_wide
17:21acrichtough forgot about that
17:21simulacrumThough perhaps we can change the APIs we call to be like std's and then use the buf
17:22simulacrumi.e. SymGetLineFromAddr64 instead of SymGetLineFromAddrW64
17:22acrichtocan we just change to the non-wide versions?
17:22simulacrumShould be able to
17:32nagisanot recommended?
17:33nagisathat is, please do change to non-wide versions only as a last effort
17:33nagisaprefer just using Vec<u16> if needed
17:39simulacrumnagisa: Unfortunately no_std
17:39simulacrumWhich feels harder and harder with time :)
17:39nagisasimulacrum: libcollections is still no_str
17:39nagisano_std
17:39simulacrumacrichto: Do we care about a dependency on libcollections?
17:39acrichtosimulacrum: ideally no
17:40acrichtonot sure how feasible that is
17:40simulacrumI think the main problem is also that the dbghelp library that backtrace depends on doesn&#39;t expose some of the symbols we want
17:45simulacrumacrichto: So basically if we want to not have libcollections as a dep (and with it, liballoc) then we need to choose a size for &#39;string&#39; data that needs to live longer
17:45simulacrumWe used to copy into OsString but no_std doesn&#39;t allow that
17:58acrichtosimulacrum: hm which data is this again?
17:59simulacrumfilename and name on msvc
18:00acrichtothat&#39;s gotta live longer?
18:00acrichtoI thought it was just one stack frame?
18:00simulacrumHard to tell
18:01simulacrumI assume so, since we copied into a OsString instead of using OsStr, but no idea
18:01acrichtoyou mean this basically, right? -- https://github.com/alexcrichton/backtrace-rs/blob/master/src/symbolize/dbghelp.rs#L55
18:01simulacrumWell, sort of
18:01simulacrumI meant the OsString::from_wide calls latter down (https://github.com/alexcrichton/backtrace-rs/blob/master/src/symbolize/dbghelp.rs#L75)
18:02simulacrumThe vec could just be [0; SIZE] -- if we don&#39;t care about the stack
18:02acrichtosure but it&#39;s in theory just a reference into the buffer
18:02simulacrumWell that buffer will go out of scope if it&#39;s the vec, right?
18:02simulacrumOr am I misunderstanding
18:02acrichtonah there&#39;s a callback you give
18:03acrichtoso the original vec! can live on teh stack
18:03simulacrumAh, okay
18:03acrichtoit doesn&#39;t need to persist beyond this one stack frame
18:03simulacrumI guess that makes sense
18:03acrichtoI&#39;d probably just switch to the A variants
18:03simulacrumSince we don&#39;t return anything and call the callback inside ourselves that&#39;ll work
18:03acrichtoand pass that along
18:03simulacrumA variants?
18:03acrichtoSymFromAddrW -> SymFromAddrA
18:03acrichtothat goes from u16 to u8
18:03simulacrumWe can&#39;t I think, those aren&#39;t defined from what I can tell in dbghelp
18:04acrichtohm
18:04acrichtowell so this is interesting actually
18:04acrichtowe can&#39;t pull in winapi as a dep for std yet
18:04simulacrumWhy?
18:04acrichtoso the windows stuff will have to all be vendored
18:04acrichtoone reason is the 0.2 license
18:04acrichtoit&#39;s not the same as rust-lang/rust, just mit
18:05acrichtoit&#39;s also absolutely massive and will likely require some thought
18:05acrichtoof which I haven&#39;t put in much yet
18:05simulacrumMakes sense
18:05simulacrumSo in theory this is probably a bad idea for the time being?
18:06acrichtoit&#39;ll probably just require some more work
18:06simulacrumI guess I can just steal the windows APIs from std
18:06simulacrumbut it feels like that shouldn&#39;t be put inside backtrace-rs (that is, we should more so just copy it into std)
18:07acrichtoI&#39;d be fine adding to backtrace-rs fwiw
18:07simulacrumJust copying them in?
18:07acrichtoyeah
18:34simulacrumacrichto: How should we deal with the non-utf8 format printing? Just call str::from_utf8 and on unwrap print the raw bytes?
18:34acrichtosimulacrum: that&#39;s in libstd right?
18:34acrichtoas in, backtrace-rs isn&#39;t printing anything?
18:34simulacrumfmt::Display
18:35simulacrumSo kind of in backtrace-rs
18:35acrichtooh
18:35acrichtoumm
18:35acrichtoideally String::from_utf8_lossy
18:35simulacrumlibstd
18:35acrichtocan it use a combination of replacement chars and whatnot?
18:35acrichtolike str::from_utf8 but handles errors
18:35acrichtoI think Utf8Error has enough info to do that nowadays?
18:36simulacrumWell, we can simply trim it to the valid utf8 part yeah
18:36simulacrumbut that&#39;s non ideal
18:36simulacrumI guess I can write my own from_utf8_lossy
18:36acrichtooh well so
18:36acrichtothe algorithm of from_utf8_lossy is what I&#39;m thinking of
18:37acrichtobut basically making a no_std variant
18:37acrichtoUtf8Error says &quot;everything up to here was valid&quot;
18:37acrichtoso we can print that
18:37acrichtoand Utf8Error alsos says &quot;the erroneous region was this big&quot;
18:37simulacrumOr... if we&#39;re okay with nightly-only, then char::decode_utf8
18:37acrichtoso we can drop those bytes, print a replacement char, and then keep going with the rest of the bytes I think?
18:38simulacrumYeah, I&#39;ll write a helper that does that
18:38simulacrumI&#39;m guessing this is not the only instance but I&#39;ll check
18:38nagisawhat you should do is `str::from_utf8`, inspect the error, take the valid part, print the valid part, print the invalid character, skip the invalid glyph, decode the rest
18:38nagisachar::decode_utf8 is gonna be slower I think
18:38simulacrumPrint the invalid character?
18:39nagisaU+FFFD
18:39simulacrumah
18:39nagisa
18:39nagisanot like symbols will have any, though?
18:40simulacrumPresumably they can?
18:40simulacrumWe could just &quot;FIXME&quot; and cut it but it doesn&#39;t seem all that hard to be correct
18:40nagisacan they, really? You can certainly just print <not valid utf8 symbol> or something
18:40nagisaIm certain that 99.999% gonna be plain ASCII
18:41nagisaif not 100%
18:41simulacrumacrichto: What do you think about that?
18:41nagisawe do already print an unknown in a bunch of cases, not like having one less line will hurt anybody very muc
18:42acrichtoseems fine by me
18:42acrichtofilenames have non-utf8 more likely
18:42acrichtosymbols are likely just ascii
18:42acrichtoI don&#39;t tink we Display filenames though?
18:42acrichtojust symbols iirc
18:42simulacrumyeah I don&#39;t see Display for filename
18:43nagisafilenames are utf-16 anyway, unless youre using ASCII API in which case it will return data in some code table
18:44nagisayou would have to decode character-by-character in the former case and almost no chance of a sane solution in the latter
18:44nagisawhich is why one should avoid the non *W APIs :)
18:44simulacrumWe&#39;ll try
18:58simulacrumacrichto: Do we have testing on all supported platforms of libbacktrace?
18:58simulacrumOr are there some holes?
18:58simulacrumplatforms and configurations, I guess
19:04acrichtosimulacrum: afaik yes
19:48simulacrumacrichto: So should default-feature = false be enough to build libc?
19:49acrichtosimulacrum: should be yeah
19:49simulacrumhmm, is there a way to track down why it&#39;s being compiled with use_std?
19:51simulacrumacrichto: I guess maybe cargo graph or something?
19:52acrichtosimulacrum: why libc is being compiled?
19:52acrichtoit may be the build script
19:53acrichtoif a build dep pulls it in
19:53simulacrumhm, maybe
19:53simulacrumBut I thought you said build scripts would have access to std?
19:54acrichtothey do yeah
19:54acrichtocargo just unifies deps
19:54acrichtowell try t
19:54acrichtothis --
19:54acrichtocargo build --target $target
19:54acrichtoinstead of just `cargo build`
19:54simulacrumWell I&#39;m trying to build with rustbuild already
19:55simulacrumJust to see if I can build at all
19:55acrichtooh
19:55acrichtoso to do that
19:55acrichtoyou&#39;ll want to overlay w/ a new Cargo.toml regardless
19:55acrichtolike libc does
19:55simulacrumlibc_shim?
19:55acrichtoyeah
19:55acrichtowe need to connect it to the in-tree libc and libcore
19:55simulacrumah
20:01simulacrumacrichto: And I need to do this for all of its dependencies? e.g. cfg-if
20:02acrichtosimulacrum: :(
20:02acrichtosimulacrum: adding deps to std isn&#39;t really don much
20:02acrichto(mainly for this reason)
20:02acrichtoI&#39;d drop the cfg-if dep
20:02acrichtojust vendor the macro
20:02acrichtoit&#39;s tiny anyway
20:03simulacrumAre we sure we can&#39;t like override the deps externally or something?
20:03simulacrumcfg_if could be no_core realistically
20:03acrichtoin theory yeah
20:03acrichtobut it&#39;s basically infeasible to do that right now
20:03acrichtothis is just an open question/bug with carog
20:03acrichtohow to add deps to libstd
20:03acrichtowe don&#39;t have a great answer no matter how you slice it right now
20:04simulacrumYeah, I&#39;m starting to realize that :)
20:04acrichtoheh
20:06simulacrumacrichto: And vendor rustc_demangle too?
20:06acrichtourgh
20:06acrichtothat one sucks
20:06simulacrumno_std too, if a little large
20:07acrichtosimulacrum: it may not be the right time to do this :(
20:07acrichtothis is a pretty big change
20:07simulacrumYeah
20:07acrichtoe.g. also adding two new submodules to the repo
20:07acrichtofor rustc_demangle and backtrace
20:07simulacrumWell, I put up the prototype PR to backtrace to make it no_std and I&#39;ll push my slight fixes to that
20:07acrichtoif we could solve the &quot;crates below std depend on core&quot; problem then I&#39;d be happy
20:07simulacrumI don&#39;t think it works
20:07acrichtoheh ok
20:08simulacrumacrichto: So I actually have another thing to bring up with you
20:08simulacrumWould we be interested in migrating to clap over getopt for arg parsing?
20:08simulacrum(to give us utf8 compat)
20:09simulacrumas well as other niceties
20:09acrichtohm
20:09acrichtoI think I&#39;d be interested yeah
20:10acrichtobut there&#39;s quite a bit of logic to mirror
20:10acrichtoand I don&#39;t think we have a lot of testing for compiler arguments
20:10simulacrumWell, I&#39;m thinking of starting with compiletest
20:10acrichtoso I&#39;d be wary of regressions
20:10acrichtothat seems plausible yeah
20:10acrichtowe&#39;ve already got clap vendored for stuff like the rls
20:10* simulacrum actually has a branch already
20:10nagisasubmodules are a terrible way to vendor stuff
20:11nagisayou have to check out the history of every vendored thing ever
20:11simulacrumCompletely agree but I don&#39;t think we have a better way right now
20:11simulacrumOther than like tarballs on s3 or something
20:11nagisait took me some 5 to 10 minutes to fetch and checkout the whole LLVM submodule on windows a few days ago
20:11nagisaand its a pretty powerful windows machine too
20:12nagisapretty terrible experience if you ask me :)
20:13nagisaI must admit that having access to history is also useful sometimes
20:38simulacrumacrichto: So is there a way to override a third-party crate&#39;s dependency to update it in a semver compatible way?
20:38simulacrumSpecifically, racer depends on clap 2.19 and we need 2.24
20:39acrichtosimulacrum: not currently, no
20:39simulacrum:(
20:39acrichtosimulacrum: although we can updat ethe rls
20:39acrichtowhich has an updated racer
20:39acrichtowhich has a less restrictive dep
20:39simulacrumI don&#39;t think so?
20:39simulacrumLike, racer&#39;s Cargo.toml is 2.19
20:40simulacrumacrichto: So we can&#39;t update rls to change racer in any way
20:41simulacrumI can file a PR with racer to update it&#39;s dep on clap, then file with rls to update racer, then finally we can work
20:41acrichtolooking
20:42acrichtosimulacrum: https://github.com/phildawes/racer/pull/715
20:42acrichtohttps://crates.io/crates/racer
20:42simulacrumAh, but that hasn&#39;t been released yet
20:42acrichtoit has?
20:43acrichto2.0.7 looks irght
20:43acrichtoright*
20:43simulacrum^2.19 ?
20:43simulacrumIs that correct? I guess it&#39;ll work?
20:43acrichtoyeah that&#39;s right
20:43acrichto~2.19 is the &quot;wrong&quot; one
20:43simulacrumHm, makes sense
20:44simulacrumThough ~2.19 is what clap recommends, IIRC
20:44simulacrumhttps://github.com/kbknapp/clap-rs#compatibility-policy
20:44acrichtohttps://github.com/kbknapp/clap-rs/issues/964
20:45acrichtoit technically only recommends if you care about rust version compat
20:45acrichtonot for general clap compat
20:45simulacrumThat&#39;s true
20:45simulacrumI think I managed to get it working
20:45simulacrumAt least, I&#39;m compiling things now
20:45simulacrumThanks!
20:55tomprince_webfrewsxcv: Thanks for the reviews.
22:45simulacrumIf someone who can run forge locally easily could figure out why the first few list items on https://forge.rust-lang.org/pr-triage-procedure.html aren&#39;t rendering that would be amazing
22:45simulacrumI&#39;m at a loss
23:16jmanHello, I&#39;ve discovered a minor glitch in the detail page of crate &quot;https://crates.io/crates/enigma_machine&quot;
23:16jmanThe &quot;repo&quot; link redirects to &quot;https://github.com/takatori/rawst/&quot; which I think is not the intended repo code for that crate
23:17jmanjust reporting here, I don&#39;t where such minor things should be reported
23:17jman*know
23:19IcefozThat does appear to be the case.
23:19IcefozWeird.
23:29simulacrumacrichto: I&#39;m not quite clear on the status of https://github.com/rust-lang/rust/pull/42020
23:30simulacrumIt looks like it&#39;s a &quot;I guess&quot;?
23:32simulacrumacrichto: Also, should we just r+ https://github.com/rust-lang/rust/pull/42040? It&#39;s not really clear to me why we even need review for submodule updates
23:32acrichtosimulacrum: #42020 is probably not going to be merged
23:32acrichtoI just feel bad closing prs
23:33simulacrumI know :/
23:33acrichtolike the state of the world won&#39;t change if we merge it basically
23:33acrichtoalso yeah approving the submodule is fine
23:33simulacrumI&#39;m wondering if it&#39;s worth merging it just so that it&#39;s there
23:33acrichtoeh I guess we could go either way
23:34* simulacrum is doing PR triage
23:45simulacrumacrichto: Actually, I was going to mention: I thought it might be a good idea to get everything our PR builds download saved somewhere, even if we don&#39;t use it for now
23:45simulacrumThat way we don&#39;t have to wait long if something gets removed
23:47simulacrumHuh. I wonder what would happen if I rolled up a beta backport PR
23:47simulacrumThough it&#39;s a weekend, I shouldn&#39;t.
21 May 2017
No messages
   
Last message: 4 days and 22 hours ago