mozilla :: #media

7 Sep 2017
00:08jesupjya: padenot: the PushCaptureData() patch (to avoid External audio API we un-deleted) for sending data on a peerconnection now is green - was missing a change to not multiply by channels since we're passing channels as a param now.
00:09jesupjya: padenot: Now we just need to do something similar (bit more complex) to MediaEngineWebRTCAudio.cpp and we can drop the External audio API
02:57cpearcebwu: I'm on PTO tomorrow, so won't make our regular meeting.
03:05bwucpearce: got it. God bless you that you got to deal with your kids!
03:10dmajorpadenot: do you remember why this EXPORT is necessary? http://searchfox.org/mozilla-central/source/media/libsoundtouch/moz-libsoundtouch.patch#86
03:21dmajoroh; xul.dll imports these from lgpllibs
03:22dmajorso really it should only be EXPORT when we're building this class for lgpllibs
03:23dmajorbut when xul.dll code includes this header, it then causes libxul to export some default things like copy ctor etc.
03:23dmajorI guess I don't care enough to file a bug :/
03:25dmajoralternatively we could move the EXPORT to the methods (setTempo etc.) which would be easy if this were mozilla code but I don't want to pollute the .patch file just for this
05:24jyamattwoodrow: I'm not sure I understand what you want me to do with bug 1392143. I'm my current test, the render test does not fail, only the decoding one)
05:24firebothttps://bugzil.la/1392143 NEW, jyavenard@mozilla.com Video decoding isn't hardware accelerated inside GPU process
05:25jyaAnd if we want to enable HW decoding on those box, we must make the test visible.
05:28jyadmajor: not sure if that's what your talking about. But to determine the path of the mozavcodec.dll lib, that are lazily loaded, I needed to read a publicly exported symbol lgpllibs as its found in the same location
07:00bakujesup: pong?
08:00mattwoodrowjya: Well now Im very confused then
08:00mattwoodrowIf the readback was being broken due to window position, Id expect it to be equally broken for pixels over the background color, as well as pixels over the video
08:01mattwoodrowWhy when we readback the window, does part of it readback fine, but pixels that happen to have a video in them readback broken?
08:01mattwoodrowThat seems like the video is just not rendered, and has nothing to do with window position
08:01jyaDid you look at my screen capture when I hovered the mouse over the application icon in the task bar?
08:02mattwoodrowNope, where is that?
08:02jyaIt looks like it's painting crap, with wrong sizes. The stuff is all read.
08:02jyaIn the bug
08:02jyaTwo attachments
08:03mattwoodrowIt looks like we got the scaling wrong...
08:05jyathe rendering test is only checking that we're reading red
08:06jyaand it is red.
08:06jyajust the wrong size :)
08:06jyaso the test pass
08:06jyamaybe if we swapped the position of the video and the canvas it would be a better test (at least for this condition)
08:07jyawhat I'm fairly certain however, is that HW decoding has been broken with amd graphic cards since forever :)
08:07jyawell, 2 years
08:07mattwoodrowWe probably should make the rendering test more complex (a pattern like the video)
08:07mattwoodrowso that it fails here
08:07mattwoodrowanyway, looks like some scaling is being applied and were not accounting for it
08:12jyaWho or what does the scaling?
08:14jyaanyhow, for my issue at hand (hoping to land this asap, I have a long pending queue I need in for 57). What exactly do you want me to do? i don't really understand how what you're asking would make the test less intrusive. we're doing it that way because the previous test at this location failed.
08:14jyaor should we just assume that not seeing the test window is better than having dxva disabled?
08:26jyamattwoodrow: ?
08:51bencfirefox is supposed to fire localTrack.onended when I disconnect a device (mic/cam) ?
08:51benctested with ff 55 on windows and it doesn't fire
08:53fippopehrsons: ^^ (I think you know about that)
08:55mattwoodrowjya: Im just not convinced that moving the window really solves the issue (it might on some machines, but not others), and it does introduce a known regression (the test window being visible)
08:55mattwoodrowI dont know who does the scaling, but thats what we need to fix for the real solution
09:00pehrsonsbenc: "ended" on physical disconnect is not hooked up in 55. We have bug 1272588 for this.
09:00firebothttps://bugzil.la/1272588 NEW, nobody@mozilla.org Unplugging a device for an active gUM capture doesn't end the track
09:01bencpehrsons: thanks
09:01pehrsonsThough I wonder if our full duplex audio implementation fixed this for audio
09:01jyamattwoodrow: ok, I'll separate the two first patches and open a new bug to do the 3rd
09:02jyaI can't see why moving the window wouldn't solve it for all machines. If it failed on other machines with a normal window being displayed, that would be a much bigger problem
09:02jyaand if it's still broken with "normal" window, then we can certainly say that decoding is broken
09:03bencpehrsons: just tested unplugging audio jack in ff 55 and "ended" didn't fire. not sure about full duplex
09:03jyathe window only appears for a tiny amount of time, on update. for most people that would show once every 6 weeks, and only if the test had first failed
09:08pehrsonsbenc: yeah, same thing in 57. You can see the doorhanger hanging around on linux (they look a bit different) after unplugging, which is the root problem.
09:10mattwoodrowjya: I dont see why moving the window correlates with scaling
09:10mattwoodrowis this multi-monitor?
09:10jyait is.
09:10bencpehrsons: ok. I'll track the bug
09:10mattwoodrowjya: and theyre at different DPIs?
09:10jyamattwoodrow: yes
09:11jyaone is 5K 27" the other 27" normal
09:11mattwoodrowThats probably very related then :)
09:13jyayesh, now thinking about it, which dpi settings to use in this case, is a tough choice
09:13mattwoodrowI think our code is getting confused somewhere
09:13mattwoodrowpossibly picking one choice at one point and the other at a different one
09:14jyai'll retry with a screen disabled, either the 5K one or the 1440p one
09:16jyaaren't the coordinates you give to moveTo, or the dimensions you provided for the size of the window relative ones?
09:16jyae.g. regardless on how the video decoded frames is scaled, the coordinates use to know where to read are always the same?
09:18jyamattwoodrow: what about this?
09:18jyacould we check if we had multiple screens at different dpi?
09:19jyaand if so, only then attempt to relocate the window?
09:19jyathat should reduce greatly the problem of the test window being visible
09:20jyaon another topic, with bug 1397214, do you have some ideas, or should I do a quick and dirty hack such as "if we're shutting down, don't complete this operation" ?
09:21mattwoodrowjya: I really have no idea! Id love to know why it regressed
09:21mattwoodrowA hack works I guess, but it still seems bad to be doing this during shutdown
09:22jyasome timing issue I gather... the puzzling thing is that when reading people commenting on those crashes, they are opening a new tab, not closing their browser. so why would it crash during shutdown
09:23jyamattwoodrow: can I query the list of screen available in JS and check their dpi ?
09:24mattwoodrowIm not sure, but I guess so
09:24mattwoodrowcould be a multiprocess thing?
09:25mattwoodrowWere shutting down a child process when it crashes, not necessarily shutting down the browser
10:32jyamattwoodrow: so what about my dual screen question?
10:32jyacould limit the change to dual screen setup, regardless of the dpi
10:33mattwoodrowjya: I guess? Id still like to see the layers test pick up that things are super broken
10:33mattwoodrowThe fact that its passing even though weve drawn everything at 2x scale is bad
10:33jyawould you like me running some specific test to see how that goes?
10:34jyato confirm your doubts I mean
10:34jya0,0 is my main screen, a 5K 27" with the scaling set at 200%
10:34jyathe second screen, located on the right is a 27" at 2560x1440, 100% scaling
10:35jyathat the layers are broken, tbh isn't my primary concern at this stage
10:35jyaI just want video playback to work in the most efficient possible fashion :)
10:36mattwoodrowjya: If the test is broken (which it is), then we can choose to not use the results to affect video
10:36mattwoodrowThats what Id prefer, over moving windows around in the hope of finding a location where its not broken
10:37mattwoodrowwhich seems like it could very easily work on your setup and nowhere else
10:37jyathe issue is that the result of this test is used to determine if h264 decoding is broken
10:37mattwoodrowRight, which is why I said do the layers test first
10:37jyawhen half the test has nothing to do with decoding h264
10:37jyaok.. so I do the layer test first.
10:37mattwoodrowif that fails, then dont do the video test, since its meaningless
10:37jyaif it's broken we abort
10:38mattwoodrowright
10:38jyaand if video failed okay to try again at a different location?
10:38jya(happy to limit that "do the test twice" only if we have dual screen (if indeed that is the reason why the test is failing)
10:38mattwoodrowI cant see why layers would pass and video fail
10:38jyaI'll confirm that as soon as I reboot under windows
10:38mattwoodrow(once we make the layers test a bit more robust)
10:39mattwoodrowso moving and repeating should be unnecessary
10:39jyawasn't the test about the decoder returning rubbish thing like a green frame?
10:39mattwoodrowsure, but video is actually broken then, so moving shouldnt affect it
10:39jyait wasn't about rendering, it was about the HW h264 decoder being broken
10:39jyalike they were returning green when watching youtube
10:40jyaI'm confused. you don't know if video is broken, if only the scaling is bad
10:40mattwoodrowI guess you mean the case where someone has both incorrect scaling *and* a broken video decoder?
10:40jyalet's assume that the video decoding works
10:40jyait's just the rendering that is crap
10:41jyathat set the pref that decoding failed. dxva is now disabled
10:41jyait will assume that decoding failed.
10:41mattwoodrowright, but if we do the layers test first, it fails and we abort
10:41mattwoodrowthen we wont change the decoding pref
10:41jyaah you don't even want to perform the decoding test if layers failed?
10:41mattwoodrowcorrect
10:42jyaeven if layers is broken but only at those coordinates?
10:42jyalike here, it's obvious it's the scaling that is wrong
10:42mattwoodrowif were not rendering anything correctly, then trying to test video is pointless, its always going to fail
10:42jyabut now let's assume we do have a buggy decoder
10:42jyabut you assume that we have a valid rendering test
10:42jyawhich we do not
10:42mattwoodrowwell lets just also fix the scaling bug and then that bit is no longer a problem
10:43pehrsonsmchiang: ping. is "devicechange" broken? I'm seeing signals suggesting that from a couple of places (including testing track.onended on linux -- I now saw code for devicechange that will stop capture, so should trigger ended)
10:43jyaI'm looking at getting around the problem in today's situation: that is scaling is buggy
10:43jyaand as is it, the test is also broken.
10:43mattwoodrowjya: Only with specific multi-monitor setups, which is super rare
10:43jyaso the render test will pass, even if it painted rubbish
10:44jyais it really that super rare? I'm fairly certain that it's pretty common to have people with a laptop and an external screen
10:45jyaI don't want to be pushy, but as it is... I don't see how what you're asking will make the test more accurate, based on the established conditions that scaling is broken *and* the render test is not accurate to detect such failure
10:45pehrsonsbenc: Actually looks like we have hooked up "ended" with the help of "devicechange". Though it also looks like "devicechange" is broken.
10:46jyaassuming that I can't fix the scaling, nor will modify the render test (i dont have time for that right now)
10:46mattwoodrowjya: I want 3 things. 1) Make the render test more accurate (by checking if the red square ends when it is supposed to), 2) Skip the video test if the render test failed 3) Fix the scaling bug
10:46jyathat leaves me with a video test only , which I have to design getting around those two problems
10:46mattwoodrowThe first two should be super easy
10:47jyaI'll let you prove that to me :)
10:47mattwoodrowand basically fixes everything except for multi-monitor/multi-dpi users, who also happen to have a gpu with broken decoder
10:47mchiangpehrsons: I am not aware of what. Blake mentioned it but he cannot find a bugzilla bug. I will test it tomorrow
10:48mattwoodrowwhich Im hoping is so obscure itll be nobody
10:48mchiangWe have a jsfiddle to test it.
10:48jyawhat about I do the following: keep the render test (because really, I don't want to touch it). I test the decoder and if it fails and *if* it's multi-screen, I set the position of the window to the top left corner of the screen declared secondary
10:49pehrsonsmchiang: ok, I'll run mozregression later and file a bug
10:49jyathat way people are less likely to notice, and I can go on and live to look at the next fire
10:49pehrsonsI saw it through an untriaged bug... We really need a test.
10:49jyai now just need to know how to test the existance of multiple screen, and the dimensions of each
10:51mattwoodrowjya: https://pastebin.mozilla.org/9031683
10:52mattwoodrowThats part 1
10:52bencpehrsons: yes it is :)
10:53* jya knew that annoying matt will make it prove his point by doing it :)
10:53mattwoodrowI hate that youre right
10:54mattwoodrowjya: https://pastebin.mozilla.org/9031684
10:54jyaare we sure to have a white point in the bottom right corner?
10:54jyawhat if the video touches the thing
10:54mattwoodrowThe video is under the red square
10:55mattwoodrowI just made the red square half-width, and added a check that the right-half is now white (the default background color)
10:56jyaok.
10:57jyaso if render/scaling is broken we no longer care if we can do HW decoding
10:57jyashould we even bother with a video decode test ?
10:58mattwoodrowThat second patch skips the decode test
10:58mattwoodrowif rendering is broken
10:59jyayes, but do we want that?
10:59jyaespecially when here the rendering is broken, but not really
11:00mattwoodrowit is really broken though, the results of the video test are meaningless because the video isnt even visible in your screenshot
11:00mattwoodrowso trying to use the results to determine if video is working will never give us a useful answer
11:01jyasure, but that's where the hack I put in place worth doing no?
11:01mattwoodrowWe have literally no way of knowing if theres a problem with video, so we might as well hope for the best (and fix the scaling bug!)
11:01mattwoodrowYou can do the hack if you want, but we should back it out again once we fix the real bug
11:01jyayou and I know that the scaling bug will still be there in a year time
11:02mattwoodrowI dunno if thats true
11:02mattwoodrowmy patch will make the rendering test fail when the scaling bug happens, and thatll show up in telemetry
11:02mattwoodrowif its happening to any real number of people, then itll get fixed
11:02jyavery well, I'll do as you say, just ignore the decoding part of things
11:03mattwoodrowIf its just you, then maybe youre out of luck haha
11:03jyaat least I understand know why i never had HW acceleration on this box. I had mentioned that to you a few years back.. that the stuff always showed up as having failed
11:03jyabut never looked into it
11:08mattwoodrowAnyway, I need to sleep
11:08mattwoodrowgood luck!
11:12jyathanks. I'm now back under windows, and this idiot thinks I have 3 screen connected now
11:22padenotachronop, I've received some sound cards and optical cables
11:22mattwoodrowjya: Can you try adding this.win.devicePixelRatio = 1;
11:22mattwoodrowto the start of runSanityTest()
11:23achronoppadenot: great, that was fast ...
11:23padenotamazon prime in paris is usually quite fast
11:23achronoppadenot: can you test if they work on mac?
11:23padenotyep, doing that now
11:24mattwoodrowMaybe that doesnt do anything
11:29jyamattwoodrow: good idea.
11:29jyait's recompiling right now
11:33achronopdminor: can you check if unconfirmed bugs 1393556 and 1393537 are applicable? at least on second one the code looks different from branch 54
11:39achronoppadenot: do you think 1393689 is real ?
11:43padenotachronop, yes I've written a patch yesterday that will make things better, I need to clean it up and put it up for review
11:47achronoppadenot: should I confirm it? or dup it to the other one?
11:47padenotwhich one is the other one ?
11:47padenotah the one marco reported and where sole added details as well ?
11:47soleWho pings me? :D
11:47padenothi sole !
11:47achronopthe one that you are writing the patch
11:48padenotachronop, yeah I haven't opened a bug yet, it's fresh, I'll do that today
11:48padenotsole, talking about your quality issue on webrtc, it's better already but we've started a project to really make it good
11:48achronopok I will leave it for later
11:48mattwoodrowjya: Also, try sanityTest.resizeTo(PAGE_WIDTH, PAGE_HEIGHT) right after the moveTo call
11:48mattwoodrow(and check if the snapshots you took change)
11:49jyamattwoodrow: will do
11:49mattwoodrowI need to actually sleep, but Ill look more in the morning
11:49jyaI'm glad you're staying awake to follow my progress and speed up the time to do the review
11:49jyaI have two more things I want you to look into :)
11:49jyain particular the use of CanUseDXVA, i totally messed things up in 55 :(
11:50* robswain-M wonders what sole's quality issue was for context and then what the new project mentioned by padenot is
11:50solerobswain-M: I was in a call using appear.in and both of us sounded like Daleks
11:51robswain-Msole: and video was fine or do you think it was due to poor connections?
11:52soleI think video was fine, and I was using the office's connection
11:52soleI wouldn't classify it as poor
11:53robswain-Moffice wifi can often be quite bad :)
11:53robswain-Mok, so...
11:53* robswain-M wonders to what padenot is referring with this new project
11:53robswain-M:)
11:54soleFYI http://beta.speedtest.net/result/6604200174
11:54soleusing wifi too
11:54padenotrobswain-M, we're doing some work to improve webrtc call quality in general
11:54padenotnothing concrete just yet
11:55robswain-Mmmm, fair enough - bandwidth tests don't show jitter and packet loss, but that looks like too good of a connection to actually be so bad
11:56padenotyeah mozilla's office connection are usually pretty good
11:56padenotwe're using level3 directly iirc
11:56padenotat least in paris
11:56robswain-Mpadenot: such as? transporty-stuff? media engine-y stuff? capture-y stuff? something else?
11:57padenothm, underrun protection, better integration for AEC and AGC and noise cancellation, better device enumeration, using new and more efficient APIs for some stuff
11:57padenotso I'd say capture and media engine
11:58robswain-Mwhich buffers would you be protecting against underruns?
12:42padenotrobswain-M, not protecting buffers, just making sur audio buffer underruns are dealt with
12:42padenotachronop, https://docs.google.com/document/d/1_4t5N3udQV7IQXDQZAygnTCg6xs-o-Ao1EJ82aF0T5A/edit?usp=sharing
12:42padenotachronop, works on OSX, Windows, Linux, 7.1 output, 2.0 input
12:55achronoppadenot: whne you plug SPDIF (in) to usb how many channels do you get on the input device?
12:55padenot2
12:55padenoton all OSes
12:55padenotthe product description was a bit ambiguous
12:56achronophhmm we still need a multichannel input device then
12:56padenotyeah I think I'll order the terratec one
12:56achronopyeah I know I went through those device
12:56achronopI will get it
12:56padenotat least I'll be able to do proper testing
12:56padenotok
12:59achronoppadenot: what cables did you buy?
13:00padenotachronop, basic amazon once
13:00padenot-c
13:00padenothttps://www.amazon.fr/gp/product/B00NH11H38/ref=oh_aui_detailpage_o01_s00?ie=UTF8&psc=1
13:00padenotsorry for the french link
13:00achronopno worries
13:01achronopare they optic to usb?
13:09dminorachronop: checking those bugs out now, sorry I missed your message earlier
13:09achronopthanks
13:11jesupsole: there was a bug last week (regression from some multichannel patches) that caused serious perf issues with audio on Windows (and mac, not linux) that was fixed in friday's (?) nightly - what build were you using? What OS?
13:11solejesup: I don't remember, it was at the time I added the comment on the bug (whose bug number I don't remember either); I was using Mac OS
13:12jesuppadenot: first patch was green. I have a patch that builds but doesn't quite work yet to convert gUM over to APM
13:12jesupsole: this week? last week?
13:13padenotjesup, sole was seeing the big regression of last week, the other person in the 1:1 was on windows, nightly
13:13jesuppadenot: aha. Ok, that'd explain it
13:16jesuppadenot: drno|irccloud: The mac I was on that was having all the gUM problems yesterday was in fact running with full_duplex set to false (manually); I'd been testing something a while back IIRC
13:24dminorachronop: both are in upstream webrtc.org code that we don't build or use. I asked him to report upstream and closed as WONTFIX.
13:25achronopdminor: sure, thanks for taking care
13:25dminornp
13:27pehrsonsbenc: mchiang: I was running through mozregression and I think I understand this now. "devicechange" still works. But only while a capture is in progress (user gave consent). Also, "ended" works (audio, linux), except when you're capturing the default input device, and unplugging means there's a new default input device (so it didn't end, just switched
13:27pehrsonsdevice).
13:27pehrsonsbenc: so if you have another case where "ended" doesn't work as expected I'm all ears
13:29padenotjesup, oh ok
13:29jesuppadenot: drno said in the late standup that someone had broken full_duplex == false (at least on mac)
13:30padenotjesup, interesting, haven't tested in ages, especially not on mac, aggregate is so much better
13:31jesuppadenot: don't BSD and a few others still require non-full-duplex?
13:31padenotjesup, yes, it still works for them iirc
13:31jesuplikely it's something trivial in any case
13:32jesupit still can be handy occasionally to validate the cause of a bug, perhaps, in addition to bsd/etc. We may want to get them to move sometime soonish
13:33padenotalsa has code to do duplex, sndio does not
13:33padenotin cubeb I mean
13:34padenotI've told their maintainer that I'll CC him to the relevant bugs if we decide to do something that breaks them
13:34padenotit's the least we can do
13:34DuClareYou mean the sndio backend doesn't?
13:34pehrsonssomeone here has win7 and wants to run a test?
13:34DuClareThe library should have support for it
13:35padenotDuClare, afaik our input/output library does not implement input for sndio
13:35DuClareAh
13:36padenothappy to take patches, though, but I can't work on it myself, don't have the time nor the knowledge
13:41bencpehrsons: 1. I have only 1 mic plugged to the audio 3.5 jack a
13:41bencpehrsons: 2. I call getUserMedia, on success I set track.onended callback
13:41benc3. I unplug the mic
13:41bencpehrsons: should I get a onended callback?
13:42pehrsonsbenc: are you saying that after unplugging and doing gUM again you have no devices in the list?
13:42pehrsonsif you even get a list
13:43bencpehrsons: I'll try doing gum again but I shouldn't have a device
13:43pehrsonsprobably it'd take a reload in between as well
13:43bencpehrsons: https://pastebin.mozilla.org/9031690
13:43bencpehrsons: reload the page?
13:44pehrsonsbenc: thanks, that could be a bug. Choosing the default device and unplugging the last available one.
13:44mchiangpehrsons: that's great. Phew!
13:44pehrsonsbenc: I basically mean start from fresh. Best is probably to launch firefox with the device unplugged and see what gUM gives you
13:45bencpehrsons: yes, that's my case. I only have one mic, so it's also the default
13:46mchiangYou can also give consent by granting permanent permission.
13:46bencpehrsons: without the mic I'm getting MediaStreamError "NotFoundError"
13:46padenotpehrsons, we know of this
13:47pehrsonspadenot: ok, do we have a bug on file?
13:47padenotpehrsons, yes
13:47padenotthat said, it _should_ fall back properly, and does for me
13:47padenotbut I've heard issues
13:47jibpadenot: what's the bug #
13:48pehrsonspadenot: well, this problem seems to be when there's nothing to fall back on
13:48padenotpehrsons, yes, we do detect this and use a system thread to continue running the MSG
13:48padenotjib, one sec
13:49pehrsonspadenot: though the devicechange logic is in MediaManager/CamerasParent afaict
13:49jibif there's no more mics in the system then continuing without error is an error
13:50jibit seems to me
13:50padenotjib, in theory, yes, in practice, we did this because it was breaking a large client (intuit, iirc, jesup?) and we needed a quick workaround
13:51padenotpehrsons, yes, for now, this is an automatic fallback for emergency situations in graphdriver.cpp
13:51bencthis bug?
13:51benchttps://bugzilla.mozilla.org/show_bug.cgi?id=1272588
13:51firebotBug 1272588 NEW, nobody@mozilla.org Unplugging a device for an active gUM capture doesn't end the track
13:51padenotthat's one
13:52padenothttps://bugzilla.mozilla.org/show_bug.cgi?id=1355773 is for another issue
13:52padenotbut related
13:53pehrsonspadenot: the first one is from before we had devicechange, I was planning to dupe it now
13:53padenotok
13:53padenotpehrsons, how do we implement the audio part, since we don't know which device is being used ?
13:53padenotor do we ?
13:53jibpehrsons: before you dup it, isn't this what benc is still experiencing?
13:53padenotin MediaManager.cpp
13:54jesuppadenot: yes, intuit was using Citrix which has an audio emulation layer, and there were issues when the client had a temporary disconnect
13:54pehrsonsjib: well, the bug is all generic, and benc's case seems like one of the last holes
13:54jibThere's also a potential privacy issue here if we fall back to a different mic the user technically never shared
13:54jesupfirefox running on a Citrix server
13:54padenotjesup, plus they had no other audio input devices, iirc
13:54padenotjesup, which was the issue, since it was likely server hardware
13:55jesupand doing screensharing
13:55padenotwhich is why it wasn't falling back
13:55pehrsonsjib: probably the more common one though (pulling the default device)
13:55jesupright
13:55jibI may not know all the details, but fallback seems wrong to me in any situation
13:55padenotjib, fully agree, I have a fully designed solution in the cubeb issue list for that
13:56jibgood
13:56padenotjib, https://github.com/kinetiknz/cubeb/issues/167
13:56pehrsonspadenot: logic is here: http://searchfox.org/mozilla-central/rev/67f38de2443e6b613d874fcf4d2cd1f2fc3d5e97/dom/media/MediaManager.cpp#2090
13:57padenotjib, note that the behaviour for output devices is not specced, and my issues in the relevant standards were aggressively closed
13:58jibhmm that doesn't sound good
13:58padenotjib, now that web audio api spec work is less urgent/heavy, I'm planning to teach the HTML standard about all this
13:58jibno pun intended
13:58padenotI've been designing/thinking about this for ages, but I couldn't reallistically do everything at once
13:59jibit's good we know how to fix this, and now's a good time (soon)
14:00padenotjib, https://github.com/w3c/mediacapture-output/issues/50, https://github.com/w3c/mediacapture-output/issues/49, so you have some context
14:05jibso the last one they agree is a problem, but closed for bureaucratic reasons it seems
14:07jibbtw you might want to add "@guidou" - whether he ever read your responses (in closed isssues) may depend on how he reads github updates.
14:25bencI'm getting in the console: "TURNS is not yet supported"
14:25bencI thought it was supported
14:25benchttps://bugzilla.mozilla.org/show_bug.cgi?format=default&id=1056934
14:31Caspy7anyone know if opus is directly mentioned in the HTML spec?
14:32DuClareNot in https://html.spec.whatwg.org/
14:33DuClareMaybe in webrtc?
14:33DuClare.. I don't see it here either https://www.w3.org/TR/webrtc/
14:34padenotCaspy7, why would it?
14:34padenotopus is mentionned in rfc 7874,
14:35Caspy7padenot: nothing important, was just having a conversation w/ people and someone claimed that opus got some sort of recommendation for audio - in contrast to video for instance
14:35DuClareAh
14:35padenotslightly more complicated, but not incorrect per se
14:35padenotOpus is MTI (mandatory to implement) for WebRTC
14:36padenotso if you claim to have a webrtc implementation (that all major browser now have), you are required to be able to deal with WebRTC
14:36padenotfor webrtc things
14:36padenotnot strictly for HTMLMediaElement, Web Audio API, etc.
14:37Caspy7padenot: ok, thanks
14:37Caspy7so opus is mandatory for WebRTC
14:38padenotsee https://datatracker.ietf.org/doc/rfc7874/, search for Opus
14:38padenotbottom of the document
14:38jibhttps://tools.ietf.org/html/rfc7874#section-3
14:38padenotactually not the bottom, it was not being displayed entirely
14:44jesuppadenot: APM input works (including AEC it appears; haven't hooked up agc or ns, though it's easy)
14:45jesuppadenot: do you have a bug open for this work?
14:45padenotyeah it's easy with APM
14:45padenotjesup, no
14:45jesupk
14:45padenotjesup, I'm finishing up some very rudimentary underrun protection for windows, haven't had time to open bugs
14:48Caspy7padenot, jib: thanks for the links :)
14:51jesuppadenot: I'll file and attach the patch
14:53achronopjib: do you think you can give the answers on unconfirmed bug 1395310?
14:54firebothttps://bugzil.la/1395310 UNCONFIRMED, nobody@mozilla.org WebRTC should be behind permission dialog like microphone access
14:54achronopjib: is it a spec issue?
14:54jibjesup: did you get a response from abr on this? ^
15:00jibWhat I'd like to say in that bug is that webrtc datachannels are merely an optimization of what web sites can already do. I.e. they can implement file sharing over web sockets already, even without webrtc. Arguably, the battle over resources on your machine was lost with the introduction of JavaScript and XMLHttpRequest
15:02jesupjib: no, I haven't. I think he missed it
15:02padenotit's mostly an IP exposure and fingerprinting issue, is it not?
15:04jesupjib: however, there's minimal incentive to use people for webtorrents without consent/knowledge if it has to go via a server. DataChannels creates the incentive, which creates the concern about legal exposure for users
15:05jesuppadenot: actually creating datachannels isn't an exposure and fingerprinting issue; that's mostly non-connected channels (and thus the IETF draft)
15:05padenotyeah
15:06padenotI mean, even web audio is used to fingerprint these days
15:06padenotbut it's not as severe
15:07padenotjib, there is also some very recent effort about preventing abuse from websites: https://wicg.github.io/budget-api
15:08padenotjib, this is mostly about background operations, though
15:10jibright, I've seen web sites contemplate bitcoin mining as an alternative(supplement?) to ad revenue
15:10padenotyes, however it seems useless even at scale, these days
15:11padenot(also it has happened on real websites, it's not theoretical)
15:18jibjesup: about ip exposure, I was going to refer to https://tools.ietf.org/html/draft-ietf-rtcweb-ip-handling-04#section-5 but then I still see host candidates when I test things, so I'm not sure where we are there these days
15:19jibsince that's mentioned in the bug
15:20jesupdrno|irccloud: ^
15:22jesupthe WG considered the "it's equivalent to websockets" argument, it didn't really consider the "it can be used to silently make someone a torrent provider" case - which involves bandwidth too, which some people have metered (and can slow down other uses from the same location)
15:23jibpadenot: FWIW I think browsers will have to own and solve both the CPU use and bandwidth problems for their users going forward, though I'm not sure that web API specs are necessarily where we'll see solutions to this. Blanket demanding powerful features be off by default doesn't seem like the best answer to me
15:25jibjesup: I understand the incentive argument, but the same could be said for offering users faster uplink speeds
15:25jesupjib: it may not be. The real argument would be for visibility and/or prior consent before starting data connections, and/or monitoring actively for background un-consented high-bandwidth use
15:26jesupnot sure I follow "faster uplink speeds"
15:26jibI mean we don't blame ISPs for this
15:27jibthis user has a monumental upload speeds, that's bad because they can be used as storage for torrent content (even over web sockets)
15:27jibI'm equating webrtc with progress
15:28jibalso, isn't the concern about storage?
15:28fippoi wonder how browsers like brave that advertise "built in webtorrent support" handle the ip leak stuff...
15:29jesupjib: the concern if is they have a cap, torrent (without consent) could blow through it and cost users $$$$$)
15:29jiblocalStorage APIs and such *should* concern people imho
15:29jibwell cost and legal exposure are different concerns, I read mostly the latter concern in the bug
15:29jesupjib: you can torrent without localStorage - download, cache in JS, and retransmit
15:29jibdoes that cause legal exposure?
15:30jesupI'm making the logical extension if we re-discuss it in the WG
15:30jesupyes
15:30jibsounds like a TOR node
15:31jesupIP compliance firms pull torrents, then record the IPs and can file suit on users who are redistributing content that's copyrighted
15:31jesuptor makes it much harder to trace, of course
15:32jesuppadenot: https://treeherder.mozilla.org/#/jobs?repo=try&revision=d41c19cd9cb70a9d0038f55fc1292cc107e6a390
15:32jyamattwoodrow|away: you're a genius...
15:33padenotjesup, I'll have a look later
15:34jibjesup: ok can I assume you or abr will comment on that bug then?
15:36* jib would love to see the wording of the permission prompt that explains that risk. :-/
15:36jesuppadenot: bug 1397793
15:36firebothttps://bugzil.la/1397793 NEW, rjesup@jesup.org Remove webrtc.org "External" audio interface and switch gUM audio input to APM
15:37jesupLooking green on Try (Linux, which has fake audio devices)
15:38jesupjib: yeah, that's the problem. Some sort of activity indicator or bandwidth-use indicator and/or alert (ala the JS-hang alert) might be in order. But the issue bears discussing
15:38jesuppadenot: still waiting on mda2 though
15:39padenotjesup, I'll have a look tomorrow I think
15:39jesupcool. I'll add ns and agc, and then we can delete the External* interfaces we un-deleted :-)
15:40catleeIs there anything I can do to debug broken AAC audio decoding on ARM for desktop? Firefox ESR on raspberry pi plays a loud hiss for videos with AAC encoded audio
15:46jesuppadenot: green on mda2 now :-)
16:02fippobwc: drno|irccloud: i just stumbled over https://bugzilla.mozilla.org/show_bug.cgi?id=1300863 and I think this came up again recently (maybe inaki on twitter). did you just forget about this? :-)
16:02firebotBug 1300863 NEW, nobody@mozilla.org recvonly m-sections in offer are rejected when no local track is assigned to them
16:02bwcHaven't forgotten. Transceivers will fix.
16:03fippowfm :-)
16:09jesuppadenot: well darn, orange on some (but not all) mda2s. Still pretty good. 1 test permafails
16:13fippoalso in "how slow is chrome in implementing unified plan": https://twitter.com/creslin287/status/905452637889363968 -- asterisk beat them to it.
17:01jesupfippo: ha! and :-(
17:12fippojesup: for reasons of chrome being stupid about rendering <video> only once a frame arrives i am considering splitting up things into a muted video and a separate audio. will it have any performance drawbacks to do that in firefox or can i keep my hack simple?
17:12bencdoes ff 55 supports TURNS? {urls: &quot;turns:example.com:443?transport=tsp&quot;, credential: &quot;...&quot;, username: &quot;...&quot;}
17:12bencI&#39;m getting a warning in the console: TURNS is not yet supported.
17:13jesupfippo: if they&#39;re in separate streams, you may lose sync at times (likely small). no other impacts I can think of.
17:14jesupdrno|irccloud: ^ benc
17:14jesupb