mozilla :: #media

18 Apr 2017
00:42bwukentuckyfriedtakahe: I missed a train, so i will be late by 15 mins for our meeting. Sorry about that.
01:21kentuckyfriedtakahebwu: no worries
01:45bwukentuckyfriedtakahe: I am ready to talk now.
01:47kentuckyfriedtakahebwu: rangitoto?
01:48bwuok
01:51kentuckyfriedtakahebwu: bug 1357133
01:51firebothttps://bugzil.la/1357133 UNCONFIRMED, nobody@mozilla.org Opus + Widevine fails to decode; CDM crashes in Nightly 55 [@ widevinecdm.dll@0xc82bf]
02:45Caspy7...does Netflix use Opus?
03:07kentuckyfriedtakaheCaspy7: The reporter works for Google on Shaka Player
03:08Caspy7ah
03:16Caspy7I know Netflix is the primary and most popular widevine consumer. The only other large consumer I was familiar with was Amazon VOD
04:17andrenatalhey guys, I have a simple question: is there any way to capture audio in 16khz with gUM?
04:21andrenatalpadenot: do you have any tips, please?
04:22jesup|PTOandrenatal: no. capture is done at the 'preferred' device rate, typically 44100 or 48000 hz. We use full-duplex in almost all (soon all but edge) cases, so input rate == output rate. padenot or karlt might be able to suggest some webaudio ideas, if they know what you're trying to do and why you want 16000Hz.
04:23andrenatalthanks jesup|PTO
04:26andrenataljesup|PTO: I am using the vad from webrtc to detect silence for one of our speech projects, and it supports only 48k but not 44 (https://github.com/mozilla/gecko-dev/blob/master/media/webrtc/trunk/webrtc/common_audio/vad/webrtc_vad.c#L77)
04:27andrenatalmaybe 48 will make it
04:42jesup|PTOthe graph (and thus getUserMedia) may run at other rates. We could look at updating the VAD code to support 44.1... the current vad code uses some icky-looking downsamples (see media/webrtc/trunk/webrtc/common_audio/signal_processing/resample_48khz.c: WebRtcSpl_Resample48khzTo8khz()) The 32->8 downsampler is a /2 downsampler applied twice...
04:48andrenatalyes, I notice that.
04:48andrenatalI am porting it to emscripten actually, because our code is all js
04:49jesup|PTOjmspeex: ^ you may have thoughts on VAD in JS here.
04:50kentuckyfriedtakahebwu: do you have someone who can take a look at bug 1335098 this week?
04:50firebothttps://bugzil.la/1335098 NEW, nobody@mozilla.org "Media couldn't be read" after hitting play/pause a lot of times in Twitter video player
04:52kentuckyfriedtakahebwu: scratch that - it reproduces in edge
05:01bwugood to know. :-)
05:18jmspeexjesup|PTO: what's the issue exactly?
05:20jesup|PTOjmspeex: andre wants to do silence detection using a VAD for some speech work, and is going it in JS. He was porting the webrtc VAD code to emscriptem, but it's 48/32/16/8 only, and we often use 44100 for getUserMedia (and MSG) if that's the device's preferred output rate
05:21jmspeexFor something like a VAD, I'm pretty sure you can pretend that your 44.1 kHz signal is 48 kHz
05:22jmspeexotherwise you can always resample it, but I kinda doubt it would make much of a difference
05:22jesup|PTOmy non-signal-processing-expert glance at the code in media/webrtc/trunk/webrtc/common_audio/signal_processing/resample_48khz.c WebRtcSpl_Resample48khzTo8khz() (and media/webrtc/trunk/webrtc/common_audio/vad/vad_sp.c in WebRtcVad_Downsampling() for 32 and 16->8) makes me kinda queasy, though perhaps for silence detection and a fixed set of easy-multiple rates it makes sense
05:22jmspeexthat's for voicebank I presume?
05:23jesup|PTOyeah, the pitch distortion of 44.1 vs 48 isn't likely to matter for VAD I'd guess
05:23jmspeexjesup|PTO: what's the issue?
05:25jesup|PTOperf for one (32->8 by doing /2 twice, and 48->8 by doing 48->24, 24 low-pass -> 24, 24 -> 16, then 16 -> 8 (phew))
05:26jesup|PTOagain, quality is likely no relevant here (I presume)
05:27jmspeexperformance is probably not as bad as you'd think
09:56robswain-Mi have some code that makes 3 gUM requests in quick succession
09:56robswain-Monly one resolves
09:56robswain-Mand there are no errors
09:56robswain-Mwhy?
09:57robswain-Mi've also seen a case where none resolve
09:57robswain-Malso with no errors
10:00robswain-Mobserved in both 52 and 54 on macOS
10:01achronoprobswain-M: there is bug 1355306 which might be the reason
10:01firebothttps://bugzil.la/1355306 NEW, achronop@gmail.com getUserMedia doesn't work in more than one tab at a time now
10:01robswain-Mthis is. in one tab
10:01robswain-M-.
10:02padenottime to talk to jib, then :-)
10:02* robswain-M looks at jib
10:02achronopyeah but the root cause might be the same
10:02robswain-Mcould be
10:03robswain-Min case it makes a difference, the code is still using navigator.mozGetUserMedia for now
10:08robswain-Mit could be this one: https://bugzilla.mozilla.org/show_bug.cgi?id=861716
10:08firebotBug 861716 NEW, nobody@mozilla.org Multiple async calls to getUserMedia before user response causes earlier requests to be lost
10:12robswain-Myeah, i'm pretty sure it is that
10:12robswain-Mjib: ^ is it possible to at least make sure all getUserMedia requests resolve/reject or success/error callback?
10:31robswain-Mgiven i sometimes see no gUMs resolving, i could be (also?) hitting https://bugzilla.mozilla.org/show_bug.cgi?id=1308304
10:31firebotBug 1308304 NEW, jib@mozilla.com After multiple calls to getUserMedia() in a tab, it eventually stops doing anything at all
10:41padenotachronop, I take it it works ?
10:41achronoppadenot: yeah, testing with your code
10:42achronopincluding the modifications
10:42padenotachronop, cool, what was the issue ? simply needing the splitter / merger ?
10:42achronopyeah let me send you the file
10:45achronoppadenot: if I need to test more let me know
10:45padenotyeah I can test on my macbook here, it's got a stereo mic
10:47achronopcool, a second look is necessary for something like that ... I have a feeling that I might miss something
10:53padenotdunno, I'll see
12:58padenotjib, which list is the one mentionned in https://github.com/w3c/mediacapture-main/pull/445#issuecomment-293838238 ?
13:05* jesup checks logs since nightly crashed overnight
13:07jesupGood excuse to rebuild m-c (I run local jprof-enabled builds)
13:22padenotachronop, what was the issue about putting a variable in the cubeb struct for the multiple duplex initialization thing ?
13:23achronoppadenot: I used the existing variable active streams and it was always 1
13:24padenotno you need another one that always increment
13:24padenotactive_stream is just the number of active stream
13:24achronopbut this one should be 2 on second stream
13:26padenotyeah, why is it not the case ?
13:26padenotin any case, the active count and an ever increasing index of aggregate devices is not the same thing
13:26padenotmreavy, ping
13:26mreavypadenot: pong
13:27achronoppadenot: you are right it's not the same
13:29achronoppadenot: did you test the stereo patch?
13:33padenotachronop, not yet, but it looks like it would work
14:47achronoppadenot: every gUM call is a new cubeb context with e10s
14:48padenotgood point
14:48achronophere: https://dxr.mozilla.org/mozilla-central/source/dom/media/CubebUtils.cpp#329
14:48padenotI hadn't think of that
14:48padenotthought, even
14:48achronopthe sCubebState is a global var in file CubebUtil.cpp
14:49padenotit's a different variable for each content process, but yeah
14:50achronoppadenot: cubeb logging through env var is broken
14:50padenotis it really ?
14:50padenotI've used it recently
14:50achronopcan you please try it there?
14:50achronopI have to use the avout:config var
14:51achronopotherwise it's always 0 even if I set MOZ_LOG
14:52padenotI need to clobber
14:52achronopok later does not matter now
14:53achronopwhat do you want me to do with the PR?
14:57padenotI'll r+ soon
14:57padenotI'm in the middle of figuring out flights and hotels for a bunch of stuff while it's clobbering
15:29padenotachronop, merged
15:29achronopthanks
17:21bsmedbergI have an MP4 file which plays fine in Chrome but not in Firefox, when served locally
17:21bsmedbergBut... when I save it to google drive and load it via their server, it works fine in Firefox!
17:21bsmedbergDoes this sound familiar to anyone?
17:22padenotjya, ^ ?
17:22bsmedbergalso loaded directly via file: doesn't play
17:23padenotmaybe google drive uses MSE
17:23padenotbsmedberg, jya would know but it's a bit late here in france
17:24bsmedbergNo, this isn&#39;t MSE. It uses a <video src=&quot;&quot;> pointing to their CDN
17:24padenotok
17:26bsmedbergI will file this as a bug. Should I NI?cya?
17:26bsmedbergsorry jya
17:27padenotbsmedberg, yes, :jya, in Core :: Audio/Video: Playback
17:49mreavydrno, drno|irccloud : ping
17:50drnomreavy: pong
18:44abrjib: ping
18:44jibabr: pong
18:45abrjib: This is probably either a question for you or bwc, but I figured Id start with you. ;)
18:45jibsure
18:45abrIn JSEP, the procedure that is outlined specificies that a=fingerprint has to occur on the m= level, not the session level
18:45abrBut thats not what we do
18:46abrSo, is this a problem with JSEP being too prescriptive, or does our implementation need to be adjusted?
18:47jibabr: that sounds like more like a bwc question
18:48abrjib thanks.
18:48abrbwc: ^^^ ???
18:49bwcabr: So, I guess JSEP can require what it wants, but it seems kinda silly.
18:50abrMaybe? Double-check me here, though if youre talking to a remote host that does not support bundle, do the local fingerprints potentially vary among media sections?
18:50* jib just commented on how stream.id and track.id will no longer match on a webrtc call https://github.com/w3c/webrtc-pc/issues/1128#issuecomment-294943194
19:03fippojib: i think the stream/track ids are already meaningless.
19:03jibfippo: today? how so?
19:04fippojib: lets say we have a videotrack. then I acquire my screen and use replaceTrack to send it to you instead of my face. you will still receive the same track as before.
19:04fippoand no renegotiation is required.
19:04fippoer... say we have a videochat
19:05jibfippo: sure, but that&#39;s as designed
19:05jibi.e. an app could conceivably keep track of that
19:05jib(no pun intended)
19:07fipporight. but does the app care? its local state will be different from the remote state
19:07* jib is trying to wrap his head around the new way to correlate streams and tracks across a peer connection
19:08fippowhat about subsequent offers. i called replaceTrack and replaced track &#39;video&#39; with track &#39;screen&#39;. does the new offer contain a=msid:stream screen? no...
19:08fipporeplaceTrack replaces the track, it does not change a tracks source. by design
19:09fippo(which is a pity actually but... too late)
19:12fippotaking a step back. when setRemoteDescription is called with an offer, we had onaddstream. that came with a stream and that stream had an id taken from the msid.
19:12fipposo we knew which m-line in the sdp it correlated to
19:18fippoand now we have the mid which identifies the mediaSection
19:18fipposo do we still need the msid?
19:20jibI mean it doesn&#39;t do much, except in the edge-case where the answerer calls SRD before adding anything
19:21jib?
19:21jibso probably not
19:22* fippo wonders how anyone still dares to call &quot;webrtc 1.0&quot; a &quot;1.0&quot; given the amount of breaking changes
19:23jibIs that 1.0 or &quot;Edge&quot; 1.0?
19:23fippoi think &quot;edge 1.0&quot; is what i expected 1.0 to be.
19:24jibyeah
19:24fippostream based, no objects. the only thing I miss from that is replaceTrack.
19:24jibseems like the wg got hooked on ortc
19:24fippowhich otoh is responsible for creating those problems :-)
19:25fippowell no. the 1.0+objects still requires you to go through the createOffer-setLocalDescription-setRemoteDescription hoop.
19:25jiball the complexity at once!
19:26jib:)
19:26fippowithout the benefits :-)
19:26jibright
19:27fippoabr: did you notice how chrome seems to implement rtpreceivers with PlanB? :-)
19:31abrfippo: No, I didnt look at that. Thats a level of dedication to deprecated behavior that I never would have expected.
19:34robswain-Mjib: did you see my pokings in the backlog?
19:36jesupfippo: jib: &quot;replaceTrack replaces the track, it does not change a tracks source. by design&quot; <- not what I intended.... replaceTrack replaces the source for an RTP stream and thus the source for the far-end track. It&#39;s not MediaStreamTrack.replaceTrack(), afterall
19:37jesupfar end was not supposed to ever be able to determine replaceTrack had occurred (infer from resolution/rate change at most)
19:37jibrobswain-M: hi, I just looked
19:38jibjesup: right, that&#39;s how it&#39;s spec&#39;ed as well I believe
19:38jesupgood. fippo implied otherwise
19:38fippojesup: yes, but it means that getSenders() at the senders will diverge from getReceivers() at the receivers
19:39jesupyou mean for trackids? (this goes back to &quot;why do we sync trackids across to the receiver?&quot;)
19:39fippoyeah
19:40jibrobswain-M: I saw bug activity there earlier but got distracted before I could comment. Do you still see the problem if you answer &quot;Always Allow&quot;?
19:40fippoit would be ok if we considered track ids to be opaque strings but we don&#39;t.
19:40jesupIMHO the received trackId should should have always been an abstract generated by the peerconnection, not the thing fed into the peerconnection - i.e. a clean separation that allows replaceTrack to not surprise anyone
19:41jibwell since stream and track ids are moot in the new spec we get our wish :)
19:41jesupyeah! :-)
19:42fippothat is the situation we have now. i&#39;m ok with that if we agree that this is a good thing and say so in some spec (which would be jsep referring to msid?)
19:42robswain-Mjib: I will need to verify that but I think it doesn&#39;t help
19:42* fippo kills https://github.com/w3c/web-platform-tests/blob/master/webrtc/rtcpeerconnection/setRemoteDescription.html because it does something meaningless
19:43jibI just wonder how much effort we should put into implementing the edge-case where the answerer calls SRD before adding things. That&#39;s the remaining case where the ids make any sense
19:43robswain-MOn macOS the Firefox dock icon bounces when a gUM request is made and I think I&#39;ve noticed that nothing happens until you focus the window/tab
19:43jibOr rather bwc wondered, and make me wonder the same
19:44fippojib: so SRD -> GUM -> createAnswer?
19:44robswain-MSo maybe unless you have always allow and the tab focused...?
19:44fippojib: so SRD -> GUM -> addTrack -> createAnswer?
19:44jibfippo: yeah, in that case the ids would sorta work on one side
19:44jibonly
19:45fippojib: in that case the track id on the senders would not match the ids of the tracks added because they have been created by SRD?
19:45jesuprobswain-M: I think I recall resource requests in unfocused tabs are delayed until they get focus to avoid focus-stealing, but perhaps I mis-remember (florian isn&#39;t here... jib?)
19:46jibfippo: no, in that case the track id on the answerer side&#39;s remote tracks WOULD match the ids on the offerer&#39;s sender.track. And that&#39;s the one case where it would AFAIK
19:47robswain-Mjesup: that seems reasonable to do tbh. It&#39;s a bit awkward for testing video conf on one laptop though :B
19:47fippojib: oh, the case i am testing in the wpt?
19:47fippojib: which would be invalid if addTrack was called before SRD?
19:47jesuprobswain-M: not a primary usecase.... ;-)
19:47robswain-MThe problem I have is rather calling gUM multiple times (3-4) and one or none of them resolve
19:48jibfippo: The issue is the instant a side calls addTransceiver (directly or indirectly through addTrack/Stream) then that side gets dummy tracks in a dummy stream.
19:48robswain-Mjesup: in one tab that is
19:48jibfippo: which ids won&#39;t ever change to match the other side
19:48robswain-Mjesup: and sure, it&#39;s fine. I&#39;ll just use chrome for all the other participants ;p
19:48fippojib: yeah. so where does the spec say so in language even I can understand?
19:49jesupOr another firefox instance (./firefox -P -no-remote)
19:49jibfippo: you&#39;re asking too much ;)
19:49fipponot hidden in step 22d of some algorithm
19:50jesupnote that sharing cameras can be a problem on one machine even across browsers
19:51jyabsmedberg: we had an issue a while back with google server, they were reporting buggy range request fragment. We put a workaround and they were supposed to have fixed it.
19:52bsmedbergjya, opposite problem ;-). Works on google *not* via file:
19:52jibfippo: here http://w3c.github.io/webrtc-pc/#dfn-create-an-rtcrtpreceiver
19:52jibstep 2
19:52jyaYes.. I reread your earlier statement. Are you on Linux?
19:54jibrobswain-M: Is this bug 1308304?
19:54firebothttps://bugzil.la/1308304 NEW, jib@mozilla.com After multiple calls to getUserMedia() in a tab, it eventually stops doing anything at all
19:55fippojib: oh... i complained about the lack of being able to create a rtpreceiver with a track id in ORTC ~18 months ago. looks like its fixed... in the wrong spec :-)
19:55jibrobswain-M: like does it reproduce e.g. in https://jsfiddle.net/jib1/srn9db4h/
19:55jibeven if you say &quot;Remember this decision&quot;/&quot;Allow always&quot;
20:11bsmedbergjya, mostly windows although I reproduced on Linux also. I filed a bug with NI for you, with more details.
20:13jyabsmedberg: the reason I asked is that on Linux, the rust MP4 demuxer is enabled by default. That could have explained differences. I&#39;m on PTO until next Monday, will check upon my return.
20:13jyaVery weird symptoms though.. if we don&#39;t play a file, it&#39;s because it&#39;s invalid.
20:13jyaThere are zero exceptions to that... chrome uses ffmpeg which is much more lenient on those files.
20:14jyaBut that it plays remotely but not locally. Now that is super weird.
20:22robswain-Mjib: thanks, i&#39;ll check tomorrow. i noted bug 1308304
20:22robswain-Myes
20:22firebothttps://bugzil.la/1308304 NEW, jib@mozilla.com After multiple calls to getUserMedia() in a tab, it eventually stops doing anything at all
20:22jibok thanks
20:23robswain-Mbut also bug 861716
20:23firebothttps://bugzil.la/861716 NEW, nobody@mozilla.org Multiple async calls to getUserMedia before user response causes earlier requests to be lost
20:23robswain-Mthey seem potentially related or overlapping in the code from how the discussions went
20:23jibright, that&#39;s why I was asking about &quot;always allow&quot; to differentiate between those two
20:23* robswain-M nods
20:28Caspy7I&#39;m seeing multiple reports of users on Radeon GPUs with the most up to date drivers (recently updated I believe) and now when they go fullscreen it starts to stutter
20:28Caspy7should I just file a bug? Is this already on the radar?
20:41rilliangerald: Does moving a C++ `const size_t foo` inside a function to hide it from the linker have performance implications?
20:41geraldrillian: Sorry, I don&#39;t understand the question. Code sample please?
20:43rilliangerald: if I do https://pastebin.mozilla.org/9019378 to avoid a linker conflict with the same constant in libvpx, does it slow the code down?
20:43rillianI guess this is C, not C++
20:43jesuprillian: you mean accessing foo? (and I assume it was something like &#39;const size_t foo = sizeof(bar);&#39; or equiv)
20:43* jesup looks
20:43rillianjesup: yes. foo is only accessed in the one function.
20:43jesupshould have no impact on perf I can think of
20:44rillianok, thanks.
20:44rillianI mean, we could make it a #define if it did.
20:44geraldwhat jesup said :-)
20:45geraldI would suggest avoiding #define&#39;s whenever possible, please use properly-typed C++ constants instead.
20:46rillianwell, see it being C code. we&#39;re lucky to have `const`
20:46gerald:-)
21:29geraldkentuckyfriedtakahe: I&#39;m thinking of limiting WebCompat notifications to NS_ERROR_DOM_MEDIA_DEMUXER_ERR and NS_ERROR_DOM_MEDIA_METADATA_ERR for now (in a pref). Ok with you? The list of errors is there: http://searchfox.org/mozilla-central/source/xpcom/base/ErrorList.py#1073-1088
21:29kentuckyfriedtakahegerald: sgtm
21:29geraldta
21:57andrenataljesup: jmspeex 48k worked well. thank you guys.
22:04SingingTreeAnyone know of any resources detailing interaction with the cycle collector aside from https://developer.mozilla.org/en-US/docs/Mozilla/Tech/XPCOM/Interfacing_with_the_XPCOM_cycle_collector ?
22:05SingingTreeI&#39;m looking at modifying a MediaRecorder which has a swag of cycle collection stuff. Don&#39;t think I need to touch those bits, but would like to understand it better as unknowingly breaking any of it seems potentially embarrassing
22:21ngjib: ping
23:05jibng: pong
23:12kamidphishrillian: ping
23:18kentuckyfriedtakahedebianuser: maybe you or damo22 want to do bug 1357594
23:18firebothttps://bugzil.la/1357594 NEW, nobody@mozilla.org Require sandbox disabling for tier-3 audio backends
23:26debianuserHm... (not really familiar with sandboxing yet) Is it possible to make it work under the sandbox? Jack probably only needs access to a /dev/shm/. Alsa only needs access to SYSV IPCs and /dev/snd/* files. Or those are not considered safe for sandboxed process?
23:41kamidphishrillian: ping?
23:59TD-Linuxdebianuser, libalsa can load arbitrary code so it&#39;s hard to know for sure. but otherwise it&#39;s possible to sandbox, if there was a maintainer.
19 Apr 2017
No messages
   
Last message: 158 days and 3 hours ago