mozilla :: #datapipeline

17 Jul 2017
09:26Dextermreid, amiyaguchi -> potentially interesting: differential privacy enabled SQL queries from Uber -> https://medium.com/uber-security-privacy/differential-privacy-open-source-7892c82c42b6 (via vanessa )
09:27Dexternot super useful client side, but might be somehow useful on the pipeline side
09:27Dexteralso harter ^
09:36amiyaguchiI wonder what motivated uber to design a differential privacy system
09:37amiyaguchithey want to query about trip (location?) data without knowing specific locations of individuals
10:19FallenSo, I have a redash query that seems to be stuck. I fixed it for me by creating a new one and archiving the old, but in case anyone needs to fix the queue for this query, https://sql.telemetry.mozilla.org/queries/4936/
12:27gfritzscheharter: hm, doing one search in my nightly fx (search box), i end up with two search counts: "google.searchbar" & "google.sap:unknown:firefox-b"
12:27gfritzschedo you know if that is intentional?
12:28gfritzschecan this result in double counting searches for our analysis?
12:44amiyaguchigfritzsche: It is intentional (bug 1367554), searches are currently filtered in search rollups and topline datasets
12:44firebotBug https://bugzil.la/1367554 is not accessible
13:00gfritzscheah, ok, confusing
13:01gfritzschethe sum of sum of search counts is larger then the actual sum of searches
13:01gfritzschebut good to know its expected
14:02mreidsunahsuh: experiments mtg?
14:03sunahsuhYeah sorry, computer is frozen?
14:04sunahsuhLogging on via mobile
15:00frankharter: can we postpone for 5 minutes
15:01hartersure
15:01harterfrank: ^
15:09spenroseatmo is down for me // back in 15
15:10robotblakespenrose: That was my fault, should be fixed now
15:33spenroserobotblake np, thanks for quick turnaround
16:32FallenSo I have a redash query that keeps getting "stuck". Updating it makes it spin forever. I already archived one and recreated it which fixed the situation for a few hours, but now it happened again. Is there something about the query causing trouble or is it something else? https://sql.telemetry.mozilla.org/queries/5694
16:33FallenI have a few similar queries for week/month that don't have that problem
17:08mreidrobotblake: ^
17:10robotblakeLooking :\
17:17Fallenrobotblake: this was the old query https://sql.telemetry.mozilla.org/queries/4936/source
17:18robotblakeI'm going to clear the query queue
17:22Fallenthat seems to have fixed it for now. Anything I can do to help figure out what the issue is if it shows up again?
17:24robotblakeFile a bug :\
17:24robotblakeBy "spin forever" do you mean in the UI or just that running it never completed?
17:25Fallenit runs and never completes. If I run it on the query page, then the blue bar showing the query is running/pending stays forever.
17:25Fallen(it runs and never completes = when I'm looking at it on a dashboard)
17:27robotblakeAlright, so my guess is that the "scheduled" queue was backed up really badly, if you run a query that has a schedule set (even if you're running it manually), it'll go into that queue
17:27robotblakeAs opposed to the "adhoc" queue
17:28Fallenah, that makes sense!
17:29Fallenso if I remove the schedule then it should go into the adhoc queue, this would confirm, right?
17:30robotblakeYeah, remove the schedule and add something to the text of the query (like a sql comment "-- something something") to invalidate the cache
18:05mreidrobotblake: ping
18:06mreid^ re bug 1381563
18:06firebothttps://bugzil.la/1381563 NEW, nobody@mozilla.org main_summary data stops at 20170706
18:08robotblakeYeah, looking into it :(
18:10mreidthanks
18:10mreidcan I assign to you in bz?
18:10mreidrobotblake: ^
18:13robotblakeYeah
23:07sunahsuhanyone else having issues with `Dataset.select()` hanging in a notebook?
18 Jul 2017
No messages
   
Last message: 67 days and 5 hours ago