mozilla :: #websectools

20 Apr 2017
07:26mgm_rain0rmornin
07:27mgm_rain0rthc202, I did as you say (re: https://github.com/zaproxy/zaproxy/pull/3427)
07:27mgm_rain0r*said
07:29psiinonmorning
07:30thc202mgm_rain0r, thanks
07:32mgm_rain0rwill do the same here: https://github.com/zaproxy/zaproxy/pull/3428/commits
07:33thc202:)
08:18mgm_rain0rIs it possible to rebase those 14 commits into a single one? https://github.com/zaproxy/zap-extensions/pull/837/commits
08:18mgm_rain0rbecause I already merged "
08:18mgm_rain0rMerge branch 'beta' of github.com:rain0r/zap-extensions into beta"
08:20thc202sure, rebase interactive will get rid of the merge commits and you can also fixup the others
08:21mgm_rain0rI tried but couldn't do it
08:21thc202git rebase -i upstream/beta ?
08:22thc202sorry, should be alpha not beta
08:22mgm_rain0rbut I'm working on beta
08:22thc202oops, yeah, it's beta :/
08:22mgm_rain0rfatal: Needed a single revision
08:22mgm_rain0r - invalid upstream upstream/beta
08:23mgm_rain0rhowever, when I do this:
08:23mgm_rain0rgit rebase -i HEAD~2
08:23mgm_rain0rI get the last 12 commits
08:23mgm_rain0rbut why?
08:24mgm_rain0rand I can't merge the 2 commits: 0be444e and 076e591
08:24mgm_rain0r(I mean squash)
08:24mgm_rain0rthen I get this: https://paste.hihn.org/M7SV/raw
08:25thc202what's the name of the remote for zaproxy repo?
08:26mgm_rain0r[remote "origin"]
08:26mgm_rain0r url = git@github.com:rain0r/zap-extensions.git
08:26mgm_rain0r fetch = +refs/heads/*:refs/remotes/origin/*
08:26mgm_rain0rthat?
08:28thc202that's yours, you should add one for zaproxy (to obtain the latest changes, sync branches) https://help.github.com/articles/configuring-a-remote-for-a-fork/
08:30mgm_rain0rOk, I did that
08:31thc202ok, did you fetch the "upstream" branches?
08:32mgm_rain0ryes, with git fetch upstream
08:32thc202ok, git rebase -i upstream/beta should work now
08:33mgm_rain0ruh that takes long
08:33mgm_rain0r:)
08:35thc202hm, it shouldn't take that much
08:35mgm_rain0rbut now I only have the master branch
08:36thc202ok, in which branch are you right now?
08:37mgm_rain0rah okay, I have origin/* and upstream/*
08:38thc202yep, with the latter you can keep your local/origin branches up-to-date
08:39thc202did you squash/fixup the commits?
08:41mgm_rain0rI can't do it
08:42mgm_rain0rSee: https://paste.hihn.org/Fpgh/raw
08:42mgm_rain0rCommit # 16409c219a885c56fbf9d89b701b2a349987cf3f is the first here: https://github.com/zaproxy/zap-extensions/pull/837/commits
08:48thc202ok, be8f713 and 1c55cad should be removed, we don't want that in that branch
08:48thc202(they end up empty because the changes are already done)
08:51thc202(you could also skip that commit, with git rebase --skip)
08:55mgm_rain0rOkay,I dismissed those 2 commits
08:55mgm_rain0rbut now...
08:55mgm_rain0rerror: could not apply 8c23be3... Added more HTTP methods
08:55mgm_rain0rbut whyyyy
08:56thc202yeah, that's interesting, let me check out your branch
09:02thc202ok, it has conflicts, because of the formatting changes
09:06mgm_rain0rwhy am I in a detached state when I checkout origin/beta?
09:07thc202you need to checkout a local branch (beta?)
09:07thc202origin/beta and beta point to the same commit?
09:08mgm_rain0rThere is no branch "beta", just origin/beta and upstream/beta
09:09thc202but you were working on that one, right? was it deleted?
09:10mgm_rain0rIt disappeared after I added the upstream repo
09:10mgm_rain0rthis is my git config https://paste.hihn.org/MqnG/raw
09:11mgm_rain0rorigin/beta is the former beta branch
09:12thc202ok, git checkout -b beta origin/beta
09:14mgm_rain0rgit checkout -b beta origin/beta
09:14thc202re the conflict, you can either resolve it (you just need to delete the chunk that you don't want) or, instead of rebase, reset and commit
09:14mgm_rain0rI pushed something!
09:15mgm_rain0rhttps://github.com/zaproxy/zap-extensions/pull/837/commits
09:15mgm_rain0rIt doesnt really work
09:16mgm_rain0rMaybe I should just delete the PR and create a new one? In the end there are just 2 files that are changed
09:16mgm_rain0rand I (stupidly) commited formatting changes
09:16thc202is 2e2932b2b743618273d64bc1a665cf2665f66d27 what you want to keep?
09:17mgm_rain0ryes
09:18thc202ok, git rebase -i upstream/beta
09:18thc202then remove all commits except the first
09:19thc202start the rebase and done, should be ready to push
09:19mgm_rain0rfrom a fresh "git clone"?
09:19mgm_rain0r"git rebase ..."
09:19mgm_rain0r?
09:19thc202from your beta branch
09:20mgm_rain0rgit checkout -b beta origin/beta
09:20mgm_rain0r?
09:20thc202git status
09:20thc202I think you are already in the beta branch
09:20mgm_rain0rgit checkout -b beta origin/beta
09:20mgm_rain0rOn branch beta
09:20mgm_rain0rYour branch is up-to-date with 'origin/beta'.
09:20thc202ok
09:20cyactually if you want to remove all commits except the first you could just "git reset 2e2932b2b743618273d"
09:21thc202yeah, that does it too
09:21cyim a total fan of rebase if you want to fix commits, like add tests to the corresponding change
09:22thc202:)
09:22cybut for this its actually harder for someone not so fancy with git :-)
09:23thc202yeah, reset is a lot simpler
09:23mgm_rain0rfatal: Could not parse object '2e2932b2b743618273d64bc1a665cf2665f66d27'.
09:23mgm_rain0r:-(
09:26thc202with which command?
09:26mgm_rain0rgit reset 2e2932b2b743618273d64bc1a665cf2665f66d27
09:26thc202git log 2e2932b2b743618273d64bc1a665cf2665f66d27
09:26thc202does it show the history?
09:27mgm_rain0rfatal: bad object 2e2932b2b743618273d64bc1a665cf2665f66d27
09:27mgm_rain0rno
09:27thc202git status
09:28mgm_rain0rOn branch beta
09:28mgm_rain0rYour branch is up-to-date with 'origin/beta'.
09:28mgm_rain0rnothing to commit, working tree clean
09:28mgm_rain0rah
09:28mgm_rain0rI wasn't up to date...
09:28mgm_rain0rgit pull helped
09:29mgm_rain0rokay git reset 2e.... worked
09:29thc202cool
09:30thc202were you using other clone?
09:31thc202you can now (force) push that
09:32mgm_rain0rI have a local copy of my github repo (so I can easily delete my working dir and start "fresh" without cloning from github)
09:32mgm_rain0rgit push origin/beta HEAD --force
09:32mgm_rain0r?
09:34thc202seems right
09:34mgm_rain0rfatal: 'origin/beta' does not appear to be a git repository
09:34mgm_rain0rfatal: Could not read from remote repository.
09:34mgm_rain0rPlease make sure you have the correct access rights
09:34mgm_rain0rand the repository exists
09:34mgm_rain0rstrange
09:34thc202oh, git push origin beta -f
09:35mgm_rain0rgit push origin beta -f
09:35mgm_rain0rhttps://github.com/zaproxy/zap-extensions/pull/837/commits
09:35mgm_rain0ryessssssssssss
09:35mgm_rain0rthanks big time!
09:36mgm_rain0rnow it's lunchtime (my coworker were already waiting for me ;) )
09:36thc202np
09:36thc202enjoy :)
14:12mgm_rain0ranother PR! :P
14:12thc202:)
14:22mgm_rain0rkingthorin doesn't make it easy for commiter
14:22thc202:D
14:23thc202well, better not introduce noise with formatting changes (which are not yet defined, so that's changing now to change later ;)
14:23thc202it's easier to review
14:24mgm_rain0rSo you don't want formatting changes in the same commit as code-changes but you also want the PR to only contain 1 commit? That's .... I don't know how to say it
14:26thc202you can format in another PR (if you really need to)
14:30thc202(that applies to everyone, I don't reformat existing classes to my own style/criteria, it might happen that some formatting slips when changing the code, but not more than the lines affected and normally to normalise the indentation with surrounding code)
14:34thc202stephend|mtg, ping
14:34stephend|mtghowdy!
14:34stephend|mtgtrying to get my poor home setup to work, sorry
14:35stephend|mtgthc202: can you access Vidyo? or should we try to use another client?
14:35thc202hi, which room is it? (also, I'm not sure I'm able to connect, I tried in the past without much success)
14:35stephend|mtgI'll grab the public room URL
14:35thc202ok
14:35stephend|mtgif not, perhaps we can use Google Hangouts or something similar
14:36stephend|mtgthc202: how about https://v.mozilla.com/flex.html?roomdirect.html&key=ZAlDIwL9AJcf ?
14:39thc202"This page is no longer supported" "For enhanced security, Flash-based functionality is no longer supported by this system. Instead, this system now supports HTML-based functionality. Please contact your system administrator for details."
14:39thc202it also points me to download VidyoDesktop
14:40thc202which requires a "VydioPortal" and an account to connect
14:40stephend|mtgthc202: let's just use Google Hangouts
14:40stephend|mtgI'll try to quickly set that up
14:41thc202ok
14:41thc202stephend|mtg, https://hub.docker.com/r/thc202/zap2docker-dns/ image that logs the DNS requests and the state set to the cache policy
14:41stephend|mtghttps://hangouts.google.com/call/2qgrcfzuqnabtomk3fybpvydyyu
14:41stephend|mtgclaudijd: psiinon
14:46stephend|mtghttps://github.com/stephendonner/docker-zap/commit/5213e631ff062e85937618e68922d3a2c907d32b
14:46stephend|mtghttp://webqa-ci-staging-temp1.qa.scl3.mozilla.com:8080/job/Docker-ZAP/ to use that ^^^
14:47stephend|mtghttp://webqa-ci-staging-temp1.qa.scl3.mozilla.com:8080/job/Docker-ZAP/107/console
14:49stephend|mtgit's using the SCL3 dmz/proxy
14:49stephend|mtgthe IP for the Jenkins box is static, though
14:49claudijdyeah, I have all the logs at my disposal, need source IP of the host you're coming from
14:50claudijdie. the worker or master, where ever the job is running
14:50claudijdchecking for: webqa-ci-staging-temp1.qa.scl3.mozilla.com
14:50stephend|mtgclaudijd: PING webqa-ci-staging-temp1.qa.scl3.mozilla.com (10.22.73.155): 56 data bytes
14:50stephend|mtgthat's its class A internal
14:51stephend|mtg07:48:48 requests.exceptions.ProxyError: HTTPConnectionPool(host=&#39;127.0.0.1&#39;, port=2375): Max retries exceeded with url: http://zap/JSON/ascan/action/scan/?url=https%3A%2F%2Fwww.allizom.org&apikey=&recurse=True (Caused by ProxyError(&#39;Cannot connect to proxy.&#39;, NewConnectionError(&#39;<requests.packages.urllib3.connection.HTTPConnection object at 0x7fdb1b320b50>:
14:51stephend|mtgFailed to establish a new connection: [Errno 111] Connection refused&#39;,)))
14:52thc202stephend|mtg, note that the image does not have any add-ons, you would have to add the command line argument -addoninstallall or -addonuninstall ascanrules
14:52thc202(to add some scanners)
14:52stephend|mtghttps://pastebin.mozilla.org/9019526
14:53thc202ok, just a sec
14:53stephend|mtgthx
14:53stephend|mtgshould https://github.com/stephendonner/docker-zap/commit/5213e631ff062e85937618e68922d3a2c907d32b have changed anything else?
14:53thc202stephend|mtg, you might need to add -config api.addrs.addr.name=.\* -config api.addrs.addr.regex=true
14:54claudijdbrb, need power!@
14:54thc202yeah, add it after -config connection.dnsTtlSuccessfulQueries=-1
14:54stephend|mtgok, thx
14:54psiinonhttps://github.com/zaproxy/zaproxy/wiki/FAQapikey
14:55thc202it might be just .*
14:55thc202(the slash was to escape the *)
14:56stephend|mtghow&#39;s this?
14:56stephend|mtghttps://github.com/stephendonner/docker-zap/commit/9e2f8ae2bc8adc367943b96ec1b3b67c2a7da086
14:56thc202looks right
14:58thc202it&#39;s also using -config api.disablekey=true
14:58thc202so it should be working
14:58stephend|mtghmm
14:58thc202better to also add to the command -addoninstall ascanrules (otherwise there&#39;s no scanners)
14:58stephend|mtghttps://pastebin.mozilla.org/9019527
15:00thc202ok, one thing I forgot to mention, the image should be run with root instead of zap
15:00stephend|mtghttps://github.com/stephendonner/docker-zap/commit/f1f29fae8792de40dc5b9a42bf371b48236945b3
15:01stephend|mtghttps://github.com/stephendonner/docker-zap/commit/deb4ce675167db1df3d848f877843add34d38974
15:02thc202claudijd, do you know why it&#39;s failing to resolve &quot;example.org&quot; ?
15:03thc202&quot;UnknownHostException: example.org: Temporary failure in name resolution&quot;
15:03claudijdresolves locally on the proxy...
15:03stephend|mtgstill no dice
15:03claudijdhttps://irccloud.mozilla.com/pastebin/AkvQ5744/
15:03stephend|mtghttps://pastebin.mozilla.org/9019528
15:04thc202yes, I added that one during startup
15:04stephend|mtgok, thanks
15:04thc202ok, that&#39;s fine, just curiosity
15:04stephend|mtgcool
15:07psiinondocker run -u root -t owasp/zap2docker-stable /usr/bin/curl example.co
15:08thc202psiinon, ZAP does not seem to be fully starting? last log message is &quot;ExtensionDynSSL - Creating new root CA certificate&quot;
15:10stephend|mtghttps://github.com/stephendonner/docker-zap/commit/ef082016eed534fdaa20d54bd300659d6a299bdc
15:14stephend|mtgno proxy host for: localhost,127.0.0.1,localaddress,.localdomain.com, 10.0.0.0/8, .scl3.mozilla.com, .phx1.mozilla.com, *.scl3.mozilla.com, *.phx1.mozilla.com, 169.254.169.254
15:15stephend|mtgam I frozen on video?
15:15stephend|mtgyikes
15:16claudijdhttps://irccloud.mozilla.com/pastebin/vxhejuoC/
15:16stephend|mtgwell, here&#39;s a potential problem
15:16psiinonstephend|mtg: yeah, you&#39;re frozen
15:18stephend|mtghttps://irccloud.mozilla.com/file/yBVmDWwe/Screen%20Shot%202017-04-20%20at%208.18.22%20AM.png
15:18stephend|mtgthat red error there...
15:19stephend|mtggit is, yes
15:19stephend|mtgso is there a whitelist for hosts?
15:22claudijdstephend|mtg: you&#39;re cutting out a lot for me
15:22stephend|mtgarg
15:23stephend|mtgnot sure what else to try
15:24stephend|mtghttps://pastebin.mozilla.org/9019533
15:25thc202stephend|mtg, could you increase the start timeout in the script to, say, 120 ? (just in case, it needs to download the add-on)
15:25thc202assuming it&#39;s able to download it
15:25stephend|mtgI can sure try - where?
15:25thc202status -t 120
15:26stephend|mtgin this line?
15:26stephend|mtghttps://github.com/stephendonner/docker-zap/blob/bb5f95add6330b744345855394df0266a307dc73/run-docker.sh#L3
15:26stephend|mtgoh sorry
15:26stephend|mtgI see
15:26stephend|mtgnvm
15:27stephend|mtghttps://github.com/stephendonner/docker-zap/commit/20d256511642d572a6cdfb388a79300a322b6829
15:29stephend|mtgZOMG
15:29stephend|mtghttps://irccloud.mozilla.com/pastebin/pI3JSVWy/
15:30* stephend|mtg crosses fingers
15:30stephend|mtgclaudijd: traffic to akamai now?
15:30stephend|mtg(still)
15:30stephend|mtger
15:30stephend|mtgallizom
15:30claudijdyes
15:30stephend|mtgit&#39;s spidering
15:30stephend|mtggreat!
15:30claudijdhttps://irccloud.mozilla.com/pastebin/O0j6jCWN/
15:30stephend|mtgyay
15:30claudijd^ MISS == GOOD
15:31stephend|mtgjust means a re-fetch request, right?
15:31stephend|mtgfull log
15:31stephend|mtghttps://irccloud.mozilla.com/pastebin/eGVnNIhB/
15:31claudijdstephend|mtg: no, TCP_MISS is synonymous with &quot;ALLOWED&quot;
15:31stephend|mtgah
15:33thc202check for updates failed, which means it was not able to download the add-on
15:33thc202line 189 of the log
15:34stephend|mtghmm
15:36claudijdall clear on the proxies ATM
15:36claudijdno failures
15:36stephend|mtgcool
15:36stephend|mtgit should be hitting http://demo.testfire.net/ soon
15:36stephend|mtgI&#39;m happy to keep hanging on, but want to be cognizant of time
15:37claudijdI see not hits for demo.....
15:37claudijdon the upstream proxy
15:37stephend|mtgspidering now, hrm
15:37thc202psiinon, yeah, it&#39;s able to connect (but not scanning)
15:38stephend|mtg08:36:53 [INFO] ZAP is running
15:38stephend|mtg08:36:53 [INFO] Accessing URL http://demo.testfire.net/
15:38stephend|mtg08:37:35 [INFO] Running spider...
15:38stephend|mtg08:37:45 [INFO] Running an active scan...
15:38thc202the UnknownHostException happened when scanning :/
15:38stephend|mtgbut no hits to proxy?
15:38stephend|mtgthat&#39;s weird
15:39claudijdideally, ZAP will not be scanning through the upstream proxy, that&#39;s a lot of unnecessary traffic IMO
15:41claudijdmissed a lot of that, but yeah AWS vs. SCL3 on proxy load stuffs
15:41stephend|mtgwe&#39;re moving hosts to AWS
15:42stephend|mtgand changing what we do with ZAP - going deeper with it
15:43thc202psiinon, yeah, the ZAP settings look correct
15:43stephend|mtgI&#39;m heading into work soon
15:43stephend|mtgcan you let me know what I need to transplant from the changes, back to master?
15:44stephend|mtgI was just thanking everyone
15:44stephend|mtgfor helpiing and staying on a bit longer :-)
15:45stephend|mtgcheers
15:47thc202stephend|mtg, I can update the image to bundle some scanner add-ons, to try reproduce the issue if you want
15:49stephend|mtgthc202: thx; sounds good!
15:51thc202cool, will do that and let you know when ready (since the image is already logging the DNS requests it should give more info about the problem, if it&#39;s really a DNS issue or not)
16:00stephendgreat
18:28stephendthc202: should I take all the changes I made this morning and put them back into my script&#39;s master?
18:28stephendhttps://github.com/stephendonner/docker-zap/blob/20d256511642d572a6cdfb388a79300a322b6829/run-docker.sh#L4
18:28stephendhttps://github.com/stephendonner/docker-zap/compare/dns
18:38thc202better leave in a separate branch
18:38thc202that&#39;s just for debugging
18:45thc202but -config api.addrs.addr.name=.* -config api.addrs.addr.regex=true should be added (to be able to run with newer ZAP versions, 2.6.0 and weeklies)
20:14thc202stephend|lunch, just pushed the new version of the docker image that contains the add-ons
20:15thc202the command line argument addoninstall can be removed
20:22stephend|lunchthc202: perfect; thx!
20:24stephend|lunchthc202: should I also take this change? https://github.com/stephendonner/docker-zap/commit/deb4ce675167db1df3d848f877843add34d38974
20:25thc202no, that one is still needed to log the DNS queries
20:26stephendthe change to run as root user, rather than zap? ok, so leave it in the debug branch, but don&#39;t change master?
20:27thc202yes, use root and don&#39;t change master
20:27stephendthx
21 Apr 2017
No messages
   
Last message: 7 days and 19 hours ago