Commit Graph

19 Commits

Author SHA1 Message Date
Justin Bastress
6618920234 add some tighter bounds checking in MSSQL scanner, and if there is an uncaught panic, log the body that caused it 2018-10-01 11:08:26 -04:00
Alex Halderman
1f5e4b0f22 Provides a framework for modularized target input and result output functions. 2018-07-05 12:41:34 -04:00
Justin Bastress
ccbf497b84 resolve conflict with utility.go 2018-06-28 16:05:48 -04:00
justinbastress
724d02d90d read full banners (#103)
* read full banners

* account for cases where smaller packets are returned
2018-06-28 15:06:39 -04:00
Alex Halderman
1147e936f9 Switch input processing to CSV and add tests 2018-06-28 08:04:50 -04:00
J. Alex Halderman
9df97e03cf Add support for tags and triggers. 2018-06-26 13:51:10 -04:00
justinbastress
b06d9bbc85
pull zflags from the zmap project (#69) 2018-03-07 11:04:16 -05:00
David Adrian
9b835ad162
Remove pointer-to-interface in zgrab2 framework (#40)
Interfaces are effectively already pointers, see
https://research.swtch.com/interfaces.

https://github.com/zmap/zgrab2/pull/40
2018-01-29 08:48:07 -05:00
justinbastress
a1fca68c04
Port FTP module from zgrab (#32)
* remove unnecessary indirection on net.Conn

* Ignore *.pyc

* fix NPE on nil handshake

* refactoring -- move status to status.go; add Open() methods for ScanTarget

* cherry-pick .gitignore fix

* pull in TLS fix

* status.go comments

* trim over-generalizations

* use /usr/bin/env bash instead of absolute path

* remove debug tcpwrap

* add integration tests for postgres

* hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib

* cleanup should actually stop the process though

* comments / rearrange

* Bump up timeout in postgres tests; only pass user if explicitly requested to do so

* add schema stubs to new.sh

* Integration test fixes -- use /usr/bin/env bash; log all validation failures

* add postgres schemas

* fill out zcrypto.client_hello schema

* handle early get of TLSLog

* postgres: return SCAN_SUCCESS on success

* cleanup

* fix new.sh

* fix typo

* postgres container cleanup

* build.sh docs

* standardize container/image names

* add not to check for success

* shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts

* catch schematically-valid but non-successful scans

* postgres: clean up output format; more scanning

* cleanup; better error handling; get detailed protocol version error

* refactor modules

* clean up dangling connections

* split gigantic postgres.go

* remove unused

* ServerParams gets its own type

* refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more

* rename entrypoint; remove duplicate postgres tests

* comments for postgres schema

* port over ftp support from the original zgrab; add schema / integration tests for FTP; fix log line in ssh test

* Use param expansion to check for env variable [minor]

This is a *very* minor change to `docker-runner/docker-run.sh` checks to
see if the environment variable required to run the script has been set
to a non-empty string. If not, the script exits with a non-zero status
code and displays a default message:

```
❯ docker-runner/docker-run.sh
docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set
```

This was the behavior before, but just uses a one-liner declarative bash
idiom.

For further reading on parameter expansion, see
https://stackoverflow.com/a/307735.

@justinbastress can tell me if I did something wrong and broke the
intent of the script :-)

* Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter

* use make in docker-runner entrypoint

* add .integration_test_setup to .gitignore

* cleanup

* add ftp schema

* more .gitignore items

* Makefile updates: Windows support; add docker-runner target; better cleanup.

* docker-runner Dockerfile: start from zgrab2_runner_base image

* cleanup postgres setup

* make travis use make

* add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least

* fix folder name in Makefile

* update go (one of our dependencies now works only with >= 1.9)

* From travis: `I don't have any idea what to do with '1.9.0'.`

* explicit clean make

* fix dep order

* fix build.sh location

* popd

* use make to ensure zgrab2_runner exists

* Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test

* use explicit tag name for zgrab2_runner

* Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images

* use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?)

* remove overzealous cleanup

* let setup continue even if some containers are already running

* ftp: use zgrab2_service_base for ftp container; restart container if already running

* FTP: remove redundant prefix; add non-authtls test

* zgrab depends on *.go

* docker-runner depends on zgrab2 binary

* clean output before running integration tests

* address Zakir's comments: rename auth_*_resp -> auth_*, 'a FTP banner' -> 'an FTP banner'

* fix schema

* update/rebuild when necessary in container
2018-01-18 13:41:51 -05:00
justinbastress
f49887290d
Implements postgres zgrab2 module (#30)
* remove unnecessary indirection on net.Conn

* Ignore *.pyc

* fix NPE on nil handshake

* refactoring -- move status to status.go; add Open() methods for ScanTarget

* cherry-pick .gitignore fix

* pull in TLS fix

* status.go comments

* trim over-generalizations

* use /usr/bin/env bash instead of absolute path

* remove debug tcpwrap

* add integration tests for postgres

* hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib

* cleanup should actually stop the process though

* comments / rearrange

* Bump up timeout in postgres tests; only pass user if explicitly requested to do so

* add schema stubs to new.sh

* Integration test fixes -- use /usr/bin/env bash; log all validation failures

* add postgres schemas

* fill out zcrypto.client_hello schema

* handle early get of TLSLog

* postgres: return SCAN_SUCCESS on success

* cleanup

* fix new.sh

* fix typo

* postgres container cleanup

* build.sh docs

* standardize container/image names

* add not to check for success

* shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts

* catch schematically-valid but non-successful scans

* postgres: clean up output format; more scanning

* cleanup; better error handling; get detailed protocol version error

* refactor modules

* clean up dangling connections

* split gigantic postgres.go

* remove unused

* ServerParams gets its own type

* refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more

* rename entrypoint; remove duplicate postgres tests

* comments for postgres schema

* Use param expansion to check for env variable [minor]

This is a *very* minor change to `docker-runner/docker-run.sh` checks to
see if the environment variable required to run the script has been set
to a non-empty string. If not, the script exits with a non-zero status
code and displays a default message:

```
❯ docker-runner/docker-run.sh
docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set
```

This was the behavior before, but just uses a one-liner declarative bash
idiom.

For further reading on parameter expansion, see
https://stackoverflow.com/a/307735.

@justinbastress can tell me if I did something wrong and broke the
intent of the script :-)

* Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter

* use make in docker-runner entrypoint

* add .integration_test_setup to .gitignore

* more .gitignore items

* Makefile updates: Windows support; add docker-runner target; better cleanup.

* docker-runner Dockerfile: start from zgrab2_runner_base image

* cleanup postgres setup

* make travis use make

* add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least

* fix folder name in Makefile

* update go (one of our dependencies now works only with >= 1.9)

* From travis: `I don't have any idea what to do with '1.9.0'.`

* explicit clean make

* fix dep order

* fix build.sh location

* popd

* use make to ensure zgrab2_runner exists

* Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test

* use explicit tag name for zgrab2_runner

* Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images

* use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?)

* remove overzealous cleanup

* let setup continue even if some containers are already running

* zgrab depends on *.go

* docker-runner depends on zgrab2 binary

* clean output before running integration tests
2018-01-15 14:24:57 -05:00
justinbastress
a244ec15e6 TLS scan prototype + Travis integration (#25)
* Fix typo

* Actually call init per sender for each goroutine

* split out TLS handling

* Rename tls_handshake; update docs

* fix comments

* format fixes

* merge updates

* fix path

* refactor heartbleed logging (now the same as original zgrab)

* add ScanStatus, update modules to return it

* fix threaded for loop; fix styling of dict literal

* fix compile errors, note un-bubbled handshake error

* initial schema commit

* fix comment

* schema cleanup

* comments

* fix TODOs

* first attempt at docker integration in travis; also add schema validation

* add integration_tests.sh

* need sudo?

* try pip install --user

* revert regression

* add docker service again

* chmod +x integration_tests.sh

* fix path of binary

* Dump output file to stdout

* travis work

* use jp's build-self-contained script

* use go get/go build to get jp

* fix jp path

* switch from bogus regex to wildcard

* do all mysql versions; fix version comparison

* re-enable notifications; fix successful version check log message; comment TryGetScanStatus

* move to conf.d layout for integration tests

* update README

* add missing scripts

* add ./ to path
2017-12-15 09:25:17 -05:00
Alex
3545d223e9 gofmt, golint, add docs 2017-10-03 23:57:56 -04:00
Alex
b7f50acf5d redo modules/make command line parsing sane and easy to understand 2017-09-29 18:59:37 -04:00
Alex
7f71198296 utility.go changes 2017-09-26 14:15:15 -04:00
Alex
54817aa08d change structure of zgrab2 2017-09-26 14:02:27 -04:00
Alex
57e6357da9 review name changes, list of ips to ipnet 2017-08-31 17:24:05 -04:00
Alex
11de980f43 timeout to base module, parse ip/domain and do lookup 2017-08-28 21:23:16 -04:00
Alex
6d926d2568 make zgrab2 great again, allow ability to scan multiple of same modules (mods to zflags as well), monitor now supports monitoring all modules, start of metadata implementation 2017-08-25 12:04:16 -04:00
Alex
41eb0fd7d4 refactor structure, monitor, modules (further work to be done on modules) 2017-08-16 12:09:20 -04:00