Commit Graph

18 Commits

Author SHA1 Message Date
Alex Halderman
1f5e4b0f22 Provides a framework for modularized target input and result output functions. 2018-07-05 12:41:34 -04:00
Justin Bastress
ccbf497b84 resolve conflict with utility.go 2018-06-28 16:05:48 -04:00
justinbastress
724d02d90d read full banners (#103)
* read full banners

* account for cases where smaller packets are returned
2018-06-28 15:06:39 -04:00
Alex Halderman
1147e936f9 Switch input processing to CSV and add tests 2018-06-28 08:04:50 -04:00
J. Alex Halderman
9df97e03cf Add support for tags and triggers. 2018-06-26 13:51:10 -04:00
justinbastress
b06d9bbc85
pull zflags from the zmap project (#69) 2018-03-07 11:04:16 -05:00
David Adrian
9b835ad162
Remove pointer-to-interface in zgrab2 framework (#40)
Interfaces are effectively already pointers, see
https://research.swtch.com/interfaces.

https://github.com/zmap/zgrab2/pull/40
2018-01-29 08:48:07 -05:00
justinbastress
a1fca68c04
Port FTP module from zgrab (#32)
* remove unnecessary indirection on net.Conn

* Ignore *.pyc

* fix NPE on nil handshake

* refactoring -- move status to status.go; add Open() methods for ScanTarget

* cherry-pick .gitignore fix

* pull in TLS fix

* status.go comments

* trim over-generalizations

* use /usr/bin/env bash instead of absolute path

* remove debug tcpwrap

* add integration tests for postgres

* hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib

* cleanup should actually stop the process though

* comments / rearrange

* Bump up timeout in postgres tests; only pass user if explicitly requested to do so

* add schema stubs to new.sh

* Integration test fixes -- use /usr/bin/env bash; log all validation failures

* add postgres schemas

* fill out zcrypto.client_hello schema

* handle early get of TLSLog

* postgres: return SCAN_SUCCESS on success

* cleanup

* fix new.sh

* fix typo

* postgres container cleanup

* build.sh docs

* standardize container/image names

* add not to check for success

* shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts

* catch schematically-valid but non-successful scans

* postgres: clean up output format; more scanning

* cleanup; better error handling; get detailed protocol version error

* refactor modules

* clean up dangling connections

* split gigantic postgres.go

* remove unused

* ServerParams gets its own type

* refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more

* rename entrypoint; remove duplicate postgres tests

* comments for postgres schema

* port over ftp support from the original zgrab; add schema / integration tests for FTP; fix log line in ssh test

* Use param expansion to check for env variable [minor]

This is a *very* minor change to `docker-runner/docker-run.sh` checks to
see if the environment variable required to run the script has been set
to a non-empty string. If not, the script exits with a non-zero status
code and displays a default message:

```
❯ docker-runner/docker-run.sh
docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set
```

This was the behavior before, but just uses a one-liner declarative bash
idiom.

For further reading on parameter expansion, see
https://stackoverflow.com/a/307735.

@justinbastress can tell me if I did something wrong and broke the
intent of the script :-)

* Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter

* use make in docker-runner entrypoint

* add .integration_test_setup to .gitignore

* cleanup

* add ftp schema

* more .gitignore items

* Makefile updates: Windows support; add docker-runner target; better cleanup.

* docker-runner Dockerfile: start from zgrab2_runner_base image

* cleanup postgres setup

* make travis use make

* add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least

* fix folder name in Makefile

* update go (one of our dependencies now works only with >= 1.9)

* From travis: `I don't have any idea what to do with '1.9.0'.`

* explicit clean make

* fix dep order

* fix build.sh location

* popd

* use make to ensure zgrab2_runner exists

* Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test

* use explicit tag name for zgrab2_runner

* Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images

* use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?)

* remove overzealous cleanup

* let setup continue even if some containers are already running

* ftp: use zgrab2_service_base for ftp container; restart container if already running

* FTP: remove redundant prefix; add non-authtls test

* zgrab depends on *.go

* docker-runner depends on zgrab2 binary

* clean output before running integration tests

* address Zakir's comments: rename auth_*_resp -> auth_*, 'a FTP banner' -> 'an FTP banner'

* fix schema

* update/rebuild when necessary in container
2018-01-18 13:41:51 -05:00
justinbastress
f49887290d
Implements postgres zgrab2 module (#30)
* remove unnecessary indirection on net.Conn

* Ignore *.pyc

* fix NPE on nil handshake

* refactoring -- move status to status.go; add Open() methods for ScanTarget

* cherry-pick .gitignore fix

* pull in TLS fix

* status.go comments

* trim over-generalizations

* use /usr/bin/env bash instead of absolute path

* remove debug tcpwrap

* add integration tests for postgres

* hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib

* cleanup should actually stop the process though

* comments / rearrange

* Bump up timeout in postgres tests; only pass user if explicitly requested to do so

* add schema stubs to new.sh

* Integration test fixes -- use /usr/bin/env bash; log all validation failures

* add postgres schemas

* fill out zcrypto.client_hello schema

* handle early get of TLSLog

* postgres: return SCAN_SUCCESS on success

* cleanup

* fix new.sh

* fix typo

* postgres container cleanup

* build.sh docs

* standardize container/image names

* add not to check for success

* shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts

* catch schematically-valid but non-successful scans

* postgres: clean up output format; more scanning

* cleanup; better error handling; get detailed protocol version error

* refactor modules

* clean up dangling connections

* split gigantic postgres.go

* remove unused

* ServerParams gets its own type

* refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more

* rename entrypoint; remove duplicate postgres tests

* comments for postgres schema

* Use param expansion to check for env variable [minor]

This is a *very* minor change to `docker-runner/docker-run.sh` checks to
see if the environment variable required to run the script has been set
to a non-empty string. If not, the script exits with a non-zero status
code and displays a default message:

```
❯ docker-runner/docker-run.sh
docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set
```

This was the behavior before, but just uses a one-liner declarative bash
idiom.

For further reading on parameter expansion, see
https://stackoverflow.com/a/307735.

@justinbastress can tell me if I did something wrong and broke the
intent of the script :-)

* Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter

* use make in docker-runner entrypoint

* add .integration_test_setup to .gitignore

* more .gitignore items

* Makefile updates: Windows support; add docker-runner target; better cleanup.

* docker-runner Dockerfile: start from zgrab2_runner_base image

* cleanup postgres setup

* make travis use make

* add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least

* fix folder name in Makefile

* update go (one of our dependencies now works only with >= 1.9)

* From travis: `I don't have any idea what to do with '1.9.0'.`

* explicit clean make

* fix dep order

* fix build.sh location

* popd

* use make to ensure zgrab2_runner exists

* Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test

* use explicit tag name for zgrab2_runner

* Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images

* use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?)

* remove overzealous cleanup

* let setup continue even if some containers are already running

* zgrab depends on *.go

* docker-runner depends on zgrab2 binary

* clean output before running integration tests
2018-01-15 14:24:57 -05:00
justinbastress
a244ec15e6 TLS scan prototype + Travis integration (#25)
* Fix typo

* Actually call init per sender for each goroutine

* split out TLS handling

* Rename tls_handshake; update docs

* fix comments

* format fixes

* merge updates

* fix path

* refactor heartbleed logging (now the same as original zgrab)

* add ScanStatus, update modules to return it

* fix threaded for loop; fix styling of dict literal

* fix compile errors, note un-bubbled handshake error

* initial schema commit

* fix comment

* schema cleanup

* comments

* fix TODOs

* first attempt at docker integration in travis; also add schema validation

* add integration_tests.sh

* need sudo?

* try pip install --user

* revert regression

* add docker service again

* chmod +x integration_tests.sh

* fix path of binary

* Dump output file to stdout

* travis work

* use jp's build-self-contained script

* use go get/go build to get jp

* fix jp path

* switch from bogus regex to wildcard

* do all mysql versions; fix version comparison

* re-enable notifications; fix successful version check log message; comment TryGetScanStatus

* move to conf.d layout for integration tests

* update README

* add missing scripts

* add ./ to path
2017-12-15 09:25:17 -05:00
Alex
3545d223e9 gofmt, golint, add docs 2017-10-03 23:57:56 -04:00
Alex
b7f50acf5d redo modules/make command line parsing sane and easy to understand 2017-09-29 18:59:37 -04:00
Alex
7f71198296 utility.go changes 2017-09-26 14:15:15 -04:00
Alex
54817aa08d change structure of zgrab2 2017-09-26 14:02:27 -04:00
Alex
57e6357da9 review name changes, list of ips to ipnet 2017-08-31 17:24:05 -04:00
Alex
11de980f43 timeout to base module, parse ip/domain and do lookup 2017-08-28 21:23:16 -04:00
Alex
6d926d2568 make zgrab2 great again, allow ability to scan multiple of same modules (mods to zflags as well), monitor now supports monitoring all modules, start of metadata implementation 2017-08-25 12:04:16 -04:00
Alex
41eb0fd7d4 refactor structure, monitor, modules (further work to be done on modules) 2017-08-16 12:09:20 -04:00