Commit Graph

148 Commits

Author SHA1 Message Date
Justin Bastress
4ee7c2a91f add oracle.go stub 2018-02-21 13:08:33 -05:00
Justin Bastress
84eaa0f88e Implement Connection.Connect; add command-line flags; implement Native Service Negotiation (NSN / 0xdeadbeef) data type 2018-02-21 09:25:33 -05:00
Justin Bastress
b30c7f45e1 split TNSHeader from the rest of the packet 2018-02-19 12:19:10 -05:00
Justin Bastress
3b262fea28 initial oracle scanner work 2018-02-19 11:19:19 -05:00
justinbastress
8d8bdeac93 if docker build fails, fail the build after deleting the *.id file (#57)
https://github.com/zmap/zgrab2/pull/57
2018-02-13 13:47:04 -05:00
justinbastress
f02bd59f8a Add redis zgrab2 module (#53) 2018-02-13 13:25:47 -05:00
justinbastress
bd952d9809
golint and godocs for mysql module (#56) 2018-02-12 10:42:45 -05:00
Zakir Durumeric
33d01cda40
Merge pull request #55 from zmap/feature/lintPostgres
Run golint on postgres module
2018-02-09 13:52:43 -05:00
justinbastress
2766a7ae08
Port http from zgrab (#31) 2018-02-09 13:45:50 -05:00
Justin Bastress
9eb7235e40 re-add package postgres 2018-02-09 13:23:59 -05:00
Justin Bastress
d242ad65f4 run gofmt on files 2018-02-09 11:49:15 -05:00
Justin Bastress
2acd6aaccb add package documentation 2018-02-09 11:25:03 -05:00
Justin Bastress
d55e385692 run golint on postgres code 2018-02-09 11:09:44 -05:00
Justin Bastress
1d72bbb2cd use hyphens, not underscores, for command-line parameters 2018-02-09 09:39:24 -05:00
justinbastress
2f7dfd76fe
Add NTP zgrab2 module (#39) 2018-02-07 12:25:46 -05:00
David Adrian
f993263100
Implement TLS module (#42)
This module just uses zgrab2.BaseTLSFlags to implement a simple TLS
module.
2018-02-02 15:54:05 -05:00
justinbastress
8eb958e22c
Add MSSQL zgrab2 module (#38) 2018-02-02 14:52:03 -05:00
justinbastress
a8b4461d29
Merge #45 into master (#52)
Remove integration-test-setup (no longer needed)
2018-01-31 13:01:14 -05:00
Zakir Durumeric
f47026adb3
Merge pull request #44 from zmap/feature/updateIntegrationTestScriptTemplates
Update integration test script templates
2018-01-31 11:05:34 -05:00
Zakir Durumeric
5d70688b66
Merge pull request #47 from jordan-wright/docs/fix-readme
Fix README install instructions pointing to previous versions of zgrab
2018-01-31 10:57:41 -05:00
Zakir Durumeric
bbecc9647a
Merge pull request #46 from zmap/feature/useFewerSimultaneousContainers
Use fewer simultaneous containers
2018-01-31 10:57:15 -05:00
Zakir Durumeric
df5d86d687
Merge pull request #49 from jordan-wright/issue/remove-zgrab-dependency
Changed tools/keys dependencies to point to zgrab2.
2018-01-31 10:56:45 -05:00
Zakir Durumeric
5c535a205e
Merge pull request #51 from jordan-wright/docs/add-readme-testing
Add information on running integration tests to README.md
2018-01-31 10:56:26 -05:00
justinbastress
e1f04217a9
log scan target on panic (#43) 2018-01-31 10:14:52 -05:00
Jordan Wright
78e7125661 Added information on how to run integration tests to README.md. Fixes #50 2018-01-31 00:09:05 -06:00
Jordan Wright
162138dac3 Changed tools/keys dependencies to point to zgrab2. 2018-01-30 23:45:52 -06:00
Jordan Wright
11d1602622 Fix README install instructions pointing to previous versions of zmap 2018-01-30 21:46:40 -06:00
Justin Bastress
e6739acff9 fix service test scripts -- remove useless i flags to docker run; if container is already running, use it; smarter wait conditions for postgres and mysql 2018-01-29 16:53:50 -05:00
Justin Bastress
c5747b7198 sandwich each module's test.sh with its setup.sh and cleanup.sh; fix some relative paths in test.sh 2018-01-29 16:48:57 -05:00
Justin Bastress
59aa3d8571 Update README; fix output file in test 2018-01-29 16:36:45 -05:00
Justin Bastress
da294f7f1e Add .templates folder in lieu of inline scripts; update templates to match more common scenarios 2018-01-29 16:03:32 -05:00
David Adrian
18728b6ce9 Remove swp file 2018-01-29 09:03:41 -05:00
David Adrian
9b835ad162
Remove pointer-to-interface in zgrab2 framework (#40)
Interfaces are effectively already pointers, see
https://research.swtch.com/interfaces.

https://github.com/zmap/zgrab2/pull/40
2018-01-29 08:48:07 -05:00
David Adrian
1ecec6d7b2
Remove cmd/zgrab2 from .gitignore (#41)
We don't want to ignore all things named zgrab2, just something named
zgrab2 in the top directory.

https://github.com/zmap/zgrab2/pull/41
2018-01-29 08:47:42 -05:00
justinbastress
a1fca68c04
Port FTP module from zgrab (#32)
* remove unnecessary indirection on net.Conn

* Ignore *.pyc

* fix NPE on nil handshake

* refactoring -- move status to status.go; add Open() methods for ScanTarget

* cherry-pick .gitignore fix

* pull in TLS fix

* status.go comments

* trim over-generalizations

* use /usr/bin/env bash instead of absolute path

* remove debug tcpwrap

* add integration tests for postgres

* hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib

* cleanup should actually stop the process though

* comments / rearrange

* Bump up timeout in postgres tests; only pass user if explicitly requested to do so

* add schema stubs to new.sh

* Integration test fixes -- use /usr/bin/env bash; log all validation failures

* add postgres schemas

* fill out zcrypto.client_hello schema

* handle early get of TLSLog

* postgres: return SCAN_SUCCESS on success

* cleanup

* fix new.sh

* fix typo

* postgres container cleanup

* build.sh docs

* standardize container/image names

* add not to check for success

* shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts

* catch schematically-valid but non-successful scans

* postgres: clean up output format; more scanning

* cleanup; better error handling; get detailed protocol version error

* refactor modules

* clean up dangling connections

* split gigantic postgres.go

* remove unused

* ServerParams gets its own type

* refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more

* rename entrypoint; remove duplicate postgres tests

* comments for postgres schema

* port over ftp support from the original zgrab; add schema / integration tests for FTP; fix log line in ssh test

* Use param expansion to check for env variable [minor]

This is a *very* minor change to `docker-runner/docker-run.sh` checks to
see if the environment variable required to run the script has been set
to a non-empty string. If not, the script exits with a non-zero status
code and displays a default message:

```
❯ docker-runner/docker-run.sh
docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set
```

This was the behavior before, but just uses a one-liner declarative bash
idiom.

For further reading on parameter expansion, see
https://stackoverflow.com/a/307735.

@justinbastress can tell me if I did something wrong and broke the
intent of the script :-)

* Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter

* use make in docker-runner entrypoint

* add .integration_test_setup to .gitignore

* cleanup

* add ftp schema

* more .gitignore items

* Makefile updates: Windows support; add docker-runner target; better cleanup.

* docker-runner Dockerfile: start from zgrab2_runner_base image

* cleanup postgres setup

* make travis use make

* add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least

* fix folder name in Makefile

* update go (one of our dependencies now works only with >= 1.9)

* From travis: `I don't have any idea what to do with '1.9.0'.`

* explicit clean make

* fix dep order

* fix build.sh location

* popd

* use make to ensure zgrab2_runner exists

* Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test

* use explicit tag name for zgrab2_runner

* Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images

* use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?)

* remove overzealous cleanup

* let setup continue even if some containers are already running

* ftp: use zgrab2_service_base for ftp container; restart container if already running

* FTP: remove redundant prefix; add non-authtls test

* zgrab depends on *.go

* docker-runner depends on zgrab2 binary

* clean output before running integration tests

* address Zakir's comments: rename auth_*_resp -> auth_*, 'a FTP banner' -> 'an FTP banner'

* fix schema

* update/rebuild when necessary in container
2018-01-18 13:41:51 -05:00
justinbastress
683e9041cd add GitHub PR template (#37)
refs https://github.com/zmap/zgrab2/pull/37
2018-01-18 12:30:12 -05:00
Zakir Durumeric
0e754eca47
Merge pull request #36 from zmap/feature/fixZCryptoSchemas
Fix zcrypto schema: address issues #33, #34, #35
2018-01-18 12:27:29 -05:00
Justin Bastress
ce583441b4 address issues #33, #34, #35 -- pull EKU, CP and NameConstraints from extensions.go 2018-01-15 16:37:02 -05:00
justinbastress
f49887290d
Implements postgres zgrab2 module (#30)
* remove unnecessary indirection on net.Conn

* Ignore *.pyc

* fix NPE on nil handshake

* refactoring -- move status to status.go; add Open() methods for ScanTarget

* cherry-pick .gitignore fix

* pull in TLS fix

* status.go comments

* trim over-generalizations

* use /usr/bin/env bash instead of absolute path

* remove debug tcpwrap

* add integration tests for postgres

* hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib

* cleanup should actually stop the process though

* comments / rearrange

* Bump up timeout in postgres tests; only pass user if explicitly requested to do so

* add schema stubs to new.sh

* Integration test fixes -- use /usr/bin/env bash; log all validation failures

* add postgres schemas

* fill out zcrypto.client_hello schema

* handle early get of TLSLog

* postgres: return SCAN_SUCCESS on success

* cleanup

* fix new.sh

* fix typo

* postgres container cleanup

* build.sh docs

* standardize container/image names

* add not to check for success

* shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts

* catch schematically-valid but non-successful scans

* postgres: clean up output format; more scanning

* cleanup; better error handling; get detailed protocol version error

* refactor modules

* clean up dangling connections

* split gigantic postgres.go

* remove unused

* ServerParams gets its own type

* refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more

* rename entrypoint; remove duplicate postgres tests

* comments for postgres schema

* Use param expansion to check for env variable [minor]

This is a *very* minor change to `docker-runner/docker-run.sh` checks to
see if the environment variable required to run the script has been set
to a non-empty string. If not, the script exits with a non-zero status
code and displays a default message:

```
❯ docker-runner/docker-run.sh
docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set
```

This was the behavior before, but just uses a one-liner declarative bash
idiom.

For further reading on parameter expansion, see
https://stackoverflow.com/a/307735.

@justinbastress can tell me if I did something wrong and broke the
intent of the script :-)

* Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter

* use make in docker-runner entrypoint

* add .integration_test_setup to .gitignore

* more .gitignore items

* Makefile updates: Windows support; add docker-runner target; better cleanup.

* docker-runner Dockerfile: start from zgrab2_runner_base image

* cleanup postgres setup

* make travis use make

* add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least

* fix folder name in Makefile

* update go (one of our dependencies now works only with >= 1.9)

* From travis: `I don't have any idea what to do with '1.9.0'.`

* explicit clean make

* fix dep order

* fix build.sh location

* popd

* use make to ensure zgrab2_runner exists

* Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test

* use explicit tag name for zgrab2_runner

* Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images

* use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?)

* remove overzealous cleanup

* let setup continue even if some containers are already running

* zgrab depends on *.go

* docker-runner depends on zgrab2 binary

* clean output before running integration tests
2018-01-15 14:24:57 -05:00
justinbastress
a5d8d0b57a
remove unnecessary indirection on net.Conn (#27) 2017-12-19 16:21:16 -05:00
Zakir Durumeric
02f94e2f49
Merge pull request #23 from zmap/feature/addMySQLZGrabModule
add MySQL module
2017-12-19 13:17:52 -05:00
justinbastress
e0bf14f645
add ssh integration tests (#26)
* add SSH integration tests

* README updates; better sshd container based on a docs.docker.com example
2017-12-19 10:09:43 -05:00
Justin Bastress
81a14b3654 typo fix 2017-12-18 11:44:05 -05:00
Justin Bastress
e7591817c6 update README.md for @andrewsardone's new.sh script 2017-12-18 11:43:46 -05:00
Justin Bastress
81c514c55e use global logger; clarify results in mysql.Scan(); use real CSV parser in tls.getCSV() 2017-12-18 11:30:22 -05:00
Justin Bastress
7bca221a9a use global logger; clarify results in mysql.Scan(); use real CSV parser in tls.getCSV() 2017-12-18 11:30:14 -05:00
Justin Bastress
ad785aba24 note travis notifications 2017-12-18 11:29:24 -05:00
Justin Bastress
3f77b4e720 remove empty integration_tests.py 2017-12-18 09:21:39 -05:00
Justin Bastress
031129c45c Merge remote-tracking branch 'origin/aps/feature/addMySQLZGrabModule' into feature/addMySQLZGrabModule 2017-12-18 09:10:43 -05:00
Andrew Sardone
f38d85b71c
Add script to generate new-protocol tests [minor]
Just a minor little utility script for generating shell scripts around
the setup/test/cleanup rhythm of the zgrab integration tests.
2017-12-18 00:56:57 -05:00