zgrab2/zgrab2_schemas/zgrab2/postgres.py

72 lines
2.3 KiB
Python
Raw Normal View History

Implements postgres zgrab2 module (#30) * remove unnecessary indirection on net.Conn * Ignore *.pyc * fix NPE on nil handshake * refactoring -- move status to status.go; add Open() methods for ScanTarget * cherry-pick .gitignore fix * pull in TLS fix * status.go comments * trim over-generalizations * use /usr/bin/env bash instead of absolute path * remove debug tcpwrap * add integration tests for postgres * hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib * cleanup should actually stop the process though * comments / rearrange * Bump up timeout in postgres tests; only pass user if explicitly requested to do so * add schema stubs to new.sh * Integration test fixes -- use /usr/bin/env bash; log all validation failures * add postgres schemas * fill out zcrypto.client_hello schema * handle early get of TLSLog * postgres: return SCAN_SUCCESS on success * cleanup * fix new.sh * fix typo * postgres container cleanup * build.sh docs * standardize container/image names * add not to check for success * shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts * catch schematically-valid but non-successful scans * postgres: clean up output format; more scanning * cleanup; better error handling; get detailed protocol version error * refactor modules * clean up dangling connections * split gigantic postgres.go * remove unused * ServerParams gets its own type * refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more * rename entrypoint; remove duplicate postgres tests * comments for postgres schema * Use param expansion to check for env variable [minor] This is a *very* minor change to `docker-runner/docker-run.sh` checks to see if the environment variable required to run the script has been set to a non-empty string. If not, the script exits with a non-zero status code and displays a default message: ``` ❯ docker-runner/docker-run.sh docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set ``` This was the behavior before, but just uses a one-liner declarative bash idiom. For further reading on parameter expansion, see https://stackoverflow.com/a/307735. @justinbastress can tell me if I did something wrong and broke the intent of the script :-) * Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter * use make in docker-runner entrypoint * add .integration_test_setup to .gitignore * more .gitignore items * Makefile updates: Windows support; add docker-runner target; better cleanup. * docker-runner Dockerfile: start from zgrab2_runner_base image * cleanup postgres setup * make travis use make * add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least * fix folder name in Makefile * update go (one of our dependencies now works only with >= 1.9) * From travis: `I don't have any idea what to do with '1.9.0'.` * explicit clean make * fix dep order * fix build.sh location * popd * use make to ensure zgrab2_runner exists * Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test * use explicit tag name for zgrab2_runner * Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images * use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?) * remove overzealous cleanup * let setup continue even if some containers are already running * zgrab depends on *.go * docker-runner depends on zgrab2 binary * clean output before running integration tests
2018-01-15 19:24:57 +00:00
# zschema sub-schema for zgrab2's postgres module
# Registers zgrab2-postgres globally, and postgres with the main zgrab2 schema.
from zschema.leaves import *
from zschema.compounds import *
import zschema.registry
2018-05-04 15:15:09 +00:00
import zcrypto_schemas.zcrypto as zcrypto
2018-04-23 14:47:34 +00:00
import zgrab2
Implements postgres zgrab2 module (#30) * remove unnecessary indirection on net.Conn * Ignore *.pyc * fix NPE on nil handshake * refactoring -- move status to status.go; add Open() methods for ScanTarget * cherry-pick .gitignore fix * pull in TLS fix * status.go comments * trim over-generalizations * use /usr/bin/env bash instead of absolute path * remove debug tcpwrap * add integration tests for postgres * hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib * cleanup should actually stop the process though * comments / rearrange * Bump up timeout in postgres tests; only pass user if explicitly requested to do so * add schema stubs to new.sh * Integration test fixes -- use /usr/bin/env bash; log all validation failures * add postgres schemas * fill out zcrypto.client_hello schema * handle early get of TLSLog * postgres: return SCAN_SUCCESS on success * cleanup * fix new.sh * fix typo * postgres container cleanup * build.sh docs * standardize container/image names * add not to check for success * shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts * catch schematically-valid but non-successful scans * postgres: clean up output format; more scanning * cleanup; better error handling; get detailed protocol version error * refactor modules * clean up dangling connections * split gigantic postgres.go * remove unused * ServerParams gets its own type * refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more * rename entrypoint; remove duplicate postgres tests * comments for postgres schema * Use param expansion to check for env variable [minor] This is a *very* minor change to `docker-runner/docker-run.sh` checks to see if the environment variable required to run the script has been set to a non-empty string. If not, the script exits with a non-zero status code and displays a default message: ``` ❯ docker-runner/docker-run.sh docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set ``` This was the behavior before, but just uses a one-liner declarative bash idiom. For further reading on parameter expansion, see https://stackoverflow.com/a/307735. @justinbastress can tell me if I did something wrong and broke the intent of the script :-) * Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter * use make in docker-runner entrypoint * add .integration_test_setup to .gitignore * more .gitignore items * Makefile updates: Windows support; add docker-runner target; better cleanup. * docker-runner Dockerfile: start from zgrab2_runner_base image * cleanup postgres setup * make travis use make * add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least * fix folder name in Makefile * update go (one of our dependencies now works only with >= 1.9) * From travis: `I don't have any idea what to do with '1.9.0'.` * explicit clean make * fix dep order * fix build.sh location * popd * use make to ensure zgrab2_runner exists * Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test * use explicit tag name for zgrab2_runner * Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images * use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?) * remove overzealous cleanup * let setup continue even if some containers are already running * zgrab depends on *.go * docker-runner depends on zgrab2 binary * clean output before running integration tests
2018-01-15 19:24:57 +00:00
# modules/postgres/scanner.go - decodeError() (TODO: Currently an unconstrained
# map[string]string; it is possible to get "unknown (0x%x)" fields, but it
# would probably be proper to reject those at this point)
# These are defined in detail at
# https://www.postgresql.org/docs/10/static/protocol-error-fields.html
postgres_error = SubRecord({
"severity": String(required=True),
"severity_v": String(),
"code": String(required=True),
"message": String(),
"detail": String(),
"hint": String(),
"position": String(),
"internal_position": String(),
"internal_query": String(),
"where": String(),
"schema": String(),
"table": String(),
"data": String(),
"file": String(),
"line": String(),
"routine": String(),
})
# modules/postgres/scanner.go - decodeAuthMode()
AUTH_MODES = [
"kerberos_v5", "password_cleartext", "password_md5", "scm_credentials",
"gss", "sspi", "sasl", "ok", "gss-continue", "sasl-continue", "sasl-final"
]
# modules/postgres/scanner.go: AuthenticationMode
postgres_auth_mode = SubRecord({
"mode": Enum(values=AUTH_MODES, required=True),
"Payload": Binary(),
})
# modules/postgres/scanner.go: BackendKeyData
postgres_key_data = SubRecord({
"process_id": Unsigned32BitInteger(),
"secret_key": Unsigned32BitInteger(),
})
# modules/postgres/scanner.go: PostgresResults
postgres_scan_response = SubRecord({
"result": SubRecord({
"tls": zgrab2.tls_log,
"supported_versions": String(),
"protocol_error": postgres_error,
"startup_error": postgres_error,
"is_ssl": Boolean(required=True),
"authentication_mode": postgres_auth_mode,
# TODO FIXME: This is currendly an unconstrained map[string]string
"server_parameters": String(),
"backend_key_data": postgres_key_data,
"transaction_status": String(),
})
}, extends=zgrab2.base_scan_response)
zschema.registry.register_schema("zgrab2-postgres", postgres_scan_response)
zgrab2.register_scan_response_type("postgres", postgres_scan_response)