zgrab2/schemas/postgres.py
justinbastress f49887290d
Implements postgres zgrab2 module (#30)
* remove unnecessary indirection on net.Conn

* Ignore *.pyc

* fix NPE on nil handshake

* refactoring -- move status to status.go; add Open() methods for ScanTarget

* cherry-pick .gitignore fix

* pull in TLS fix

* status.go comments

* trim over-generalizations

* use /usr/bin/env bash instead of absolute path

* remove debug tcpwrap

* add integration tests for postgres

* hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib

* cleanup should actually stop the process though

* comments / rearrange

* Bump up timeout in postgres tests; only pass user if explicitly requested to do so

* add schema stubs to new.sh

* Integration test fixes -- use /usr/bin/env bash; log all validation failures

* add postgres schemas

* fill out zcrypto.client_hello schema

* handle early get of TLSLog

* postgres: return SCAN_SUCCESS on success

* cleanup

* fix new.sh

* fix typo

* postgres container cleanup

* build.sh docs

* standardize container/image names

* add not to check for success

* shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts

* catch schematically-valid but non-successful scans

* postgres: clean up output format; more scanning

* cleanup; better error handling; get detailed protocol version error

* refactor modules

* clean up dangling connections

* split gigantic postgres.go

* remove unused

* ServerParams gets its own type

* refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more

* rename entrypoint; remove duplicate postgres tests

* comments for postgres schema

* Use param expansion to check for env variable [minor]

This is a *very* minor change to `docker-runner/docker-run.sh` checks to
see if the environment variable required to run the script has been set
to a non-empty string. If not, the script exits with a non-zero status
code and displays a default message:

```
❯ docker-runner/docker-run.sh
docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set
```

This was the behavior before, but just uses a one-liner declarative bash
idiom.

For further reading on parameter expansion, see
https://stackoverflow.com/a/307735.

@justinbastress can tell me if I did something wrong and broke the
intent of the script :-)

* Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter

* use make in docker-runner entrypoint

* add .integration_test_setup to .gitignore

* more .gitignore items

* Makefile updates: Windows support; add docker-runner target; better cleanup.

* docker-runner Dockerfile: start from zgrab2_runner_base image

* cleanup postgres setup

* make travis use make

* add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least

* fix folder name in Makefile

* update go (one of our dependencies now works only with >= 1.9)

* From travis: `I don't have any idea what to do with '1.9.0'.`

* explicit clean make

* fix dep order

* fix build.sh location

* popd

* use make to ensure zgrab2_runner exists

* Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test

* use explicit tag name for zgrab2_runner

* Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images

* use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?)

* remove overzealous cleanup

* let setup continue even if some containers are already running

* zgrab depends on *.go

* docker-runner depends on zgrab2 binary

* clean output before running integration tests
2018-01-15 14:24:57 -05:00

72 lines
2.3 KiB
Python

# zschema sub-schema for zgrab2's postgres module
# Registers zgrab2-postgres globally, and postgres with the main zgrab2 schema.
from zschema.leaves import *
from zschema.compounds import *
import zschema.registry
import schemas.zcrypto as zcrypto
import schemas.zgrab2 as zgrab2
# modules/postgres/scanner.go - decodeError() (TODO: Currently an unconstrained
# map[string]string; it is possible to get "unknown (0x%x)" fields, but it
# would probably be proper to reject those at this point)
# These are defined in detail at
# https://www.postgresql.org/docs/10/static/protocol-error-fields.html
postgres_error = SubRecord({
"severity": String(required=True),
"severity_v": String(),
"code": String(required=True),
"message": String(),
"detail": String(),
"hint": String(),
"position": String(),
"internal_position": String(),
"internal_query": String(),
"where": String(),
"schema": String(),
"table": String(),
"data": String(),
"file": String(),
"line": String(),
"routine": String(),
})
# modules/postgres/scanner.go - decodeAuthMode()
AUTH_MODES = [
"kerberos_v5", "password_cleartext", "password_md5", "scm_credentials",
"gss", "sspi", "sasl", "ok", "gss-continue", "sasl-continue", "sasl-final"
]
# modules/postgres/scanner.go: AuthenticationMode
postgres_auth_mode = SubRecord({
"mode": Enum(values=AUTH_MODES, required=True),
"Payload": Binary(),
})
# modules/postgres/scanner.go: BackendKeyData
postgres_key_data = SubRecord({
"process_id": Unsigned32BitInteger(),
"secret_key": Unsigned32BitInteger(),
})
# modules/postgres/scanner.go: PostgresResults
postgres_scan_response = SubRecord({
"result": SubRecord({
"tls": zgrab2.tls_log,
"supported_versions": String(),
"protocol_error": postgres_error,
"startup_error": postgres_error,
"is_ssl": Boolean(required=True),
"authentication_mode": postgres_auth_mode,
# TODO FIXME: This is currendly an unconstrained map[string]string
"server_parameters": String(),
"backend_key_data": postgres_key_data,
"transaction_status": String(),
})
}, extends=zgrab2.base_scan_response)
zschema.registry.register_schema("zgrab2-postgres", postgres_scan_response)
zgrab2.register_scan_response_type("postgres", postgres_scan_response)