zgrab2/integration_tests/postgres/setup.sh

73 lines
1.9 KiB
Bash
Raw Normal View History

Implements postgres zgrab2 module (#30) * remove unnecessary indirection on net.Conn * Ignore *.pyc * fix NPE on nil handshake * refactoring -- move status to status.go; add Open() methods for ScanTarget * cherry-pick .gitignore fix * pull in TLS fix * status.go comments * trim over-generalizations * use /usr/bin/env bash instead of absolute path * remove debug tcpwrap * add integration tests for postgres * hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib * cleanup should actually stop the process though * comments / rearrange * Bump up timeout in postgres tests; only pass user if explicitly requested to do so * add schema stubs to new.sh * Integration test fixes -- use /usr/bin/env bash; log all validation failures * add postgres schemas * fill out zcrypto.client_hello schema * handle early get of TLSLog * postgres: return SCAN_SUCCESS on success * cleanup * fix new.sh * fix typo * postgres container cleanup * build.sh docs * standardize container/image names * add not to check for success * shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts * catch schematically-valid but non-successful scans * postgres: clean up output format; more scanning * cleanup; better error handling; get detailed protocol version error * refactor modules * clean up dangling connections * split gigantic postgres.go * remove unused * ServerParams gets its own type * refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more * rename entrypoint; remove duplicate postgres tests * comments for postgres schema * Use param expansion to check for env variable [minor] This is a *very* minor change to `docker-runner/docker-run.sh` checks to see if the environment variable required to run the script has been set to a non-empty string. If not, the script exits with a non-zero status code and displays a default message: ``` ❯ docker-runner/docker-run.sh docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set ``` This was the behavior before, but just uses a one-liner declarative bash idiom. For further reading on parameter expansion, see https://stackoverflow.com/a/307735. @justinbastress can tell me if I did something wrong and broke the intent of the script :-) * Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter * use make in docker-runner entrypoint * add .integration_test_setup to .gitignore * more .gitignore items * Makefile updates: Windows support; add docker-runner target; better cleanup. * docker-runner Dockerfile: start from zgrab2_runner_base image * cleanup postgres setup * make travis use make * add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least * fix folder name in Makefile * update go (one of our dependencies now works only with >= 1.9) * From travis: `I don't have any idea what to do with '1.9.0'.` * explicit clean make * fix dep order * fix build.sh location * popd * use make to ensure zgrab2_runner exists * Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test * use explicit tag name for zgrab2_runner * Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images * use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?) * remove overzealous cleanup * let setup continue even if some containers are already running * zgrab depends on *.go * docker-runner depends on zgrab2 binary * clean output before running integration tests
2018-01-15 19:24:57 +00:00
#!/usr/bin/env bash
set -e
versions="9.3 9.4 9.5 9.6 10.1"
types="ssl nossl"
function doSetup() {
VERSION=$1
TYPE=$2
CONTAINER_NAME="zgrab_postgres_${VERSION}-${TYPE}"
IMAGE_TAG="zgrab_postgres:${VERSION}-${TYPE}"
if docker ps --filter "name=$CONTAINER_NAME" | grep -q $CONTAINER_NAME; then
echo "postgres/setup: Container $CONTAINER_NAME already running -- nothing to do."
return
fi
if grep -q "$IMAGE_TAG" < images.tmp > /dev/null && [ -x $REBUILD ]; then
Implements postgres zgrab2 module (#30) * remove unnecessary indirection on net.Conn * Ignore *.pyc * fix NPE on nil handshake * refactoring -- move status to status.go; add Open() methods for ScanTarget * cherry-pick .gitignore fix * pull in TLS fix * status.go comments * trim over-generalizations * use /usr/bin/env bash instead of absolute path * remove debug tcpwrap * add integration tests for postgres * hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib * cleanup should actually stop the process though * comments / rearrange * Bump up timeout in postgres tests; only pass user if explicitly requested to do so * add schema stubs to new.sh * Integration test fixes -- use /usr/bin/env bash; log all validation failures * add postgres schemas * fill out zcrypto.client_hello schema * handle early get of TLSLog * postgres: return SCAN_SUCCESS on success * cleanup * fix new.sh * fix typo * postgres container cleanup * build.sh docs * standardize container/image names * add not to check for success * shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts * catch schematically-valid but non-successful scans * postgres: clean up output format; more scanning * cleanup; better error handling; get detailed protocol version error * refactor modules * clean up dangling connections * split gigantic postgres.go * remove unused * ServerParams gets its own type * refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more * rename entrypoint; remove duplicate postgres tests * comments for postgres schema * Use param expansion to check for env variable [minor] This is a *very* minor change to `docker-runner/docker-run.sh` checks to see if the environment variable required to run the script has been set to a non-empty string. If not, the script exits with a non-zero status code and displays a default message: ``` ❯ docker-runner/docker-run.sh docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set ``` This was the behavior before, but just uses a one-liner declarative bash idiom. For further reading on parameter expansion, see https://stackoverflow.com/a/307735. @justinbastress can tell me if I did something wrong and broke the intent of the script :-) * Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter * use make in docker-runner entrypoint * add .integration_test_setup to .gitignore * more .gitignore items * Makefile updates: Windows support; add docker-runner target; better cleanup. * docker-runner Dockerfile: start from zgrab2_runner_base image * cleanup postgres setup * make travis use make * add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least * fix folder name in Makefile * update go (one of our dependencies now works only with >= 1.9) * From travis: `I don't have any idea what to do with '1.9.0'.` * explicit clean make * fix dep order * fix build.sh location * popd * use make to ensure zgrab2_runner exists * Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test * use explicit tag name for zgrab2_runner * Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images * use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?) * remove overzealous cleanup * let setup continue even if some containers are already running * zgrab depends on *.go * docker-runner depends on zgrab2 binary * clean output before running integration tests
2018-01-15 19:24:57 +00:00
echo "postgres/setup: docker image $IMAGE_TAG already exists -- skipping."
else
echo "postgres/setup: docker image $IMAGE_TAG does not exist -- creating..."
./build.sh $TYPE $VERSION
fi
echo "postgres/setup: Starting container $CONTAINER_NAME..."
docker run -td --rm --name $CONTAINER_NAME -e POSTGRES_PASSWORD=password $IMAGE_TAG
Implements postgres zgrab2 module (#30) * remove unnecessary indirection on net.Conn * Ignore *.pyc * fix NPE on nil handshake * refactoring -- move status to status.go; add Open() methods for ScanTarget * cherry-pick .gitignore fix * pull in TLS fix * status.go comments * trim over-generalizations * use /usr/bin/env bash instead of absolute path * remove debug tcpwrap * add integration tests for postgres * hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib * cleanup should actually stop the process though * comments / rearrange * Bump up timeout in postgres tests; only pass user if explicitly requested to do so * add schema stubs to new.sh * Integration test fixes -- use /usr/bin/env bash; log all validation failures * add postgres schemas * fill out zcrypto.client_hello schema * handle early get of TLSLog * postgres: return SCAN_SUCCESS on success * cleanup * fix new.sh * fix typo * postgres container cleanup * build.sh docs * standardize container/image names * add not to check for success * shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts * catch schematically-valid but non-successful scans * postgres: clean up output format; more scanning * cleanup; better error handling; get detailed protocol version error * refactor modules * clean up dangling connections * split gigantic postgres.go * remove unused * ServerParams gets its own type * refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more * rename entrypoint; remove duplicate postgres tests * comments for postgres schema * Use param expansion to check for env variable [minor] This is a *very* minor change to `docker-runner/docker-run.sh` checks to see if the environment variable required to run the script has been set to a non-empty string. If not, the script exits with a non-zero status code and displays a default message: ``` ❯ docker-runner/docker-run.sh docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set ``` This was the behavior before, but just uses a one-liner declarative bash idiom. For further reading on parameter expansion, see https://stackoverflow.com/a/307735. @justinbastress can tell me if I did something wrong and broke the intent of the script :-) * Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter * use make in docker-runner entrypoint * add .integration_test_setup to .gitignore * more .gitignore items * Makefile updates: Windows support; add docker-runner target; better cleanup. * docker-runner Dockerfile: start from zgrab2_runner_base image * cleanup postgres setup * make travis use make * add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least * fix folder name in Makefile * update go (one of our dependencies now works only with >= 1.9) * From travis: `I don't have any idea what to do with '1.9.0'.` * explicit clean make * fix dep order * fix build.sh location * popd * use make to ensure zgrab2_runner exists * Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test * use explicit tag name for zgrab2_runner * Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images * use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?) * remove overzealous cleanup * let setup continue even if some containers are already running * zgrab depends on *.go * docker-runner depends on zgrab2 binary * clean output before running integration tests
2018-01-15 19:24:57 +00:00
echo "...started."
}
function waitFor() {
VERSION=$1
TYPE=$2
PORT=$3
CONTAINER_NAME="zgrab_postgres_${VERSION}-${TYPE}"
echo "postgres/setup: Waiting for postgres process to come up on $CONTAINER_NAME..."
if [ "10.1" == "$VERSION" ]; then
while ! (docker logs --tail all $CONTAINER_NAME | grep -q " listening on IPv4 address"); do
echo -n "."
sleep 1
done
else
while ! (docker exec $CONTAINER_NAME ps -Af | grep -q "postgres: logger process" > /dev/null); do
echo -n "*"
sleep 1
done
while ! (docker exec -t $CONTAINER_NAME cat //var/lib/postgresql/data/pg_log/postgres.log | grep -q "STARTED; state"); do
echo -n "."
sleep 1
done
fi
sleep 1
Implements postgres zgrab2 module (#30) * remove unnecessary indirection on net.Conn * Ignore *.pyc * fix NPE on nil handshake * refactoring -- move status to status.go; add Open() methods for ScanTarget * cherry-pick .gitignore fix * pull in TLS fix * status.go comments * trim over-generalizations * use /usr/bin/env bash instead of absolute path * remove debug tcpwrap * add integration tests for postgres * hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib * cleanup should actually stop the process though * comments / rearrange * Bump up timeout in postgres tests; only pass user if explicitly requested to do so * add schema stubs to new.sh * Integration test fixes -- use /usr/bin/env bash; log all validation failures * add postgres schemas * fill out zcrypto.client_hello schema * handle early get of TLSLog * postgres: return SCAN_SUCCESS on success * cleanup * fix new.sh * fix typo * postgres container cleanup * build.sh docs * standardize container/image names * add not to check for success * shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts * catch schematically-valid but non-successful scans * postgres: clean up output format; more scanning * cleanup; better error handling; get detailed protocol version error * refactor modules * clean up dangling connections * split gigantic postgres.go * remove unused * ServerParams gets its own type * refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more * rename entrypoint; remove duplicate postgres tests * comments for postgres schema * Use param expansion to check for env variable [minor] This is a *very* minor change to `docker-runner/docker-run.sh` checks to see if the environment variable required to run the script has been set to a non-empty string. If not, the script exits with a non-zero status code and displays a default message: ``` ❯ docker-runner/docker-run.sh docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set ``` This was the behavior before, but just uses a one-liner declarative bash idiom. For further reading on parameter expansion, see https://stackoverflow.com/a/307735. @justinbastress can tell me if I did something wrong and broke the intent of the script :-) * Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter * use make in docker-runner entrypoint * add .integration_test_setup to .gitignore * more .gitignore items * Makefile updates: Windows support; add docker-runner target; better cleanup. * docker-runner Dockerfile: start from zgrab2_runner_base image * cleanup postgres setup * make travis use make * add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least * fix folder name in Makefile * update go (one of our dependencies now works only with >= 1.9) * From travis: `I don't have any idea what to do with '1.9.0'.` * explicit clean make * fix dep order * fix build.sh location * popd * use make to ensure zgrab2_runner exists * Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test * use explicit tag name for zgrab2_runner * Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images * use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?) * remove overzealous cleanup * let setup continue even if some containers are already running * zgrab depends on *.go * docker-runner depends on zgrab2 binary * clean output before running integration tests
2018-01-15 19:24:57 +00:00
echo "...postgres is up."
}
pushd container
docker images --format {{.Repository}}:{{.Tag}} > images.tmp
for version in $versions; do
for type in $types; do
doSetup $version $type
done
done
rm -f images.tmp
popd
echo "postgres/setup: Waiting for all postgres containers to start up..."
for version in $versions; do
for type in $types; do
waitFor $version $type
done
done
echo "postgres/setup: Containers started."