Commit Graph

71 Commits

Author SHA1 Message Date
Elliot Cubit
a1d5e930ca
smb: allow dynamic length fields of exact length (#323) 2021-07-26 15:57:24 -04:00
David Adrian
4e047842a6 stm: go get -u, fix compile errors 2021-04-06 16:49:27 -06:00
Elliot Cubit
17a5257565
Add NativeOS, NTLM, and GroupName to SMBv1 results (#286)
* add smbv1 session setup scan

* remove unused values

* rename os_name to native_os to match smb documentation

* remove superfluous comment

* update zschema to include new SMB fields

* improve clarity on bounds checking for SMBv1 requests
2021-03-15 09:23:55 -04:00
Elliot Cubit
3c17bf32e8
Fix SMB scanner marshaller panic (#299) 2021-02-11 14:44:36 -05:00
Tim Schumacher
fc82b6b673 Capture BodyTextLength 2020-09-17 15:01:27 -04:00
Tim Schumacher
30e6cdb73f Add BodyHash field with hash type prefix 2020-09-17 11:04:27 -04:00
Benjamin Wireman
d2e503881f
Allow for programaitc access of ssh result types (#260)
https://github.com/zmap/zgrab2/pull/260
2020-03-31 15:56:44 -04:00
thegwan
31645233a5 reduce code duplication 2019-10-23 17:45:40 +00:00
thegwan
422cdc7b9f prevent running key exchange if hello-only flag set 2019-10-21 08:31:57 +00:00
Zakir Durumeric
28cde1a5a6
Add SSH Client Hello Only flag (#215) 2019-08-15 17:01:24 -07:00
Jeff Cody
b69c22c532
SMB: Probe for SMB1 in addition to SMB2
If the probe for SMB2 fails, close the connection and then try probing
for SMB1 as a backup.

Since there are more SMB2 servers in the wild, that is the first
attempt.
2019-06-10 17:17:20 -04:00
Jeff Cody
1232ca4e60
SMB: Add Negotiation Req & Response for v1
Send SMB1 header, and Negotiation Request message for SMB1.

This brings the zgrab2 smb1 scanner to parity with the zgrab smb1
scanner, with presence detection via smbv1_support.

We check the ProtocolID in the raw data response, for two reasons:

1. Even if the full unmarshal fails for the message, we will log
   that it is an smbv1 server

2. We need to add more response types structs, because the format
   is different for various SMB1 dialects.

The negotiation response v1 structure is for the SMB1 "NT LM 0.12"
dialect, and is essentially placeholder for now for future parsing.

TODO: Unmarshal into the appropriate message struct based on
SMB1 dialect, and parse dialect and capabilities, and return those
results.
2019-06-10 17:17:20 -04:00
Jeff Cody
f2b76412fb
SMB: Combine 'GetSMBLog()' and 'GetSMBBanner()'
These two functions are largely duplicates, and only differ in the
boolean option passed to LoggedNegotiateProtocol().  Combine the
functions, and just take that option in as an argument to pass along.
2019-06-07 16:54:26 -04:00
Jeff Cody
c0f5084a6a
Merge pull request #205 from codyprime/jcody/lib-smbfix
SMB: Bounds check slice index before access
2019-06-05 14:02:20 -04:00
Jeff Cody
2e67991362
SMB: Bounds check slice index before access
The smb library bounds checks for a message size that is too large, but
does not check for a message size that is way too small.  Error out if
the message size is not at least as large as the ProtocolID 4-byte
preamble.

This fixes slice out of bound panics when checking the buffer for the
protID string for certain hosts.

Signed-off-by: Jeff Cody <jcody@censys.io>
2019-06-04 22:26:27 -04:00
Jeff Cody
37bb923ce4
SMB: Parse capabilties flags for the server
This parses the capabilities flags for the server, masking off invalid
flags based on dialect.

While both the NegotiationLogs and SessionSetupLog contain capabilities
flags, we extract the flags from the NegotiationLogs to represent the
server, as it is explicitly stated in [MS-SMB2] that those are the ones
that represent the capabilities of the server.
2019-05-24 15:47:22 -04:00
Jeff Cody
79a96f08ae
SMB: Parse SMB Versions and Dialects.
This parses the SMB Version response, and the dialect, to determine the
full SMB version.  This is done in accordance to "[MS-SMB2] - v20190430"
from Microsoft, Section 2.2.4.
2019-05-24 15:47:10 -04:00
Jeff Cody
2e71f25c85
SMB: Omit Negotiation adn Session logs if empty
Currently, we will output 'null' json fields for either of
these logs if they are not present; instead, let's just omit
them entirely.
2019-05-24 15:33:25 -04:00
Jeff Cody
da90d489b1
SMB: Formatting, line-wrap comments 2019-05-23 13:45:18 -04:00
justinbastress
9f19df743e
Merge pull request #176 from sdnewhop/master
SSH: Add banner grabbing
2019-01-23 15:25:29 -05:00
Justin Bastress
50b77f285a don't trigger constant index-out-of-bounds exceptions 2018-10-24 12:58:11 -04:00
Justin Bastress
28626f0b94 address @parkanzky's comments in #178 2018-10-24 10:29:07 -04:00
Justin Bastress
7294e41bce catch nil body 2018-10-22 13:23:07 -04:00
Oleg Broslavsky
d5aaeca283 Add SSH banner grabbing when userauth enabled
- Bump lib/ssh from golang.org/x/crypto/ssh (just banner hadling)
- Add a banner field to the SSH module info
2018-10-20 03:39:24 -04:00
Justin Bastress
06b72315df add initial work 2018-10-17 16:07:12 -04:00
Justin Bastress
c51faaf8f5 use functions introduced in https://github.com/zmap/zcrypto/pull/178 in place of functions removed in https://github.com/zmap/zcrypto/pull/173/files 2018-09-05 16:26:05 -04:00
justinbastress
713150b089 merge client/server header list (#134)
* merge client/server header list

* fix formatting of header.go
2018-06-28 14:46:19 -04:00
Justin Bastress
320f6f4a8a in SSH kex ECDH JSONLog, don't create a field unless it's going to be populated (to allow omitempty to work properly) 2018-05-31 16:29:59 -04:00
Justin Bastress
05841e9ab4 raw_packets is debug; server_version is omitempty 2018-05-22 16:05:56 -04:00
Justin Bastress
434ccaa3f4 make all debug fields omitempty; make capability_flags/status_flags omitempty, in the aux type as well. 2018-05-22 16:03:00 -04:00
Justin Bastress
486dc517a3 fix required/non-required in schemas; add error_ids, and code to get them; throw ERRPacket as APPLICATION_ERROR 2018-05-21 12:15:17 -04:00
Justin Bastress
74c8b16e7a httplex -> httpguts 2018-05-07 16:22:37 -04:00
Justin Bastress
5d6c0ef7a3 fix comment re PR review 2018-04-26 11:58:08 -04:00
Justin Bastress
7a013ca261 Add output unit tests; add output.Process to strip debug; make MySQL debug fields omitempty too; use processor to strip data in Process() 2018-04-03 17:15:20 -04:00
Justin Bastress
e4bd0bcc89 move process tests 2018-04-02 16:10:01 -04:00
Justin Bastress
e82d5b4cfd add process tests 2018-04-02 16:09:41 -04:00
Justin Bastress
53e35bbaa7 prevent multiple concurrent usages of the same processor; copy all fields, nil out debug ones; 2018-04-02 16:09:27 -04:00
Justin Bastress
58601e045a add output processing 2018-03-30 16:28:24 -04:00
Luke Valenta
b4ac16e630 add H field to xssh server signature to allow signature verification (#95) 2018-03-29 10:17:39 -04:00
Justin Bastress
199fe5d8e5 PEP8 updates; fix NTLM typo 2018-03-16 14:39:06 -04:00
Justin Bastress
1024fb4619 break out early if !setup-session; make debug configurable 2018-03-16 14:34:15 -04:00
Justin Bastress
586992cb15 More comments; make old zgrab behavior the default 2018-03-16 14:00:12 -04:00
Justin Bastress
65d1789860 comments; update schema 2018-03-16 13:21:05 -04:00
Justin Bastress
c05c00a743 Pull in updated SMB library, modify it to take logs, use it in scanner 2018-03-16 12:08:59 -04:00
Justin Bastress
70314ce92b Port SMB (???) scanner from ZGrab 2018-03-15 16:58:57 -04:00
justinbastress
075924400f
Add shared FlagsToSet function in output.go (#62)
* Add shared FlagsToSet function in output.go, use it in mysql. Add examples / tests.

* Add utility functions to widen map keys
2018-03-01 14:32:12 -05:00
justinbastress
bd952d9809
golint and godocs for mysql module (#56) 2018-02-12 10:42:45 -05:00
justinbastress
2766a7ae08
Port http from zgrab (#31) 2018-02-09 13:45:50 -05:00
Jordan Wright
162138dac3 Changed tools/keys dependencies to point to zgrab2. 2018-01-30 23:45:52 -06:00
justinbastress
f49887290d
Implements postgres zgrab2 module (#30)
* remove unnecessary indirection on net.Conn

* Ignore *.pyc

* fix NPE on nil handshake

* refactoring -- move status to status.go; add Open() methods for ScanTarget

* cherry-pick .gitignore fix

* pull in TLS fix

* status.go comments

* trim over-generalizations

* use /usr/bin/env bash instead of absolute path

* remove debug tcpwrap

* add integration tests for postgres

* hack for cleanup.sh to work on mingw -- use //var/lib instead of /var/lib

* cleanup should actually stop the process though

* comments / rearrange

* Bump up timeout in postgres tests; only pass user if explicitly requested to do so

* add schema stubs to new.sh

* Integration test fixes -- use /usr/bin/env bash; log all validation failures

* add postgres schemas

* fill out zcrypto.client_hello schema

* handle early get of TLSLog

* postgres: return SCAN_SUCCESS on success

* cleanup

* fix new.sh

* fix typo

* postgres container cleanup

* build.sh docs

* standardize container/image names

* add not to check for success

* shift mysql's connection management to ScanTarget.Open(); wrap Read/Write methods returned by ScanTarget.Open() to enforce timeouts

* catch schematically-valid but non-successful scans

* postgres: clean up output format; more scanning

* cleanup; better error handling; get detailed protocol version error

* refactor modules

* clean up dangling connections

* split gigantic postgres.go

* remove unused

* ServerParams gets its own type

* refactor integration tests: run zgrab2 in its own container, which is linked to the service containers, so that we don't need to keep track of unique ports on the host any more

* rename entrypoint; remove duplicate postgres tests

* comments for postgres schema

* Use param expansion to check for env variable [minor]

This is a *very* minor change to `docker-runner/docker-run.sh` checks to
see if the environment variable required to run the script has been set
to a non-empty string. If not, the script exits with a non-zero status
code and displays a default message:

```
❯ docker-runner/docker-run.sh
docker-runner/docker-run.sh: line 7: CONTAINER_NAME: parameter null or not set
```

This was the behavior before, but just uses a one-liner declarative bash
idiom.

For further reading on parameter expansion, see
https://stackoverflow.com/a/307735.

@justinbastress can tell me if I did something wrong and broke the
intent of the script :-)

* Add integration_test targets to makefile; use makefile instead of directly calling go build everywhere; run postgres schema through PEP8 linter

* use make in docker-runner entrypoint

* add .integration_test_setup to .gitignore

* more .gitignore items

* Makefile updates: Windows support; add docker-runner target; better cleanup.

* docker-runner Dockerfile: start from zgrab2_runner_base image

* cleanup postgres setup

* make travis use make

* add .gitattributes, try to prevent it from overriding lfs with crlfs in shell scripts at least

* fix folder name in Makefile

* update go (one of our dependencies now works only with >= 1.9)

* From travis: `I don't have any idea what to do with '1.9.0'.`

* explicit clean make

* fix dep order

* fix build.sh location

* popd

* use make to ensure zgrab2_runner exists

* Make docker-runner an order-dependency for integration-test-cleanup; don't do a cleanup after each integration test

* use explicit tag name for zgrab2_runner

* Add container-clean target to Makefile, to remove cyclic dependency on docker; use .id files to track docker images; add servce-base image; use Make to build / track images

* use LF in Makefiles; update .gitignore; use zgrab_service_base image in ssh container; fix line endings (?)

* remove overzealous cleanup

* let setup continue even if some containers are already running

* zgrab depends on *.go

* docker-runner depends on zgrab2 binary

* clean output before running integration tests
2018-01-15 14:24:57 -05:00