Go to file
dependabot[bot] 121e56cbea
Bump git.tcp.direct/kayos/common from 0.7.6 to 0.7.7
Bumps git.tcp.direct/kayos/common from 0.7.6 to 0.7.7.

---
updated-dependencies:
- dependency-name: git.tcp.direct/kayos/common
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-12-19 15:10:01 +00:00
.github Update go.yml 2022-12-06 21:53:44 -08:00
example Fix: SOCKS5 upstream server 2022-10-19 06:15:16 -07:00
internal Feat: configuration for autoscaler + tests 2022-12-04 17:39:19 -08:00
logger SOCKS5 upstream server refactor 2022-10-19 05:36:31 -07:00
.gitignore Chore: tidy up 2022-07-25 00:14:26 -07:00
conductor.go Chore: rename 2022-10-16 03:53:04 -07:00
daemons.go Feat: Implement AutoScale 2022-10-16 05:56:40 -07:00
debug.go small debugging tweak, add email to license 2022-11-22 21:26:11 -08:00
defs.go SOCKS5 upstream server refactor 2022-10-19 05:36:31 -07:00
dispense.go Fix: combat conditional high cpu usage 2022-11-26 06:53:27 -08:00
getters.go Feat: Implement AutoScale 2022-10-16 05:56:40 -07:00
go.mod Bump git.tcp.direct/kayos/common from 0.7.6 to 0.7.7 2022-12-19 15:10:01 +00:00
go.sum Bump git.tcp.direct/kayos/common from 0.7.6 to 0.7.7 2022-12-19 15:10:01 +00:00
LICENSE small debugging tweak, add email to license 2022-11-22 21:26:11 -08:00
list_management.go Chore: rename 2022-10-16 03:53:04 -07:00
mr_worldwide.go Fix odd timeout issues 2022-11-30 11:30:59 -08:00
mystery_dialer.go SOCKS5 upstream server refactor 2022-10-19 05:36:31 -07:00
mystery_resolver.go Chore: rename 2022-10-16 03:53:04 -07:00
parse_test.go Improve parsing 2022-10-11 17:15:57 -07:00
parse.go Use pool implementation from common package 2022-10-16 00:28:26 -07:00
proto.go Use pool implementation from common package 2022-10-16 00:33:23 -07:00
proxy.go Feat: configuration for autoscaler + tests 2022-12-04 17:39:19 -08:00
README.md Update: README.md 2022-12-13 15:23:11 -08:00
setters.go Feat: configuration for autoscaler + tests 2022-12-04 17:39:19 -08:00
socks5_server.go Fix: SOCKS5 upstream server 2022-10-19 06:15:16 -07:00
stats.go Chore: rename 2022-10-16 03:53:04 -07:00
validator_engine.go Chore: rename 2022-10-16 03:53:04 -07:00

Prox5

SOCKS5/4/4a validating proxy pool + SOCKS5 server

Animated Screenshot of Prox5 Example

GoDoc Go Report Card IRC Test Status five

Prox5 is a golang library for managing, validating, and utilizing a very large amount of arbitrary SOCKS proxies.

Notably it features interface compatible dialer functions that dial out from different proxies for every connection, and a SOCKS5 server that utilizes those functions.


WARNING

You very well may end up causing denial of service when using this library with certain tools.

It is fairly easy to end up with a slowloris type effect with this library when paired with efficient tools that try to reuse http connections or that tend to use keepalive. Because the tool has no idea what the proxy server is doing (dialing with different connections often) you may end up leaving a ton of open connections from a website to the proxy servers. This is either a bug or a feature... That much is for you to decide.


Table of Contents

  1. Overview
    1. Validation Engine
    2. Auto Scaler
    3. Rate Limiting
    4. Accessing Validated Proxies
  2. Additional info
    1. The Secret Sauce
    2. External Integrations
  3. Status and Final Thoughts

Overview

Validation Engine

  1. TCP Dial to the endpoint
  2. HTTPS GET request to a list of IP echo endpoints
  3. Store the IP address discovered during step 2
  4. Instantiate a pointer to a prox5.Proxy type
  5. Enqueue the pointer for future use

Auto Scaler

The validation has an optional auto scale feature that allows for the automatic tuning of validation worker count as more proxies are dispensed. This feature is still new, but seems to work well. It can be enabled with [...].EnableAutoScaler().

Please refer to the autoscale related items within the documentation for more info.

Rate Limiting

Using Rate5, prox5 naturally reduces the frequency of proxies that fail to validate. It does this by reducing the frequency proxies are accepted into the validation pipeline the more they fail to verify or fail to successfully connect to an endpoint. This is not yet adjustable, but will be soon. See the documentation for Rate5, and the source code for this project (defs.go is a good place to start) for more info.

Accessing Validated Proxies

  • Retrieve validated 4/4a/5 proxies as simple strings for generic use
  • Use one of the dialer functions with any golang code that calls for a net.Dialer
  • Spin up a SOCKS5 server that will then make rotating use of your validated proxies

Additional info

The Secret Sauce

What makes Prox5 special is largely the Mystery Dialer. This dialer satisfies the net.Dialer interface. Upon using the dialer to connect to and endpoint, Prox5:

  • Loads up a previously verified proxy
  • Attempts to make connection with the dial endpoint using said proxy
  • Upon failure, prox5:
    • repeats this process mid-dial
    • does not drop connection to the client
  • Once a proxy has been successfully used to connect to the target endpoint, prox5 passes the same net.Conn onto the client

External Integrations

Mullvad

Take a look at mullsox for an easy way to access all of the mullvad proxies reachable from any one VPN endpoint. It is trivial to feed the results of GetAndVerifySOCKS into prox5.

Here's a snippet that should just about get you there:

p5 := NewProxyEngine()
mc := mullsox.NewChecker()

if err := mc.Update(); err != nil {
	println(err.Error())
        return
}

incoming, _ := mc.GetAndVerifySOCKS()

var count = 0
for line := range incoming {
        if p5.LoadSingleProxy(line.String()) {
                count++
        }
}

if count == 0 {
        println("failed to load any proxies")
        return
}

if err := p5.Start(); err != nil {
        println(err.Error())
        return
}
ProxyBonanza

Take a look at ProxyGonanza

(TODO: code example here)


Status and Final Thoughts

This project is in development.

It "works" and has been used in "production", but still needs some love.

Please break it and let me know what broke.

The way you choose to use this lib is yours. The API is fairly extensive for you to be able to customize runtime configuration without having to do any surgery.

Things like the amount of validation workers that are concurrently operating, timeouts, and proxy re-use policies may be tuned in real-time.


Please see the docs and the example for more details.