You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
kayos@tcp.direct 25c45c57d5
Fix: missing proxies
4 days ago
.github CI: fix go version 4 days ago
example Complete teardown (broken atm) 4 days ago
internal Scaling adjustments 1 month ago
logger Chores: docs+misc 1 month ago
.gitignore Chore: tidy up 6 months ago
LICENSE small debugging tweak, add email to license 2 months ago
README.md Fix: typo 1 month ago
conductor.go Complete teardown (broken atm) 4 days ago
daemons.go Complete teardown (broken atm) 4 days ago
debug.go Optimize: use concurrent map 1 month ago
defs.go Fix: stack overflow 4 days ago
dispense.go Fix: missing proxies 4 days ago
getters.go Complete teardown (broken atm) 4 days ago
go.mod Complete teardown (broken atm) 4 days ago
go.sum Complete teardown (broken atm) 4 days ago
list_management.go Complete teardown (broken atm) 4 days ago
mr_worldwide.go Fix: tls verify bug 4 days ago
mystery_dialer.go Fix: missing proxies 4 days ago
mystery_resolver.go Chore: rename 4 months ago
parse.go Use pool implementation from common package 4 months ago
parse_test.go Improve parsing 4 months ago
proto.go Use pool implementation from common package 4 months ago
proto_test.go Testing: add immutable proto test 4 weeks ago
proxy.go Scaling adjustments 1 month ago
scale_util.go Complete teardown (broken atm) 4 days ago
setters.go Complete teardown (broken atm) 4 days ago
socks5_server.go Complete teardown (broken atm) 4 days ago
stats.go Complete teardown (broken atm) 4 days ago
validator_engine.go Complete teardown (broken atm) 4 days ago

README.md

Prox5

SOCKS5/4/4a validating proxy pool + SOCKS5 server

Animated Screenshot of Prox5 Example

GoDoc Go Report Card IRC Test Status five

import git.tcp.direct/kayos/prox5

Prox5 is a golang library for managing, validating, and utilizing a very large amount of arbitrary SOCKS proxies.

Notably it features interface compatible dialer functions that dial out from different proxies for every connection, and a SOCKS5 server that utilizes those functions.


WARNING

You very well may end up causing denial of service when using this library with certain tools.

It is fairly easy to end up with a slowloris type effect with this library when paired with efficient tools that try to reuse http connections or that tend to use keepalive. Because the tool has no idea what the proxy server is doing (dialing with different connections often) you may end up leaving a ton of open connections from a website to the proxy servers. This is either a bug or a feature... That much is for you to decide.


Table of Contents

  1. Overview
    1. Validation Engine
    2. Auto Scaler
    3. Rate Limiting
    4. Accessing Validated Proxies
  2. Additional info
    1. The Secret Sauce
    2. External Integrations
  3. Status and Final Thoughts

Overview

Validation Engine

  1. TCP Dial to the endpoint, if successful, reuse net.Conn down for step 2
  2. HTTPS GET request to a list of IP echo endpoints
  3. Store the IP address discovered during step 2
  4. Allocate a new prox5.Proxy type || update an existing one, store latest info
  5. Enqueue a pointer to this proxy.Proxy instance, instantiating it for further use

Auto Scaler

The validation has an optional auto scale feature that allows for the automatic tuning of validation worker count as more proxies are dispensed. This feature is still new, but seems to work well. It can be enabled with [...].EnableAutoScaler().

Please refer to the autoscale related items within the documentation for more info.

Rate Limiting

Using Rate5, prox5 naturally reduces the frequency of proxies that fail to validate. It does this by reducing the frequency proxies are accepted into the validation pipeline the more they fail to verify or fail to successfully connect to an endpoint. This is not yet adjustable, but will be soon. See the documentation for Rate5, and the source code for this project (defs.go is a good place to start) for more info.

Accessing Validated Proxies

  • Retrieve validated 4/4a/5 proxies as simple strings for generic use
  • Use one of the dialer functions with any golang code that calls for a net.Dialer
  • Spin up a SOCKS5 server that will then make rotating use of your validated proxies

Additional info

The Secret Sauce

What makes Prox5 special is largely the Mystery Dialer. This dialer satisfies the net.Dialer and ContextDialer interfaces. The implementation is a little bit different from your average dialer. Here's roughly what happens when you dial out with a ProxyEngine;

  • Loads up a previously verified proxy
  • Attempts to make connection with the dial endpoint using said proxy
  • Upon failure, prox5:
    • repeats this process mid-dial
    • does not drop connection to the client
  • Once a proxy has been successfully used to connect to the target endpoint, prox5 passes the same net.Conn onto the client

External Integrations

Mullvad

Take a look at mullsox for an easy way to access all of the mullvad proxies reachable from any one VPN endpoint. It is trivial to feed the results of GetAndVerifySOCKS into prox5.

Here's a snippet that should just about get you there:

package main

import (
    "os"
    "time"

    "git.tcp.direct/kayos/mullsox"
    "git.tcp.direct/kayos/prox5"
)

func main() {
	p5 := prox5.NewProxyEngine()
	mc := mullsox.NewChecker()

	if err := mc.Update(); err != nil {
		println(err.Error())
		return
	}

	incoming, _ := mc.GetAndVerifySOCKS()

	var count = 0
	for line := range incoming {
		if p5.LoadSingleProxy(line.String()) {
			count++
		}
	}

	if count == 0 {
		println("failed to load any proxies")
		return
	}

	if err := p5.Start(); err != nil {
		println(err.Error())
		return
	}
	
	go func() {
		if err := p5.StartSOCKS5Server("127.0.0.1:42069", "", ""); err != nil {
			println(err.Error())
			os.Exit(1)
		}
	}()
	
	time.Sleep(time.Millisecond * 500)
	
	println("proxies loaded and socks server started")
}
ProxyBonanza

Take a look at ProxyGonanza

(TODO: code example here)


Status and Final Thoughts

This project is in development.

It "works" and has been used in "production", but still needs some love.

Please break it and let me know what broke.

The way you choose to use this lib is yours. The API is fairly extensive for you to be able to customize runtime configuration without having to do any surgery.

Things like the amount of validation workers that are concurrently operating, timeouts, and proxy re-use policies may be tuned in real-time.


Please see the docs and the example for more details.