You cannot select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
|
1 year ago | |
---|---|---|
.gitignore | 1 year ago | |
README.md | 1 year ago | |
proxy_lists.txt | 1 year ago | |
pull_lists.py | 1 year ago | |
pulled_proxies.txt | 1 year ago | |
run.py | 1 year ago |
README.md
Dependencies
- UVLoop
- python-socks
- ssl
Most of these libraries do not support windows
Solely for checking if a proxy is alive, checks ~1200 proxies a minute with 100 workers, cpu load and memory being negligable
UVLoop isn't necessary, but does give far greater performance
Theoretically you could run thousands of workers, you would just need to increase the maximum file descriptors allowed
Drop the CHECK_TIMEOUT
if you would like to increase time, this would likely ensure that the less responsive proxies are verified
1000 Workers uses roughly 5% cpu on a dual core, completes checking 10000 proxies in roughly 4 minutes.