Commit Graph

174 Commits

Author SHA1 Message Date
Costa Tsaousis (ktsaou)
a88a9b7b27 can now load the ipset optimized (10-20% faster); files are closed after being read (was leaving them open) 2015-08-09 15:31:19 +03:00
Costa Tsaousis (ktsaou)
ca826e3bdb fixed the update frequencies 2015-08-08 15:23:31 +03:00
Costa Tsaousis (ktsaou)
f297a10e82 added asprox_c2 2015-08-05 21:02:09 +03:00
Costa Tsaousis (ktsaou)
811afea3a6 added proxyspy, bds_atif 2015-08-05 20:05:41 +03:00
Costa Tsaousis (ktsaou)
eda86d80c2 added sblam, myip, haleys_ssh, nt_malware_dns 2015-08-04 23:16:01 +03:00
Costa Tsaousis (ktsaou)
0f2aa42aca comments update for current status; json arrays are printed one value per line, removed an obsolete cp that was complaining 2015-08-04 14:35:28 +03:00
Costa Tsaousis (ktsaou)
4b9c8eed6b limited history chart to 1000 entries and moved full history to CACHE_DIR 2015-08-02 19:33:40 +03:00
Costa Tsaousis (ktsaou)
62ffffe43f removed time from sitemap.xml lastmod 2015-08-02 16:25:05 +03:00
Costa Tsaousis (ktsaou)
c7b9c1a4d6 added category to each ipset; added sitemap.xml 2015-08-02 16:21:05 +03:00
Costa Tsaousis (ktsaou)
01ad5f7bc6 fixed comments for certain lists 2015-08-02 13:09:14 +03:00
Costa Tsaousis (ktsaou)
a1a0f9a2b5 added "any" gategory to badips.com and new lists: botscout, greensnow and all bambenek lists 2015-07-31 00:16:14 +03:00
Costa Tsaousis (ktsaou)
912d2fcc48 stopped re-distribution of bluetack lists 2015-07-26 15:13:54 +03:00
Costa Tsaousis (ktsaou)
4f9377799c when history was requested the base set was not updated 2015-07-19 03:12:28 +03:00
Costa Tsaousis (ktsaou)
ec070910f1 rewrote history part 2015-07-18 22:40:19 +03:00
Costa Tsaousis (ktsaou)
7ac1f15020 moved retention status dir to /var/lib; fixed complete retention detection 2015-07-12 23:33:49 +03:00
Costa Tsaousis (ktsaou)
da25e1be9a added dshield_1d 2015-07-05 02:48:31 +03:00
Costa Tsaousis (ktsaou)
b741774c87 retention now counts only IPs that are added and removed after we started monitoring the ipset 2015-06-19 21:54:43 +03:00
Costa Tsaousis (ktsaou)
1062ca07d4 added incomplete status and start timestamp to retention histogram 2015-06-19 10:46:27 +03:00
Costa Tsaousis (ktsaou)
79ce864c0f was not calculating age of existing IPs if no new IPs were added to the ipset 2015-06-19 10:19:59 +03:00
Costa Tsaousis (ktsaou)
1d421145f7 generate script to git to restore the timestamps 2015-06-19 01:32:08 +03:00
Costa Tsaousis (ktsaou)
d1c0be6eba added option --enable-all 2015-06-19 01:01:12 +03:00
Costa Tsaousis (ktsaou)
c3e12f9902 support for tracking age per IP per blocklist; tracking retention policy; added support for the site json and csv files; allowed running as non-root (without loading the files into kernel) 2015-06-19 00:13:00 +03:00
Costa Tsaousis (ktsaou)
0f9ead1542 many improvements mainly for the self maintainance of the github repo; now it publishes json and csv files for the github pages; a lot faster and now it compares all the files with both geo country databases 2015-06-15 02:29:02 +03:00
Costa Tsaousis (ktsaou)
cd225be1ac keyword "as" was not working for the all the *-next files 2015-06-15 02:27:37 +03:00
Costa Tsaousis (ktsaou)
c70466eec5 added wormrbl.imp.ch 2015-06-12 20:15:22 +03:00
Costa Tsaousis (ktsaou)
f83a24e3bd added more dragon sets 2015-06-12 20:14:45 +03:00
Costa Tsaousis (ktsaou)
9b43ef43b6 fixed help and added option aliases 2015-06-12 20:14:01 +03:00
Costa Tsaousis (ktsaou)
9727741d55 added --exclude-next option that merges all the files before the parameter excluding all IP in the files after the parameter 2015-06-11 23:16:26 +03:00
Costa Tsaousis (ktsaou)
53b61b1edf comparisons are now made using the COMMON IPs mode, which is 20+ times faster than mode COMBINE 2015-06-11 01:33:21 +03:00
Costa Tsaousis (ktsaou)
b62a017d8f comparisons are now made using the COMMON IPs mode, which is 20+ times faster than mode COMBINE 2015-06-11 01:30:30 +03:00
Costa Tsaousis (ktsaou)
30c37b32da added: iw_spamlist, iw_wormlist, dragon_http, tor_exits 2015-06-11 00:34:25 +03:00
Costa Tsaousis (ktsaou)
06b3997ba0 fixed line numbers reported in --common mode 2015-06-10 01:23:03 +03:00
Costa Tsaousis (ktsaou)
d97df52e69 changed the copyright header to follow the other firehol tools 2015-06-10 01:04:06 +03:00
Costa Tsaousis (ktsaou)
ffb0a46916 added option --common that prints the common IP ranges in all files given 2015-06-10 00:05:45 +03:00
Costa Tsaousis (ktsaou)
62cc7456b1 added ipdeny.com geolocation country db 2015-06-09 02:02:49 +03:00
Costa Tsaousis (ktsaou)
8af07601c1 added version 2015-06-09 00:06:05 +03:00
Costa Tsaousis (ktsaou)
d91174c108 added version 2015-06-08 23:55:42 +03:00
Costa Tsaousis (ktsaou)
d7d721d754 added option --prefixes to enable only specific CIDR prefixes 2015-06-08 23:53:55 +03:00
Costa Tsaousis (ktsaou)
dcb4c57f0a some rework to get statistics and timings in verbose 2015-06-08 22:32:42 +03:00
Costa Tsaousis (ktsaou)
e7c8b2c07b added firehol lists 2015-06-08 09:13:53 +03:00
Costa Tsaousis (ktsaou)
0ee49203f8 added help for option --compare-next 2015-06-08 00:24:33 +03:00
Costa Tsaousis (ktsaou)
d5bc03c2f0 was reporting 1 unique IP on empty files - fixed 2015-06-08 00:13:11 +03:00
Costa Tsaousis (ktsaou)
448df325d8 fixed a bug that appears only when IP ranges are given, the original iprange.c did not merge overlapping ranges that are not bound to any specific netmask - fixed it 2015-06-07 23:57:30 +03:00
Costa Tsaousis (ktsaou)
9fada68a92 info for compiling and installing iprange 2015-06-07 23:18:16 +03:00
Costa Tsaousis (ktsaou)
ce9684aaa6 added option --comprare-next to compare two different sets of files with each other; solved a number overflow issue in print-outs 2015-06-07 23:11:24 +03:00
Costa Tsaousis (ktsaou)
ad4de2d8a9 fix for not reducing netsets by default (touch .reduce to enable it) and keeping the unreduced file 2015-06-07 19:59:04 +03:00
Costa Tsaousis (ktsaou)
0e766f2b58 added support for latest iprange 2015-06-07 19:10:37 +03:00
Costa Tsaousis (ktsaou)
7383b67dde added --print-prefix and --print-suffix options only for IPs or subnets 2015-06-07 16:20:44 +03:00
Costa Tsaousis (ktsaou)
ee6624f9e2 removed comments that were left by accident; added --print-prefix and --print-suffix; fixed --min-prefix that was not operational 2015-06-07 15:39:39 +03:00
Costa Tsaousis (ktsaou)
7706d4f541 added default prefix option 2015-06-07 01:37:16 +03:00
Costa Tsaousis (ktsaou)
335a7637ad renamed an option 2015-06-06 23:29:37 +03:00
Costa Tsaousis (ktsaou)
11e3454bdd fixed internal comparison function 2015-06-06 21:44:48 +03:00
Costa Tsaousis (ktsaou)
7c2922fe14 added function to reduce the prefixes/subnets produced, for optimized iptables ipsets 2015-06-06 20:23:29 +03:00
Costa Tsaousis (ktsaou)
277d38e014 new updated version of iprange, capable of merging and comparing ipsets 2015-06-06 15:41:11 +03:00
Costa Tsaousis (ktsaou)
3d89d9b12f added virbl lashback_ubl 2015-06-03 23:23:25 +03:00
Costa Tsaousis (ktsaou)
b0b67e40fc sped up comparisons significantly by added in -C flag to iprange to print the number of unique IPs in an ipset 2015-05-31 20:38:03 +03:00
Costa Tsaousis (ktsaou)
847d77d222 more renames; added maxmind_proxy_fraud; added badips.com 2015-05-30 19:55:42 +03:00
Costa Tsaousis (ktsaou)
ec9cc7ac19 better curl error handling; renamed stopforumspam and cleanmx 2015-05-29 01:55:59 +03:00
Costa Tsaousis (ktsaou)
981d90465b added proxz.com, proxyrss.com; source ipv4 filtering is now strict 2015-05-29 00:06:04 +03:00
Costa Tsaousis (ktsaou)
7a1c8a48fb added xroxy; this required to set the referrer of the curl command 2015-05-28 19:24:57 +03:00
Costa Tsaousis (ktsaou)
17dc909a30 added timeout and others options to curl; changed the comments on a few ipsets 2015-05-28 17:10:45 +03:00
Costa Tsaousis (ktsaou)
8fa876f2c6 added all blocklist.de ipsets 2015-05-28 01:19:48 +03:00
Costa Tsaousis (ktsaou)
03f93b44ec fixed an issue where the output format of netsets was not correct and was filtering out single IPs from netsets 2015-05-27 14:08:16 +03:00
Costa Tsaousis (ktsaou)
3c8bcb74d1 allowed spliting voipbl for older systems 2015-05-27 12:04:58 +03:00
Costa Tsaousis (ktsaou)
be81785345 minor fixes in comparison with geolite2 2015-05-26 22:46:47 +03:00
Costa Tsaousis (ktsaou)
90b52c3662 added checks for required commands to run 2015-05-26 22:14:49 +03:00
Costa Tsaousis (ktsaou)
63cce35a16 a few more ipset renames; added checks for required commands to run 2015-05-26 22:14:32 +03:00
Costa Tsaousis (ktsaou)
760f4456eb renamed ipsets to have a uniform naming scheme 2015-05-26 11:21:34 +03:00
Costa Tsaousis (ktsaou)
b697137fef spamhaus whitelisting added properly 2015-05-25 23:13:03 +03:00
Costa Tsaousis (ktsaou)
7806a957e2 added voipbl; fixed an issue when comparing ipsets that were updated in a previous run without the comparison enabled 2015-05-25 22:02:20 +03:00
Costa Tsaousis (ktsaou)
85320022d1 added warning about aggregate and large netsets 2015-05-25 03:09:50 +03:00
Costa Tsaousis (ktsaou)
8bbb14b908 added iprange, just in case someone needs it 2015-05-25 03:08:33 +03:00
Costa Tsaousis (ktsaou)
cb0e6e56dd added support for comparing ipsets and generating a report 2015-05-25 02:00:51 +03:00
Costa Tsaousis (ktsaou)
e771e8da89 added httpbl.org; re-organized scores to apply changes more easily 2015-05-24 16:22:30 +03:00
Costa Tsaousis (ktsaou)
c029501805 added maxmind geolite2 country databases 2015-05-23 19:03:04 +03:00
Costa Tsaousis (ktsaou)
c585705a2f fixed bug that was preventing an update; preparations to add maxmind geolite2 database 2015-05-23 03:49:57 +03:00
Costa Tsaousis (ktsaou)
f8e689f7a1 added syslog support 2015-05-22 23:53:04 +03:00
Costa Tsaousis (ktsaou)
6ec6ee3da7 now all files have a header with info; prevented repeatily checking a file that has not been modified on the server - now it will check again as if it was downloaded; added option -i to disable this feature 2015-05-22 23:34:21 +03:00
Costa Tsaousis (ktsaou)
80cc04b005 added example in blacklist_check() 2015-05-22 13:13:48 +03:00
Costa Tsaousis (ktsaou)
2351ee4297 fix for whitelisting IPs from custom function 2015-05-22 13:00:41 +03:00
Costa Tsaousis (ktsaou)
30de402230 added function in config for approving all blacklisted IPs; added command like argument "flush" to emtpy the blacklist and cache ipsets; added check before starting that verifies that the exclusion ipsets do exist. 2015-05-22 12:52:54 +03:00
Costa Tsaousis (ktsaou)
e8f3ecbf90 added command line options 2015-05-22 00:03:42 +03:00
Costa Tsaousis (ktsaou)
cf1591345e added documentation 2015-05-21 23:43:08 +03:00
Costa Tsaousis (ktsaou)
3a2274279b code cleanup and optimizations; DNSBL weights favor dynamic IP users 2015-05-21 20:28:56 +03:00
Costa Tsaousis (ktsaou)
6fc19b713a fixed split mode 2015-05-21 13:10:31 +03:00
Costa Tsaousis (ktsaou)
85e01d9538 renamed 2015-05-21 13:04:40 +03:00
Costa Tsaousis (ktsaou)
66e252b305 renamed to dnsbl-ipset 2015-05-21 13:01:56 +03:00
Costa Tsaousis (ktsaou)
2142bd027e first commit of DNSBL->IPSET script 2015-05-21 02:35:21 +03:00
Costa Tsaousis (ktsaou)
8041e0089d added nixspam; fixed issues with projecthoneypot not updating its sets 2015-05-20 23:15:29 +03:00
Costa Tsaousis (ktsaou)
f6138ee03d added sslbl 2015-05-19 23:53:05 +03:00
Costa Tsaousis (ktsaou)
2c7f66eb7c added links for the generated tables 2015-05-17 21:27:53 +03:00
Costa Tsaousis (ktsaou)
d25fe961d0 added projecthoneypot.org rss lists 2015-05-17 18:21:03 +03:00
Costa Tsaousis (ktsaou)
885717da90 comment fix 2015-05-17 16:07:16 +03:00
Costa Tsaousis (ktsaou)
aa9ec38cd2 updated docs and info 2015-05-17 14:30:06 +03:00
Costa Tsaousis (ktsaou)
2ccf75646a added support for IF_MODIFIED_SINCE to save bandwidth on the servers 2015-05-17 14:13:55 +03:00
Costa Tsaousis
3f575ec403 it now counts the number of unique IPs in a netset 2015-05-17 04:46:32 +03:00
Costa Tsaousis (ktsaou)
647fc5691d added many more lists; fixed a bug from the last commit that it would not add IPs when the first component of the IP included a zero; added locking; better generation of the README.md file 2015-05-17 00:21:05 +03:00
Costa Tsaousis (ktsaou)
45c6b5a877 added iblocklist.com lists 2015-05-16 06:30:02 +03:00
Costa Tsaousis (ktsaou)
05667c88e4 added support for history merging, needed for rss feeds 2015-05-16 03:52:08 +03:00
Costa Tsaousis
b00916ccb1 one git commit per run 2015-05-15 23:23:22 +03:00