threat-intelligence auto-update

This commit is contained in:
Zelo72 2021-10-30 09:24:44 +02:00
parent 61c85c3e7b
commit a71c41e7c8
2 changed files with 440 additions and 158 deletions

File diff suppressed because it is too large Load Diff

View File

@ -8,9 +8,8 @@ Config:
adblock
localpush
-- Excludelists - 2:
-- Excludelists - 1:
../../fake.txt
../../doh-vpn-proxy-bypass.txt
-- Deadlists - 1:
@ -34,24 +33,24 @@ Initialize ...
# Download and convert Sourcelists ...
Nr | Count | Format | Source | Status | File | URL/File
1 | 6780 | hosts | http | online | changed | https://curben.gitlab.io/malware-filter/phishing-filter-hosts.txt
2 | 419 | hosts | http | online | changed | https://curben.gitlab.io/malware-filter/pup-filter-hosts.txt
3 | 10295 | hosts | http | online | changed | https://curben.gitlab.io/malware-filter/urlhaus-filter-hosts.txt
1 | 6780 | hosts | http | online | unchanged | https://curben.gitlab.io/malware-filter/phishing-filter-hosts.txt
2 | 419 | hosts | http | online | unchanged | https://curben.gitlab.io/malware-filter/pup-filter-hosts.txt
3 | 10295 | hosts | http | online | unchanged | https://curben.gitlab.io/malware-filter/urlhaus-filter-hosts.txt
4 | 3496 | hosts | http | online | unchanged | https://gitlab.com/ZeroDot1/CoinBlockerLists/raw/master/hosts_browser
5 | 34375 | hosts | http | online | unchanged | https://hole.cert.pl/domains/domains_hosts.txt
6 | 550 | hosts | http | online | unchanged | https://paulgb.github.io/BarbBlock/blacklists/hosts-file.txt
7 | 5987 | hosts | http | online | unchanged | https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate%20versions%20Anti-Malware%20List/AntiMalwareHosts.txt
8 | 2204 | hosts | http | online | unchanged | https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Risk/hosts
9 | 59 | hosts | http | online | unchanged | https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Spam/hosts
10 | 145 | hosts | http | online | changed | https://raw.githubusercontent.com/davidonzo/Threat-Intel/master/lists/latestdomains.piHole.txt
11 | 1056 | hosts | http | online | changed | https://raw.githubusercontent.com/durablenapkin/scamblocklist/master/hosts.txt
10 | 145 | hosts | http | online | unchanged | https://raw.githubusercontent.com/davidonzo/Threat-Intel/master/lists/latestdomains.piHole.txt
11 | 1056 | hosts | http | online | unchanged | https://raw.githubusercontent.com/durablenapkin/scamblocklist/master/hosts.txt
12 | 8624 | hosts | http | online | unchanged | https://raw.githubusercontent.com/guardicore/labs_campaigns/master/Autodiscover/autodiscover-tlds.txt
13 | 695 | hosts | http | online | unchanged | https://raw.githubusercontent.com/hoshsadiq/adblock-nocoin-list/master/hosts.txt
14 | 3686 | hosts | http | online | unchanged | https://raw.githubusercontent.com/infinitytec/blocklists/master/scams-and-phishing.txt
15 | 1072 | hosts | http | online | unchanged | https://raw.githubusercontent.com/metamask/eth-phishing-detect/master/src/hosts.txt
16 | 1386 | hosts | http | online | unchanged | https://raw.githubusercontent.com/mitchellkrogza/Badd-Boyz-Hosts/master/hosts
17 | 13465 | hosts | http | online | unchanged | https://raw.githubusercontent.com/mitchellkrogza/The-Big-List-of-Hacked-Malware-Web-Sites/master/hosts
18 | 1753 | hosts | http | online | changed | https://urlhaus.abuse.ch/downloads/hostfile/
18 | 1723 | hosts | http | online | unchanged | https://urlhaus.abuse.ch/downloads/hostfile/
19 | 4467 | hosts | http | online | unchanged | https://threatfox.abuse.ch/downloads/hostfile
20 | 883 | adblock | http | online | unchanged | https://raw.githubusercontent.com/piperun/iploggerfilter/master/filterlist
21 | 915 | domains | http | online | changed | https://azorult-tracker.net/api/list/domain?format=plain
@ -62,13 +61,13 @@ Initialize ...
26 | 34375 | domains | http | online | unchanged | https://hole.cert.pl/domains/domains.txt
27 | 73595 | domains | http | online | unchanged | https://joewein.net/dl/bl/dom-bl-base.txt
28 | 858 | domains | http | online | unchanged | https://joewein.net/dl/bl/dom-bl.txt
29 | 1999 | domains | http | online | changed | https://kriskintel.com/feeds/ktip_covid_domains.txt
30 | 1996 | domains | http | online | changed | https://kriskintel.com/feeds/ktip_malicious_domains.txt
29 | 1999 | domains | http | online | unchanged | https://kriskintel.com/feeds/ktip_covid_domains.txt
30 | 1996 | domains | http | online | unchanged | https://kriskintel.com/feeds/ktip_malicious_domains.txt
31 | 397 | domains | http | online | unchanged | https://kriskintel.com/feeds/ktip_ransomware_feeds.txt
32 | 2245 | domains | http | online | unchanged | https://orca.pet/notonmyshift/domains.txt
33 | 145 | domains | http | online | changed | https://osint.digitalside.it/Threat-Intel/lists/latestdomains.txt
34 | 46194 | domains | http | online | changed | https://phishing.army/download/phishing_army_blocklist.txt
35 | 56141 | domains | http | online | changed | https://phishing.army/download/phishing_army_blocklist_extended.txt
33 | 145 | domains | http | online | unchanged | https://osint.digitalside.it/Threat-Intel/lists/latestdomains.txt
34 | 46194 | domains | http | online | unchanged | https://phishing.army/download/phishing_army_blocklist.txt
35 | 56141 | domains | http | online | unchanged | https://phishing.army/download/phishing_army_blocklist_extended.txt
36 | 1406 | domains | http | online | unchanged | https://raw.githubusercontent.com/AmnestyTech/investigations/master/2021-07-18_nso/domains.txt
37 | 27 | domains | http | online | unchanged | https://raw.githubusercontent.com/DRSDavidSoft/additional-hosts/master/domains/blacklist/fake-domains.txt
38 | 34689 | domains | http | online | unchanged | https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADomains.txt
@ -78,14 +77,14 @@ Initialize ...
42 | 3960 | domains | http | online | unchanged | https://raw.githubusercontent.com/bongochong/CombinedPrivacyBlockLists/master/NoFormatting/MD-ID-Fork.txt
43 | 18454 | domains | http | online | unchanged | https://raw.githubusercontent.com/cbuijs/shallalist/master/spyware/domains
44 | 13908 | domains | http | online | unchanged | https://raw.githubusercontent.com/cbuijs/ut1/master/cryptojacking/domains
45 | 169592 | domains | http | online | changed | https://raw.githubusercontent.com/cbuijs/ut1/master/malware/domains
45 | 169592 | domains | http | online | unchanged | https://raw.githubusercontent.com/cbuijs/ut1/master/malware/domains
46 | 434 | domains | http | online | unchanged | https://raw.githubusercontent.com/hpthreatresearch/iocs/main/CryptBot/domains.txt
47 | 137 | domains | http | online | unchanged | https://raw.githubusercontent.com/hpthreatresearch/iocs/main/IcedID/domains.txt
48 | 657 | domains | http | online | unchanged | https://raw.githubusercontent.com/hpthreatresearch/iocs/main/TA551/domains.txt
49 | 3729 | domains | http | online | unchanged | https://raw.githubusercontent.com/iam-py-test/my_filters_001/main/Alternative%20list%20formats/antimalware_domains.txt
50 | 2079 | domains | http | online | unchanged | https://raw.githubusercontent.com/matomo-org/referrer-spam-blacklist/master/spammers.txt
51 | 71289 | domains | http | online | unchanged | https://raw.githubusercontent.com/mitchellkrogza/Phishing.Database/master/phishing-domains-ACTIVE.txt
52 | 350 | domains | http | online | changed | https://raw.githubusercontent.com/mitchellkrogza/Phishing.Database/master/phishing-domains-NEW-today.txt
52 | 350 | domains | http | online | unchanged | https://raw.githubusercontent.com/mitchellkrogza/Phishing.Database/master/phishing-domains-NEW-today.txt
53 | 10000 | domains | http | online | unchanged | https://raw.githubusercontent.com/prodaft/malware-ioc/master/FluBot/v3.7_5000_domain.txt
54 | 10000 | domains | http | online | unchanged | https://raw.githubusercontent.com/prodaft/malware-ioc/master/FluBot/v3.7_germany.txt
55 | 10000 | domains | http | online | unchanged | https://raw.githubusercontent.com/prodaft/malware-ioc/master/FluBot/v3.8_domains.txt
@ -105,29 +104,29 @@ Initialize ...
69 | 512 | domains | http | online | unchanged | https://raw.githubusercontent.com/scafroglia93/blocklists/master/blocklists-unit42-playbook.txt
70 | 23232 | domains | http | online | unchanged | https://raw.githubusercontent.com/scafroglia93/blocklists/master/blocklists-unit42-silverterrier.txt
71 | 3791 | domains | http | online | unchanged | https://raw.githubusercontent.com/scafroglia93/blocklists/master/blocklists-zscaler.txt
72 | 150480 | domains | http | online | changed | https://raw.githubusercontent.com/stamparm/aux/master/maltrail-malware-domains.txt
72 | 150480 | domains | http | online | unchanged | https://raw.githubusercontent.com/stamparm/aux/master/maltrail-malware-domains.txt
73 | 16941 | domains | http | online | unchanged | https://raw.githubusercontent.com/stamparm/blackbook/master/blackbook.txt
74 | 500 | domains | http | online | changed | https://rescure.me/covid.txt
75 | 500 | domains | http | online | changed | https://rescure.me/rescure_domain_blacklist.txt
74 | 500 | domains | http | online | unchanged | https://rescure.me/covid.txt
75 | 500 | domains | http | online | unchanged | https://rescure.me/rescure_domain_blacklist.txt
76 | 508 | domains | http | online | unchanged | https://www.botvrij.eu/data/ioclist.domain.raw
77 | 90 | domains | http | online | unchanged | https://www.botvrij.eu/data/ioclist.hostname.raw
78 | 35108 | domains | http | online | unchanged | https://www.stopforumspam.com/downloads/toxic_domains_whole.txt
79 | 102278 | domains | http | online | changed | https://www.usom.gov.tr/url-list.txt
79 | 102278 | domains | http | online | unchanged | https://www.usom.gov.tr/url-list.txt
80 | 135 | domains | local | online | unchanged | black.list.threat-intelligence
# Build threat-intelligence Domainlist ...
Stats threat-intelligence:
** Source (raw): 1201529
== Source (unique): 867324 (-334205)
-- Exclude: 866739 (-585)
-- White: 865965 (-774)
-- White(*): 860718 (-5247)
-- Dead: 371247 (-489471)
** Source (raw): 1201499
== Source (unique): 867316 (-334183)
-- Exclude: 867311 (-5)
-- White: 866537 (-774)
-- White(*): 861290 (-5247)
-- Dead: 371817 (-489473)
371247 unique Domains - Version 2021.1030.090201
MD5 Domains RAW: 3549b2aaa72f282d5fd91266d952fc2f
371817 unique Domains - Version 2021.1030.091751
MD5 Domains RAW: ee9949ee24fc12248a0faaeef5b19958
# Convert threat-intelligence to Hostlist ...
@ -150,10 +149,10 @@ Prepare domain list for compiling ... done.
]
}
Start compiling threat-intelligence.adblock.raw
Original length is 351897
Length after applying transformations is 351897
The list was compressed from 351900 to 323358
Final length of the list is 323364
Original length is 352285
Length after applying transformations is 352285
The list was compressed from 352288 to 323641
Final length of the list is 323647
Writing output to /media/nas/git/rpi/pihole/blocklists/build/threat-intelligence/out/threat-intelligence.adblock
Finished compiling