Big: build adjustments

This commit is contained in:
Zelo72 2021-10-28 08:43:58 +02:00
parent 14641dd3a3
commit 3b6258c632
2 changed files with 36109 additions and 21721 deletions

57676
big.adblock

File diff suppressed because it is too large Load Diff

154
big.stats
View File

@ -2,22 +2,20 @@
Config: Config:
** Build - 6: ** Build - 4:
extendWWWFLD extendWWWFLD
hosts hosts
adblock adblock
adblockwhite
no_adblock_convert
localpush localpush
++ Prioritized Blocklists - 1: ++ Prioritized Blocklists - 1:
../../data/rules.list.block ../../data/rules.list.block
- White - 31: -- Deadlists - 1:
big.white ../../data/dead.list
-- Whitelists - 2: -- Whitelists - 2:
@ -34,155 +32,57 @@ Initialize ...
# Download and convert Sourcelists ... # Download and convert Sourcelists ...
Nr | Count | Format | Source | Status | File | URL/File Nr | Count | Format | Source | Status | File | URL/File
1 | 478815 | domains | local | online | changed | multi.txt 1 | 478815 | domains | local | online | unchanged | multi.txt
2 | 46809 | domains | local | online | changed | fake.txt 2 | 46809 | domains | local | online | unchanged | fake.txt
3 | 1058155 | domains | http | online | unchanged | https://dbl.oisd.nl 3 | 1058155 | domains | http | online | unchanged | https://dbl.oisd.nl
4 | 40736 | adblock | http | online | unchanged | https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt 4 | 40751 | adblock | http | online | changed | https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt
# Build big Domainlist ... # Build big Domainlist ...
Stats big: Stats big:
** Source (raw): 1624515 ** Source (raw): 1624530
== Source (unique): 1334013 (-290502) == Source (unique): 1334015 (-290515)
-- White: 1333748 (-265) -- White: 1333750 (-265)
-- White(*): 1328848 (-4900) -- White(*): 1328850 (-4900)
++ Block: 1328867 (+19) -- Dead: 1287735 (-41115)
-- Unblock: 1328815 (-52) ++ Block: 1287872 (+137)
-- Unblock(*): 1328733 (-82) -- Unblock: 1287820 (-52)
++ FLD: 1328948 (+215) -- Unblock(*): 1287738 (-82)
++ WWW: 1340839 (+11891) ++ FLD: 1302039 (+14301)
++ WWW: 1311456 (+9417)
1340839 unique Domains - Version 2021.1028.061851 1311456 unique Domains - Version 2021.1028.082222
MD5 Domains RAW: acdcc2c85704f7d477f0c4a6b2f0f12e MD5 Domains RAW: 5e08ae3f764ac3bc7a05775bf28514be
# Convert big to Hostlist ... # Convert big to Hostlist ...
# Convert big to AdBlocklist ... # Convert big to AdBlocklist ...
Prepare domain list for compiling ... done.
Starting @adguard/hostlist-compiler v1.0.12 Starting @adguard/hostlist-compiler v1.0.12
Starting the compiler Starting the compiler
Configuration: { Configuration: {
"name": "Big", "name": "big",
"sources": [ "sources": [
{ {
"source": "/media/nas/git/adguard/multi.adblock", "source": "big.adblock.raw",
"type": "adblock", "type": "adblock",
"transformations": [ "transformations": [
"Validate" "Validate"
] ]
},
{
"source": "/media/nas/git/adguard/fake.adblock",
"type": "adblock",
"transformations": [
"Validate"
]
},
{
"source": "https://abp.oisd.nl/",
"type": "adblock",
"exclusions_sources": [
"../exclusions.txt",
"../../../data/adblock.exclusions.referral",
"../../../data/adblock.exclusions.list",
"../../../data/adblock.exclusions.important",
"https://raw.githubusercontent.com/AdguardTeam/AdGuardSDNSFilter/master/Filters/exclusions.txt"
],
"transformations": [
"Validate"
]
},
{
"source": "https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt",
"type": "adblock",
"exclusions_sources": [
"../exclusions.txt",
"../../../data/adblock.exclusions.referral",
"../../../data/adblock.exclusions.list",
"../../../data/adblock.exclusions.important"
],
"transformations": [
"Validate"
]
},
{
"source": "https://raw.githubusercontent.com/AdguardTeam/AdGuardSDNSFilter/master/Filters/rules.txt",
"exclusions_sources": [
"../exclusions.txt",
"../../../data/adblock.exclusions.referral",
"../../../data/adblock.exclusions.important"
],
"transformations": [
"Validate"
],
"type": "adblock"
},
{
"source": "../rules.txt",
"exclusions_sources": [
"../exclusions.txt",
"../../../data/adblock.exclusions.referral",
"../../../data/adblock.exclusions.important"
],
"transformations": [
"Validate"
],
"type": "adblock"
},
{
"source": "https://raw.githubusercontent.com/AdguardTeam/AdGuardSDNSFilter/master/Filters/exceptions.txt",
"transformations": [
"Validate"
],
"type": "adblock"
},
{
"source": "../exceptions.txt",
"transformations": [
"Validate"
],
"type": "adblock"
} }
], ],
"transformations": [ "transformations": [
"Compress" "Compress"
] ]
} }
Start compiling /media/nas/git/adguard/multi.adblock Start compiling big.adblock.raw
Original length is 258092 Original length is 951039
Length after applying transformations is 258092 Length after applying transformations is 951037
Start compiling /media/nas/git/adguard/fake.adblock The list was compressed from 951040 to 508330
Original length is 23809 Final length of the list is 508336
Length after applying transformations is 23809
Start compiling https://abp.oisd.nl/
Original length is 371588
Filtering the list of rules using 2151 exclusion rules
Excluded 560 rules. 371028 rules left.
Length after applying transformations is 371027
Start compiling https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt
Original length is 41807
Filtering the list of rules using 1825 exclusion rules
Excluded 105 rules. 41702 rules left.
Length after applying transformations is 41702
Start compiling https://raw.githubusercontent.com/AdguardTeam/AdGuardSDNSFilter/master/Filters/rules.txt
Original length is 576
Filtering the list of rules using 918 exclusion rules
Excluded 0 rules. 576 rules left.
Length after applying transformations is 570
Start compiling ../rules.txt
Original length is 2
Filtering the list of rules using 918 exclusion rules
Excluded 0 rules. 2 rules left.
Length after applying transformations is 2
Start compiling https://raw.githubusercontent.com/AdguardTeam/AdGuardSDNSFilter/master/Filters/exceptions.txt
Original length is 179
Length after applying transformations is 179
Start compiling ../exceptions.txt
Original length is 32
Length after applying transformations is 32
The list was compressed from 695437 to 494878
Final length of the list is 494884
Writing output to /media/nas/git/rpi/pihole/blocklists/build/big/out/big.adblock Writing output to /media/nas/git/rpi/pihole/blocklists/build/big/out/big.adblock
Finished compiling Finished compiling