stop_cloudflare/instructions.md

281 lines
9.0 KiB
Markdown
Raw Normal View History

2019-11-20 07:34:51 +01:00
# List Instructions
![](image/imnotarobot.gif)
---
2020-01-11 07:57:08 +01:00
<details>
<summary>_click me_
2018-10-11 15:00:30 +02:00
2019-02-25 03:09:59 +01:00
## Website is using Cloudflare
2020-01-11 07:57:08 +01:00
</summary>
2018-10-11 15:00:30 +02:00
2019-11-20 07:34:51 +01:00
- **Cloudflare users** | [**List Directory**](cloudflare_users/)
2019-05-06 14:13:39 +02:00
2018-10-11 15:00:30 +02:00
| List name | Description |
2018-10-11 15:04:44 +02:00
| -------- | -------- |
2019-08-11 16:18:35 +02:00
| **/domain/cloudflare_?.txt** | **Split files (base domain)** |
2019-02-25 03:20:36 +01:00
| ex_cloudflare_users.txt | Domains which used Cloudflare in the past, not anymore |
2019-05-10 07:07:38 +02:00
| cloudflare_supporter.txt | who is using Cloudflare or endorsing Cloudflare. (URL only) |
2019-05-06 14:13:39 +02:00
2019-11-20 07:34:51 +01:00
- **Cloudflare Corporation** | [**List Directory**](cloudflare_inc/)
2019-05-06 14:13:39 +02:00
| List name | Description |
| -------- | -------- |
2019-02-25 03:20:36 +01:00
| cloudflare_CIDR_v4.txt | IPv4 CIDR owned by Cloudflare |
| cloudflare_CIDR_v6.txt | IPv6 CIDR owned by Cloudflare |
| cloudflare_range_v4.txt | IPv4 range owned by Cloudflare |
| cloudflare_owned_ASN.txt | AS network owned by Cloudflare |
2019-03-02 04:41:09 +01:00
| cloudflare_owned_NS.txt | Name Server owned by Cloudflare |
2018-10-11 15:17:40 +02:00
| cloudflare_owned_domains.txt | Domains owned by Cloudflare |
| cloudflare_owned_onions.txt | Tor .onions owned by Cloudflare |
2019-05-07 11:53:24 +02:00
| cloudflare_members.txt | Cloudflare employer & employee |
2018-10-11 15:00:30 +02:00
2020-01-11 07:57:08 +01:00
**How to detect Cloudflare?**
2018-10-11 15:00:30 +02:00
2019-03-03 00:01:42 +01:00
There are many ways to detect it:
2018-10-11 15:12:11 +02:00
- [These add-ons](what-to-do.md) will help your Cloudflare collection.
- Visit a website via Tor or VPN, and you will be greeted by "Attention Required! Cloudflare" webpage.
2019-11-12 06:11:14 +01:00
- Use "[Is MITM?](https://searxes.eu.org/collab/sxes/tool_ismitm.php)" webpage.
2019-03-03 00:01:42 +01:00
- Dig "[NS record](https://www.digwebinterface.com/?hostnames=emsisoft.com&type=NS&ns=resolver&useresolver=8.8.4.4&nameservers=)" of the domain.
2019-03-03 00:02:09 +01:00
2019-03-03 00:01:42 +01:00
```
emsisoft.com. 21599 IN NS bella.ns.cloudflare.com.
emsisoft.com. 21599 IN NS dom.ns.cloudflare.com.
```
2019-03-03 00:02:09 +01:00
2019-03-03 00:01:42 +01:00
- Dig "[A record](https://www.digwebinterface.com/?hostnames=dev.qubes-os.org&type=A&ns=resolver&useresolver=8.8.4.4&nameservers=)" of the FQDN, then [check the IP's owner](https://ipinfo.io/104.18.228.122).
2019-03-03 00:02:09 +01:00
2019-03-03 00:01:42 +01:00
```
dev.qubes-os.org. 299 IN A 104.18.228.122
ASN AS13335 Cloudflare, Inc.
Organization Cloudflare, Inc.
Route 104.18.224.0/20
```
2018-10-11 15:00:30 +02:00
2018-10-11 15:12:11 +02:00
```
2019-02-25 03:14:30 +01:00
IMPORTANT: Please add only "Base Domain"
2019-02-25 03:09:59 +01:00
if "community.example.com" is using Cloudflare
2019-02-25 03:14:30 +01:00
add "example.com"
if "www.example.co.uk" is using Cloudflare
add "example.co.uk"
2019-02-25 03:09:59 +01:00
if "example.net" is using Cloudflare
2019-02-25 03:14:30 +01:00
add "example.net"
... to /split/cloudflare_e.txt
2019-02-25 03:09:59 +01:00
```
2020-01-11 07:57:08 +01:00
**But the website X no longer using Cloudflare!**
*Remove* it from /split/ list and *add* to "[ex_cloudflare_users.txt](cloudflare_users/ex_cloudflare_users.txt)".
</details>
------
2018-04-18 18:27:14 +02:00
2020-01-11 07:57:08 +01:00
<details>
<summary>_click me_
2018-04-18 18:25:19 +02:00
2019-11-20 07:34:51 +01:00
## Website is NOT using Cloudflare
2020-01-11 07:57:08 +01:00
</summary>
2018-04-18 18:25:19 +02:00
2019-11-20 07:34:51 +01:00
- **Anti-Tor users** (formerly "*TorBlocker Hall of Shame Part I*") | [**List Directory**](not_cloudflare/)
2019-05-07 11:49:47 +02:00
2019-03-05 03:35:07 +01:00
| List name | Description |
| -------- | -------- |
2019-12-04 05:41:15 +01:00
| **/domain/(cdnName).txt** | **Split files (FQDN)** |
2019-12-04 05:55:43 +01:00
| tor_blocked.txt | FQDN which denied access via Tor |
2019-12-04 05:47:13 +01:00
| /cidr_data/?.txt | CIDR, ASN |
2019-12-04 05:56:52 +01:00
| ex_tor_blocked.txt | _was_ previously on one of the above tor-hostile lists |
2019-12-04 05:55:43 +01:00
2020-01-11 07:57:08 +01:00
- Add-on "[Kiu retejo malakceptis min?](about.urjm.md)" will help your domain collection.
2019-12-03 00:21:11 +01:00
2019-08-15 08:47:32 +02:00
![](image/siteground.jpg)
2019-12-04 00:02:18 +01:00
Above is how Siteground-hosted([INAP](https://www.inap.com/press-release/inap-completes-acquisition-singlehop/);[Singlehop](https://www.siteground.com/blog/siteground-partners-singlehop/)) sites often appear to Tor visitors when timeouts/tarpitting doesn't occur.
2019-12-04 05:47:13 +01:00
You can find such examples in `/domains/`.
2019-04-04 01:23:12 +02:00
2019-12-04 05:45:36 +01:00
2019-03-05 03:37:27 +01:00
```
2019-03-11 00:44:36 +01:00
2020-01-11 07:57:08 +01:00
About "CDN FQDN list"
www.example.com
---> www.example.com is using CDN.
?.akamaiedge.net
---> subdomain of akamaiedge.net is using CDN.
* unique hostname will be masked as "(subdomain)".
senate.gov
---> base domain is using CDN.
2019-03-11 00:44:36 +01:00
2019-03-05 03:37:27 +01:00
```
2019-03-05 03:35:07 +01:00
2019-02-25 03:09:59 +01:00
Some websites use other companies with the CloudFlare business model.
2018-04-18 18:25:19 +02:00
2019-02-25 03:09:59 +01:00
This is a collection of websites that ban Tor exits, other than through Cloudflare(e.g. showing access denied pages, systematic timing out connections, ...).
2019-03-05 03:38:30 +01:00
2020-01-11 07:57:08 +01:00
</details>
------
<details>
<summary>_click me_
## How to add your data
</summary>
A or B will be enough. Thank you for your contribution.
- Type A: Push to OpenPrivacy
1. Log in to *OpenPrivacy*.
2. Click "*Fork*" button. (top-left corner)
3. Edit text file.
4. Click *Double-arrow* button to create a *new pull request*.
- Type B: Just scan the FQDN
1. Scan FQDN on "[Is MITM?](https://searxes.eu.org/collab/sxes/tool_ismitm.php)" webpage. (or just use "MITM test API", "Detect CDN API")
2. It will be pushed to OpenPrivacy automatically within a week.
</details>
------
<details>
<summary>_click me_
2019-07-11 15:17:30 +02:00
2019-11-20 07:34:51 +01:00
## How to setup git
2020-01-11 07:57:08 +01:00
</summary>
2019-08-29 11:33:45 +02:00
This procedure will give you a cloudflare-tor fork with a
privacy-respecting configuration to do pushes with SSH over Tor using
codeberg.org ("CDB"). This procedure is designed for ***linux***.
The first step covers Windows too, but these instructions probably
need more adaptations for Windows and other platforms.
- Linux: `aptitude install git tor ssh`
- Windows: Download `https://github.com/git-for-windows/git/releases/PortableGit-2.21.0-64-bit.7z` & run `git-bash.exe`
1. install Git, SSH(Not Windows), and Tor (if you haven't already)
1. create a `codeberg.org` account (username "snowden" will be used for this example)
1. create an SSH key pair `$ ssh-keygen -t rsa -N '' -C 'snowden at codeberg' -f "$HOME"/.ssh/id_rsa_codeberg-snowden`
1. edit `$HOME/.ssh/config`:
```
host codeberg-*
hostname codeberg.org
ForwardX11 no
ProxyCommand connect -4 -S 127.0.0.1:9050 $(tor-resolve %h 127.0.0.1:9050) %p
host codeberg-snowden
IdentityFile /home/user/.ssh/id_rsa_codeberg-snowden
```
1. copy `"$HOME"/.ssh/id_rsa_codeberg-snowden.pub` to clipboard
1. codeberg.org > settings > SSH/GPG Keys > add key (paste from clipboard)
1. $ `firefox https://codeberg.org/crimeflare/cloudflare-tor`
1. fork it (top right corner)
1. go to the directory you want the project to be rooted in (hereafter we'll call it `$project_root`).
1. anonymously download your fork: $ `git clone git@codeberg-snowden:crimeflare/cloudflare-tor.git`
1. edit `$project_root/cloudflare-tor/.git/config` to include the account name and email address that will be on every commit, as well as the URL:
```
[user]
email = BM-yadayadayada6fgnLfybVnCcWf25AGZcgg@bitmessage.ch
name = snowden
[remote "origin"]
url = git@codeberg-snowden:snowden/cloudflare-tor.git
fetch = +refs/heads/*:refs/remotes/origin/*
[remote "upstream"]
url = git@codeberg-snowden:crimeflare/cloudflare-tor.git
fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
remote = origin
merge = refs/heads/master
```
1. make your first change
1. (from `$project_root`) $ `git add . -u -n`
1. check that the files listed are what you changed and intend to push upstream
1. if yes: `$ git add . -u`
1. $ `git commit -m 'description of first change'`
1. $ `git push origin master`
1. $ `firefox https://codeberg.org/crimeflare/cloudflare-tor`
1. make a new pull request
&nbsp;
Whenever git operates on the cloudflare-tor project, all connections
to codeberg are automatically over Tor with this configuration
(because the `url` in `.git/config` references the virtual host
`codeberg-snowden` in `~/.ssh/config`).
2019-07-11 15:17:30 +02:00
2020-01-11 07:57:08 +01:00
</details>
------
<details>
<summary>_click me_
## About Cloudflare base domain list
</summary>
2019-11-12 06:11:14 +01:00
Our mission is clear - `stay away from Cloudflare`.
If the `subdomain.example.com` is cloudflared, we add `example.com` to the database. (`subdomain.example.com` is the sub-domain of `example.com`. Only `the owner` of `example.com` can create sub-domain)
Even if `whatever.example.com` is _not_ behind cloudflare we _will_ raise a warning, because the base domain `example.com` is `cloudflare user`.
`The owner` of `example.com` can enable Cloudflare to `whatever.example.com` at any time without user's notice. It can be done from `dash.cloudflare.com` webpage or hitting `Cloudflare API`. `The owner` is supporting `Cloudflare` and this is severe `security risk`.
Until `the owner` completely stop using Cloudflare service for `example.com`, we _do not_ remove `example.com` from the database.
There is `no exception`.
2020-01-11 07:57:08 +01:00
```
"Amazon.com"
$ getweb --headonly https://pages.payments.amazon.com/robots.txt
cf-cache-status: HIT
expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
alt-svc: h2="cflare******.onion:443"; ma=86400; persist=1
server: cloudflare
cf-ray: XXXXXXXXXXX-YYY
```
2019-11-12 06:11:14 +01:00
If `the owner` moved away from `cloudflare` **completely**, you are welcome to add `example.com` to the "[ex_cloudflare_users.txt](cloudflare_users/ex_cloudflare_users.txt)" - after checking `example.com` with online tool below.
1. Open "[Is MITM?](https://searxes.eu.org/collab/sxes/tool_ismitm.php)" webpage.
2. Input `gitlab.com` and click `Skanu`.
3. Click `testo` for detailed scan.
2020-01-11 07:57:08 +01:00
4. If you got `---Finish---`, the domain _might_ stopped using Cloudflare. We'll _investigate_ and remove it - or not. (wait some days and scan again to see whether the domain is removed)
`Only a few Cloudflare user leave Cloudflare. False positive is uncommon.`
</details>
---
2019-11-12 06:11:14 +01:00
2020-01-11 07:57:08 +01:00
!["Cloudflare is not an option."](image/cfisnotanoption.jpg)