premiere
/
premiere-libtorrent
2
2
Fork 1
Code Issues Pull Requests Releases Wiki Activity

make tests not rely on delegated anymore, but instead use simple python implementations for socks and http proxy

This commit is contained in:
Arvid Norberg 2013-09-22 03:37:33 +00:00
parent fcff201356
commit 9c99220dd1
5 changed files with 521 additions and 74 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docs/index.rst
View File

@ -9,7 +9,6 @@
* examples_
* `library overview`_
* `reference documentation`_
* `running tests`_
* `tuning`_
* screenshot_
* `mailing list`_ (archive_)
@ -60,7 +59,6 @@ libtorrent
.. _examples: examples.html
.. _`library overview`: manual-ref.html
.. _`reference documentation`: reference.html
.. _`running tests`: running_tests.html
.. _`tuning`: tuning.html
.. _screenshot: client_test.png
.. _`uTP`: utp.html

35
docs/running_tests.rst
View File

@ -1,35 +0,0 @@
=================
libtorrent manual
=================
:Author: Arvid Norberg, arvid@rasterbar.com
.. contents:: Table of contents
:depth: 2
:backlinks: none
running and building tests
==========================
The tests for SOCKS and HTTP proxy relies on ``delegate`` being installed
to set up test proxies. This document outlines the requirements of the
tests as well as describes how to set up your environment to be able to run them.
.. _lighty: http://www.lighttpd.net
delegate
========
Delegate_ can act as many different proxies, which makes it a convenient
tool to use to test libtorrent's support for SOCKS4, SOCKS5, HTTPS and
HTTP proxies.
.. _Delegate: http://www.delegate.org
You can download prebuilt binaries for the most common platforms on
`deletate's download page`_. Make sure to name the executable ``delegated``
and put it in a place where a shell can pick it up, in its ``PATH``. For
instance ``/bin``.
.. _`deletate's download page`: http://www.delegate.org/delegate/download/

200
test/http.py Normal file
View File

@ -0,0 +1,200 @@
# -*- coding: cp1252 -*-
# <PythonProxy.py>
#
#Copyright (c) <2009> <Fábio Domingues - fnds3000 in gmail.com>
#
#Permission is hereby granted, free of charge, to any person
#obtaining a copy of this software and associated documentation
#files (the "Software"), to deal in the Software without
#restriction, including without limitation the rights to use,
#copy, modify, merge, publish, distribute, sublicense, and/or sell
#copies of the Software, and to permit persons to whom the
#Software is furnished to do so, subject to the following
#conditions:
#
#The above copyright notice and this permission notice shall be
#included in all copies or substantial portions of the Software.
#
#THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
#EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
#OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
#NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
#HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
#WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
#FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
#OTHER DEALINGS IN THE SOFTWARE.
"""\
Copyright (c) <2009> <Fábio Domingues - fnds3000 in gmail.com> <MIT Licence>
**************************************
*** Python Proxy - A Fast HTTP proxy ***
**************************************
Neste momento este proxy é um Elie Proxy.
Suporta os métodos HTTP:
- OPTIONS;
- GET;
- HEAD;
- POST;
- PUT;
- DELETE;
- TRACE;
- CONENCT.
Suporta:
- Conexões dos cliente em IPv4 ou IPv6;
- Conexões ao alvo em IPv4 e IPv6;
- Conexões todo o tipo de transmissão de dados TCP (CONNECT tunneling),
p.e. ligações SSL, como é o caso do HTTPS.
A fazer:
- Verificar se o input vindo do cliente está correcto;
- Enviar os devidos HTTP erros se não, ou simplesmente quebrar a ligação;
- Criar um gestor de erros;
- Criar ficheiro log de erros;
- Colocar excepções nos sítios onde é previsível a ocorrência de erros,
p.e.sockets e ficheiros;
- Rever tudo e melhorar a estrutura do programar e colocar nomes adequados nas
variáveis e métodos;
- Comentar o programa decentemente;
- Doc Strings.
Funcionalidades futuras:
- Adiconar a funcionalidade de proxy anónimo e transparente;
- Suportar FTP?.
(!) Atenção o que se segue só tem efeito em conexões não CONNECT, para estas o
proxy é sempre Elite.
Qual a diferença entre um proxy Elite, Anónimo e Transparente?
- Um proxy elite é totalmente anónimo, o servidor que o recebe não consegue ter
conhecimento da existência do proxy e não recebe o endereço IP do cliente;
- Quando é usado um proxy anónimo o servidor sabe que o cliente está a usar um
proxy mas não sabe o endereço IP do cliente;
É enviado o cabeçalho HTTP "Proxy-agent".
- Um proxy transparente fornece ao servidor o IP do cliente e um informação que
se está a usar um proxy.
São enviados os cabeçalhos HTTP "Proxy-agent" e "HTTP_X_FORWARDED_FOR".
"""
import socket, thread, select, sys
__version__ = '0.1.0 Draft 1'
BUFLEN = 8192
VERSION = 'Python Proxy/'+__version__
HTTPVER = 'HTTP/1.1'
class ConnectionHandler:
def __init__(self, connection, address, timeout):
self.client = connection
self.client_buffer = ''
self.timeout = timeout
self.method, self.path, self.protocol = self.get_base_header()
try:
if self.method=='CONNECT':
self.method_CONNECT()
elif self.method in ('OPTIONS', 'GET', 'HEAD', 'POST', 'PUT',
'DELETE', 'TRACE'):
self.method_others()
except:
self.client.send(HTTPVER+' 502 Connection failed\n'+
'Proxy-agent: %s\n\n'%VERSION)
self.client.close()
return
self.client.close()
self.target.close()
def get_base_header(self):
while 1:
self.client_buffer += self.client.recv(BUFLEN)
end = self.client_buffer.find('\n')
if end!=-1:
break
print '%s'%self.client_buffer[:end]#debug
data = (self.client_buffer[:end+1]).split()
self.client_buffer = self.client_buffer[end+1:]
return data
def method_CONNECT(self):
self._connect_target(self.path)
self.client.send(HTTPVER+' 200 Connection established\n'+
'Proxy-agent: %s\n\n'%VERSION)
self.client_buffer = ''
self._read_write()
def method_others(self):
self.path = self.path[7:]
i = self.path.find('/')
host = self.path[:i]
path = self.path[i:]
self._connect_target(host)
self.target.send('%s %s %s\n'%(self.method, path, self.protocol)+
self.client_buffer)
self.client_buffer = ''
self._read_write()
def _connect_target(self, host):
i = host.find(':')
if i!=-1:
port = int(host[i+1:])
host = host[:i]
else:
port = 80
(soc_family, _, _, _, address) = socket.getaddrinfo(host, port)[0]
self.target = socket.socket(soc_family)
self.target.connect(address)
def _read_write(self):
time_out_max = self.timeout/3
socs = [self.client, self.target]
count = 0
while 1:
count += 1
(recv, _, error) = select.select(socs, [], socs, 3)
if error:
break
if recv:
for in_ in recv:
data = in_.recv(BUFLEN)
if in_ is self.client:
out = self.target
else:
out = self.client
if data:
out.send(data)
count = 0
if count == time_out_max:
break
def start_server(host='localhost', port=8080, IPv6=False, timeout=60,
handler=ConnectionHandler):
if IPv6==True:
soc_type=socket.AF_INET6
else:
soc_type=socket.AF_INET
soc = socket.socket(soc_type)
soc.bind((host, port))
print "Serving on %s:%d."%(host, port)#debug
soc.listen(0)
while 1:
thread.start_new_thread(handler, soc.accept()+(timeout,))
if __name__ == '__main__':
listen_port = 8080
i = 1
while i < len(sys.argv):
if sys.argv[i] == '--port':
listen_port = int(sys.argv[i+1])
i += 1
else:
if sys.argv[i] != '--help': print('unknown option "%s"' % sys.argv[i])
print('usage: http.py [--port <listen-port>]')
sys.exit(1)
i += 1
start_server(port=listen_port)

104
test/setup_transfer.cpp
View File

@ -60,6 +60,11 @@ POSSIBILITY OF SUCH DAMAGE.
#include <boost/detail/atomic_count.hpp>
#ifndef _WIN32
#include <spawn.h>
#include <signal.h>
#endif
#define DEBUG_WEB_SERVER 1
#define DLOG if (DEBUG_WEB_SERVER) fprintf
@ -279,8 +284,14 @@ void test_sleep(int millisec)
libtorrent::sleep(millisec);
}
struct proxy_t
{
pid_t pid;
int type;
};
// maps port to proxy type
static std::map<int, int> running_proxies;
static std::map<int, proxy_t> running_proxies;
void stop_proxy(int port)
{
@ -290,8 +301,8 @@ void stop_proxy(int port)
// calling stop_all_proxies().
}
// returns 0 on success
int async_run(char const* cmdline)
// returns 0 on failure, otherwise pid
pid_t async_run(char const* cmdline)
{
#ifdef _WIN32
char buf[2048];
@ -304,35 +315,57 @@ int async_run(char const* cmdline)
startup.hStdInput = GetStdHandle(STD_INPUT_HANDLE);
startup.hStdOutput= GetStdHandle(STD_OUTPUT_HANDLE);
startup.hStdError = GetStdHandle(STD_INPUT_HANDLE);
int ret = CreateProcessA(NULL, buf, NULL, NULL, TRUE, 0, NULL, NULL, &startup, &pi);
int ret = CreateProcessA(NULL, buf, NULL, NULL, TRUE, CREATE_NEW_PROCESS_GROUP, NULL, NULL, &startup, &pi);
if (ret == 0)
{
int error = GetLastError();
fprintf(stderr, "failed (%d) %s\n", error, error_code(error, get_system_category()).message().c_str());
return 0;
}
return ret == 0;
return pi.dwProcessId;
#else
char buf[2048];
snprintf(buf, sizeof(buf), "%s &", cmdline);
int ret = system(buf);
pid_t p;
char arg_storage[4096];
char* argp = arg_storage;
std::vector<char*> argv;
argv.push_back(argp);
for (char const* in = cmdline; *in != '\0'; ++in)
{
if (*in != ' ')
{
*argp++ = *in;
continue;
}
*argp++ = '\0';
argv.push_back(argp);
}
*argp = '\0';
argv.push_back(NULL);
int ret = posix_spawnp(&p, argv[0], NULL, NULL, &argv[0], NULL);
if (ret != 0)
{
fprintf(stderr, "failed (%d) %s\n", errno, strerror(errno));
return ret;
return 0;
}
return p;
#endif
}
void stop_all_proxies()
{
std::map<int, int> proxies = running_proxies;
for (std::map<int, int>::iterator i = proxies.begin()
std::map<int, proxy_t> proxies = running_proxies;
for (std::map<int, proxy_t>::iterator i = proxies.begin()
, end(proxies.end()); i != end; ++i)
{
char buf[100];
snprintf(buf, sizeof(buf), "delegated -P%d -Fkill", i->first);
int ret = async_run(buf);
if (ret == 0)
running_proxies.erase(i->first);
#ifdef _WIN32
GenerateConsoleCtrlEvent(CTRL_C_EVENT, i->second.pid);
#else
printf("killing pid: %d\n", i->second.pid);
kill(i->second.pid, SIGINT);
#endif
running_proxies.erase(i->second.pid);
}
}
@ -340,50 +373,55 @@ int start_proxy(int proxy_type)
{
using namespace libtorrent;
for (std::map<int, int>::iterator i = running_proxies.begin()
for (std::map<int, proxy_t>::iterator i = running_proxies.begin()
, end(running_proxies.end()); i != end; ++i)
{
if (i->second == proxy_type) return i->first;
if (i->second.type == proxy_type) return i->first;
}
int port = 10000 + (rand() % 50000);
char const* type = "";
char const* auth = "";
char const* cmd = "";
switch (proxy_type)
{
case proxy_settings::socks4:
type = "socks4";
auth = " --allow-v4";
cmd = "python ../socks.py";
break;
case proxy_settings::socks5:
type = "socks5";
cmd = "python ../socks.py";
break;
case proxy_settings::socks5_pw:
type = "socks5";
auth = "AUTHORIZER=-list{testuser:testpass}";
auth = " --username testuser --password testpass";
cmd = "python ../socks.py";
break;
case proxy_settings::http:
type = "http";
cmd = "python ../http.py";
break;
case proxy_settings::http_pw:
type = "http";
auth = "AUTHORIZER=-list{testuser:testpass}";
auth = " AUTHORIZER=-list{testuser:testpass}";
cmd = "python ../http.py";
break;
}
char buf[512];
snprintf(buf, sizeof(buf), "delegated -P%d ADMIN=test@test.com "
"PERMIT=\"*:*:localhost\" REMITTABLE=\"*\" RELAY=proxy,delegate "
"SERVER=%s %s"
, port, type, auth);
snprintf(buf, sizeof(buf), "%s --port %d%s", cmd, port, auth);
fprintf(stderr, "%s starting delegated proxy on port %d (%s %s)...\n", time_now_string(), port, type, auth);
fprintf(stderr, "%s starting socks proxy on port %d (%s %s)...\n", time_now_string(), port, type, auth);
fprintf(stderr, "%s\n", buf);
int r = async_run(buf);
if (r != 0) exit(1);
running_proxies.insert(std::make_pair(port, proxy_type));
if (r == 0) exit(1);
proxy_t t = { r, proxy_type };
running_proxies.insert(std::make_pair(port, t));
fprintf(stderr, "%s launched\n", time_now_string());
// apparently delegate takes a while to open its listen port
test_sleep(500);
return port;
}
@ -959,7 +997,7 @@ void send_content(socket_type& s, char const* file, int size, bool chunked)
else
{
write(s, boost::asio::buffer(file, size), boost::asio::transfer_all(), ec);
DLOG(stderr, " >> %s\n", std::string(file, size).c_str());
// DLOG(stderr, " >> %s\n", std::string(file, size).c_str());
if (ec) fprintf(stderr, "*** send failed: %s\n", ec.message().c_str());
}
}
@ -1204,14 +1242,6 @@ void web_server_thread(int* port, bool ssl, bool chunked)
std::string connection = p.header("connection");
std::string via = p.header("via");
// The delegate proxy doesn't say connection close, but it expects it to be closed
// the Via: header is an indicator of delegate making the request
if (connection == "close" || !via.empty())
{
DLOG(stderr, "*** got connection close\n");
connection_close = true;
}
if (p.protocol() == "HTTP/1.0")
{
DLOG(stderr, "*** HTTP/1.0, closing connection when done\n");

254
test/socks.py Normal file
View File

@ -0,0 +1,254 @@
#!/usr/bin/python
"""Minimal non-feature complete socks proxy"""
import logging
from logging import error, info, debug
import random
import socket
from SocketServer import StreamRequestHandler, ThreadingTCPServer
from struct import pack, unpack
import threading
import sys
class MyTCPServer(ThreadingTCPServer):
allow_reuse_address = True
CLOSE = object()
logging.basicConfig(filename='/dev/stderr', level=logging.INFO)
VERSION = '\x05'
NOAUTH = '\x00'
USERPASS = '\x02'
CONNECT = '\x01'
IPV4 = '\x01'
IPV6 = '\x04'
DOMAIN_NAME = '\x03'
SUCCESS = '\x00'
password = None
username = None
allow_v4 = False
def send(dest, msg):
if msg == CLOSE:
dest.shutdown(socket.SHUT_WR)
dest.close()
return 0
else:
return dest.sendall(msg)
def recv(source, buffer):
data = source.recv(buffer)
if data == '':
return CLOSE
else:
return data
def forward(source, dest, name):
while True:
data = recv(source, 4000)
if data == CLOSE:
send(dest, CLOSE)
info('%s hung up' % name)
return
debug('Sending (%d) %r' % (len(data), data))
send(dest, data)
def spawn_forwarder(source, dest, name):
t = threading.Thread(target=forward, args=(source, dest, name))
t.daemon = True
t.start()
class SocksHandler(StreamRequestHandler):
"""Highly feature incomplete SOCKS 5 implementation"""
def close_request(self):
self.server.close_request(self.request)
def read(self, n):
data = ''
while len(data) < n:
extra = self.rfile.read(n)
if extra == '':
raise Exception('Connection closed')
data += extra
return data
def handle(self):
# IMRPOVEMENT: Report who requests are from in logging
# IMPROVEMENT: Timeout on client
info('Connection - authenticating')
version = self.read(1)
if allow_v4 and version == '\x04':
cmd = self.read(1)
if cmd != CONNECT:
error('Only supports connect method not (%r) closing' % cmd)
self.close_request()
return
raw_dest_port = self.read(2)
dest_port, = unpack('>H', raw_dest_port)
raw_dest_address = self.read(4)
dest_address = '.'.join(map(str, unpack('>4B', raw_dest_address)))
user_id = ''
c = self.read(1)
while c != '\0':
user_id += c
c = self.read(1)
outbound_sock = socket.socket(socket.AF_INET)
out_address = socket.getaddrinfo(dest_address,dest_port)[0][4]
debug("Creating forwarder connection to %r", out_address)
outbound_sock.connect(out_address)
self.send_reply_v4(outbound_sock.getsockname())
spawn_forwarder(outbound_sock, self.request, 'destination')
forward(self.request, outbound_sock, 'client')
return
if version != '\x05':
error('Wrong version number (%r) closing...' % version)
self.close_request()
return
nmethods = ord(self.read(1))
method_list = self.read(nmethods)
global password
global username
if password == None and NOAUTH in method_list:
self.send_no_auth_method()
info('Authenticated (no-auth)')
elif USERPASS in method_list:
self.send_user_pass_auth_method()
auth_version = self.read(1)
if auth_version != '\x01':
error('Wrong sub-negotiation version number (%r) closing...' % version)
self.close_request()
return
usr_len = ord(self.read(1))
usr_name = self.read(usr_len)
pwd_len = ord(self.read(1))
pwd = self.read(pwd_len)
if usr_name != username or pwd != password:
error('Invalid username or password')
self.close_request()
return
info('Authenticated (user/password)')
self.send_authenticated()
else:
error('Server only supports NOAUTH and user/pass')
self.send_no_method()
return
# If we were authenticating it would go here
version, cmd, zero, address_type = self.read(4)
if version != '\x05':
error('Wrong version number (%r) closing...' % version)
self.close_request()
elif cmd != CONNECT:
error('Only supports connect method not (%r) closing' % cmd)
self.close_request()
elif zero != '\x00':
error('Mangled request. Reserved field (%r) is not null' % zero)
self.close_request()
if address_type == IPV4:
raw_dest_address = self.read(4)
dest_address = '.'.join(map(str, unpack('>4B', raw_dest_address)))
elif address_type == IPV6:
raw_dest_address = self.read(16)
dest_address = ":".join(map(lambda x: hex(x)[2:],unpack('>8H',raw_dest_address)))
elif address_type == DOMAIN_NAME:
dns_length = ord(self.read(1))
dns_name = self.read(dns_length)
dest_address = dns_name
else:
error('Unknown addressing (%r)' % address_type)
self.close_request()
raw_dest_port = self.read(2)
dest_port, = unpack('>H', raw_dest_port)
if address_type == IPV6:
outbound_sock = socket.socket(socket.AF_INET6)
else:
outbound_sock = socket.socket(socket.AF_INET)
out_address = socket.getaddrinfo(dest_address,dest_port)[0][4]
debug("Creating forwarder connection to %r", out_address)
outbound_sock.connect(out_address)
if address_type == IPV6:
self.send_reply6(outbound_sock.getsockname())
else:
self.send_reply(outbound_sock.getsockname())
spawn_forwarder(outbound_sock, self.request, 'destination')
forward(self.request, outbound_sock, 'client')
def send_reply_v4(self, (bind_addr, bind_port)):
self.wfile.write('\0\x5a\0\0\0\0\0\0')
self.wfile.flush()
def send_reply(self, (bind_addr, bind_port)):
bind_tuple = tuple(map(int, bind_addr.split('.')))
full_address = bind_tuple + (bind_port,)
info('Setting up forwarding port %r' % (full_address,))
msg = pack('>cccc4BH', VERSION, SUCCESS, '\x00', IPV4, *full_address)
self.wfile.write(msg)
def send_reply6(self, (bind_addr, bind_port, unused1, unused2)):
bind_tuple = tuple(map(lambda x: int(x,16), bind_addr.split(':')))
full_address = bind_tuple + (bind_port,)
info('Setting up forwarding port %r' % (full_address,))
msg = pack('>cccc8HH', VERSION, SUCCESS, '\x00', IPV6, *full_address)
self.wfile.write(msg)
def send_no_method(self):
self.wfile.write('\x05\xff')
self.close_request()
def send_no_auth_method(self):
self.wfile.write('\x05\x00')
self.wfile.flush()
def send_user_pass_auth_method(self):
self.wfile.write('\x05\x02')
self.wfile.flush()
def send_authenticated(self):
self.wfile.write('\x01\x00')
self.wfile.flush()
if __name__ == '__main__':
listen_port = 8002
i = 1
while i < len(sys.argv):
if sys.argv[i] == '--username':
username = sys.argv[i+1]
i += 1
elif sys.argv[i] == '--password':
password = sys.argv[i+1]
i += 1
elif sys.argv[i] == '--port':
listen_port = int(sys.argv[i+1])
i += 1
elif sys.argv[i] == '--allow-v4':
allow_v4 = True
else:
if sys.argv[i] != '--help': info('unknown option "%s"' % sys.argv[i])
print('usage: socks.py [--username <user> --password <password>] [--port <listen-port>]')
sys.exit(1)
i += 1
info('Listening on port %d...' % listen_port)
server = MyTCPServer(('localhost', listen_port), SocksHandler)
server.serve_forever()