verify that torrent names matches the DNS name in its certificate (RFC 2818-style). Fix issues that was breaking SSL support and tidy up a bit

docs/manual.html

@@ -339,7 +339,10 @@
 <li><a class="reference internal" href="#prefer-whole-pieces" id="id260">prefer whole pieces</a></li>
 </ul>
 </li>
-<li><a class="reference internal" href="#ssl-torrents" id="id261">SSL torrents</a></li>
+<li><a class="reference internal" href="#ssl-torrents" id="id261">SSL torrents</a><ul>
+<li><a class="reference internal" href="#testing" id="id262">testing</a></li>
+</ul>
+</li>
 </ul>
 </div>
 <div class="section" id="overview">
@@ -8684,7 +8687,7 @@ as much space as has been downloaded.</p>
 </div>
 <div class="section" id="full-allocation">
 <h2>full allocation</h2>
-<p>When a torrent is started in full allocation mode, the disk-io thread (see <a href="#id262"><span class="problematic" id="id263">threads_</span></a>)
+<p>When a torrent is started in full allocation mode, the disk-io thread (see <a href="#id263"><span class="problematic" id="id264">threads_</span></a>)
 will make sure that the entire storage is allocated, and fill any gaps with zeros.
 This will be skipped if the filesystem supports sparse files or automatic zero filling.
 It will of course still check for existing pieces and fast resume data. The main
@@ -9078,6 +9081,14 @@ different port. It defaults to port 4433. This setting is only taken into accoun
 normal listen socket is opened (i.e. just changing this setting won't necessarily close
 and re-open the SSL socket). To not listen on an SSL socket at all, set <tt class="docutils literal"><span class="pre">ssl_listen</span></tt> to 0.</p>
 <p>This feature is only available if libtorrent is build with openssl support (<tt class="docutils literal"><span class="pre">TORRENT_USE_OPENSSL</span></tt>).</p>
+<p>Peer certificates must have at least one <em>SubjectAltName</em> field of type dNSName. At least
+one of the fields must <em>exactly</em> match the name of the torrent. This is a byte-by-byte comparison,
+the UTF-8 encoding must be identical (i.e. there's no unicode normalization going on). This
+the recommended way of verifying certificates for HTTPS servers according to <a class="reference external" href="http://www.ietf.org/rfc/rfc2818.txt">RFC 2818</a>. Note
+the difference that for torrents only <em>dNSName</em> fields are taken into account (not IP address fields)
+and that only <em>SubjectAltNames</em> are taken into account, not the <em>Common Name</em> fields.</p>
+<div class="section" id="testing">
+<h2>testing</h2>
 <p>To test incoming SSL connections to an SSL torrent, one can use the following <em>openssl</em> command:</p>
 <pre class="literal-block">
 openssl s_client -cert &lt;peer-certificate&gt;.pem -key &lt;peer-private-key&gt;.pem -CAfile &lt;torrent-cert&gt;.pem -debug -connect 127.0.0.1:4433 -tls1 -servername &lt;info-hash&gt;
@@ -9097,10 +9108,11 @@ the pem file to include in the .torrent file.</p>
 <p>The peer's certificate is located in <tt class="docutils literal"><span class="pre">./newcert.pem</span></tt> and the certificate's
 private key in <tt class="docutils literal"><span class="pre">./newkey.pem</span></tt>.</p>
 </div>
+</div>
 <div class="system-messages section">
 <h1>Docutils System Messages</h1>
-<div class="system-message" id="id262">
+<div class="system-message" id="id263">
-<p class="system-message-title">System Message: ERROR/3 (<tt class="docutils">manual.rst</tt>, line 8686); <em><a href="#id263">backlink</a></em></p>
+<p class="system-message-title">System Message: ERROR/3 (<tt class="docutils">manual.rst</tt>, line 8686); <em><a href="#id264">backlink</a></em></p>
 Unknown target name: &quot;threads&quot;.</div>
 </div>
     </div>

docs/manual.rst

@@ -9057,7 +9057,29 @@ different port. It defaults to port 4433. This setting is only taken into accoun
 normal listen socket is opened (i.e. just changing this setting won't necessarily close
 and re-open the SSL socket). To not listen on an SSL socket at all, set ``ssl_listen`` to 0.
 
-This feature is only available if libtorrent is build with openssl support (``TORRENT_USE_OPENSSL``).
+This feature is only available if libtorrent is build with openssl support (``TORRENT_USE_OPENSSL``)
+and requires at least openSSL version 1.0, since it needs SNI support.
+
+Peer certificates must have at least one *SubjectAltName* field of type dNSName. At least
+one of the fields must *exactly* match the name of the torrent. This is a byte-by-byte comparison,
+the UTF-8 encoding must be identical (i.e. there's no unicode normalization going on). This is
+the recommended way of verifying certificates for HTTPS servers according to `RFC 2818`_. Note
+the difference that for torrents only *dNSName* fields are taken into account (not IP address fields).
+The most specific (i.e. last) *Common Name* field is also taken into account if no *SubjectAltName*
+did not match.
+
+If any of these fields contain a single asterisk ("*"), the certificate is considered covering
+any torrent, allowing it to be reused for any torrent.
+
+The purpose of matching the torrent name with the fields in the peer certificate is to allow
+a publisher to have a single root certificate for all torrents it distributes, and issue
+separate peer certificates for each torrent. A peer receiving a certificate will not necessarily
+be able to access all torrents published by this root certificate (only if it has a "star cert").
+
+.. _`RFC 2818`: http://www.ietf.org/rfc/rfc2818.txt
+
+testing
+-------
 
 To test incoming SSL connections to an SSL torrent, one can use the following *openssl* command::
 

examples/client_test.cpp

@@ -1098,6 +1098,7 @@ int main(int argc, char* argv[])
 			"  -S <limit>            limits the upload slots\n"
 			"  -A <num pieces>       allowed pieces set size\n"
 			"  -H                    Don't start DHT\n"
+			"  -X                    Don't start local peer discovery\n"
 			"  -n                    announce to trackers in all tiers\n"
 			"  -W <num peers>        Set the max number of peers to keep in the peer list\n"
 			"  -B <seconds>          sets the peer timeout\n"
@@ -1159,6 +1160,7 @@ int main(int argc, char* argv[])
 	int refresh_delay = 1000;
 	bool start_dht = true;
 	bool start_upnp = true;
+	bool start_lsd = true;
 	int loop_limit = 0;
 
 	std::deque<std::string> events;
@@ -1364,6 +1366,7 @@ int main(int argc, char* argv[])
 				break;
 			case 'I': outgoing_interface = arg; break;
 			case 'N': start_upnp = false; --i; break;
+			case 'X': start_lsd = false; --i; break;
 			case 'Y': settings.ignore_limits_on_local_network = false; --i; break;
 			case 'v': settings.active_downloads = atoi(arg);
 				settings.active_limit = (std::max)(atoi(arg) * 2, settings.active_limit);
@@ -1383,7 +1386,9 @@ int main(int argc, char* argv[])
 	if (ec)
 		fprintf(stderr, "failed to create resume file directory: %s\n", ec.message().c_str());
 
+	if (start_lsd)
 		ses.start_lsd();
+
 	if (start_upnp)
 	{
 		ses.start_upnp();

include/libtorrent/socket_type.hpp

@@ -181,6 +181,8 @@ namespace libtorrent
 		io_service& get_io_service() const;
 		bool is_open() const;
 
+		char const* type_name() const;
+
 #ifndef BOOST_NO_EXCEPTIONS
 		void open(protocol_type const& p);
 		void close();
@@ -196,7 +198,7 @@ namespace libtorrent
 		endpoint_type remote_endpoint(error_code& ec) const;
 		void bind(endpoint_type const& endpoint, error_code& ec);
 		std::size_t available(error_code& ec) const;
-		int type();
+		int type() const;
 
 
 		template <class Mutable_Buffers>
@@ -292,6 +294,9 @@ namespace libtorrent
 
 		size_type m_data[(storage_size + sizeof(size_type) - 1) / sizeof(size_type)];
 	};
+
+	bool is_ssl(socket_type const& s);
+
 }
 
 #endif

include/libtorrent/torrent.hpp

@@ -947,6 +947,7 @@ namespace libtorrent
 #ifdef TORRENT_USE_OPENSSL
 		boost::shared_ptr<asio::ssl::context> m_ssl_ctx;
 
+		bool verify_peer_cert(bool preverified, boost::asio::ssl::verify_context& ctx);
 		void init_ssl(std::string const& cert);
 #endif
 

src/bt_peer_connection.cpp

@@ -229,7 +229,7 @@ namespace libtorrent
 		if (key.size() == 32) out_enc_policy = pe_settings::disabled;
 
 		// never try an encrypted connection when already using SSL
-		if (get_socket()->get<ssl_stream<stream_socket> >() || get_socket()->get<ssl_stream<utp_stream> >())
+		if (is_ssl(*get_socket()))
 			out_enc_policy = pe_settings::disabled;
 #endif
 #ifdef TORRENT_VERBOSE_LOGGING
@@ -2946,8 +2946,7 @@ namespace libtorrent
 #endif
 
 #ifdef TORRENT_USE_OPENSSL
-				if (get_socket()->get<ssl_stream<stream_socket> >()
+				if (is_ssl(*get_socket()))
-					|| get_socket()->get<ssl_stream<utp_stream> >())
 				{
 #ifdef TORRENT_VERBOSE_LOGGING
 					peer_log("*** SSL peers are not allowed to use any other encryption");
@@ -3030,9 +3029,10 @@ namespace libtorrent
 #ifndef TORRENT_DISABLE_ENCRYPTION
 				TORRENT_ASSERT(m_state != read_pe_dhkey);
 
-				if (!is_local() && 
+				if (!is_local()
-					(m_ses.get_pe_settings().in_enc_policy == pe_settings::forced) &&
+					&& m_ses.get_pe_settings().in_enc_policy == pe_settings::forced
-					!m_encrypted) 
+					&& !m_encrypted
+					&& !is_ssl(*get_socket()))
 				{
 					disconnect(errors::no_incoming_regular);
 					return;

src/peer_connection.cpp

@@ -235,16 +235,12 @@ namespace libtorrent
 		error_code ec;
 		m_logger = m_ses.create_log(m_remote.address().to_string(ec) + "_"
 			+ to_string(m_remote.port()).elems, m_ses.listen_port());
-		peer_log(">>> %s [ ep: %s transport: %s seed: %d p: %p ]"
+		peer_log("%s [ ep: %s type: %s seed: %d p: %p local: %s]"
-			, outgoing ? "OUTGOING_CONNECTION" : "INCOMING CONNECTION"
+			, outgoing ? ">>> OUTGOING_CONNECTION" : "<<< INCOMING CONNECTION"
 			, print_endpoint(m_remote).c_str()
-			,
+			, m_socket->type_name()
-#ifdef TORRENT_USE_OPENSSL
+			, m_peer_info ? m_peer_info->seed : 0, m_peer_info
-				m_socket->get<ssl_stream<stream_socket> >() ? "SSL/TCP" :
+			, print_endpoint(m_socket->local_endpoint(ec)).c_str());
-				m_socket->get<ssl_stream<utp_stream> >() ? "SSL/uTP" :
-#endif
-				m_socket->get<utp_stream>() ? "uTP" : "TCP"
-			, m_peer_info ? m_peer_info->seed : 0, m_peer_info);
 #endif
 #ifdef TORRENT_DEBUG
 		piece_failed = false;
@@ -387,10 +383,11 @@ namespace libtorrent
 		TORRENT_ASSERT(m_socket->remote_endpoint(ec) == m_remote || ec);
 		m_logger = m_ses.create_log(remote().address().to_string(ec) + "_"
 			+ to_string(remote().port()).elems, m_ses.listen_port());
-		peer_log("<<< %s [ ep: %s transport: %s ]"
+		peer_log("%s [ ep: %s type: %s local: %s]"
-			, outgoing ? "OUTGOING_CONNECTION" : "INCOMING CONNECTION"
+			, outgoing ? ">>> OUTGOING_CONNECTION" : "<<< INCOMING CONNECTION"
 			, print_endpoint(m_remote).c_str()
-			, (m_socket->get<utp_stream>()) ? "uTP connection" : "TCP connection");
+			, m_socket->type_name()
+			, print_endpoint(m_socket->local_endpoint(ec)).c_str());
 #endif
 		
 #ifndef TORRENT_DISABLE_GEO_IP
@@ -5379,7 +5376,7 @@ namespace libtorrent
 #endif
 		m_socket->async_connect(m_remote
 			, boost::bind(&peer_connection::on_connection_complete, self(), _1));
-		m_connect = time_now();
+		m_connect = time_now_hires();
 		m_statistics.sent_syn(m_remote.address().is_v6());
 
 		if (t->alerts().should_post<peer_connect_alert>())
@@ -5387,6 +5384,9 @@ namespace libtorrent
 			t->alerts().post_alert(peer_connect_alert(
 				t->get_handle(), remote(), pid()));
 		}
+#if defined TORRENT_VERBOSE_LOGGING
+		peer_log("*** LOCAL ENDPOINT[ e: %s ]", print_endpoint(m_socket->local_endpoint(ec)).c_str());
+#endif
 	}
 	
 	void peer_connection::on_connection_complete(error_code const& e)
@@ -5394,7 +5394,7 @@ namespace libtorrent
 #if defined TORRENT_ASIO_DEBUGGING
 		complete_async("peer_connection::on_connection_complete");
 #endif
-		ptime completed = time_now();
+		ptime completed = time_now_hires();
 
 		TORRENT_ASSERT(m_ses.is_network_thread());
 
@@ -5402,6 +5402,13 @@ namespace libtorrent
 
 		m_rtt = total_milliseconds(completed - m_connect);
 
+#ifdef TORRENT_USE_OPENSSL
+		// add this RTT to the PRNG seed, to add more unpredictability
+		boost::uint64_t now = total_microseconds(completed - m_connect);
+		// assume 12 bits of entropy (i.e. about 8 milliseconds)
+		RAND_add(&now, 8, 1.5);
+#endif
+		
 		if (m_disconnecting) return;
 
 		error_code ec;

src/session_impl.cpp

@@ -2538,7 +2538,7 @@ namespace aux {
 
 #if defined(TORRENT_VERBOSE_LOGGING) || defined(TORRENT_LOGGING)
 		(*m_logger) << time_now_string() << " *** peer SSL handshake done [ ip: "
-			<< endp << " ec: " << ec.message() << "]\n";
+			<< endp << " ec: " << ec.message() << " socket: " << s->type_name() << "]\n";
 #endif
 
 		if (ec)
@@ -2560,6 +2560,13 @@ namespace aux {
 	{
 		TORRENT_ASSERT(is_network_thread());
 
+#ifdef TORRENT_USE_OPENSSL
+		// add the current time to the PRNG, to add more unpredictability
+		boost::uint64_t now = total_microseconds(time_now_hires() - min_time());
+		// assume 12 bits of entropy (i.e. about 8 milliseconds)
+		RAND_add(&now, 8, 1.5);
+#endif
+
 		if (m_paused)
 		{
 #if defined(TORRENT_VERBOSE_LOGGING) || defined(TORRENT_LOGGING)
@@ -2584,7 +2591,8 @@ namespace aux {
 		TORRENT_ASSERT(endp.address() != address_v4::any());
 
 #if defined(TORRENT_VERBOSE_LOGGING) || defined(TORRENT_LOGGING)
-		(*m_logger) << time_now_string() << " <== INCOMING CONNECTION " << endp << "\n";
+		(*m_logger) << time_now_string() << " <== INCOMING CONNECTION " << endp
+			<< " type: " << s->type_name() << "\n";
 #endif
 
 		if (m_alerts.should_post<incoming_connection_alert>())

src/socket_type.cpp

@@ -39,6 +39,25 @@ POSSIBILITY OF SUCH DAMAGE.
 namespace libtorrent
 {
 
+	bool is_ssl(socket_type const& s)
+	{
+#ifdef TORRENT_USE_OPENSSL
+#define CASE(t) case socket_type_int_impl<ssl_stream<t> >::value:
+		switch (s.type())
+		{
+			CASE(stream_socket)
+			CASE(socks5_stream)
+			CASE(http_stream)
+			CASE(utp_stream)
+				return true;
+			default: return false;
+		};
+#undef CASE
+#else
+		return false;
+#endif
+	}
+
 	void socket_type::destruct()
 	{
 		switch (m_type)
@@ -131,6 +150,32 @@ namespace libtorrent
 		m_type = type;
 	}
 
+	char const* socket_type::type_name() const
+	{
+		static char const* const names[] =
+		{
+			"uninitialized",
+			"TCP",
+			"Socks5",
+			"HTTP",
+			"uTP",
+#if TORRENT_USE_I2P
+			"I2P",
+#else
+			"",
+#endif
+#ifdef TORRENT_USE_OPENSSL
+			"SSL/TCP",
+			"SSL/Socks5",
+			"SSL/HTTP",
+			"SSL/uTP"
+#else
+			"","","",""
+#endif
+		};
+		return names[m_type];
+	}
+
 	io_service& socket_type::get_io_service() const
 	{ return m_io_service; }
 
@@ -164,7 +209,7 @@ namespace libtorrent
 	std::size_t socket_type::available(error_code& ec) const
 	{ TORRENT_SOCKTYPE_FORWARD_RET(available(ec), 0) }
 
-	int socket_type::type() { return m_type; }
+	int socket_type::type() const { return m_type; }
 
 #ifndef BOOST_NO_EXCEPTIONS
 	void socket_type::open(protocol_type const& p)

src/torrent.cpp

@@ -1285,12 +1285,92 @@ namespace libtorrent
 #endif
 
 #ifdef TORRENT_USE_OPENSSL
-/*
+
-	bool verify_function(bool preverified, boost::asio::ssl::verify_context& ctx)
+	bool torrent::verify_peer_cert(bool preverified, boost::asio::ssl::verify_context& ctx)
 	{
+		// if the cert wasn't signed by the correct CA, fail the verification
+		if (!preverified) return false;
+
+		// we're only interested in checking the certificate at the end of the chain.
+		int depth = X509_STORE_CTX_get_error_depth(ctx.native_handle());
+		if (depth > 0) return true;
+
+		X509* cert = X509_STORE_CTX_get_current_cert(ctx.native_handle());
+
+		// Go through the alternate names in the certificate looking for matching DNS entries
+		GENERAL_NAMES* gens = static_cast<GENERAL_NAMES*>(
+			X509_get_ext_d2i(cert, NID_subject_alt_name, 0, 0));
+
+#if defined(TORRENT_VERBOSE_LOGGING) || defined(TORRENT_LOGGING)
+		std::string names;
+		bool match = false;
+#endif
+		for (int i = 0; i < sk_GENERAL_NAME_num(gens); ++i)
+		{
+			GENERAL_NAME* gen = sk_GENERAL_NAME_value(gens, i);
+			if (gen->type != GEN_DNS) continue;
+			ASN1_IA5STRING* domain = gen->d.dNSName;
+			if (domain->type != V_ASN1_IA5STRING || !domain->data || !domain->length) continue;
+			const char* torrent_name = reinterpret_cast<const char*>(domain->data);
+			std::size_t name_length = domain->length;
+
+#if defined(TORRENT_VERBOSE_LOGGING) || defined(TORRENT_LOGGING)
+			if (i > 1) names += " | n: ";
+			names.append(torrent_name, name_length);
+#endif
+			if (strncmp(torrent_name, "*", name_length) == 0
+				|| strncmp(torrent_name, m_torrent_file->name().c_str(), name_length) == 0)
+			{
+#if defined(TORRENT_VERBOSE_LOGGING) || defined(TORRENT_LOGGING)
+				match = true;
+				// if we're logging, keep looping over all names,
+				// for completeness of the log
+				continue;
+#endif
 				return true;
 			}
-*/
+		}
+
+		// no match in the alternate names, so try the common names. We should only
+		// use the "most specific" common name, which is the last one in the list.
+		X509_NAME* name = X509_get_subject_name(cert);
+		int i = -1;
+		ASN1_STRING* common_name = 0;
+		while ((i = X509_NAME_get_index_by_NID(name, NID_commonName, i)) >= 0)
+		{
+			X509_NAME_ENTRY* name_entry = X509_NAME_get_entry(name, i);
+			common_name = X509_NAME_ENTRY_get_data(name_entry);
+		}
+		if (common_name && common_name->data && common_name->length)
+		{
+			const char* torrent_name = reinterpret_cast<const char*>(common_name->data);
+			std::size_t name_length = common_name->length;
+
+#if defined(TORRENT_VERBOSE_LOGGING) || defined(TORRENT_LOGGING)
+			if (!names.empty()) names += " | n: ";
+			names.append(torrent_name, name_length);
+#endif
+
+			if (strncmp(torrent_name, "*", name_length) == 0
+				|| strncmp(torrent_name, m_torrent_file->name().c_str(), name_length) == 0)
+			{
+#if !defined(TORRENT_VERBOSE_LOGGING) && !defined(TORRENT_LOGGING)
+				return true;
+#else
+				match = true;
+#endif
+			}
+		}
+
+#if defined(TORRENT_VERBOSE_LOGGING) || defined(TORRENT_LOGGING)
+		(*m_ses.m_logger) << time_now_string() << " <== INCOMING SSL CONNECTION [ torrent: "
+			<< m_torrent_file->name() << " | n: " << names << " | match: " << (match?"yes":"no")
+			<<  " ]\n";
+		return match;
+#endif
+
+		return false;
+	}
 
 	void torrent::init_ssl(std::string const& cert)
 	{
@@ -1299,8 +1379,12 @@ namespace libtorrent
 		// this is needed for openssl < 1.0 to decrypt keys created by openssl 1.0+
 		OpenSSL_add_all_algorithms();
 
-		// TODO: come up with something better
+		boost::uint64_t now = total_microseconds(time_now_hires() - min_time());
-		RAND_seed(&info_hash()[0], 20);
+		// assume 9 bits of entropy (i.e. about 1 millisecond)
+		RAND_add(&now, 8, 1.125);
+		RAND_add(&info_hash()[0], 20, 3);
+		// entropy is also added on incoming and completed connection attempts
+
 		TORRENT_ASSERT(RAND_status() == 1);
 
 		// create the SSL context for this torrent. We need to
@@ -1331,17 +1415,17 @@ namespace libtorrent
 			return;
 		}
 
-		// this is used for debugging			
+		// the verification function verifies the distinguished name
-		/*
+		// of a peer certificate to make sure it matches the info-hash
-#error there's a bug where the async_handshake on the ssl_stream always succeeds, regardless of the certificate failing. It's not a trivial bug in asio, that's been tested with a small repro program.
+		// of the torrent, or that it's a "star-cert"
-		ctx->set_verify_callback(verify_function, ec);
+		ctx->set_verify_callback(boost::bind(&torrent::verify_peer_cert, this, _1, _2), ec);
 		if (ec)
 		{
 			set_error(ec, "SSL verify callback");
 			pause();
 			return;
 		}
-		*/
+
 		SSL_CTX* ssl_ctx = ctx->impl();
 		// create a new x.509 certificate store
 		X509_STORE* cert_store = X509_STORE_new();