diff --git a/documentation/ole.sgml b/documentation/ole.sgml
index d270917c510..ebb61506b9a 100644
--- a/documentation/ole.sgml
+++ b/documentation/ole.sgml
@@ -376,7 +376,7 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
- BASICS
+ Basics
The basic idea behind DCOM is to take a COM object and make it location
@@ -488,7 +488,7 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
- PROXIES AND STUBS
+ Proxies and Stubs
Manually marshalling and unmarshalling each method call using the NDR
@@ -535,7 +535,7 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
- INTERFACE MARSHALLING
+ Interface Marshalling
Standard NDR only knows about C style function calls - they
@@ -597,7 +597,7 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
- COM PROXY/STUB SYSTEM
+ COM Proxy/Stub System
COM proxies are objects that implement both the interfaces needing to be
@@ -611,8 +611,7 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
names. I'm not sure either, except that a running theme in DCOM is that
interfaces which have nothing to do with buffers have the word Buffer
appended to them, seemingly at random. Ignore it and don't let it
- confuse you
- :) This stuff is convoluted enough ...
+ confuse you :) This stuff is convoluted enough ...
@@ -621,8 +620,8 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
- DCOM is theoretically an internet RFC [2] and is
+ DCOM is theoretically an internet RFC
+ [2] and is
specced out, but in reality the only implementation of it apart from
ours is Microsoft's, and as a result there are lots of interfaces
which can be used if you want to customize or
@@ -673,7 +672,7 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
- RPC CHANNELS
+ RPC Channels
Remember the RPC runtime? Well, that's not just responsible for
@@ -718,7 +717,7 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
- HOW THIS ACTUALLY WORKS IN WINE
+ How this actually works in Wine
Right now, Wine does not use the NDR marshallers or RPC to implement its
@@ -743,7 +742,7 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
- TYPELIB MARSHALLER
+ Typelib Marshaller
In fact, the reason for the PSFactoryBuffer layer of indirection is
@@ -790,48 +789,321 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
- WRAPUP
+ Appartments
- OK, so there are some (very) basic notes on DCOM. There's a ton of stuff
- I have not covered:
+ Before a thread can use COM it must enter an apartment. Apartments are
+ an abstraction of a COM objects thread safety level. There are many types
+ of apartment but the only two we care about right now are single threaded
+ apartments (STAs) and the multi-threaded apartment (MTA).
+
+
+
+ Any given process may contain at most one MTA and potentially many STAs.
+ This is because all objects in MTAs never care where they are invoked from
+ and hence can all be treated the same. Since objects in STAs do care, they
+ cannot be treated the same.
+
+
+
+ You enter an apartment by calling CoInitializeEx() and
+ passing the desired thread model in as a parameter. The default if you use
+ the deprecated CoInitialize() is a STA, and this is the
+ most common type of apartment used in COM.
+
+
+
+ An object in the multi-threaded apartment may be accessed concurrently by
+ multiple threads: eg, it's supposed to be entirely thread safe. It must also
+ not care about thread-affinity, the object should react the same way no matter
+ which thread is calling it.
+
+
+
+ An object inside a STA does not have to be thread safe, and all calls upon it
+ should come from the same thread - the thread that entered the apartment in
+ the first place.
+
+
+
+ The apartment system was originally designed to deal with the disparity between
+ the Windows NT/C++ world in which threading was given a strong emphasis, and the
+ Visual Basic world in which threading was barely supported and even if it had
+ been fully supported most developers would not have used it. Visual Basic code
+ is not truly multi-threaded, instead if you start a new thread you get an entirely
+ new VM, with separate sets of global variables. Changes made in one thread do
+ not reflect in another, which pretty much violates the
+ expected semantics of multi-threading entirely but this is Visual Basic, so what
+ did you expect? If you access a VB object concurrently from multiple threads,
+ behind the scenes each VM runs in a STA and the calls are marshaled between the
+ threads using DCOM.
+
+
+
+ In the Windows 2000 release of COM, several new types of apartment were added, the
+ most important of which are RTAs (the rental threaded apartment) in which concurrent
+ access are serialised by COM using an apartment-wide lock but thread affinity is
+ not guaranteed.
+
+
+
+
+ Structure of a marshaled interface pointer
+
+
+ When an interface is marshaled using CoMarshalInterface(),
+ the result is a serialized OBJREF structure. An OBJREF actually contains a union,
+ but we'll be assuming the variant that embeds a STDOBJREF here which is what's
+ used by the system provided standard marshaling. A STDOBJREF (standard object
+ reference) consists of the magic signature 'MEOW', then some flags, then the IID
+ of the marshaled interface. Quite what MEOW stands for is a mystery, but it's
+ definitely not "Microsoft Extended Object Wire". Next comes the STDOBJREF flags,
+ identified by their SORF_ prefix. Most of these are reserved, and their purpose
+ (if any) is unknown, but a few are defined.
+
+
+
+ After the SORF flags comes a count of the references represented by this marshaled
+ interface. Typically this will be 5 in the case of a normal marshal, but may be 0
+ for table-strong and table-weak marshals (the difference between these is explained below).
+ The reasoning is this: In the general case, we want to know exactly when an object
+ is unmarshaled and released, so we can accurately control the lifetime of the stub
+ object. This is what happens when cPublicRefs is zero. However, in many cases, we
+ only want to unmarshal an object once. Therefore, if we strengthen the rules to say
+ when marshaling that we will only unmarshal once, then we no longer have to know when
+ it is unmarshaled. Therefore, we can give out an arbitrary number of references when
+ marshaling and basically say "don't call me, except when you die."
+
+
+
+ The most interesting part of a STDOBJREF is the OXID, OID, IPID triple. This triple
+ identifies any given marshaled interface pointer in the network. OXIDs are apartment
+ identifiers, and are supposed to be unique network-wide. How this is guaranteed is
+ currently unknown: the original algorithm Windows used was something like the current
+ UNIX time and a local counter.
+
+
+
+ OXIDs are generated and registered with the OXID resolver by performing local RPCs
+ to the RPC subsystem (rpcss.exe). In a fully security-patched Windows system they
+ appear to be randomly generated. This registration is done using the
+ ILocalOxidResolver interface, however the exact structure of
+ this interface is currently unknown.
+
+
+
+ OIDs are object identifiers, and identify a stub manager. The stub manager manages
+ interface stubs. For each exported COM object there are multiple interfaces and
+ therefore multiple interface stubs (IRpcStubBuffer implementations).
+ OIDs are apartment scoped. Each ifstub is identified by an IPID, which identifies
+ a marshaled interface pointer. IPIDs are apartment scoped.
+
+
+
+ Unmarshaling one of these streams therefore means setting up a connection to the
+ object exporter (the apartment holding the marshaled interface pointer) and being
+ able to send RPCs to the right ifstub. Each apartment has its own RPC endpoint and
+ calls can be routed to the correct interface pointer by embedding the IPID into the
+ call using RpcBindingSetObject. IRemUnknown, discussed below, uses a reserved IPID.
+ Please note that this is true only in the current implementation. The native version
+ generates an IPID as per any other object and simply notifies the SCM of this IPID.
+
+
+
+ Both standard and handler marshaled OBJREFs contains an OXID resolver endpoint which
+ is an RPC string binding in a DUALSTRINGARRAY. This is necessary because an OXID
+ alone is not enough to contact the host, as it doesn't contain any network address
+ data. Instead, the combination of the remote OXID resolver RPC endpoint and the OXID
+ itself are passed to the local OXID resolver. It then returns the apartment string binding.
+
+
+
+ This step is an optimisation: technically the OBJREF itself could contain the string
+ binding of the apartment endpoint and the OXID resolver could be bypassed, but by using
+ this DCOM can optimise out a server round-trip by having the local OXID resolver cache
+ the query results. The OXID resolver is a service in the RPC subsystem (rpcss.exe) which
+ implements a raw (non object-oriented) RPC interface called IOXIDResolver.
+ Despite the identical naming convention this is not a COM interface.
+
+
+
+ Unmarshaling an interface pointer stream therefore consists of
+ reading the OXID, OID and IPID from the STDOBJREF, then reading
+ one or more RPC string bindings for the remote OXID resolver.
+ Then RpcBindingFromStringBinding is used
+ to convert this remote string binding into an RPC binding handle
+ which can be passed to the local
+ IOXIDResolver::ResolveOxid implementation
+ along with the OXID. The local OXID resolver consults its list
+ of same-machine OXIDs, then its cache of remote OXIDs, and if
+ not found does an RPC to the remote OXID resolver using the
+ binding handle passed in earlier. The result of the query is
+ stored for future reference in the cache, and finally the
+ unmarshaling application gets back the apartment string binding,
+ the IPID of that apartments IRemUnknown
+ implementation, and a security hint (let's ignore this for now).
+
+
+
+ Once the remote apartments string binding has been located the
+ unmarshalling process constructs an RPC Channel Buffer
+ implementation with the connection handle and the IPID of the
+ needed interface, loads and constructs the
+ IRpcProxyBuffer implementation for that
+ IID and connects it to the channel. Finally the proxy is passed
+ back to the application.
+
+
+
+
+ Handling IUnknown
+
+
+ There are some subtleties here with respect to IUnknown. IUnknown
+ itself is never marshaled directly: instead a version of it
+ optimised for network usage is used. IRemUnknown is similar in
+ concept to IUnknown except that it allows you to add and release
+ arbitrary numbers of references at once, and it also allows you to
+ query for multiple interfaces at once.
+
+
+
+ IRemUnknown is used for lifecycle management, and for marshaling
+ new interfaces on an object back to the client. Its definition can
+ be seen in dcom.idl - basically the IRemUnknown::RemQueryInterface
+ method takes an IPID and a list of IIDs, then returns STDOBJREFs
+ of each new marshaled interface pointer.
+
+
+
+ There is one IRemUnknown implementation per apartment, not per
+ stub manager as you might expect. This is OK because IPIDs are
+ apartment not object scoped (In fact, according to the DCOM draft
+ spec, they are machine-scoped, but this implies apartment-scoped).
+
+
+
+
+ Table marshaling
+
+
+ Normally once you have unmarshaled a marshaled interface pointer
+ that stream is dead, you can't unmarshal it again. Sometimes this
+ isn't what you want. In this case, table marshaling can be used.
+ There are two types: strong and weak. In table-strong marshaling,
+ selected by a specific flag to CoMarshalInterface(),
+ a stream can be unmarshaled as many times as you like. Even if
+ all the proxies are released, the marshaled object reference is
+ still valid. Effectively the stream itself holds a ref on the object.
+ To release the object entirely so its server can shut down, you
+ must use CoReleaseMarshalData() on the stream.
+
+
+
+ In table-weak marshaling the stream can be unmarshaled many times,
+ however the stream does not hold a ref. If you unmarshal the
+ stream twice, once those two proxies have been released remote
+ object will also be released. Attempting to unmarshal the stream
+ at this point will yield CO_E_DISCONNECTED.
+
+
+
+
+ RPC dispatch
+
+
+ Exactly how RPC dispatch occurs depends on whether the exported
+ object is in a STA or the MTA. If it's in the MTA then all is
+ simple: the RPC dispatch thread can temporarily enter the MTA,
+ perform the remote call, and then leave it again. If it's in a
+ STA things get more complex, because of the requirement that only
+ one thread can ever access the object.
+
+
+
+ Instead, when entering a STA a hidden window is created implicitly
+ by COM, and the user must manually pump the message loop in order
+ to service incoming RPCs. The RPC dispatch thread performs the
+ context switch into the STA by sending a message to the apartments
+ window, which then proceeds to invoke the remote call in the right
+ thread.
+
+
+
+ RPC dispatch threads are pooled by the RPC runtime. When an incoming
+ RPC needs to be serviced, a thread is pulled from the pool and
+ invokes the call. The main RPC thread then goes back to listening
+ for new calls. It's quite likely for objects in the MTA to therefore
+ be servicing more than one call at once.
+
+
+
+
+ Message filtering and re-entrancy
+
+
+ When an outgoing call is made from a STA, it's possible that the
+ remote server will re-enter the client, for instance to perform a
+ callback. Because of this potential re-entrancy, when waiting for
+ the reply to an RPC made inside a STA, COM will pump the message loop.
+ That's because while this thread is blocked, the incoming callback
+ will be dispatched by a thread from the RPC dispatch pool, so it
+ must be processing messages.
+
+
+
+ While COM is pumping the message loop, all incoming messages from
+ the operating system are filtered through one or more message filters.
+ These filters are themselves COM objects which can choose to discard,
+ hold or forward window messages. The default message filter drops all
+ input messages and forwards the rest. This is so that if the user
+ chooses a menu option which triggers an RPC, they then cannot choose
+ that menu option *again* and restart the function from the beginning.
+ That type of unexpected re-entrancy is extremely difficult to debug,
+ so it's disallowed.
+
+
+
+ Unfortunately other window messages are allowed through, meaning that
+ it's possible your UI will be required to repaint itself during an
+ outgoing RPC. This makes programming with STAs more complex than it
+ may appear, as you must be prepared to run all kinds of code any time
+ an outgoing call is made. In turn this breaks the idea that COM
+ should abstract object location from the programmer, because an
+ object that was originally free-threaded and is then run from a STA
+ could trigger new and untested codepaths in a program.
+
+
+
+
+ Wrapup
+
+
+ Theres are still a lot of topics that have not been covered:
Format strings/MOPs
- Apartments, threading models, inter-thread marshalling
-
- OXIDs/OIDs, etc, IOXIDResolver
-
IRemoteActivation
Complex/simple pings, distributed garbage collection
Marshalling IDispatch
- Structure of marshalled interface pointers (STDOBJREFs etc)
+ ICallFrame
+
+ Interface pointer swizzling
Runtime class object registration (CoRegisterClassObject), ROT
- IRemUnknown
-
Exactly how InstallShield uses DCOM
-
- Then there's a bunch of stuff I still don't understand, like ICallFrame,
- interface pointer swizzling, exactly where and how all this stuff is
- actually implemented and so on.
-
-
-
- But for now that's enough.
-
- FURTHER READING
+ Further Reading
Most of these documents assume you have knowledge only contained in
@@ -842,6 +1114,12 @@ static ICOM_VTABLE(IDirect3D) d3dvt = {
+
+
+ http://www-csag.ucsd.edu/individual/achien/cs491-f97/projects/dcom-writeup.ps
+
+
+
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/com/htm/cmi_n2p_459u.asp