Sweden-Number/documentation/architecture.sgml

  <chapter id="architecture">
    <title>Overview</title>
    <para>Brief overview of Wine's architecture...</para>

    <sect1 id="basic-overview">
      <title>Wine Overview</title>

      <para>
        With the fundamental architecture of Wine stabilizing, and
        people starting to think that we might soon be ready to
        actually release this thing, it may be time to take a look at
        how Wine actually works and operates.
      </para>

      <sect2>
        <title>Foreword</title>
        <para>
          Wine is often used as a recursive acronym, standing for
          "Wine Is Not an Emulator". Sometimes it is also known to be
          used for "Windows Emulator". In a way, both meanings are
          correct, only seen from different perspectives. The first
          meaning says that Wine is not a virtual machine, it does not
          emulate a CPU, and you are not supposed to install 
          Windows nor any Windows device drivers on top of it; rather,
          Wine is an implementation of the Windows API, and can be
          used as a library to port Windows applications to Unix. The
          second meaning, obviously, is that to Windows binaries
          (<filename>.exe</filename> files), Wine does look like
          Windows, and emulates its behaviour and quirks rather
          closely.
        </para>
        <note>
          <title>"Emulator"</title>
          <para>
            The "Emulator" perspective should not be thought of as if
            Wine is a typical inefficient emulation layer that means
            Wine can't be anything but slow - the faithfulness to the
            badly designed Windows API may of course impose a minor
            overhead in some cases, but this is both balanced out by
            the higher efficiency of the Unix platforms Wine runs on,
            and that other possible abstraction libraries (like Motif,
            GTK+, CORBA, etc) has a runtime overhead typically
            comparable to Wine's.
          </para>
        </note>
      </sect2>

      <sect2>
	<title>Executables</title>
	<para>
	  Wine's main task is to run Windows executables under non
	  Windows operating systems. It supports different types of
	  executables:
	  <itemizedlist>
	    <listitem>
	      <para>
		DOS executable. Those are even older programs, using
		the DOS format (either <filename>.com</filename> or
		<filename>.exe</filename> (the later being also called
		MZ)).
	      </para>
	    </listitem>
	    <listitem>
	      <para>
		Windows NE executable, also called 16 bit. They were
		the native processes run by Windows 2.x and 3.x. NE
		stands for New Executable &lt;g&gt;.
	      </para>
	    </listitem>
	    <listitem>
	      <para>
		Windows PE executable. These are programs were
		introduced in Windows 95 (and became the native
		formats for all later Windows version), even if 16 bit
		applications were still supported. PE stands for
		Portable Executable, in a sense where the format of
		the executable (as a file) is independent of the CPU
		(even if the content of the file - the code - is CPU
		dependent). 
	      </para>
	    </listitem>
	    <listitem>
	      <para>
		WineLib executable. These are applications, written
		using the Windows API, but compiled as a Unix
		executable. Wine provides the tools to create such
		executables.
	      </para>
	    </listitem>
	  </itemizedlist>
	</para>
	<para>
	  Let's quickly review the main differences for the supported
	  executables:
	  <table>
	    <title>Wine executables</title>
	    <tgroup cols="5" align="left" colsep="1" rowsep="1">
	      <thead>
		<row>
		  <entry></entry>
		  <entry>DOS (.COM or .EXE)</entry>
		  <entry>Win16 (NE)</entry>
		  <entry>Win32 (PE)</entry>
		  <entry>WineLib</entry>
		</row>
	      </thead>
	      <tbody>
		<row>
		  <entry>Multitasking</entry>
		  <entry>Only one application at a time (except for TSR)</entry>
		  <entry>Cooperative</entry>
		  <entry>Preemptive</entry>
		  <entry>Preemptive</entry>
		</row>
		<row>
		  <entry>Address space</entry>
		  <entry>
		    One MB of memory, where each application is loaded
		    and unloaded.
		  </entry>
		  <entry>
		    All 16 bit applications share a single address
		    space, protected mode.
		  </entry>
		  <entry>
		    Each application has it's own address
		    space. Requires MMU support from CPU. 
		  </entry>
		  <entry>
		    Each application has it's own address
		    space. Requires MMU support from CPU. 
		  </entry>
		</row>
		<row>
		  <entry>Windows API</entry>
		  <entry>
		    No Windows API but the DOS API (like <function>Int
		      21h</function> traps).
		  </entry>
		  <entry>
		    Will call the 16 bit Windows API.
		  </entry>
		  <entry>
		    Will call the 32 bit Windows API.
		  </entry>
		  <entry>
		    Will call the 32 bit Windows API, and possibly
		    also the Unix APIs.
		  </entry>
		</row>
		<row>
		  <entry>Code (CPU level)</entry>
		  <entry>
		    Only available on x86 in real mode. Code and data
		    are in segmented forms, with 16 bit
		    offsets. Processor is in real mode.
		  </entry>
		  <entry>
		    Only available on IA-32 architectures, code and
		    data are in segmented forms, with 16 bit offsets
		    (hence the 16 bit name). Processor is in protected
		    mode.
		  </entry>
		  <entry>
		    Available (with NT) on several CPUs, including
		    IA-32. On this CPU, uses a flat memory model with
		    32 bit offsets (hence the 32 bit name).
		  </entry>
		  <entry>
		    Flat model, with 32 bit addresses.
		  </entry>
		</row>
		<row>
		  <entry>Multi-threading</entry>
		  <entry>Not available.</entry>
		  <entry>Not available.</entry>
		  <entry>
		    Available.
		  </entry>
		  <entry>
		    Available, but must use the Win32 APIs for
		    threading and synchronization, not the Unix ones.
		  </entry>
		</row>
	      </tbody>
	    </tgroup>
	  </table>
	</para>

        <para>
          Wine deals with this issue by launching a separate Wine
          process (which is in fact a Unix process) for each Win32
	  process, but not for Win16 tasks. Win16 tasks (as well as
	  DOS programs) are run as different intersynchronized
	  Unix-threads in the same dedicated Wine process; this Wine
	  process is commonly known as a <firstterm>WOW</firstterm>
	  process (Windows on Windows), referring to a similar
	  mechanism used by Windows NT.
	</para>
	<para>
          Synchronization between the Win16 tasks running in the WOW
          process is normally done through the Win16 mutex - whenever
          one of them is running, it holds the Win16 mutex, keeping
          the others from running. When the task wishes to let the
          other tasks run, the thread releases the Win16 mutex, and
          one of the waiting threads will then acquire it and let its
          task run.
        </para>
      </sect2>
    </sect1>

    <sect1>
      <title>Standard Windows Architectures</title>

      <sect2>
	<title>Windows 9x architecture</title>

	<para>
	  The windows architecture (Win 9x way) looks like this:
	  <screen>
+---------------------+        \
|     Windows EXE     |         } application
+---------------------+        /

+---------+ +---------+        \
| Windows | | Windows |         \ application & system DLLs
|   DLL   | |   DLL   |         /
+---------+ +---------+        /

+---------+ +---------+      \
|  GDI32  | | USER32  |       \
|   DLL   | |   DLL   |        \
+---------+ +---------+         } core system DLLs
+---------------------+        /
|     Kernel32 DLL    |       /
+---------------------+      / 

+---------------------+        \
|     Win9x kernel    |         } kernel space
+---------------------+        /

+---------------------+       \
|  Windows low-level  |        \ drivers (kernel space)
|       drivers       |        /
+---------------------+       /
	  </screen>
	</para>
      </sect2>
      <sect2>
	<title>Windows NT architecture</title>

	<para>
	  The windows architecture (Windows NT way) looks like the
	  following drawing. Note the new DLL (NTDLL) which allows
	  implementing different subsystems (as win32); kernel32 in NT
	  architecture implements the Win32 subsystem on top of NTDLL.
	  <screen>
+---------------------+                      \
|     Windows EXE     |                       } application
+---------------------+                      /

+---------+ +---------+                      \
| Windows | | Windows |                       \ application & system DLLs
|   DLL   | |   DLL   |                       /
+---------+ +---------+                      /

+---------+ +---------+     +-----------+   \
|  GDI32  | |  USER32 |     |           |    \
|   DLL   | |   DLL   |     |           |     \
+---------+ +---------+     |           |      \ core system DLLs
+---------------------+     |           |      / (on the left side)
|    Kernel32 DLL     |     | Subsystem |     /
|  (Win32 subsystem)  |     |Posix, OS/2|    /
+---------------------+     +-----------+   / 

+---------------------------------------+
|               NTDLL.DLL               |
+---------------------------------------+

+---------------------------------------+     \
|               NT kernel               |      } NT kernel (kernel space)
+---------------------------------------+     /
+---------------------------------------+     \
|       Windows low-level drivers       |      } drivers (kernel space)
+---------------------------------------+     /
	  </screen>
	</para>
	<para>
	  Note also (not depicted in schema above) that the 16 bit
	  applications are supported in a specific subsystem.
	  Some basic differences between the Win9x and the NT
	  architectures include:
	  <itemizedlist>
	    <listitem>
	      <para>
		Several subsystems (Win32, Posix...) can be run on NT,
		while not on Win 9x
	      </para>
	    </listitem>
	    <listitem>
	      <para>
		Win 9x roots its architecture in 16 bit systems, while
		NT is truely a 32 bit system.
	      </para>
	    </listitem>
	    <listitem>
	      <para>
		The drivers model and interfaces in Win 9x and NT are
		different (even if Microsoft tried to bridge the gap
		with some support of WDM drivers in Win 98 and above).
	      </para>
	    </listitem>
	  </itemizedlist>
	</para>
      </sect2>
    </sect1>

    <sect1>
      <title>Wine architecture</title>

      <sect2>
	<title>Global picture</title>

	<para>
	  Wine implementation is closer to the Windows NT
	  architecture, even if several subsystems are not implemented
	  yet (remind also that 16bit support is implemented in a 32-bit
	  Windows EXE, not as a subsystem). Here's the overall picture:
	  <screen>
+---------------------+                                  \
|     Windows EXE     |                                   } application
+---------------------+                                  /

+---------+ +---------+                                  \
| Windows | | Windows |                                   \ application & system DLLs
|   DLL   | |   DLL   |                                   /
+---------+ +---------+                                  /

+---------+ +---------+     +-----------+  +--------+  \
|  GDI32  | |  USER32 |     |           |  |        |   \
|   DLL   | |   DLL   |     |           |  |  Wine  |    \
+---------+ +---------+     |           |  | Server |     \ core system DLLs
+---------------------+     |           |  |        |     / (on the left side)
|    Kernel32 DLL     |     | Subsystem |  | NT-like|    /
|  (Win32 subsystem)  |     |Posix, OS/2|  | Kernel |   /
+---------------------+     +-----------+  |        |  / 
                                           |        |
+---------------------------------------+  |        |
|                 NTDLL                 |  |        |
+---------------------------------------+  +--------+

+---------------------------------------+               \
|   Wine executable (wine-?thread)      |                } unix executable
+---------------------------------------+               /
+---------------------------------------------------+   \
|                   Wine drivers                    |    } Wine specific DLLs
+---------------------------------------------------+   /

+------------+    +------------+     +--------------+   \
|    libc    |    |   libX11   |     |  other libs  |    } unix shared libraries
+------------+    +------------+     +--------------+   /  (user space)

+---------------------------------------------------+   \
|         Unix kernel (Linux,*BSD,Solaris,OS/X)     |    } (Unix) kernel space
+---------------------------------------------------+   /
+---------------------------------------------------+   \
|                 Unix device drivers               |    } Unix drivers (kernel space)
+---------------------------------------------------+   /
	  </screen>
	</para>

	<para>
	  Wine must at least completely replace the "Big Three" DLLs
	  (KERNEL/KERNEL32, GDI/GDI32, and USER/USER32), which all
	  other DLLs are layered on top of. But since Wine is (for
	  various reasons) leaning towards the NT way of implementing
	  things, the NTDLL is another core DLL to be implemented in
	  Wine, and many KERNEL32 and ADVAPI32 features will be
	  implemented through the NTDLL.
	</para>
	<para>
	  As of today, no real subsystem (apart the Win32 one) has
	  been implemented in Wine. 
	</para>
	<para>
	  The Wine server provides the backbone for the implementation
	  of the core DLLs. It mainly implementents inter-process
	  synchronization and object sharing. It can be seen, from a
	  functional point of view, as a NT kernel (even if the APIs
	  and protocols used between Wine's DLL and the Wine server
	  are Wine specific).
	</para>
	<para>
	  Wine uses the Unix drivers to access the various hardware
	  pieces on the box. However, in some cases, Wine will
	  provide a driver (in Windows sense) to a physical hardware
	  device. This driver will be a proxy to the Unix driver
	  (this is the case, for example, for the graphical part
	  with X11 or SDL drivers, audio with OSS or ALSA drivers...).
	</para>
	<para>
	  All DLLs provided by Wine try to stick as much as possible
	  to the exported APIs from the Windows platforms. There are
	  rare cases where this is not the case, and have been
	  propertly documented (Wine DLLs export some Wine specific
	  APIs). Usually, those are prefixed with
	  <function>__wine</function>.
	</para>
	<para>
	  Let's now review in greater details all of those components.
	</para>
      </sect2>

      <sect2>
	<title>The Wine server</title>
	<para>
	  The Wine server is among the most confusing concepts in
	  Wine. What is its function in Wine? Well, to be brief, it
	  provides Inter-Process Communication (IPC),
	  synchronization, and process/thread management. When the
	  Wine server launches, it creates a Unix socket for the
	  current host based on (see below) your home directory's
	  <filename>.wine</filename> subdirectory (or wherever the
	  <constant>WINEPREFIX</constant> environment variable
	  points to) - all Wine processes launched later connects to
	  the Wine server using this socket. (If a Wine server was
	  not already running, the first Wine process will start up
	  the Wine server in auto-terminate mode (i.e. the Wine
	  server will then terminate itself once the last Wine
	  process has terminated).)
	</para>
	<para>
	  In earlier versions of Wine the master socket mentioned
	  above was actually created in the configuration directory;
	  either your home directory's <filename>/wine</filename>
	  subdirectory or wherever the
	  <constant>WINEPREFIX</constant> environment variable
	  points>. Since that might not be possible the socket is
	  actually created within the <filename>/tmp</filename>
	  directory with a name that reflects the configuration
	  directory. This means that there can actually be several
	  separate copies of the Wine server running; one per
	  combination of user and configuration directory. Note that
	  you should not have several users using the same
	  configuration directory at the same time; they will have
	  different copies of the Wine server running and this could
	  well lead to problems with the registry information that
	  they are sharing.
	</para>
	<para>
	  Every thread in each Wine process has its own request
	  buffer, which is shared with the Wine server. When a
	  thread needs to synchronize or communicate with any other
	  thread or process, it fills out its request buffer, then
	  writes a command code through the socket. The Wine server
	  handles the command as appropriate, while the client
	  thread waits for a reply. In some cases, like with the
	  various <function>WaitFor???</function> synchronization
	  primitives, the server handles it by marking the client
	  thread as waiting and does not send it a reply before the
	  wait condition has been satisfied.
	</para>
	<para>
	  The Wine server itself is a single and separate Unix
	  process and does not have its own threading - instead, it
	  is built on top of a large <function>poll()</function>
	  loop that alerts the Wine server whenever anything
	  happens, such as a client having sent a command, or a wait
	  condition having been satisfied. There is thus no danger
	  of race conditions inside the Wine server itself - it is
	  often called upon to do operations that look completely
	  atomic to its clients.
	</para>
	<para>
	  Because the Wine server needs to manage processes,
	  threads, shared handles, synchronization, and any related
	  issues, all the clients' Win32 objects are also managed by
	  the Wine server, and the clients must send requests to the
	  Wine server whenever they need to know any Win32 object
	  handle's associated Unix file descriptor (in which case
	  the Wine server duplicates the file descriptor, transmits
	  it back to the client, and leaves it to the client to
	  close the duplicate when the client has finished with
	  it).
	</para>
      </sect2>

      <sect2>
	<title>
	  Wine builtin DLLs: about Relays, Thunks, and DLL
	  descriptors
	</title>
	<para>
	  This section mainly applies to builtin DLLs (DLLs provided
	  by Wine). See section <xref linkend="arch-dlls"> for the
	  details on native vs. builtin DLL handling.
	</para>
	<para>
	  Loading a Windows binary into memory isn't that hard by
	  itself, the hard part is all those various DLLs and entry
	  points it imports and expects to be there and function as
	  expected; this is, obviously, what the entire Wine
	  implementation is all about. Wine contains a range of DLL
	  implementations. You can find the DLLs implementation in the
	  <filename>dlls/</filename> directory. 
	</para>
	<para>
	  Each DLL (at least, the 32 bit version, see below) is
	  implemented in a Unix shared library. The file name of this
	  shared library is the module name of the DLL with a
	  <filename>.dll.so</filename> suffix (or
	  <filename>.drv.so</filename> or any other relevant extension
	  depending on the DLL type). This shared library contains the
	  code itself for the DLL, as well as some more information,
	  as the DLL resources and a Wine specific DLL descriptor.
	</para>
	<para>
	  The DLL descriptor, when the DLL is instanciated, is used to
	  create an in-memory PE header, which will provide access to
	  various information about the DLL, including but not limited
	  to its entry point, its resources, its sections, its debug
	  information...
	</para>
	<para>
	  The DLL descriptor and entry point table is generated by
	  the <command>winebuild</command> tool (previously just
	  named <command>build</command>), taking DLL specification
	  files with the extension <filename>.spec</filename> as
	  input. Resources (after compilation by
	  <command>wrc</command>) or message tables (after
	  compilation by <command>wmc</command>) are also added to
	  the descriptor by <command>winebuild</command>.
	</para>
	<para>
	  Once an application module wants to import a DLL, Wine
	  will look at:
	  <itemizedlist>
	    <listitem>
	      <para>
		through its list of registered DLLs (in fact, both
		the already loaded DLLs, and the already loaded
		shared libraries which has registered a DLL
		descriptor). Since, the DLL descriptor is
		automatically registered when the shared library is
		loaded - remember, registration call is put inside a
		shared library constructor - using the
		<constant>PRELOAD</constant> environment variable
		when running a Wine process can force the
		registration of some DLL descriptors.
	      </para>
	    </listitem>
	    <listitem>
	      <para>
		If it's not registered, Wine will look for it on
		disk, building the shared library name from the DLL
		module name. Directory searched for are specified by
		the <constant>WINEDLLPATH</constant> environment
		variable.
	      </para>
	    </listitem>
	    <listitem>
	      <para>
		Failing that, it will look for a real Windows
		<filename>.DLL</filename> file to use, and look
		through its imports, etc) and use the loading of
		native DLLs.
	      </para>
	    </listitem>
	  </itemizedlist>
	</para>
	<para>
	  After the DLL has been identified (assuming it's still a
	  native one), it's mapped into memory using a
	  <function>dlopen()</function> call. Note, that Wine doesn't
	  use the shared library mechanisms for resolving and/or
	  importing functions between two shared libraries (for two
	  DLLs). The shared library is only used for providing a way
	  to load a piece of code on demand. This piece of code,
	  thanks the DLL descriptor, will provide the same type of
	  information a native DLL would. Wine can then use the same
	  code for native and builtin DLL to handle imports/exports.
	</para>
	<para>
	  Wine also relies on the dynamic loading features of the Unix
	  shared libraries to relocate the DLLs if needed (the same
	  DLL can be loaded at different address in two different
	  processes, and even in two consecutive run of the same
	  executable if the order of loading the DLLs differ). 
	</para>
	<para>
	  The DLL descriptor is registered in the Wine realm using
	  some tricks. The <command>winebuild</command> tool, while
	  creating the code for DLL descriptor, also creates a
	  constructor, that will be called when the shared library is
	  loaded into memory. This constructor will actually register
	  the descriptor to the Wine DLL loader. Hence, before the
	  <function>dlopen</function> call returns, the DLL descriptor
	  will be known and registered. This also helps to deal with
	  the cases where there's still dependencies (at the ELF
	  shared lib level, not at the embedded DLL level) between
	  different shared libraries: the embedded DLLs will be
	  properly registered, and even loaded (from a Windows point
	  of view).
	</para>
	<para>
	  Since Wine is 32-bit code itself, and if the compiler
	  supports Windows' calling convention, <type>stdcall</type>
	  (<command>gcc</command> does), Wine can resolve imports
	  into Win32 code by substituting the addresses of the Wine
	  handlers directly without any thunking layer in
	  between. This eliminates the overhead most people
	  associate with "emulation", and is what the applications
	  expect anyway.
	</para>
	<para>
	  However, if the user specified <parameter>WINEDEBUG=+relay
	  </parameter>, a thunk layer is inserted between the
	  application imports and the Wine handlers (actually the
	  export table of the DLL is modified, and a thunk is
	  inserted in the table); this layer is known as "relay"
	  because all it does is print out the arguments/return
	  values (by using the argument lists in the DLL
	  descriptor's entry point table), then pass the call on,
	  but it's invaluable for debugging misbehaving calls into
	  Wine code. A similar mechanism also exists between Windows
	  DLLs - Wine can optionally insert thunk layers between
	  them, by using <parameter>WINEDEBUG=+snoop</parameter>,
	  but since no DLL descriptor information exists for
	  non-Wine DLLs, this is less reliable and may lead to
	  crashes.
	</para>
	<para>
	  For Win16 code, there is no way around thunking - Wine
	  needs to relay between 16-bit and 32-bit code. These
	  thunks switch between the app's 16-bit stack and Wine's
	  32-bit stack, copies and converts arguments as appropriate
	  (an int is 16 bit 16-bit and 32 bits in 32-bit, pointers
	  are segmented in 16 bit (and also near or far) but are 32
	  bit linear values in 32 bit), and handles the Win16
	  mutex. Some finer control can be obtained on the
	  conversion, see <command>winebuild</command> reference
	  manual for the details. Suffice to say that the kind of
	  intricate stack content juggling this results in, is not
	  exactly suitable study material for beginners.
	</para>
	<para>
	  A DLL descriptor is also created for every 16 bit
	  DLL. However, this DLL normally paired with a 32 bit
	  DLL. Either, it's the 16 bit counterpart of the 16 bit DLL
	  (KRNL386.EXE for KERNEL32, USER for USER32...), or a 16
	  bit DLL directly linked to a 32 bit DLL (like SYSTEM for
	  KERNEL32, or DDEML for USER32). In those cases, the 16 bit
	  descriptor(s) is (are) inserted in the same shared library
	  as the the corresponding 32 bit DLL. Wine will also create
	  symbolic links between kernel32.dll.so and system.dll.so
	  so that loading of either
	  <filename>kernel32.dll</filename> or
	  <filename>system.dll</filename> will end up on the same
	  shared library.
	</para>
      </sect2>
      <sect2 id="arch-dlls">
	<title>Wine/Windows DLLs</title>

	<para>
	  This document mainly deals with the status of current DLL
	  support by Wine.  The Wine ini file currently supports
	  settings to change the load order of DLLs.  The load order
	  depends on several issues, which results in different settings
	  for various DLLs.
	</para>

	<sect3>
	  <title>Pros of Native DLLs</title>

	  <para>
	    Native DLLs of course guarantee 100% compatibility for
	    routines they implement. For example, using the native USER
	    DLL would maintain a virtually perfect and Windows 95-like
	    look for window borders, dialog controls, and so on. Using
	    the built-in Wine version of this library, on the other
	    hand, would produce a display that does not precisely mimic
	    that of Windows 95.  Such subtle differences can be
	    engendered in other important DLLs, such as the common
	    controls library COMMCTRL or the common dialogs library
	    COMMDLG, when built-in Wine DLLs outrank other types in load
	    order.
	  </para>
	  <para>
	    More significant, less aesthetically-oriented problems can
	    result if the built-in Wine version of the SHELL DLL is
	    loaded before the native version of this library. SHELL
	    contains routines such as those used by installer utilities
	    to create desktop shortcuts. Some installers might fail when
	    using Wine's built-in SHELL.
	  </para>
	</sect3>

	<sect3>
	  <title>Cons of Native DLLs</title>

	  <para>
	    Not every application performs better under native DLLs. If
	    a library tries to access features of the rest of the system
	    that are not fully implemented in Wine, the native DLL might
	    work much worse than the corresponding built-in one, if at
	    all. For example, the native Windows GDI library must be
	    paired with a Windows display driver, which of course is not
	    present under Intel Unix and Wine.
	  </para>
	  <para>
	    Finally, occasionally built-in Wine DLLs implement more
	    features than the corresponding native Windows DLLs.
	    Probably the most important example of such behavior is the
	    integration of Wine with X provided by Wine's built-in USER
	    DLL. Should the native Windows USER library take load-order
	    precedence, such features as the ability to use the
	    clipboard or drag-and-drop between Wine windows and X
	    windows will be lost.
	  </para>
	</sect3>

	<sect3>
	  <title>Deciding Between Native and Built-In DLLs</title>

	  <para>
	    Clearly, there is no one rule-of-thumb regarding which
	    load-order to use. So, you must become familiar with
	    what specific DLLs do and which other DLLs or features 
	    a given library interacts with, and use this information 
	    to make a case-by-case decision.
	  </para>
	</sect3>

	<sect3>
	  <title>Load Order for DLLs</title>

	  <para>
	    Using the DLL sections from the wine configuration file, the
	    load order can be tweaked to a high degree. In general it is
	    advised not to change the settings of the configuration
	    file. The default configuration specifies the right load
	    order for the most important DLLs.
	  </para>
	  <para>
	    The default load order follows this algorithm: for all DLLs
	    which have a fully-functional Wine implementation, or where
	    the native DLL is known not to work, the built-in library
	    will be loaded first. In all other cases, the native DLL
	    takes load-order precedence.
	  </para>
	  <para>
	    The <varname>DefaultLoadOrder</varname> from the
	    [DllDefaults] section specifies for all DLLs which version
	    to try first. See manpage for explanation of the arguments.
	  </para>
	  <para>
	    The [DllOverrides] section deals with DLLs, which need a
	    different-from-default treatment. 
	  </para>
	  <para>
	    The [DllPairs] section is for DLLs, which must be loaded in
	    pairs. In general, these are DLLs for either 16-bit or
	    32-bit applications. In most cases in Windows, the 32-bit
	    version cannot be used without its 16-bit counterpart. For
	    Wine, it is customary that the 16-bit implementations rely
	    on the 32-bit implementations and cast the results back to
	    16-bit arguments. Changing anything in this section is bound
	    to result in errors.
	  </para>
	  <para>
	    For the future, the Wine implementation of Windows DLL seems
	    to head towards unifying the 16 and 32 bit DLLs wherever
	    possible, resulting in larger DLLs.  They are stored in the
	    <filename>dlls/</filename> subdirectory using the 32-bit
	    name.
	  </para>
	</sect3>
      </sect2>

      <sect2 id="arch-mem">
	<title>Memory management</title>
	<para>
	  Every Win32 process in Wine has its own dedicated native
	  process on the host system, and therefore its own address
	  space. This section explores the layout of the Windows
	  address space and how it is emulated.
	</para>

	<para>
	  Firstly, a quick recap of how virtual memory works. Physical
	  memory in RAM chips is split into
	  <emphasis>frames</emphasis>, and the memory that each
	  process sees is split into <emphasis>pages</emphasis>. Each
	  process has its own 4 gigabytes of address space (4gig being
	  the maximum space addressable with a 32 bit pointer). Pages
	  can be mapped or unmapped: attempts to access an unmapped
	  page cause an
	  <constant>EXCEPTION_ACCESS_VIOLATION</constant> which has
	  the easily recognizable code of
	  <constant>0xC0000005</constant>. Any page can be mapped to
	  any frame, therefore you can have multiple addresses which
	  actually "contain" the same memory. Pages can also be mapped
	  to things like files or swap space, in which case accessing
	  that page will cause a disk access to read the contents into
	  a free frame.
	</para>

	<sect3>
	  <title>Initial layout (in Windows)</title>
	  <para>
	    When a Win32 process starts, it does not have a clear
	    address space to use as it pleases. Many pages are already
	    mapped by the operating system. In particular, the EXE
	    file itself and any DLLs it needs are mapped into memory,
	    and space has been reserved for the stack and a couple of
	    heaps (zones used to allocate memory to the app
	    from). Some of these things need to be at a fixed address,
	    and others can be placed anywhere.
	  </para>

	  <para>
	    The EXE file itself is usually mapped at address 0x400000
	    and up: indeed, most EXEs have their relocation records
	    stripped which means they must be loaded at their base
	    address and cannot be loaded at any other address.
	  </para>

	  <para>
	    DLLs are internally much the same as EXE files but they
	    have relocation records, which means that they can be
	    mapped at any address in the address space. Remember we
	    are not dealing with physical memory here, but rather
	    virtual memory which is different for each
	    process. Therefore <filename>OLEAUT32.DLL</filename> may
	    be loaded at one address in one process, and a totally
	    different one in another. Ensuring all the functions
	    loaded into memory can find each other is the job of the
	    Windows dynamic linker, which is a part of NTDLL.
	  </para>
	  <para>
	    So, we have the EXE and its DLLs mapped into memory. Two
	    other very important regions also exist: the stack and the
	    process heap. The process heap is simply the equivalent of
	    the libc <function>malloc</function> arena on UNIX: it's a
	    region of memory managed by the OS which
	    <function>malloc</function>/<function>HeapAlloc</function>
	    partitions and hands out to the application. Windows
	    applications can create several heaps but the process heap
	    always exists.
	  </para>
	  <para>
	    Windows 9x also implements another kind of heap: the
	    shared heap. The shared heap is unusual in that
	    anything allocated from it will be visible in every other
	    process.
	  </para>
	</sect3>

	<sect3>
	  <title>Comparison</title>
	  <para>
	    So far we've assumed the entire 4 gigs of address space is
	    available for the application. In fact that's not so: only
	    the lower 2 gigs are available, the upper 2 gigs are on
	    Windows NT used by the operating system and hold the
	    kernel (from 0x80000000). Why is the kernel mapped into
	    every address space?  Mostly for performance: while it's
	    possible to give the kernel its own address space too -
	    this is what Ingo Molnars 4G/4G VM split patch does for
	    Linux - it requires that every system call into the kernel
	    switches address space. As that is a fairly expensive
	    operation (requires flushing the translation lookaside
	    buffers etc) and syscalls are made frequently it's best
	    avoided by keeping the kernel mapped at a constant
	    position in every processes address space.
	  </para>
	  
	  <para>
	    Basically, the comparison of memory mappings looks as
	    follows:
	    <table>
	      <title>Memory layout (Windows and Wine)</title>
	      <tgroup cols="4" align="left" colsep="1" rowsep="1">
		<thead>
		  <row>
		    <entry>Address</entry>
		    <entry>Windows 9x</entry>
		    <entry>Windows NT</entry>
		    <entry>Linux</entry>
		  </row>
		</thead>
		<tbody>
		  <row>
		    <entry>00000000-7fffffff</entry>
		    <entry>User</entry>
		    <entry>User</entry>
		    <entry>User</entry>
		  </row>
		  <row>
		    <entry>80000000-bfffffff</entry>
		    <entry>Shared</entry>
		    <entry>User</entry>
		    <entry>User</entry>
		  </row>
		  <row>
		    <entry>c0000000-ffffffff</entry>
		    <entry>Kernel</entry>
		    <entry>Kernel</entry>
		    <entry>Kernel</entry>
		  </row>
		</tbody>
	      </tgroup>
	    </table>
	  </para>

	  <para>
	    On Windows 9x, in fact only the upper gigabyte
	    (<constant>0xC0000000</constant> and up) is used by the
	    kernel, the region from 2 to 3 gigs is a shared area used
	    for loading system DLLs and for file mappings. The bottom
	    2 gigs on both NT and 9x are available for the programs
	    memory allocation and stack.
	  </para>
	</sect3>

	<sect3>
	  <title>Implementation</title>
	  <para>
	    Wine (with a bit of black magic) is able to map all items
	    at the correct locations as depicted above.
	  </para>
	  <para>
	    Wine also implements the shared heap so native win9x DLLs
	    can be used. This heap is always created at the
	    <constant>SYSTEM_HEAP_BASE</constant> address or
	    <constant>0x80000000</constant> and defaults to 16
	    megabytes in size.
	  </para>
	  <para>
	    There are a few other magic locations. The bottom 64k of
	    memory is deliberately left unmapped to catch null pointer
	    dereferences. The region from 64k to 1mb+64k are reserved
	    for DOS compatibility and contain various DOS data
	    structures. Finally, the address space also contains
	    mappings for the Wine binary itself, any native libaries
	    Wine is using, the glibc malloc arena and so on.
	  </para>
	</sect3>

	<sect3 id="address-space">
	  <title>Laying out the address space</title>

	  <para>
	    Up until about the start of 2004, the Linux address space
	    very much resembled the Windows 9x layout: the kernel sat
	    in the top gigabyte, the bottom pages were unmapped to
	    catch null pointer dereferences, and the rest was
	    free. The kernels mmap algorithm was predictable: it would
	    start by mapping files at low addresses and work up from
	    there.
	  </para>

	  <para>
	    The development of a series of new low level patches
	    violated many of these assumptions, and resulted in Wine
	    needing to force the Win32 address space layout upon the
	    system. This section looks at why and how this is done.
	  </para>

	  <para>
	    The exec-shield patch increases security by randomizing
	    the kernels mmap algorithms. Rather than consistently
	    choosing the same addresses given the same sequence of
	    requests, the kernel will now choose randomized
	    addresses. Because the Linux dynamic linker
	    (ld-linux.so.2) loads DSOs into memory by using mmap, this
	    means that DSOs are no longer loaded at predictable
	    addresses, so making it harder to attack software by using
	    buffer overflows. It also attempts to relocate certain
	    binaries into a special low area of memory known as the
	    ASCII armor so making it harder to jump into them when
	    using string based attacks.
	  </para>

	  <para>
	    Prelink is a technology that enhances startup times by
	    precalculating ELF global offset tables then saving the
	    results inside the native binaries themselves. By grid
	    fitting each DSO into the address space, the dynamic
	    linker does not have to perform as many relocations so
	    allowing applications that heavily rely on dynamic linkage
	    to be loaded into memory much quicker. Complex C++
	    applications such as Mozilla, OpenOffice and KDE can
	    especially benefit from this technique.
	  </para>

	  <para>
	    The 4G VM split patch was developed by Ingo Molnar. It
	    gives the Linux kernel its own address space, thereby
	    allowing processes to access the maximum addressable
	    amount of memory on a 32-bit machine: 4 gigabytes. It
	    allows people with lots of RAM to fully utilise that in
	    any given process at the cost of performance: the reason
	    behind giving the kernel a part of each processes address
	    space was to avoid the overhead of switching on each
	    syscall.
	  </para>

	  <para>
	    Each of these changes alter the address space in a way
	    incompatible with Windows. Prelink and exec-shield mean
	    that the libraries Wine uses can be placed at any point in
	    the address space: typically this meant that a library was
	    sitting in the region that the EXE you wanted to run had
	    to be loaded (remember that unlike DLLs, EXE files cannot
	    be moved around in memory). The 4G VM split means that
	    programs could receive pointers to the top gigabyte of
	    address space which some are not prepared for (they may
	    store extra information in the high bits of a pointer, for
	    instance). In particular, in combination with exec-shield
	    this one is especially deadly as it's possible the process
	    heap could be allocated beyond ADDRESS_SPACE_LIMIT which
	    causes Wine initialization to fail.
	  </para>

	  <para>
	    The solution to these problems is for Wine to reserve
	    particular parts of the address space so that areas that
	    we don't want the system to use will be avoided. We later
	    on (re/de)allocate those areas as needed. One problem is
	    that some of these mappings are put in place automatically
	    by the dynamic linker: for instance any libraries that
	    Wine is linked to (like libc, libwine, libpthread etc)
	    will be mapped into memory before Wine even gets
	    control. In order to solve that, Wine overrides the
	    default ELF initialization sequence at a low level and
	    reserves the needed areas by using direct syscalls into
	    the kernel (ie without linking against any other code to
	    do it) before restarting the standard initialization and
	    letting the dynamic linker continue. This is referred to
	    as the preloader and is found in loader/preloader.c.
	  </para>

	  <para>
	    Once the usual ELF boot sequence has been completed, some
	    native libraries may well have been mapped above the 3gig
	    limit: however, this doesn't matter as 3G is a Windows
	    limit, not a Linux limit. We still have to prevent the
	    system from allocating anything else above there (like the
	    heap or other DLLs) though so Wine performs a binary
	    search over the upper gig of address space in order to
	    iteratively fill in the holes with MAP_NORESERVE mappings
	    so the address space is allocated but the memory to
	    actually back it is not. This code can be found in libs/wine/mmap.c:reserve_area.
	  </para>
	</sect3>
      </sect2>

      <sect2>
	<title>Processes</title>
	<para>
	  Let's take a closer look at the way Wine loads and run
	  processes in memory.
	</para>
	<sect3>
	  <title>Starting a process from command line</title>
	  <para>
	    When starting a Wine process from command line (we'll get
	    later on to the differences between NE, PE and Winelib
	    executables), there are a couple of things Wine need to do
	    first. A first executable is run to check the threading
	    model of the underlying OS (see <xref linkend="threading">
	    for the details) and will start the real Wine loader
	    corresponding to the choosen threading model.
	  </para>
	  <para>
	    Then Wine graps a few elements from the Unix world: the
	    environment, the program arguments. Then the
	    <filename>ntdll.dll.so</filename> is loaded into memory
	    using the standard shared library dynamic loader. When
	    loaded, NTDLL will mainly first create a decent Windows
	    environment:
	    <itemizedlist>
	      <listitem>
		<para>create a PEB and a TEB</para>
	      </listitem>
	      <listitem>
		<para>
		  set up the connection to the Wine server - and
		  eventually launching the Wine server if none runs
		</para>
	      </listitem>
	      <listitem>
		<para>create the process heap</para>
	      </listitem>
	    </itemizedlist>
	  </para>
	  <para>
	    Then <filename>Kernel32</filename> is loaded (but now
	    using the Windows dynamic loading capabilities) and a Wine
	    specific entry point is called
	    <function>__wine_kernel_init</function>. This function
	    will actually handle all the logic of the process loading
	    and execution, and will never return from it's call.
	  </para>
	  <para>
	    <function>__wine_kernel_init</function> will undergo the
	    following tasks:
	    <itemizedlist>
	      <listitem>
		<para>
		  initialization of program arguments from Unix
		  program arguments
		</para>
	      </listitem>
	      <listitem>
		<para>
		  lookup of executable in the file system
		</para>
	      </listitem>
	      <listitem>
		<para>
		  If the file is not found, then an error is printed
		  and the Wine loader stops.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  We'll cover the non-PE file type later on, so assume
		  for now it's a PE file. The PE module is loaded in
		  memory using the Windows shared library
		  mechanism. Note that the dependencies on the module
		  are not resolved at this point.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  A new stack is created, which size is given in the
		  PE header, and this stack is made the one of the
		  running thread (which is still the only one in the
		  process). The stack used at startup will no longer
		  be used.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  Which this new stack,
		  <function>ntdll.LdrInitializeThunk</function> is
		  called which performs the remaining initialization
		  parts, including resolving all the DLL imports on
		  the PE module, and doing the init of the TLS slots.
		</para>
	      </listitem>
	      <listitem>
		<para>
		  Control can now be passed to the
		  <function>EntryPoint</function> of the PE module,
		  which will let the executable run.
		</para>
	      </listitem>
	    </itemizedlist>
	  </para>
	</sect3>
	<sect3>
	  <title>Creating a child process from a running process</title>
	  <para>
	    The steps used are closely link to what is done in the
	    previous case.
	  </para>
	  <para>
	    There are however a few points to look at a bit more
	    closely. The inner implementation creates the child
	    process using the <function>fork()</function> and
	    <function>exec()</function> calls. This means that we
	    don't need to check again for the threading model, we can
	    use what the parent (or the grand-parent process...)
	    started from command line has found.
	  </para>
	  <para>
	    The Win32 process creation allows to pass a lot of
	    information between the parent and the child. This
	    includes object handles, windows title, console
	    parameters, environment strings... Wine makes use of both
	    the standard Unix inheritance mechanisms (for environment
	    for example) and the Wine server (to pass from parent to
	    child a chunk of data containing the relevant information).
	  </para>
	  <para>
	    The previously described loading mechanism will check in
	    the Wine server if such a chunk exists, and, if so, will
	    perform the relevant initialization.
	  </para>
	  <para>
	    Some further synchronization is also put in place: a
	    parent will wait until the child has started, or has
	    failed. The Wine server is also used to perform those
	    tasks.
	  </para>
	</sect3>
	<sect3>
	  <title>Starting a Winelib process</title>
	  <para>
	    Before going into the gory details, let's first go back to
	    what a Winelib application is. It can be either a regular
	    Unix executable, or a more specific Wine beast. This later
	    form in fact creates two files for a given executable (say
	    <filename>foo.exe</filename>). The first one, named
	    <filename>foo</filename> will be a symbolic link to the
	    Wine loader (<filename>wine</filename>). The second one,
	    named <filename>foo.exe.so</filename>, is the equivalent
	    of the <filename>.dll.so</filename> files we've already
	    described for DLLs. As in Windows, an executable is, among
	    other things, a module with its import and export
	    information, as any DLL, it makes sense Wine uses the same
	    mechanisms for loading native executables and DLLs.
	  </para>
	  <para>
	    When starting a Winelib application from the command line
	    (say with <command>foo arg1 arg2</command>), the Unix
	    shell will execute <command>foo</command> as a Unix
	    executable. Since this is in fact the Wine loader, Wine
	    will fire up. However, will notice that it hasn't been
	    started as <command>wine</command> but as
	    <command>foo</command>, and hence, will try to load (using
	    Unix shared library mechanism) the second file
	    <filename>foo.exe.so</filename>. Wine will recognize a 32
	    bit module (with its descriptor) embedded in the shared
	    library, and once the shared library loaded, it will
	    proceed the same path as when loading a standard native PE
	    executable.
	  </para>
	  <para>
	    Wine needs to implement this second form of executable in
	    order to maintain the order of initialization of some
	    elements in the executable. One particular issue is when
	    dealing with global C++ objects. In standard Unix
	    executable, the call of the constructor to such objects is
	    stored in the specific section of the executable
	    (<function>.init</function> not to name it). All
	    constructors in this section are called before the
	    <function>main</function> function is called. Creating a
	    Wine executable using the first form mentionned above will
	    let those constructors being called before Wine gets a
	    chance to initialize itself. So, any constructor using a
	    Windows API will fail, because Wine infrastructure isn't
	    in place. The use of the second form for Winelib
	    executables ensures that we do the initialization using
	    the following steps:
	    <itemizedlist>
	      <listitem>
		<para>
		  initialize the Wine infrastructure
		</para>
	      </listitem>
	      <listitem>
		<para>
		  load the executable into memory
		</para>
	      </listitem>
	      <listitem>
		<para>
		  handle the import sections for the executable
		</para>
	      </listitem>
	      <listitem>
		<para>
		  call the global object constructors (if any). They
		  now can properly call the Windows APIs
		</para>
	      </listitem>
	      <listitem>
		<para>
		  call the executable entry point
		</para>
	      </listitem>
	    </itemizedlist>
	  </para>
	  <para>
	    The attentive reader would have noted that the resolution
	    of imports for the executable is done, as for a DLL, when
	    the executable/DLL descriptor is registered. However, this
	    is done also by adding a specific constructor in the
	    <function>.init</function> section. For the above describe
	    scheme to function properly, this constructor must be the
	    first constructor to be called, before all the other
	    constructors, generated by the executable itself. The Wine
	    build chain takes care of that, and also generating the
	    executable/DLL descriptor for the Winelib executable.
	  </para>
	</sect3>
	<sect3>
	  <title>Starting a NE (Win16) process</title>
	  <para>
	    When Wine is requested to run a NE (Win 16 process), it
	    will in fact hand over the execution of it to a specific
	    executable <filename>winevdm</filename>. VDM stands for
	    Virtual DOS Machine. This <filename>winevdm</filename>
	    will in fact set up the correct 16 bit environment to run
	    the executable. Any new 16 bit process created by this
	    executable (or its children) will run into the same
	    <filename>winevdm</filename> instance. Among one instance,
	    several functionalities will be provided to those 16 bit
	    processes, including the cooperative multitasking, sharing
	    the same address space, managing the selectors for the 16
	    bit segments needed for code, data and stack.
	  </para>
	  <para>
	    Note that several <filename>winevdm</filename> instances
	    can run in the same Wine session, but the functionalities
	    described above are only shared among a given instance,
	    not among all the instances. <filename>winevdm</filename>
	    is built as Winelib application, and hence has access to
	    any facility a 32 bit application has.
	  </para>
	  <para>
	    The behaviour we just described also applies to DOS
	    executables, which are handled the same way by
	    <filename>winevdm</filename>.
	  </para>
	</sect3>
      </sect2>
      <sect2>
	<title>Wine drivers</title>
	<para>
	  Wine will not allow running native Windows drivers under
	  Unix. This comes mainly because (look at the generic
	  architecture schemas) Wine doesn't implement the kernel
	  features of Windows (kernel here really means the kernel,
	  not the KERNEL32 DLL), but rather sets up a proxy layer on
	  top of the Unix kernel to provide the NTDLL and KERNEL32
	  features. This means that Wine doesn't provide the inner
	  infrastructure to run native drivers, either from the Win9x
	  family or from the NT family.
	</para>
	<para>
	  In other words, Wine will only be able to provide access to
	  a specific device, if and only if, 1/ this device is
	  supported in Unix (there is Unix-driver to talk to it), 2/
	  Wine has implemented the proxy code to make the glue between
	  the API of a Windows driver, and the Unix interface of the
	  Unix driver.
	</para>
	<para>
	  Wine, however, tries to implement in the various DLLs
	  needing to access devices to do it through the standard
	  Windows APIs for device drivers in user space. This is for
	  example the case for the multimedia drivers, where Wine
	  loads Wine builtin DLLs to talk to the OSS interface, or the
	  ALSA interface. Those DLLs implement the same interface as
	  any user space audio driver in Windows.
	</para>
      </sect2>
    </sect1>
  </chapter>


<!-- Keep this comment at the end of the file
Local variables:
mode: sgml
sgml-parent-document:("wine-devel.sgml" "set" "book" "part" "chapter" "")
End:
-->