Wine and Multimedia
This file contains information about the implementation of the
multimedia layer of Wine.
The implementation can be found in the dlls/winmm/ directory (and in
many of its subdirectories), but also in dlls/msacm/ (for the
audio compression/decompression manager) and dlls/msvideo/ (for the
video compression/decompression manager).
Written by &name-eric-pouech; &email-eric-pouech;
(Last updated: 02/16/2001)
Overview
The multimedia stuff is split into 3 layers. The low level (device
drivers), mid level (MCI commands) and high level abstraction layers.
The low level layer has also some helper DLLs (like the MSACM/MSACM32
and MSVIDEO/MSVFW32 pairs).
The low level layer may depend on current hardware and OS services
(like OSS on Unix). Mid level (MCI) and high level layers must be
written independently from the hardware and OS services.
There are two specific low level drivers (one for wave input/output,
another one for MIDI output only), whose role is:
help choosing one low level driver between many
add the possibility to convert streams (ie ADPCM => PCM)
add the possibility to filter a stream (adding echo, equalizer...
to a wave stream), or modify the instruments that have to be
played (MIDI).
All of those components are defined as DLLs (one by one).
Low level layers
Please note that native low level drivers are not currently supported
in Wine, because they either access hardware components or require
VxDs to be loaded; Wine does not correctly supports those two so far.
The following low level layers are implemented (as built-in DLLs):
(Wave form) Audio
MMSYSTEM and WINMM call the real low level audio driver using the
wodMessage/widMessage which handles the different requests.
OSS implementation
The low level audio driver is currently only implemented for the
OpenSoundSystem (OSS) as supplied in the Linux and FreeBSD kernels by
4Front Technologies.
The presence of this driver is checked by configure (depends on the
<sys/soundcard.h> file). Source code resides in
dlls/winmm/wineoss/audio.c.
The implementation contains all features commonly used, but has
several problems (see TODO list).
Note that some Wine specific flag has been added to the wodOpen function,
so that the dsound DLL can share the /dev/dsp access. Currently, this
only provides mutual exclusion for both DLLs. Future extension could add
a virtual mixer between the two output streams.
TODO:
verify all functions for correctness
Add virtual mixer between wave-out and dsound interfaces.
Other sub systems
No other is available. Could think of Sun Audio, remote audio systems
(using X extensions, ...), ALSA, EsounD, ArTs...
MIDI
MMSYSTEM and WINMM call the low level driver functions using the
midMessage and the modMessage functions.
OSS driver
The low level audio driver is currently only implemented for the
OpenSoundSystem (OSS) as supplied in the Linux and FreeBSD kernels by
4Front Technologies.
The presence of this driver is checked by configure (depends on the
<sys/soundcard.h> file, and also some specific defines because
MIDI is not supported on all OSes by OSS). Source code resides in
dlls/winmm/wineoss/midi.c
Both Midi in and Midi out are provided. The type of MIDI devices
supported is external MIDI port (requires an MIDI capable device -
keyboard...) and OPL/2 synthesis (the OPL/2 patches for all
instruments are in midiPatch.c).
TODO:
use better instrument definition for OPL/2 (midiPatch.c) or use
existing instrument definition (from playmidi or kmid) with a
.winerc option
have a look at OPL/3 ?
implement asynchronous playback of MidiHdr
implement STREAM'ed MidiHdr (question: how shall we share the code
between the midiStream functions in MMSYSTEM/WINMM and the code
for the low level driver)
use a more accurate read mechanism than the one of snooping on
timers (like select on fd)
Other sub systems
Could support other MIDI implementation for other sub systems (any
idea here ?)
Could also implement a software synthesizer, either inside Wine or
using using MIDI loop back devices in an external program (like
timidity). The only trouble is that timidity is GPL'ed...
Mixer
MMSYSTEM and WINMM call the low level driver functions using the
mxdMessage function.
OSS implementation
The current implementation uses the OpenSoundSystem mixer, and resides
in dlls/winmm/wineoss/mixer.c
TODO:
implement notification mechanism when state of mixer's controls
change
Other sub systems
TODO:
implement mixing low level drivers for other mixers (ALSA...)
Aux
The AUX low level driver is the predecessor of the mixer driver
(introduced in Win 95).
OSS driver
The implementation uses the OSS mixer API, and is incomplete.
TODO:
verify the implementation
check with what is done in mixer
open question: shall we implement it on top of the low level mixer
functions ?
Wine OSS
All the OSS dependent functions are stored into the WineOSS DLL. It still
lack a correct installation scheme (as any multimedia device under Windows),
so that all the correct keys are created in the registry. This requires
an advanced model since, for example, the number of wave out devices can
only be known on the destination system (depends on the sound card driven
by the OSS interface). A solution would be to install all the multimedia
drivers through the SETUPX DLL; this is not doable yet (the multimedia
extension to SETUPX isn't written yet).
Joystick
The API consists of the joy* functions found in dlls/winmm/joystick/joystick.c.
The implementation currently uses the Linux joystick device driver
API. It is lacking support for enhanced joysticks and has not been
extensively tested.
TODO:
better support of enhanced joysticks (Linux 2.2 interface is available)
support more joystick drivers (like the XInput extension)
should load joystick DLL as any other driver (instead of hardcoding)
the driver's name, and load it as any low lever driver.
Wave mapper (msacm.drv)
The Wave mapper device allows to load on-demand codecs in order to
perform software conversion for the types the actual low level driver
(hardware). Those codecs are provided through the standard ACM
drivers.
Built-in
A first working implementation for wave out as been provided (wave in
exists, but doesn't allow conversion).
Wave mapper driver implementation can be found in dlls/winmm/wavemap/
directory. This driver heavily relies on MSACM and MSACM32 DLLs which
can be found in dlls/msacm and dlls/msacm32. Those DLLs load ACM
drivers which provide the conversion to PCM format (which is normally
supported by low level drivers). ADPCM, MP3... fit into the category
of non PCM formats.
There is currently no built-in ACM driver in Wine, so you must use
native ones if you're looking for non PCM playback.
TODO:
check for correctness and robustness
Native
Seems to work quite ok (using of course native MSACM/MSACM32 DLLs)
Some other testings report some issues while reading back the registry
settings.
MIDI mapper
Midi mapper allows to map each one of 16 MIDI channels to a specific
instrument on an installed sound card. This allows for example to
support different MIDI instrument definition (XM, GM...). It also
permits to output on a per channel basis to different MIDI renderers.
Built-in
A built-in MIDI mapper can be found in dlls/winmm/midimap/. It partly
provides the same functionality as the Windows' one. It allows to pick up
destination channels (you can map a given channel to a specific playback
device channel (see the configuration bits for more details).
TODO:
implement the Midi mapper features (instrument on the fly modification)
if it has to be done as under Windows, it required parsing the midi
configuration files (didn't find yet the specs)
Native
The native midimapper from Win 98 works, but it requires a bunch of
keys in the registry which are not part of the Wine source yet.
TODO:
add native midimapper keys to the registry to let it run. This
will require proper multimedia driver installation routines.
Mid level drivers (MCI)
The mid level drivers are represented by some common API functions,
mostly mciSendCommand and mciSendString. See status in chapter 3 for
more information. Wine implements several MCI mid level drivers
(status is given for both built-in and native implementation):
TODO: (apply to all built-in MCI drivers)
use MMSYSTEM multitasking caps instead of the home grown
CDAUDIO
Built-in
The currently best implementation is the MCI CDAUDIO driver that can
be found in dlls/winmm/mcicda/mcicda.c. The implementation is mostly
complete, there have been no reports of errors. It makes use of
misc/cdrom.c Wine internal cdrom interface.
This interface has been ported on Linux, FreeBSD and NetBSD. (Sun
should be similar, but are not implemented.)
A very small example of a cdplayer consists just of the line
mciSendString("play cdaudio",NULL,0,0);
TODO:
add support for other cdaudio drivers (Solaris...)
add support for multiple cdaudio devices (plus a decent
configuration scheme)
The DLL is not cleanly separated from the NTDLL DLL. The CDROM
interface should be exported someway (or stored in a Wine only DLL)
Native
Native MCICDA works also correctly... It uses the MSCDEX traps (on int
2f). However, some commands (like seeking) seem to be broken.
MCIWAVE
Built-in
The implementation is rather complete and can be found in
dlls/winmm/mciwave/audio.c. It uses the low level audio API (although
not abstracted correctly).
FIXME:
The MCI_STATUS command is broken.
TODO:
check for correctness
better use of asynchronous playback from low level
better implement non waiting command (without the MCI_WAIT flag).
Native
Native MCIWAVE works also correctly.
MCISEQ (MIDI sequencer)
Built-in
The implementation can be found in dlls/winmm/mciseq/mcimidi.c. Except
from the Record command, should be close to completion (except for non
blocking commands, as many MCI drivers).
TODO:
implement it correctly
finish asynchronous commands (especially for reading/record)
better implement non waiting command (without the MCI_WAIT flag).
implement the recording features
Native
Native MCIMIDI has been working but is currently blocked by scheduling
issues (mmTaskXXX no longer work).
FIXME:
midiStreamPlay get from time to time an incorrect MidiHdr when
using the native MCI sequencer
MCIANIM
Built-in
The implementation is in dlls/winmm/mcianim/.
TODO:
implement it, probably using xanim or something similar.
Native
Native MCIANIM is reported to work (but requires native video DLLs
also, even though the built-in video DLLs start to work correctly).
MCIAVI
Built-in
The implementation is in dlls/winmm/mcianim/. Basic features are present,
simple playing is available, even if lots remain to be done. It rather
heavily relies on MSVIDEO/MSVFW32 DLLs pair to work.
TODO:
finish the implementation
fix the audio/video synchronization issue
Native
Native MCIAVI is reported to work (but requires native video DLLs
also). Some files exhibit some deadlock issues anyway.
High level layers
The rest (basically the MMSYSTEM and WINMM DLLs entry points). It also
provides the skeleton for the core functionality for multimedia
rendering. Note that native MMSYSTEM and WINMM do not currently work
under Wine and there is no plan to support them (it would require to
also fully support VxD, which is not done yet).
Moreover, native DLLs require 16 bit MCI and low level drivers. Wine
implements them as 32 bit drivers.
MCI and low level drivers can either be 16 or 32 bit for Wine.
TODO:
it seems that some program check what's installed in registry
against value returned by drivers. Wine is currently broken
regarding this point.
add clean-up mechanisms when process detaches from MM DLLs
prepare for the 16/32 big split
check thread-safeness for MMSYSTEM and WINMM entry points
unicode entry points are badly supported
MCI skeleton
Implementation of what is needed to load/unload MCI drivers, and to
pass correct information to them. This is implemented in
dlls/winmm/mci.c. The mciSendString function uses command strings,
which are translated into normal MCI commands as used by
mciSendCommand with the help of command tables. The API can be found
in dlls/winmm/mmsystem.c and dlls/winmm/mci.c. The functions there
(mciOpen,mciSysInfo) handle mid level driver allocation and calls. The
implementation is not complete.
MCI drivers are seen as regular Wine modules, and can be loaded (with
a correct load order between builtin, native, so), as any
other DLL. Please note, that MCI drivers module names must bear the
.drv extension to be correctly understood.
The list of available MCI drivers is obtained as follows:
1. key 'mci' in [option] section from .winerc (or wineconf)
mci=CDAUDIO:SEQUENCER gives the list of MCI drivers (names, in
uppercase only) to be used in Wine.
2. This list, when defined, supersedes the mci key in
c:\windows\system.ini
Note that native VIDEODISC crashes when the module is loaded, which
occurs when the MCI procedures are initialized. Make sure that this is
not in the list from above. Try adding:
mci=CDAUDIO:SEQUENCER:WAVEAUDIO:AVIVIDEO:MPEGVIDEO
to the [options] section of the wine config file.
TODO:
correctly handle the MCI_ALL_DEVICE_ID in functions.
finish mapping 16 <=> 32 of MCI structures and commands
MCI_SOUND is not handled correctly (should not be sent to MCI
driver => same behavior as MCI_BREAK)
implement auto-open feature (ie, when a string command is issued
for a not yet opened device, MCI automatically opens it)
MCI multi-tasking
Multi-tasking capabilities used for the MCI drivers are provided in
dlls/winmm/mmsystem.c.
TODO:
mmTaskXXX functions are currently broken because the 16 loader does
not support binary command lines => provide Wine's own mmtask.tsk not
using binary command line.
the Wine native MCI drivers should use the mmThreadXXX API (but since
all built-in MCI drivers are 32 bit, this would require a special
flag to mark 32 bit entry points)
Timers
It currently uses a service thread, run in the context of the calling
process, which should correctly mimic Windows behavior.
TODO:
Check if minimal time is satisfactory for most programs.
current implementation may let a timer tick (once) after it has
been destroyed
MMIO
The API consists of the mmio* functions found in dlls/winmm/mmio.c.
Seems to work ok in most of the cases. There's some linear/segmented
issues with 16 bit code. There are also some bugs when writing MMIO
files.
sndPlayXXX functions
Seem to work correctly.
Multimedia configuration
Currently, multimedia configuration heavily relies on Win 3.x
configuration model.
Drivers
Since all multimedia drivers (MCI, low level ones, ACM drivers,
mappers) are, at first, drivers they need to appear in the [mci] or
[mci32] section of the system.ini file.
Since all drivers are, at first, DLLs, you can choose to load their
Wine's (built-in) or Windows (native) version.
MCI
A default [mci] section (in system.ini) looks like (see the note above
on videodisc):
[mci]
cdaudio=mcicda.drv
sequencer=mciseq.drv
waveaudio=mciwave.drv
avivideo=mciavi.drv
videodisc=mcipionr.drv
vcr=mcivisca.drv
MPEGVideo=mciqtz.drv
By default, the list of loadable MCI drivers will be made of those
drivers (in the [mci] section).
The list of loadable (recognized) MCI drivers can be altered in the
[option] section of the wine config file, like:
mci=CDAUDIO:SEQUENCER:WAVEAUDIO:AVIVIDEO:MPEGVIDEO
TODO:
use a default registry setting to bypass this (ugly) configuration
model
Low level drivers
Configuration of low level drivers is done with the Wine configuration file.
Default keys are provided in winedefault.reg.
The registry keys used here differ from the Windows' one. Using the Windows' one
would require implementing something equivalent to a (real) driver installation.
Even if this would be necessary in a few cases (mainly using MS native multimedia)
modules, there's no real need so far (or it hasn't been run into yet).
See the configuration part of the User's Guide for more details.
Midi mapper
The Midi mapper configuration is the same as on Windows 9x. Under the key
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Multimedia\MIDIMap
if the 'UseScheme' value is not set, or is set to a null value, the midi
mapper will always use the driver identified by the 'CurrentInstrument'
value. Note: Wine (for simplicity while installing) allows to define
'CurrentInstrument' as "#n" (where n is a number), whereas Windows only
allows the real device name here. If UseScheme is set to a non null value,
'CurrentScheme' defines the name of the scheme to map the different channels.
All the schemes are available with keys like
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\MediaProperties\PrivateProperties\Midi\Schemes\%name_of_scheme%
For every scheme, under this key, will be a sub-key (which name is usually
a two digit index, starting at 00). Its default value is the name of the
output driver, and the value 'Channels' lists all channels (of the 16
standard MIDI ones) which have to be copied to this driver.
To provide enhanced configuration and mapping capabilities, each driver
can define under the key
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\MediaProperties\PrivateProperties\Midi\Ports\%driver_name%
a link to and .IDF file which allows to remap channels internally (for
example 9 -> 16), to change instruments identification, event
controllers values. See the source file dlls/winmm/midimap/midimap.c
for the details (this isn't implemented yet).
ACM
To be done (use the same mechanism as MCI drivers configuration).
VIDC
To be done (use the same mechanism as MCI drivers configuration).
Multimedia architecture
Windows 9x multimedia architecture
|
Kernel space | Client applications
|
| | | ^ ^ | | | |
| 16>| |<32 16>| |<32 16>| |<32 16>| |<32
| | v | | | v | v
| +----|-----------|---------|------------|-------+
| | | | | | | WinMM.dll
| | | | | | | 32 bit
| +----|-----------|---------|------------|-------+
| | | | ^ | | |
| +------+ | |<16 | | | |<16 |
| | 16>| | | | | | | |
| | v v v | | v v v
| | +---------------+---+-------------+-------------+
| | | waveInXXX | | mciXXX | *playSound* |
| | | waveOutXXX | | | mmioXXX |
| | | midiInXXX | | | timeXXX |
| | | midiOutXXX | | | driverXXX |
| | | midiStreamXXX | | | | MMSystem.dll
| | | mixerXXX | | | | 16 bit
+--------+ | | | auxXXX +---+ +---+ mmThread| |
|MMDEVLDR|<------->| joyXXX | Call back | mmTask | |
+--------+ | | +-----------+-----------+---------+-------------+
^ | | | ^ ^ | ^
| | | 16>| |<16>| 16>| |<16
v | | v | | v |
+--------+ | | +-------------+ +----------+
| VxD |<------->| *.drv | | mci*.drv |
+--------+ | | +--------------+ +-----------+
| | | msacm.drv | | mciwave |
| | +--------------+ +-----------+
| | | midimap.drv | | mcimidi |
| | +-------------+ +-----------+
| | Low-level drivers | ... | MCI drivers
| | +----------+
| | |
| | |<16
| +-------------------------------+
|
The important points to notice are:
all drivers (and most of the core code) is 16 bit
all hardware (or most of it) dependent code reside in the kernel
space (which is not surprising)
Wine multimedia architecture
|
Kernel space | Client applications
|
| | | ^ ^ | | | |
| 16>| |<32 16>| |<32 16>| |<32 16>| |<32
| | | | | | | | |
| +------+ | | | | | | | |
| |32/16>| | | | | | | | |
| | v v v | | v v v v
| | +---------------+---+-------------+-------------+
| | | waveInXXX | | mciXXX | *playSound* |
| | | waveOutXXX | | | mmioXXX | WinMM.dll
| | | midiInXXX | | | timeXXX | 32 bit
| | | midiOutXXX | | | driverXXX |
| | | midiStreamXXX | | | | MMSystem.dll
| | | mixerXXX | | | | 16 bit
| | | auxXXX +---+ +---+ mmThread| |
| | | joyXXX | Call back | mmTask | |
| | +-----------+-----------+---------+-------------+
| | || ^ ^ || ^^
| | 16>||<32 |<16>| 16>||<32>||<16
| | vv |<32>| vv ||
+---------+ | | +-------------+ +----------+
|HW driver|<------->| *.drv | | mci*.drv |
+---------+ | | +--------------+ +-----------+
| | | msacm.drv | | mciwave |
| | +--------------+ +-----------+
| | | midimap.drv | | mcimidi |
| | +-------------+ +-----------+
| | Low-level drivers | ... | MCI drivers
| | +----------+
| | |
| | |<32/16
| +-------------------------------+
|
From the previous drawings, the most noticeable differences are:
low-level drivers can either be 16 or 32 bit
MCI drivers can either be 16 or 32 bit
MMSystem and WinMM will be hosted in a single elfglue library
no link between the MMSystem/WinMM pair on kernel space shall
exist. For example, there will be a low level driver to talk to a
UNIX OSS (Open Sound System) driver
all built-in drivers (low-level and MCI) will be written as 32 bit
drivers
all native drivers will be 16 bits drivers
MS ACM Dlls
Contents
tbd
Status
tbd
Caching
The MSACM/MSACM32 keeps some data cached for all known ACM
drivers. Under the key
Software\Microsoft\AudioCompressionManager\DriverCache\<driver
name>
are kept for values:
aFormatTagCache which contains an array of
DWORD. There are two DWORDs per cFormatTags
entry. The first DWORD contains a format tag
value, and the second the associated maximum
size for a WAVEFORMATEX structure.
(Fields dwFormatTag and cbFormatSize from
ACMFORMATDETAILS)
cFilterTags contains the number of tags supported by the driver
for filtering.
cFormatTags contains the number of tags support
by the driver for conversions.
fdwSupport (the same as the one returned from
acmDriverDetails).
The cFilterTags, cFormatTags, fdwSupport are the same
values as the ones returned from acmDriverDetails
function.