Q-Midi 1.11
====== ====

This is the Q MIDI module, which allows you to create simple MIDI applications
with the Q programming language. The module runs on top of Grame's excellent
portable MidiShare package, see http://www.grame.fr/MidiShare/. The currently
supported platforms are Linux, Mac OS X and Windows.

As of release 1.6, this module has been dubbed "stable". There's no real
manual yet, but see below for essential information to get you started. Please
also refer to the the midi.q script and the MidiShare developer documentation
for more information.  Some basic examples can be found in the midi_examp.q
script provided with the package.

This package is distributed under the GPL V2 (see COPYING), and requires the Q
programming system, which can be found at the Q homepage:

	http://www.musikwissenschaft.uni-mainz.de/~ag/q.

Make sure you get Q version 4.2 or later.

Under the above URL you'll also find the latest Q-Midi sources and binary
packages. Currently SuSE 8.x rpm and a Windows installer (msi) files are
available; the latter also includes the necessary dlls and config files for
running MidiShare under Windows. A binary package for OS X will soon be
available, too.


INSTALLATION - LINUX
============ = =====

You first have to get MidiShare at http://www.grame.fr/MidiShare/ and install
it on your system. You can also get a SuSE 8.x rpm for the latest stable
MidiShare version from the Q homepage. Then, if you have a SuSE 8.x system
with Q installed under /usr, you can simply grab q-midi-<version>.rpm and
install it with rpm. (If you have installed MidiShare from the sources, rpm
may complain about an unsatisfied dependency on libMidiShare.so. In this case
you can disable rpm's dependency checks to install the package anyway.)

If you want to compile from the sources, you'll need both the MidiShare and
POSIX thread libraries. See INSTALL for generic UNIX installation
instructions. If you have the required libraries installed in the standard
locations, running

	./configure; make; make install

should do the job. Make sure that you get the installation prefix right (it
should match Q's installation prefix). If necessary, you can specify the
prefix with configure's --prefix option. You might also wish to add some
compilation options. E.g.:

	CFLAGS=-O3 ./configure --prefix=/usr; make; make install

Note that the Linux version of MidiShare consists of both a shared library and
a kernel module. Q-Midi won't work unless the kernel module is running on your
system. You can either start up the module manually (with modprobe) or install
a startup script in your /etc/init.d directory which starts the module at boot
time; see the MidiShare docs for details. (This is all taken care of
automatically when you install the midishare rpm from the Q homepage, see
above.)

To verify that everything has been installed properly, fire up the midi.q
script and evaluate `midi_ready'; it should return `true'. If the midi module
does not load (you'll see an error message and complaints about a lot of
unresolved externals in this case) then probably libMidiShare.so is missing.
If the module loads, but `midi_ready' returns `false' then maybe the MidiShare
kernel module is not running. You can verify this with `lsmod | grep
MidiShare'.

You might also wish to take a look at some of the tools distributed with
MidiShare, like the msconnect and msdisplay programs, which are *very* useful
for debugging purposes.

Another useful program is fluidsynth (formerly called iiwusynth), available
from http://www.fluidsynth.org/. This is a SoundFont-compatible software
synthesizer which can be run as a MidiShare driver client, in order to render
MIDI on your soundcard instead of an external MIDI device. (Note that
fluidsynth does not require a SoundFont-compatible soundcard, so any type of
soundcard will do. You just need a suitable SoundFont file. If you do have a
SoundFont-compatible soundcard like the SBLive, you should be able to find
suitable SoundFont files on your driver CD; otherwise you can find some on the
Internet, or buy them from Emu or Creative.)

Make sure you configure fluidsynth with MidiShare support (./configure
--enable-midishare) when compiling it from the sources. Run fluidsynth as a
MidiShare driver as follows:

	$ fluidsynth -m midishare soundfont.sf2

(Replace soundfont.sf2 with the pathname of your SoundFont file.) The driver
can then be accessed as client "fluidsynth" from MidiShare and Q-Midi
applications. Or you can use the mididev.q script which allows you to access
fluidsynth as MIDIDEV!1 (see "PORTABLE MIDI DEVICES" below.)


INSTALLATION - OS X
============ = == =

Get MidiShare at http://www.grame.fr/MidiShare/ and install it on your
system. There is a binary release available for OS X, thus you might wish to
use that (if you compile Q-Midi yourself then you need both the user and the
developer kit). Follow the instructions in the application directory to define
the MIDI ports with the msDrivers application.

To compile from the sources, you need Apple's developer toolkit (gcc and
friends). Unpack the Q-Midi sources, configure and compile as usual:

	./configure; make; make install

Make sure that you get the installation prefix right (it should match Q's
installation prefix). If necessary, you can specify the prefix with
configure's --prefix option. You might also wish to add some compilation
options. E.g.:

	CFLAGS=-O3 ./configure --prefix=/usr; make; make install


INSTALLATION - WINDOWS
============ = =======

(I've tested this on Windows 98 and 2000, but hopefully it works on other 32
bit Windows flavours as well.)

Get Q-Midi-<version>.msi, double-click that file and go through the install
procedure. Select the installation folder into which you already installed the
Qpad package.

If you have not installed MidiShare before, you'll have to configure the MIDI
driver ports with the msDrivers application. To do this, go to the Qpad
folder, run msDrivers.exe and follow the instructions in the MidiShare
Docs\Setup.html file. Once you've completed this step, you should have a
msMMSystem.ini file. Copy that file along with the other *.ini files and
msDrivers.exe to your WINNT (or WINDOWS) folder. Make a shortcut to
WINNT\msDrivers.exe on your desktop so that you can run it quickly when you
want to change the port definitions.

If you are going to use the portable device definitions in mididev.q, you
should also make sure that the your msDrivers configuration provides some
"standard" port assignments; see "PORTABLE MIDI DEVICES" below for details.


PLAYER
======

This release comes bundled with Player 1.1, a sample Q-Midi script which
implements a full graphical MIDI player/recorder. To run this script, you'll
need Tcl/Tk. See README-Player for further information.


USAGE
=====

Once Q-Midi has been installed, you can give it a try. Go to the examples/midi
directory and run the midi_examp.q script with the Q interpreter. Evaluate
`record' and play some notes on your keyboard. Press <CR> to finish the
recording. If everything has been configured properly, the recorded sequence
should be printed as a list of (TIME,MSG) pairs, and you can play back the
sequence with `play _'; press <CR> again to terminate the playback. To play
the prelude3.mid file included in the distribution, evaluate `play (load
"prelude3.mid")'.

To use the MIDI interface in your own scripts, just include/import the midi.q
script. Take midi_examp.q as a start and modify it for your purposes. Have
fun. :)


PROGRAMMING
===========

Most of the basic MidiShare functionality is implemented in this release, and
appropriate data types and external functions are provided to handle MidiShare
clients and connections, MIDI I/O, and MIDI file access. The interface is
divided into several different sections described in the following.


GENERAL FUNCTIONS
======= =========

The `midi_ready' and `midi_version' functions are used to check that MidiShare
is up and running, and which MidiShare version is installed on your system.
Note that if `midi_ready' returns false, then *all* other operations of this
module will fail.

Some functions for managing MidiShare's internal event memory are also
provided. The `midi_total_space' function returns the total number of event
cells in the memory pool, `midi_free_space' the number of currently unused
cells available to clients. At startup, Q-Midi makes sure that a reasonably
large number (usually 60000) of cells is available. This should suffice for
most applications. However, some functions of this module may fail if not
enough internal memory is available to complete the operation. This can
happen, e.g., when reading huge tracks from a MIDI file. In this case you can
increase the size of the internal memory pool with the `midi_grow_space'
function.


MIDI MESSAGE TYPE
==== ======= ====

For convenience, MIDI messages are not represented as raw byte sequences, but
are mapped into constructor terms of the `MidiMsg' type. Take a look at the
definition of this type in midi.q to see which types of messages are supported
and how they are encoded. The messages listed there should cover just about
anything that you'll ever encounter in a MIDI file or on an instrument, plus
some MidiShare-specific extensions. For instance, a "note on" message will be
represented as `note_on CHAN PITCH VEL', and a "system exclusive" message will
look like `sysex [BYTE1,BYTE2,...]'. Pretty obvious, isn't it? The other types
of messages are encoded in a fairly straightforward manner as well.

We do not describe the semantics of the different messages here; you can find
a closer description of all standard MIDI messages (albeit in raw format) on
http://www.borg.com/~jglatt/. The MidiShare-specific messages supported in
this release are:

- `note CHAN PITCH VEL DUR': a note sounding for a duration DUR. On output to
  a MIDI device, MidiShare will expand this to a sequence of "note on" and
  "off" messages with the appropriate timing.

- `ctrl14b_change CHAN CTRL VAL': set the value of a 14 bit controller CTRL in
  the range 0..31. This can be used to replace two separate `ctrl_change'
  messages affecting corresponding coarse and fine controllers.

- `non_reg_param CHAN PARAM VAL', `reg_param CHAN PARAM VAL': set (non-)
  registered parameters with a single message. This expands to a corresponding
  sequence of controller messages.

- `midi_stream [BYTE1,BYTE2,...]': this message allows you to send raw
  (unencoded) MIDI messages as a literal byte stream.

Other (mostly internal) MidiShare-specific messages are encoded as the
constant `unknown' if they are encountered in a MIDI stream.

Besides the MidiMsg type itself, the module also provides various functions
for determining the category of a message (is_voice, is_system_common, etc.),
and to check for specific messages (is_note_on, is_note_off, ...). These are
useful, e.g., for filtering purposes.


DRIVER MANAGEMENT
====== ==========

These functions allow you to determine which MidiShare drivers are loaded,
which hardware ports ("slots") they have, and how the 256 logical MidiShare
ports are connected to the driver ports. Note that only the application-side
subset of MidiShare's driver management functions are provided. Also note that
currently (as of MidiShare 1.86) these operations are *not* supported in the
Linux version. For the time being, a portable application should instead use
the device table implemented by the mididev.q script, see "PORTABLE MIDI
DEVICES" below.

The `midi_drivers' function returns the reference numbers of all loaded
drivers. Information about a specific driver can be retrieved with
`midi_driver_info'. Driver information is represented as a triple
(NAME,VERSION,SLOTS) denoting the name of the driver, its version number (an
integer) and a list of the slots it provides. Each slot is given as a pair
consisting of the driver number and a slot number.

Information about a driver slot is encoded as a triple (NAME,DIR,PORTS)
denoting the name of the slot, its direction (1 = input only, 2 = output only,
3 = input and output), and a list of the MidiShare ports it is connected
to. This information can be retrieved with `midi_slot_info'.

The `midi_slot_connect' and `midi_slot_disconnect' operations allow you to
specify how the 256 logical MidiShare ports are connected to the physical
driver slots. The current status of a connection can be verified with
`midi_slot_connected'.

Moreover, some convenience functions (`midi_ports', `midi_slots' and
`midi_port_info') are provided to see which ports and slots are available on
your system, and which slots a given port is connected to.


SESSION MANAGEMENT
======= ==========

In order to perform MIDI I/O, you first have to register a client (also called
an "application" in MidiShare terminology) with MidiShare. Each client has a
name (a string) and a reference number (an integer) by which it is known to
the MidiShare system. The name is set with the `midi_open' function. The
reference number returned by `midi_open' can then be used in MIDI I/O
operations. A client is removed with the `midi_close' function, or
automatically when exiting the Q interpreter. Multiple clients can be
registered in a single Q script, which is useful if several tasks each
performing their own MIDI I/O have to be executed.

Note that most of the MIDI file and I/O related operations of this module will
only work properly if at least one client has been registered. This is
achieved most easily by including a definition like

	def MYREF = midi_open "my application name";

in your script.

The list of (the reference numbers of) all clients currently known to the
MidiShare system -- including clients registered by other application programs
-- can be retrieved with the `midi_clients' function. (Note that you'll only
see MidiShare clients registered by other processes in the Linux version.
MidiShare NT/2K can only access clients within the same address space, i.e.,
the clients must reside in the same process. The same applies to the MidiShare
"monoapp" version under OS X.) You can also get and set the name of a client
with the `midi_client_name' and `midi_set_client_name' routines. Finally, the
reference number of a client given by its name can be obtained with
`midi_client_ref'.

Moreover, the module also provides a convenience function for accessing an
"external" client, starting up the client with a shell command if necessary.
This is most useful for managing driver clients under Linux (see mididev.q for
some examples). The `midi_client' function looks for the reference number of a
client given by its name, and, if the client is not present already, it will
start the client with the given shell command and wait until it shows up in
the client list. In either case, the function returns the client's reference
number. Note that the client's startup command should be run in the background
(if necessary, use `&' at the end of the command to achieve this).


CONNECTION MANAGEMENT
========== ==========

The `midi_connect' and `midi_disconnect' functions are used to specify how
MIDI messages are to be routed between different clients (including clients
hosted by other application programs). When a MIDI message is sent from a
given client, it will be routed to all target clients to which the source
client is connected. The status of existing connections can be checked with
the `midi_connected' function. Moreover, the `midi_in_connections' and
`midi_out_connections' convenience functions return the lists of clients for
incoming and outgoing connections of the given client, respectively.

These routines are also used to connect a client to physical MIDI devices.
Under OS X and Windows, it is sufficient to connect with client #0 (the
predefined "MidiShare" client) which manages the MIDI I/O drivers according to
the port definitions specified with the msDrivers application. Under Linux,
drivers are currently (as of MidiShare 1.86) implemented as ordinary clients,
such as the msRawMidi driver distributed with the Linux version of
MidiShare. Here you have to connect directly with the MIDI driver client which
you want to use. To achieve some degree of portability, this should be done
using the device descriptions in mididev.q; see "PORTABLE MIDI DEVICES" below
for details.


FILTERING
=========

These functions allow you to determine the messages a client wants to receive
with `midi_get' (see MIDI MESSAGING, below). By default, all ports, channels
and message types are received. The `midi_accept_port', `midi_accept_chan' and
`midi_accept_type' functions can be used to restrict the messages which are
received by a client, on the basis of the source/destination port, channel,
and type of a message. Message types are specified using the corresponding
constructor symbol, as in `midi_accept_type REF ctrl_change false' which will
cause all ctrl_change messages to be suppressed. For convenience, some
predefined lists of message constructors let you specify the various message
types. E.g., filtering out all voice messages for a given client REF can be
done as follows:

	do (flip (midi_accept_type REF) false) voice

The current filter states can be checked with `midi_is_accepted_port',
`midi_is_accepted_chan' and `midi_is_accepted_type'. To turn off all filtering
you can use the `midi_accept_all' function.


TIMING
======

MidiShare maintains an internal 32 bit time value which counts the time in
milliseconds which elapsed since MidiShare was last activated. The current
value of this counter can be accessed with the `midi_time' function. A client
can also wait for a specific time to arrive with the `midi_wait' function,
which sleeps until the given time and then returns that time value. (The TIME
argument of `midi_wait' is taken modulo 0x100000000 automatically, so you do
not have to worry about wrapover at the end of the time range.) Applications
typically use this function to perform scheduling of MIDI events.


MIDI MESSAGING
==== =========

The `midi_send' and `midi_get' routines provide the basic functionality of
sending and receiving MIDI events.

Use `midi_send REF PORT MSG' to send a MIDI message MSG (of type `MidiMsg')
immediately, where REF denotes the sending client and PORT the destination
port. (Under OS X and Windows, the ports are associated with suitable hardware
devices with the msDrivers application. Under Linux, the port number is
currently ignored by the drivers and can always be set to zero. See "PORTABLE
MIDI DEVICES" below for details.) Instead of MSG, you can also specify a
(TIME,MSG) pair to send MSG at the specified TIME value, which is taken modulo
0x100000000 automagically. The message will then be sent by MidiShare when the
current `midi_time' (see above) becomes equal to TIME.

Incoming MIDI messages are received with the `midi_get' function which returns
a (REF,PORT,TIME,MSG) tuple denoting the reference number of the client the
message was received from, the port of the message (which will be the source
port if the message was received from a MIDI I/O driver, and the destination
port specified with `midi_send' if the message was received from a regular
client), the time when the message was received, and the message (of type
`MidiMsg') itself. Note that unlike the corresponding MidiShare function
MidiGetEv, this function is blocking, i.e., it waits for a message to become
available. You can also check beforehand whether a message is currently
available with the `midi_avail' function, and empty a client's input queue
with the `midi_flush' function.


REALTIME ISSUES
======== ======

When implementing a realtime loop (such as a sequencer) it is often essential
that timing is performed very accurately and MIDI events are always processed
immediately. This can only be guaranteed when the thread executing the
realtime loop runs at a high priority, which can be achieved with clib's
setsched function (see the Clib section in the Q manual for details). An
example for the use of setsched in realtime Q-Midi programs can be found in
midi_examp.q.


MIDI FILE I/O
==== ==== ===

These functions let you read and write MIDI files. MIDI files are represented
using the extern `MidiFile' type. Objects of this type are returned by the
`midi_file_open', `midi_file_append' and `midi_file_create' functions, which
open a file for reading, append to an existing file and create a new file,
respectively. Each of these functions takes the name of the file to be opened
as its (first) argument. A `MidiFile' object can be closed with the
`midi_file_close' function (this operation is also performed automatically
when a `MidiFile' object is garbage-collected).

MIDI files can have one of three formats, 0 = single track, 1 = multiple
tracks/single song, 2 = multiple tracks/multiple songs. By definition, a
format 0 file consists of a single track, into which all MIDI events go. A
format 1 file (the most common format used by sequencers) contains several
tracks forming a single piece; the tempo map of the piece is to be found in
the first track. A format 2 file has several independent tracks which are
to be played separately; in this case each track contains its own tempo
map.

The MIDI file header information also describes how many tracks the file
contains and which "division" the timestamps of events in the file are based
upon. The division can be either a number denoting pulses per quarter note
(PPQN), or an SMPTE type division consisting of a number of frames per seconds
(24, 25, 29 or 30) together with the number of subdivisions of each frame. In
Q land, division is represented either as an integer (PPQN) or as a
(FRAMES_PER_SEC,TICKS_PER_FRAME) pair.

The MIDI file header information can be determined with the
`midi_file_format', `midi_file_division' and `midi_file_num_tracks'
functions. Moreover, the "current" time of the file (time of the last event
read or written) can be retrieved with the `midi_file_time' function, and you
can check whether the file has been opened for reading or writing with the
`midi_file_mode' function.

For both reading and writing, the events in each track are represented as
(TIME,MSG) pairs, where TIME is the time of the event and MSG is the MIDI
message of type `MidiMsg'. Tracks are represented as event lists. Note that
the time values are always absolute (zero-based) timestamps, whereas they are
actually stored as "deltas" (time difference w.r.t. the previous event) in a
MIDI file. Moreover, the timestamps are measured in "ticks", i.e., in the unit
specified by "division", thus you'll have to use the value of "division" and
the tempo map of the piece to tranform these values to milliseconds, if you
want to play back the sequence.

The tracks in a MIDI file opened for reading can be accessed in sequence,
either by reading whole tracks at once with `midi_file_read_track', or by
opening a track with `midi_open_track' and reading individual events from it
with `midi_file_read'. The `midi_file_read' function will fail (and the track
will be closed, advancing the file pointer to the next track) when no more
events are available. (In case of another error condition, like a format
error, `midi_file_read' will fail as well, but the track will remain open. You
can check for this condition with the `midi_file_track_open' function.) You
can also close a track explicitly with the `midi_file_close_track' function.

Moreover, when a  MIDI file has been opened for reading, you can position the
file pointer at an arbitrary track with `midi_file_choose_track', which takes
a zero-based track index as argument.

Creating a new or appending to an existing MIDI file works analogously. A new
track is written at the end of the file with the `midi_file_write_track'
function. You can also use `midi_file_new_track', `midi_file_write' and
`midi_file_close_track' to write a track event by event.

Note that random track access is not supported while writing, i.e., the
`midi_file_choose_track' function can only be used while reading a file. New
tracks can only be added at the end of the file.


PORTABLE MIDI DEVICES
======== ==== =======

We have already pointed out that in order to actually perform I/O from/to real
MIDI devices, you have to connect to a corresponding driver client. Moreover,
when sending a MIDI message to a device you also have to specify the right
port value in the call to midi_send.

Unfortunately, the current MidiShare version treats driver clients differently
depending on which system you use. Under OS X and Windows, all MIDI devices are
accessed via MidiShare client #0, and the particular device a message should
be sent to is specified using the corresponding port value; the port values in
turn are configured with the msDrivers application. Under Linux, however,
different devices are accessed using different clients and the port values
don't matter at all so you can always set them to zero.

This makes it hard to handle MIDI devices in a truly portable manner. The
described incompatibility is expected to be rectified in a future MidiShare
version. For the time being, Q-Midi includes an additional script named
`mididev.q' which implements a portability layer for accessing MIDI devices on
all supported platforms. In particular, the script provides the MIDIDEV
variable, which is a list of MIDI device descriptions. Each entry of the
device table is a (NAME,REF,PORT) tuple which specifies a symbolic name NAME
for the device and the MidiShare client and port numbers REF and PORT with
which the device can be adressed. To simplify matters, it is recommended to
always have some "standard" device entries at the beginning of the table which
are available to all applications. The current implementation uses the
following setup:

- MIDIDEV!0 is the external MIDI synthesizer (input/output)
- MIDIDEV!1 is the internal MIDI synthesizer (input only)
- MIDIDEV!2 is the network (input/output)

The third entry is only required for networked applications. Under OS X and
Windows, you should make sure that the first two or three port assignments in
your msDrivers setup match the above specifications. Then your applications
can use the same standard entries for all platforms. For instance, for a
typical setup your script might contain the following definitions:

	def (_,IN,_) = MIDIDEV!0, (_,OUT,PORT) = MIDIDEV!1;

This will assign the client number of the external MIDI driver to the variable
IN, and the client and port numbers of the internal MIDI driver to the
variables OUT and PORT. You can then connect IN to your application's input to
get incoming MIDI messages from the external MIDI interface with midi_get, and
connect your application's output to OUT to output MIDI messages on the
internal synth with midi_send (specifying PORT as the port number with
midi_send).

An example for the use of mididev.q can be found in midi_examp.q.


Enjoy! :)

June 2003
Albert Graef
ag@muwiinfa.geschichte.uni-mainz.de, Dr.Graef@t-online.de
http://www.musikwissenschaft.uni-mainz.de/~ag