# Libquicktime dynamic codec interface

Programmed in 2002 by Burkhard Plaum (gmerlin_at_users.sourceforge.net)

1 Notes for application users
2 API extensions for application programmers
3. API for Codec developers
3.1 Introduction
3.2 Creating and deleting codecs
3.3 Presenting ourselves to the outer world
3.4 Passing codecs to libquicktime
3.5 Encoding video
3.6 Decoding video
3.7 Encoding audio
3.8 Decoding audio

### 1. Notes for application users

Libquicktime has dynamic codecs, which are installed in <prefix>/lib/libquicktime. Here, prefix is the same prefix, you used, when you compiled libquicktime (/usr/local by default). If you installed a binary package, you will find the plugins in /usr/lib/libquicktime. The plugin directory is currently hardcoded in the library and can only be changed by recompiling.

When your application tries to open a quicktime file with libquicktime, it will first search for the file .libquicktime_codecs in your home directory. Then it scans the plugin directory for files lqt_*.so. (The prefix lqt_ was choosen, to prevent confusion with regular shared libraries).

For all dynamic plugins, it checks, if they are in the database (.libquicktime_codecs). If they are not there, or if the dynamic plugin is younger, than the database entry, libquicktime opens the plugin and reads the information (name, capabilities of each codec etc) from the plugin. If the database entry however is still valid, the information is taken from the database to avoid the time consuming loading of the plugin. After the whole directory is scanned, the .libquicktime_plugins file is written with the updated codec data.

With this mechanism, you will always have the new codecs enabled automatically when you start a new libquicktime application. If your application supports it, you can even register new codecs without restarting the program (See below, how application programmers can implement this feature).

### 2. API extensions for application programmers

2.1 Maintaining the registry

Libquicktime has a codec registry (see above) defined in a private linked list. It allows applications to build configuration dialogs for codecs dynamically at runtime depending on the codecs, the user has installed. This is necessary for the separate distribution of libquicktime codecs.

The registry is initialized automatically by the first access to codec informations, with the exception of the non thread save functions (see below). If you want to access the registry with the non thread save functions before your application openend the first file, call:
`lqt_registry_init();`
If you want to enable newly installed codecs without restarting your application, destroy the registry with:
`lqt_registry_destroy();`
and call lqt_registry_inif() to rebuild it again. By default, the file .libquicktime_codecs is written once each time the registry is initialized. If you want to write the file at a time of your choice, call:
`lqt_registry_write();`
Libquicktime also has an (not completely finished) API extension for handling default values for codec parameters. To store a certain default value for a codec in the registry, use:
`void lqt_set_default_parameter(lqt_codec_type type, int encode,``
` `const char * codec_name,``
` `const char * parameter_name,``
` `lqt_parameter_value_t * val);`
Remember that you need to call lqt_registry_write() to save the new default value permanently.

2.2 The codec info structure

The structure for all codec data is defined as follows:

`struct lqt_codec_info_s`
`{`
`/* These are set by the plugins */`
``
`char * name; /* Name of the codec */`
`char * long_name; /* Long name of the codec */`
`char * description; /* Description */```

`lqt_codec_type type;`
`lqt_codec_direction direction;`
``
`int num_fourccs; /* Fourccs, this codec can handle */`
`char ** fourccs;`
``
`int num_encoding_parameters;`
`lqt_parameter_info_t * encoding_parameters;```

`int num_decoding_parameters;`
`lqt_parameter_info_t * decoding_parameters;```

`/* Colormodels this codec can handle */`
``
`int num_encoding_colormodels;`
`int * encoding_colormodels;```

`/*`
`* Colormodel for decoding.`
`* Must be set to LQT_COLORMODEL_NONE if the stream colormodel`
`* must be obtained at runtime by the codec`
`*/`
``
`int decoding_colormodel;`
``
`/* The following members are set by libquicktime */`
``
`char * module_filename; /* Filename of the module */`
`int module_index; /* Index inside the module */`
``
`uint32_t file_time; /* File modification time */`
``
`struct lqt_codec_info_s * next; /* For chaining */`
`};```

`typedef struct lqt_codec_info_s lqt_codec_info_t;`

Most of these members are self explaining. The only thicky thing are the colormodels. They indicate the the native colormodels of the codec, which can be read and written directly without colorspace conversion. A codec can have more than one native colormodel (e.g. the png codec supports RGB as well as RGBA), but in this case the best decoding colormodel depends on the other stream parameters (e.g. on the video depth for png).
If a codec supports more colormodels, it has all of them in the encoding_colormodels array. The application can then choose one these, if a file is written. In this case, the decoding_colormodel is set to LQT_COLORMODEL_NONE. This indicates, that libquicktime will try to get the stream colormodel from the codec if the file is open (see below).
In most cases, however, there will be only one native colormodel, so there will be only one encoding colormodel, and the decoding colormodel will be known at compile time.
The next pointer should not be used by applications. To simplify the access to the data, all codec informations are passed as NULL terminated arrays to the outer world.

You can get the encoding and decoding parameters in the parameter_info_t structure:

`typedef struct`
`{`
`char * name; /* Parameter name (to be passed to quicktime_set_parameter() ) */`
`char * real_name; /* Other name (for making dialogs) */`
``
`lqt_parameter_type_t type; /* LQT_INT, LQT_STRING or LQT_STRINGLIST */`
``
`lqt_parameter_value_t val_default;```

`/*`
`* Minimum and maximum values:`
`* These are only valid for numeric types and if val_min < val_max`
`*/```

`int val_min;`
`int val_max;```

`/*`
`* Possible options (only valid for LQT_STRINGLIST)`
`*/`
``
`int num_stringlist_options;`
`char ** stringlist_options;`
``
`} lqt_parameter_info_t;`
``

The minimum and maximum values are only valid for integer parameters and if val_min < val_max. If integer parameters have a minimum of 0 and a maximum of 1, they should be considered as boolean (applications will make a check button instead of a slider for this). The stringlist is for enumerated types, the possible options are in stringlist_options.

lqt_parameter_value_t is a union, which can carry values of different datatypes:

`typedef union`
`{`
`int val_int;`
`char * val_string;`
`} lqt_parameter_value_t;`

2.3 Getting codec informations (the easy way):

If you can make sure, that no quicktime files are openend or closed, while another thread rebuilds the registry, you can access the registry directly. Note, that these won't initialize the registry automatically. Get the number of audio- and videocodecs with:

`int lqt_get_num_audio_codecs();`

`int lqt_get_num_video_codecs();`

For each codec index (starting with zero), you can obtain an info structure with:

`const lqt_codec_info_t * lqt_get_audio_codec_info(int index);`

`const lqt_codec_info_t * lqt_get_video_codec_info(int index);`

These functions bypass the locking machanism of the registry and return pointers to the internal data.

2.4 Getting codec informations (the harder way):

There is another group of registry access functions, which lock the registry, copy the informations into the return value and unlock the registry. They are thread save, but you must free the retruned data expicitly. All these functions return a NULL terminated lqt_codec_info_t* array, even if they retrun always only one codec.
For getting any combination of audio/video en/decoders, use:

`lqt_codec_info_t ** lqt_query_registry(int audio, int video, int encode, int decode);`

You can use it to show the commanline user a list of supported codecs or for building dialogs. To find a codec by it's individual name, use:

`lqt_codec_info_t ** lqt_find_audio_codec_by_name(const char * name);```

`lqt_codec_info_t ** lqt_find_video_codec_by_name(const char * name);```

If you want to know something about the codecs of an open quicktime file, use:

`lqt_codec_info_t ** lqt_audio_codec_from_file(quicktime_t *, int track);```

`lqt_codec_info_t ** lqt_video_codec_from_file(quicktime_t *, int track);```

There is one function, which will free all the data returned from the above functions:

`void lqt_destroy_codec_info(lqt_codec_info_t ** info);`
``

Currently, there is no codec with support for decoding parameters, but this might change in future versions.

### 3. API for Codec developers

3.1 Introduction

The codecs itself work the same way as quicktime4linux codecs. Several codecs (any combination of audio/video en- and decoders) can be in one shared module. How to define the codecs and how to write the interface functions for access by libquicktime is described here. For the codec interface specific parts of your module, you always need to include the file lqt_codecapi.h.

A quicktime codec is defined in qtprivate.h as:``

`typedef struct
{
int (*delete_vcodec)(quicktime_video_map_t *vtrack);
int (*delete_acodec)(quicktime_audio_map_t *atrack);
int (*decode_video)(quicktime_t *file, unsigned char **row_pointers, int track);
int (*encode_video)(quicktime_t *file, unsigned char **row_pointers, int track);
int (*decode_audio)(quicktime_t *file, int16_t *output_i, float *output_f,
long samples, int track, int channel);
int (*encode_audio)(quicktime_t *file, int16_t **input_i, float **input_f,
int track, long samples);
int (*reads_colormodel)(quicktime_t *file, int colormodel, int track);
int (*writes_colormodel)(quicktime_t *file, int colormodel, int track);
int (*set_parameter)(quicktime_t *file, int track, char *key, void *value);
void (*flush)(quicktime_t *file, int track);
void *priv;
void *module; /* Needed by libquicktime for dynamic loading */`
`char *name;
} quicktime_codec_t;`

The module pointer is used by libquicktime to dlclose() us, if we are not longer needed. It should not be touched. The functions pointers are your actual en- and decoding functions (how they are written and what they do is described below).

3.2 Creating and deleting codecs

The interface function, which creates one codec sets the members of the quicktime_codec_t structure to the functions defined in our module. It must be defined in the sourcecode for each codec separately.

Example: You have written a video codec blup, which keeps additional data in a structure blup_t. The structure is created by blup_create_codec() and destroyed by blup_delete_codec(). Your initialization function will look like this (if blup is a video codec):

`void quicktime_init_codec_blup(quicktime_video_map_t *vtrack)`
`{`
`blup_t *codec = blup_create_codec();`
`/* Init public items */`
`((quicktime_codec_t*)vtrack->codec)->priv = codec;`
`((quicktime_codec_t*)vtrack->codec)->delete_vcodec = delete_codec;`
`((quicktime_codec_t*)vtrack->codec)->decode_video = decode;`
`((quicktime_codec_t*)vtrack->codec)->encode_video = encode;`
`((quicktime_codec_t*)vtrack->codec)->reads_colormodel = reads_colormodel;`
`((quicktime_codec_t*)vtrack->codec)->writes_colormodel = writes_colormodel;`
`((quicktime_codec_t*)vtrack->codec)->set_parameter = set_parameter;`
`}`

The functions delete_codec(), encode(), decode(), reads_colormodel(), writes_colormodel() and set_parameter() should be defined in your source (ideally as static functions in the same sourcefile). If your codec has no parameters, you don't need to define set_parameter(). The function pointer is NULL by default and libquicktime won't try to set parameters then. For an audio codec, you set just set delete_acodec(), encode_audio(), decode_audio(), set_parameter() and flush(). Note that track->codec is defined as a void pointer in qtprivate.h (for whatever reason), so you always need to cast it to quicktime_codec_t. The priv member of the codec is for storing private data of the codec (like our blup_t).

Note:

The quicktime4linux codecs don't do any internal initialization in this function. Instead, the private codec structures have a flag, which is set to zero at at the beginning. Depending on this flag, the real initialization is then done at the beginning of the first call of one of the encoding or decoding functions. The reason for this could be, that it should be possible to call quicktime_set_parameter() after quicktime_set_audio() or quicktime_set_video() when writing or after quicktime_open() when reading.

When playback or encoding is finished a call to quicktime_close() will release all memory associated with the audio and video tracks of the file. For your codec, this will mean, that our delete_vcodec() or delete_acodec() are called. Since we have some private data allocated, we should free it then. The delete function will look like this for our blup video codec:

`static int delete_codec(quicktime_video_map_t *vtrack)`
`{`
`blup_t *codec;`

`codec = (blup_t*)(((quicktime_codec_t*)vtrack->codec)->priv);`
`blup_delete_codec(codec);`
`}`

3.3 Presenting ourselves to the outer world

Libquicktime users now want to know something about the codecs contained in the module, e.g. their descriptions, parameters, your developer website etc. This is done by functions, which pass information structures to libquicktime. Imagine, you have written blup and blop (which share many routines, so they are in one module). You need to define some static data structures. These are the fourccs of each codec, the encoding parameters, the decoding parameters and structures for the other informations. These are defined as follows:

`static char * fourccs_blup[] = { "BLUP", "BLU1", (char*)0 };```

`static int encoding_colormodels_blup[] =`
`{`
`BC_YUV420P,`
`LQT_COLORMODEL_NONE`
`};`
`` ``

`static lqt_parameter_info_static_t encode_parameters_jpeg[] =`
`{`
`{`
`"blup_quality",`
`"Quality",`
`LQT_PARAMETER_INT,`
`{95 },`
`1,`
`100,`
`(char**)0`
`},`
`{`
`"blup_motion_search_radius",`
`"Motion search radius",`
`LQT_PARAMETER_INT,`
`{ 0 },`
`0,`
`15,`
`(char**)0`
`},`
`{ /* End of parameters */ }`
`};```

`static lqt_codec_info_static_t codec_info_blup =`
`{`
`name: "blup",`
`long_name: "Blup",`
`description: "Video codec based on the bluplet transform algorithm",`
`fourccs: fourccs_blup,`
`type: LQT_CODEC_VIDEO,`
`direction: LQT_DIRECTION_BOTH,`
`encoding_parameters: (lqt_parameter_info_static_t*)0,`
`decoding_parameters: (lqt_parameter_info_static_t*)0,`
`encoding_colormodels: encoding_colormodels_blup,`
`decoding_colormodel: BC_YUV420P`
`};`
``
``

The structures lqt_codec_parameter_info_static_t and lqt_codec_info_static_t have the same members as their nonstatic counterparts (see above), with the exception that all arrays must be NULL terminated (the num_* members are missing) and the members, which are set by libquicktime, aren't there.

Imagine you have similar structures defined for the blop codec also. You now need to program 2 functions, which pass this information to libquicktime. These look as follows:

`extern int get_num_codecs() { return 2; }`

`extern lqt_codec_info_static_t * get_codec_info(int index)`
`{`
`switch(index)`
`{`
`case 0:`
`return &codec_info_blup;`
`case 1:`
`return &codec_info_blop;`
`}`
`return (lqt_codec_info_static_t*)0; /* Keep gcc happy */`
`}`

The function get_num_codecs() is self explaining. The function get_codec_info() returns the lqt_codec_info_static_t structure, which will then be converted into a lqt_codec_info_t structure by libquicktime.

3.4 Passing codecs to libquicktime

When libqucktime wants to create a codec from the module, it calles one the following functions, which must be defined in the module:

`extern lqt_init_video_codec_func_t get_video_codec(int index);`
`extern lqt_inif_audio_codec_func_t get_audio_codec(int index);`

The return values are the same as the function prototypes, which create codecs in quicktime4linux:

`typedef void (* lqt_init_video_codec_func_t)(quicktime_video_map_t *);`
`typedef void (* lqt_init_audio_codec_func_t)(quicktime_audio_map_t *);`

You need only one of these functions if your module has only audio or only video codecs. For the blup and blop codecs we would write the following:

`extern lqt_init_video_codec_func_t get_video_codec(int index)`
`{`
`switch(index)`
`{`
`case 0:`
`return quicktime_init_codec_blup;`
`case 1:`
`return quicktime_inif_codec_blop;`
`}`
`return (lqt_init_video_codec_func_t)0;`
`}`

The return values are the functions for creating single codecs as described above.

A note for all, who think, this is too complicated:

Remember the flexibility, we can have with this interface. We can port codecs between libquicktime and quicktime4linux by just adding/removing the libquicktime structures and functions described above. Since you must define the interface functions yourself, you can give them an arbitrary level of intellegence. The mjpa and jpeg codecs for example, share the functions and encoding parameters but have different names and descriptions. All this is possible.

3.5 Encoding video

This part is currently unknown to the author. Use the codec source to reseach this.

3.6 Decoding video

This part is currently unknown to the author. Use the codec source to reseach this.

3.7 Encoding audio

This part is currently unknown to the author. Use the codec source to reseach this.

3.8 Decoding audio

This part is currently unknown to the author. Use the codec source to reseach this.