Resource Loading Hooks

Hooks for providing alternative means of fetching game resources - e.g. a map, image, sound, model, from custom or even proprietary formats like PNG, GIF, MP3, 3DS. Enable the community to extend default resource loading from a PAK or ZIP archive, as well as adding additional archive formats (outdated WAD, WAD2, WAL, new BZ2).

Advantages:

flexibility, extensibility, compatibility
fallback for built-in code
built-in code can be replaced with means for autodownload, accessing file from the 'net or a LAN (e.g. FTP), or generating resources on the fly (e.g. just-in-time compilation of code)

Disadvantages:

The resource handling is part of the engine, used by client and server. Implementing fallbacks and extensions in the Game DLL does not change the client. While certain load features might make sense server side only, most demands for alternative resource files are made with respect to data used for rendering: images, sounds, cinematics, models. Further, a server-side only modification would require two different archives, one for server load, one for client load.

requires a client side DLL, or client side Java
additional performance problems relating to uncompression
threading required for retrieving resources from a network or other slow means
conversion to known formats might as well, or might better, be done externally

A server-side only implementation makes sense with respect to file formats predominantly or exclusively used server side. Example: Java JAR files, which include manifest files, and (in JDK1.2) will support package versioning.

Effort:

ZIP handling code is being added to Q3, replacing the PAK format is out. Hooking into custom code could be handled in a two ways :

Fallback: If the builtin mechanism does not find the requested resource or can not load it, the game engine makes a call to an exported DLL function which dispatches to whatever load mechanisms added. The default behaviour is to return NULL, and the engine will report a "Not Found" error and/or fall back to a default texture/sound/whatever. If successfull, the dispatcher will return a reference to a buffer area returned by the plugin.
External first: this allows for replacing handling as done by builtin functions, even if the code of the builtin functions is not available for modifications. In this case, the DLL should import all functions builtin the main engine, and export the dispatcher to use them, or override them.

There are different aspects:

Uncompression: to implement alternative means of compression for a single chunk, data can be returned as a lump of bytes.
Conversion: To support additional images, sound etc. formats, the resulting data has to be converted to a format known in the engine.
Archiving: to use other archive formats (including different archive compressions), a second stage is needed.
Recursion: ZIP of PAKs of WALs?

A sketchy implementation that does not expose the data structures used by the engine to store sounds, RGB, and RGBA mipmaps, would be

        resource_t * (*LoadResource)(char *resourcename, char* type);

        typedef struct resource_s 
        {
           int         size;
           byte*       data;

        } resource_t;

Here, the engine indicates the desired format of the raw data stream as a MIME type, e.g. "image/tga", "sound/wav" or "application/x-q3map". The plugin identifies the actual format of the resource (file extension or more sophisticated means), reads, and converts.

Comments:

It is possible to implement different search patterns (CLASSPATH-like, see Path Flexibility) for resources as part of the extensions. This would be a mere workaround if the builtin search is not adequate.

Server-side only implementation (e.g. as part of the Game DLL) could be combined with an autodownload mechanism to transfer data chunks to the client, with the known problems related to autodownload from a dedicated Game Server in general. It would still ease development of textures, models etc. in proprietary formats, as autodownload in single player would not come with a penalty, and conversion of a given file to a Q3 format could be delayed till final distribution.

A project like Q2Java would be enabled to use the powerful builtin networking mechanisms to get files from potentially anywhere, even threaded in parallel, and return it to the game engine.

Submitted 980430, revised 980508, revised 980509, revised 980510, contact Withnails.