Java in Q3

Java can be used for both extensions of the Q3 client, and to implement a portable Q3 server. Java has been used by Barry Pederson for an outstanding port of the Q2 GameDLL. I maintain that Q3, to be released no earlier than 4Q98, can and should be done with the basic Java support structure in place now.

Advantage:

id Software would gain an inherently more stable server (due to the design of the Java language), a flexible security management system for both server side and client side modifications, and builtin support for download on runtime. More features, especially from the client, could be exposed to modifications.

The community would gain a language for powerful for Internet applications then any other, portable without the need for verification or recompilation, a language that is more elegant than C or C++, while as easy to start with as QuakeC, and made for object oriented programming from ground up.

Potential applications include:

Game Subsystem as modular Plugin set
configurable client side rendering effects
client side hooks for autodownload requests, redirected to dedicated download servers using builtin HTTP/FTP support
client side hooks for post-frame blits into framebuffer
modification of client side processing shortcuts,
e.g. Player Movement Prediction, Entity Dead Reckoning, inteprolation and extrapolation between updates
modification of client side executed inputs, e.g. for movement and orientation
easier implementation for server-server communication and connected multi-server
network protocol extensions
Client-side "applet" GUI extensions

It is important to note that e.g. custom rendering effects are not possible without client-downloadable Java.

Disadvantage:

id Software needs a JRE (Java Runtime Environment) available for commercial distribution with the product (Q3). JIT (Just-In-Time) compilation is in its early beginnings, thus Java code might not yet be competitive to native C in all cases. Maintaining large parts of the codebase in Java will be a step soon be irreversible. The JRE is currently available only for Win32 and Solaris, and (unsupported) for Linux, meaning less platforms than available for ANSI C and OpenGL, effectively. Clean-room implemetations of the JVM are not yet complete, and currently do not target the main target platform of Q3, Win32. Using JNI to glue Java and native C code together might introduce unacceptable performance penalties in some cases.

Implementation

Q2Java - Proof Of Concept

Parts of the Q2 Game DLL have been implemented in Java and executed on a Java Virtual Machine by the server, in the Q2Java port by Barry Pederson. This is a fully valid Proof Of Concept for the Win32 platform. A Java-based deathmatch server has been running for some time now at 134.129.166.11, with a server-builtin telnet (at port 1100). CTF and VWep ports are already under way. Previous modifications of the scheduling of nextthink events provided huge gains, so performance is more then sufficient (however, currently not all DM events are fully executed):

"JDK 1.1.6 seems way faster - server frames on my machine
are now less that .2ms (used to be more around .5ms, with
the stock C game around ..65ms, +set deathmatch 1 +map base1)"

Q2Java is currently in revision v0.5, and fully available for download at the Q2Java site. Linux and Solaris ports are in preparation, the former hampered by the current state of the inofficial JDK for Linux ports with respect to the libc5/glibc migration.

In summary, even with the current server-DLL interaction and the additional overhead introduced that way,

JRE - Availability of Runtime Environment

The following requirements have to be met by the Java Runtime Environment (JRE) on which Java in Q3 depends:

Java Native Interface (JNI), to glue C and Java code together, importing main engine functions, and exporting wrappers around Java methods.
Java Virtual Machine (JVM) invocation at runtime. This has to cooperate with the mechanism to load a DLL at runtime.
license free distribution of the JRE for Win32.

The following are not mandatory, but will add to the performance:

Just-In-Time (JIT) compilation, which lets repeated execution of Java bytecode approach C generated code execuition speed.

JRE - Ditribution with Q3

Distribution of the JRE that includes the JVM, JNI, and in upcoming releases also the JIT, is possible under the terms of the license set by SMI, and has no obvious strings attached except an EULA-like termination mechanism. Quote:

It [JRE] can be legally distributed with your product. The JDK
cannot. Distributing the JRE with your app also ensures that the
correct VM is used with your product.

and

Termination. This license shall automatically 
terminate 180 days after production release of 
the next version of the Software by Sun.

See the full text of the current JRE license here.

JDK - Availability of Development Kit

The JDK can not be distributed along with the game. The JDK is available for download for Windows NT 4.0, Windows95, Solaris, and Linux at no charge, to both id Software and authors of homebew Q3 mods. See Sun's Java site for details. A CDROM is available for USD30 plus shipping.

Free clean-room implementations

Aside from commercial and semi-commercial packages (e.g. Magician), a number of implementations available under GNU GPL or BSD License can be found. References are:

Note that the implementations above are clean room implementations of the JVM (Japhar, Kaffe) and the core classes (Kore), performed with the goal of creating a freely distributable JVM in mind. Using DJGPP for Quake set a precedence for relying on freely available software in open game architecture development.

None of the implemtations above has (yet) been ported to Win32. A port would possibly be Cygwin32 based, and might lack the AWT component.

Improvements of JIT compilation in the freely distributable JVM's encouraged by the Q3 experimentations might add to the feasibility of full Java in Trinity (another precedence for this would be Mesa+Voodoo for Linux GLQuake).

Decompiler and Obsfuscation

There are freeware, shareware and commercial tools like Mocha and Decaf allowing for reverse engineering Java bytecode. Due to the bytecode structure and the symbol information carried for late binding/RTTI, additional obfuscation for client/server code not to be released to the public might be required. Commercial tools include:

An obfuscator has to be applied to the Java source only once, prior to shipping, so it has to be available only on one platform, during development (meaning Windows NT, as Solaris is not in use at id Software). Note that reverse engineering is always in violation of the common end user license agreements, independend of the availability of decompilers, whether obfuscation has been applied or not.

Alternatives:

This proposal adresses the most important part of the Client Side DLL proposal, and would fulfill the same purpose with respect to making client side and engine functions accessible to server-driven modification, while avoiding the security concerns raised by Client Side DLL.

Maintaining parts of the Q3 database in Java, and using java2c conversion tools is not an option for a commercial product. Maintaining a C codebase first, converting it to Java manually later is a waste of time, and will possibly never happen even if planned.

Dave Kirsch suggested a Forth-like scripting language for custom client side rendering effects for Q2. This would be a special solution that seems to be as expensive, implementation-wise, while not being as flexible and generic as a Java based solution.

Submitted 980407, revised 980420, revised 980430, revised 980507, comments to bk@gamers.org.