OpenACS and Java - The Technical Argument

by Ben Adida in OpenACS Community.
DRAFT - DO NOT DISTRIBUTE - DO NOT LINK

Executive Summary

While many software companies are rushing to convert anything and everything they do to Sun's Java programming language, the OpenACS project chooses to limit its use of Java. Because Java and the J2EE platform are far from ideal for web development, the OpenACS project is not using this technology as its primary development environment. Instead, because the Java programming language is used by an estimated 2 million programmers worldwide, the OpenACS toolkit provides an efficient and easy-to-use interface to Java libraries which allows rapid integration of existing functionality built in Java.

The OpenACS project aims to leverage existing Java resources without wasting time on straight porting or on using a language and platform not perfectly suited to the purpose at hand. The goal of the OpenACS toolkit remains enabling the rapid development of innovative, enterprise-scale, collaborative web applications. Java is by no means the all-encompassing solution to this challenge.

The OpenACS team also notes that, while J2EE is currently the most marketed web development environment, Microsoft's .NET platform and specifically the Common Language Runtime will surely challenge J2EE and Java in the next few months. OpenACS is not in the business of keeping up with the latest technology hype without strong technical justification.

This paper describes the technical reasoning behind the OpenACS decision to stick with the AOLserver/Tcl environment while making limited and well-targeted use of Java.

On Standards Compliance and Marketing

OpenACS is a toolkit for building enterprise-level collaborative web applications. OpenACS relies on two major technology components:

an ACID-compliant Relational Database Management System (RDBMS). Examples of ACID-compliant RDBMSs include Oracle and PostgreSQL.
The AOLserver Tcl scripting environment, which makes multi-threaded use of the Tcl programming language.

Today, Sun's Java programming language and the J2EE platform are all the rage. There is a desire to move just about every technological component to Java. In fact, ArsDigita has chosen to build the next version of ACS Classic (which is, today, a different product from OpenACS) in Java. They are not alone. Vignette is moving from its Tcl environment to Java. Art Technology Group is moving from its non-standard use of Java to the somewhat-more-standard J2EE specification.

Interestingly enough, this massive and unequivocal exodus to Java is mostly a marketing decision: it is much easier to sell a Java solution by leveraging the hundreds of millions of dollars in Sun marketing than it is to pitch the quality of a technology solution.

The idea that these companies are "standardizing" is false. Some are following the Java Servlet standard, which is little more than a barebones "server-side application skeleton." Others are using the full J2EE EJB specifications. The many EJB containers available on the market are (surprisingly) not quite compatible with one another. In addition, all of these companies are adding their proprietary application-specific APIs on top of the Servlet and EJB standards. This is certainly to be expected. However, it's difficult to see how these proprietary APIs, simply because they are written in the Java language, are any more "standard" than OpenACS's or AOLserver's Tcl API. The only real common point across these various Java systems is the marketing: Java everywhere.

The OpenACS project is not subject to the market hype that these commercial entities must consider. We can and will continue to focus on practical, technological superiority. We have considered Java. We have examined how it might be useful to the OpenACS platform. We have chosen to provide specific means of integrating Java into our existing platform. However, we have chosen not to port our existing code to Java. We have also chosen to continue to develop significant new applications in Tcl.

On The Equality of Programming Languages & Platforms

"All programming languages are created equal, but some are more equal than others."

In theory, every programming language is Turing-complete, which means it can accomplish the work of a Universal Turing Machine, which is to say that it can perform any work that is theoretically computable. This argument is often used to argue that any decent software engineer can simply pick up a programming language and go with it: language bias is said to be the battle of the weak engineer. This is true to some degree: any serious software engineer should be able to pick up any programming language fairly quickly.

In practice, however, programming languages are usually targeted to a certain type of application. All programming languages are functionally capable of doing any task, but some are more appropriate than others. In addition, a programming language exists within the context of a development platform. The development environment significantly affects the way a language serves a particular purpose. Thus, a programming language and platform's usefulness should be measured with respect to the following criteria, in order of importance:

The programming language and platform's inherent adequacy to the task at hand: If the task involves manipulating large amounts of text, for example, the programming language should be a particularly good at manipulating string variables, specifically with regards to the efficiency of writing the code and the ease of reading the resulting code. If the task involves serving multiple simultaneous requests, the platform and language should provide efficient and powerful mechanisms for multi-threading.
The extensibility of the language platform: Extensibility of the language platform involves both the neat package-ability of functionality within the confines of the language, and the ability to easily link in other libraries of code often written in a different programming language. The AOLserver Tcl environment, for example, is particularly strong at linking in new libraries of functionality. It is easy to link in any library written in C or C++. It is also easy to link in code written in Python and Java while maintaining all existing Tcl code and calling the proper Python and Java libraries directly from Tcl. Tcl, the language, is a particularly good "glue language" to bring together all of these various code libraries.
The availability of proven, useful packages of functionality: The availability of libraries usually scales with the number of developers working with the language and platform in question. There are 2 million Java developers worldwide. This ensures that XML parsers, SOAP servers, Jabber clients and the like will be available as Java libraries fairly quickly. The same can be said of the Perl programming language, with the CPAN (Comprehensive Perl Archive Network) site that lists thousands of available Perl libraries.
The general popularity of the language and platform: If one needs to quickly start a project, picking a popular language and platform may help recruit engineers more efficiently. This is significantly less important than the previous issues raised, however, because this is only a transitional issue: using a less popular environment will require a one-time training period for engineers, after which the issue of popularity is clearly surpassed by other factors.

The AOLserver/Tcl Platform

Properties

Consider, thus, the properties of the AOLserver Tcl platform:

Tcl is a scripting language. Like Perl, it can manipulate textual information in a simple and clear manner. This is crucial for rapidly developing web applications.
AOLserver is single-process and multi-threaded, which allows for a simple-to-use shared global cache and memory-efficient scaling.
AOLserver includes an efficient, DB-independent connectivity scheme that allows rapid database integration and interaction.
ADP, AOLserver's templating scheme, combines the power of Tcl with the standard ASP/JSP templating format for integrating code within a presentation layer. Powerful new presentation tags can be added to further enable reusability of content components in the context of a markup language, a format already well understood by UI and graphic designers.
Many high-traffic sites like Moviefone, MapQuest, FoxSports.com and AOL Digital City rely on the extreme scalability of AOLserver.

There is no doubt, neither theoretical nor practical, that the AOLserver/Tcl platform is a particularly good fit for web application development.

Extending the Platform

The AOLserver platform is highly extensible. Any thread-safe C library can easily expose a Tcl API within AOLserver. This ability has allowed the creation of the following AOLserver modules:

ns_xml: the ability to parse and generate XML using the standard GNOME XML parser. The efficiency of this native library combined with the flexibility of a safe Tcl API is unmatched.
PyWX: the ability to integrate Python code. Tcl code can call Python code, and vice-versa. This enables the immediate use of all existing Python code libraries.
ns_java: the ability to integrate Java code. Java objects can be instantiated and methods called directly from Tcl code. Since AOLserver Tcl is threaded, each Tcl thread is matched to one Java thread, providing a scalable integration. All Java libraries can easily be integrated and used within the confines of a flexible scripting language.
ns_soap: the ability to process SOAP requests using simple AOLserver/Tcl. AOLserver/Tcl immediately becomes a serious player in the world of Web Services.

The Java/J2EE Platform

J2EE: Not that Revolutionary

The J2EE platform, after 3 years of constant development by Sun and their partners, offers no particular technical advantages over the simple APIs that the AOLserver/Tcl environment has provided since 1995. There is no single part of web site development that is made easier on the J2EE framework.

AOLserver/Tcl provides:

efficient database connectivity and connection pooling - equivalent to (if not better than) J2EE's JDBC.
a templating scheme with custom-tags and customizable parser - significantly more powerful than the JSP templating scheme.
a shared cache with efficient locking capability - equivalent to (and more efficient than) Java's general architecture and built-in threading/locking.
an automated LRU cache API - not part of the J2EE platform
an architecture that significantly facilitates session management - equivalent to the J2EE servlet session model.
a complete interface to email functionality - equivalent to J2EE's java.mail package.

In short, a move from the AOLserver/Tcl to the J2EE platform brings little real functionality to the table, and in fact significantly complicates the streamlined development process of AOLserver/Tcl.

Java: Not that Web-Friendly

Java is a strongly-typed compiled development environment. While extremely useful in many situations, the Java language is seriously handicapped in dealing with web applications for two important reasons:

type matching: in most web applications today, the application logic interfaces with the RDBMS on one end, and the web browser's HTML parser on the other. In many cases, the SQL interface to the RDBMS is mostly typeless, or at least deals with data types that don't match Java types consistently. SQL is basically a string-oriented language, with some additional hacks to perform type-specific argument binding. HTML generation and form processing is essentially all string manipulation. Java is not particularly efficient at dealing with strings. In developing a Java web application, one spends a significant amount of time dealing with type conversions from HTML forms to Java, from Java to SQL, from SQL back to Java, and from Java back to HTML. This amount of work adds up and can become a serious impediment to clean, fast implementation. A smart templating system can cure part of the problem, but not the entire issue.
tedious development cycle: Java is not very compatible with the per-page model of web applications. JSPs, a hack which permits single-page Java code, introduces significant complications in the debugging process.

The Conflict of Abstraction and RDBMSs

The relational database's most interesting feature is the ability to join, correlate, and generally massage data from many different sources using highly efficient data structures. Individual entities can be correlated to others they are related to, thus the idea of modeling the world in terms of entities and relations.

In a more abstracted model like that of the typical Java Object Oriented approach, this ability to provide flexible means of storing and retrieving data is destroyed by the layering of single-entity data abstractions on top of the relational model. The individual data abstractions behave like walls between the various entities, preventing true relational operations. In fact, layering a Java Object Model on top of a relational model actually breaks the semantics of SQL.

The result is straight-forward: where one SQL statement could do the job, a highly abstracted system like Java requires many, sometimes increasing the number of database interactions by a factor of 10 or 20. The back-and-forth communication between the application and the database adds significant overhead to performing the raw data manipulation. Using a highly-abstracted system like Java, The system becomes inherently less scalable.

The Future

The OpenACS project is certainly not set on Tcl as the be-all and end-all of programming languages. Tcl is a valid programming language made extremely powerful by the AOLserver/Tcl platform and APIs. In a web environment, a scripting language - a glue of sorts - has its well-established place.

The OpenACS team is considering combining Tcl and Java (using ns_java) in order to mix the flexibility of per-page Tcl scripting with the reusability and package-ability of certain specific Java libraries for well-contained functionality. This approach only justifies integrating existing Java code with Tcl, however. Converting existing Tcl code to Java is a marketing game that we consider unnecessary.

The OpenACS team notes with great interest that Microsoft's .NET platform includes the use of a strongly-typed, Java-like object-oriented system (C#) combined with a programming language (VB.NET) that is much more script-like. In fact, the interactions between VB.NET and C# are much like those of Tcl and Java in ns_java: the reusable, well-packaged functionality is built in the more structured, object-oriented language, while the specific process flow and basic interface manipulations are done in a more flexible "language for the masses."

With the market hype surrounding Java expected to drop significantly with .NET, the OpenACS team will wait patiently for the bloody marketing battle to end while it continues to provide better ways to:

rapidly build web applications
extend web applications using as many existing libraries as possible
provide the simplest, most flexible, and most scalable architecture by keeping things simple.