Forum OpenACS Q&A: some nsjava benchmarking

Collapse
1: some nsjava benchmarking
Posted by Jonathan Ellis on 03/16/02 11:26 PM
First, despite what the JNI documentation says, GetStaticMethodID is
not slow enough to bother optimizing (through caching).  We're
talking a handful of MICROseconds.  And I can't imagine how they
could've made it worse than linear with the number of classes needed
to be searched.  So for OpenACS purposes I think this is not an issue.

Second, JIT compilation ROCKS.  The first time my methods run they're
2-3 times slower than the equivalent tcl; subsequently, they're 10-20
times FASTER.  I'm impressed.  Of course optimizing slow stuff in C
has always been an option but it's rediculously painful to develop.
ns_java could be a big win here.

Third, the current (NsTcl.eval()) method for returning results back
to the tcl interpreter is inadequate.  Over 1/10 of a second on my
machine to do a simple eval().  I'll have to see what kind of
improvement letting java methods return a String buys us but I
suspect several orders of magnitude.

Fourth...  anyone been able to reproduce ns_java screwing over
execs? :)

Collapse
2: Response to some nsjava benchmarking (response to 1)
Posted by Don Baccus on 03/17/02 06:59 PM
Well ... before getting overly excited about JIT one needs some information about where the bottlenecks in OpenACS 4.5 lie.  Thus far, bottlenecks that have been uncovered lie in the realm of the RDBMS.

An apparent 10-20x win (which is easy to believe since JIT code, though typically dumb compiled code is at least compiled) sounds fantastic, but of course if you're speeding up something that accounts for just a couple of percent of the time involved in building dynamic pages then you've won essentially nothing.

So ... as usual my concern with enthusiasm for ns_java is the same concern I have with enthusiasm for adding Python pieces or more Perl peices.  We really want to avoid a situtation where a newcomer needs to be fluent in Java, Tcl, Perl, Python and SQL in order to use our toolkit.

Collapse
3: Response to some nsjava benchmarking (response to 1)
Posted by Jonathan Ellis on 03/18/02 04:34 AM
I'm certainly not advocating rewriting large parts of the toolkit in java.  As I've said before, I <i>like</i> tcl.  And I like keeping the work in the DB where possible.  But where non-DB logic is taking up a significant amount of CPU on a small amount of pages, nsjava is the most appealing alternative to me (vs C or PyWX).
Collapse
4: Response to some nsjava benchmarking (response to 1)
Posted by Don Baccus on 03/18/02 06:21 AM
Well ... first, document where Tcl code is causing us to take a substantial hit.  As a "for instance" I've wondered whether or not the RP should be re-written as an AOLserver module (written in C, guaranteed to be faster than JIT, but then again I've *written* C compilers so this idea doesn't scare me).

Monitoring and documenting performance problems with our Tcl-based core is the first step towards building enthusiasm for rewrites in *any* language, IMO.  Without specifics that not only document faster execution time, but also manage to convince one that the decrease is significant (the 90%/10% rule), will probably just result in repeated calls to "prove it".

Once proven, one can start looking at optimization.  Perhaps language choice is the answer, perhaps better use at existing tools is the answer, but with real data no one can say.

Ironically (at least from my POV) my background is as an optimizing compiler freak who more-or-less ruled the earth for smaller machines in the late 1970s-mid 1980s.  But our targets were different than in this environment (i.e. we targetted assembly coders for process-control systems) and it was all fairly analytical.

So I'll want some "fairly analytical" information before I'll change my mind.  But my mind's open, of course!

Collapse
5: Response to some nsjava benchmarking (response to 1)
Posted by Jonathan Ellis on 03/21/02 05:17 PM
Just to correct my numbers...  I was drunk or something when I posted originally and divided the numbers for NsTcl.eval() by 10 rather than 10000.  oops.  So yeah we're talking 0.15 ms, not 150 ms.  That makes a lot more sense.  Rewriting nsjava to take a String returned from from the java method directly buys us about a factor of 2 speedup.