Forum OpenACS Q&A: A Template-Based ACS

Posted by Brent Fulgham on 04/10/00 10:45 PM

This message is based on some private e-mail I've had with Ben, so he
may find it repetitive, but others might have interesting comments.

I have been attempting to figure out the best way to present the ACS
to Joe Debian-User, who wants to install the software and create his
site, but also wants the bugfixes and upgrades that will come along.

I was crestfallen to discover how the ACS is constructed with very
little separation between data and presentation. I.e., there
doesn't seem to be an easy way to swap out the mechanisms of the
site (say the pieces that handle user validation, message posting,
etc.) without hand-editing your original files. To do so is likely
to result in loss of any customization you may have done.

This (obviously) makes it very hard (if not impossible) to automate
an upgrade. This is especially true where changes in the data
model are required.

I submit that by-and-large this does not have to be the case. It
seems that the data models, presentation layers, and functionality
could all be provided through a system of separate file/directory
structures. More regular use of cascading style sheets could help
lessen the burden of customizing a site (in terms of its
presentation), and help keep it uniform when changes are made.

I really see two abstractions that could be made to help this. We
could (1) auto-generate the hierarchy based on a set of user-editable
templates to produce the presentation layer, and plug-in the Tcl code
into that template. And (2) separate the data modelling from the Tcl
interface as much as possible. That way different RDBM's could be
swapped with less impact on the site itself.

I'm sure others of you have far more insight into this than I do
(based on my limited experience). I'd like to see what you think.

2: Response to A Template-Based ACS (response to 1)

Posted by Gregory McMullan on 04/11/00 12:12 AM

I like the idea, Brent, since I have gotten to liking the Debian approach to upgrading your system. Unfortunately, there is good news and bad news on that front. The good news is that at least one ArsDigita is working on templates for the ACS, I think in much the way that you propose. It is described as:

The dynamic publishing system is a substrate for team development of web applications. It maintains a strict separation between data processing and presentation, allowing developers and designers to work together more effectively. The system handles both content pages as well as most forms.

For more on that, see http://karl.arsdigita.com/projects/template/doc/ (I sure hope that Karl doesn't mind people getting links to his stuff!)

The bad news is that at least one of the other ArsDigitans (Jin) has said that separating out Oracle-specific SQL is "a particularly pernicious idea" (his words, not mine - see http://photo.net/bboard/q-and-a-fetch-msg.tcl?msg_id=000WhO for more - look for "pernicious" 😊. He says that that kind of separation (not quite what you propose, but still something that would be handy for us ACS/pg types) slows you down in producing new sites, and that being able to swap new RDBMSs in and out isn't particularly useful.

That said, I'd love to see the kind of abstraction that you propose done. I confess to finding it a daunting prospect, unlike some folks here who seem to eat, drink, and breathe ACS (I'm working on it, I promise!). I also have some trouble seeing how we could manage to do it on our own and manage to keep up with ArsDigita's paid army of programmers. Of course, on the gripping hand, we do seem to be growing in numbers of downloads pretty quickly!

3: Response to A Template-Based ACS (response to 1)

Posted by Brent Fulgham on 04/11/00 12:52 AM

First, thanks for the kind words about Debian -- it's nice to hear from people who like what we're doing. Since I've stopped reading the debian-user lists all I tend to see nowadays are bug reports 😊

At any rate, aD's concept is a good step -- separating content from processing should be a design concept with easy-to-grasp consquences. Anyone who has had to work with "Product Designers" will know what I mean. 😊 (no offense to all the fine Product
Designers out there)

And this is important in more areas than simply upgrading. Take for
example "http://www.debian.org". Now, this is a set of static HTML pages generated from XML and then hand-tweaked. But this system allows us to provide translations to 7 or 8 languages from the same source template, and makes it a bit easier for non-catalan-speaking people to update the whole set of pages when changes are made. It would be nice to generate all of this on the fly -- and a templated display system would seem a good way to go about it.

The aD's concept seems more advanced that this, but would certainly be a big step in the right direction. As long as a site had not made changes to the functionality of a site, we could easily upgrade their system.

As for Jin Choi's arguments -- I certainly can't argue. I have no expertise in this area and would be foolish to second guess. However, I will point out that plenty of Perl programs are written using the DBI specification which allows them to map to a variety of
RDBMS's. I think this kind of abstraction is worth the potentially non-trivial performance hit. But that's just me. I've found that being completely shackled to a particular platform (even an RDBM) is generally not wise.

Under a system like the ACS, I imagine that a lot of time would shift to writing plumbing code that connects the ACS "generic code" to the particular database "driver". This "driver" is where the SQL would reside. This could just be another Tcl file that gets sourced by AOLserver.

What experiences have the rest of you had with database systems -- do you tend to have to move from one to another fairly often? Or are you able to dictate to your clients what they will use?

4: Response to A Template-Based ACS (response to 1)

Posted by Don Baccus on 04/11/00 03:35 PM

It does appear that aD is finally tackling the first problem, via a re-thought templating system and, if one believes the murmurings, by actually using for most of, if not all of, the modules.

As far as Jin's comments regarding supporting more than one database, well, do keep in mind that aD has a very narrow mission, to become the dominant developer of complex database-driven web sites. Most of their potential customers use or will be using Oracle. aD is tight with Oracle. Supporting multiple databases would simply slow them down, without tangibly increasing their business and potentially putting their relationship with Oracle at risk. I suspect the latter reason's the most important, to be honest. Philip loves Oracle. Oracle bought an earlier company of his. Several months ago, when a curious individual e-mailed him about the desirability of making the ACS available on a low-cost or free database like Postgres, Philip lambasted him in his response, telling him that only "losers" would consider using anything other than Oracle. No, it wasn't me, but I did get to see the e-mail exchange...

I think aD is starting to wake up to the fact that a Postgres-based ACS might turn out to be very popular, indeed. While I don't envision ACS/PG taking over huge sites built on load-balancing server farms (the fact is that PG has zero support for this level of scalability), I do envision the ACS/PG being used to implement a large number of more modest sites. You could do something like scorecard.org using ACS/PG, for instance...

The bottom line is that any such abstraction of queries would have to come from us, would probably not be folded back into the mainstream ACS, wouldn't be maintained for new modules by aD anyway (we'd be porting to this abstraction), and the needs of ACS/PG users may not track mainstream ACS forever anyway. We have no way, at the moment, to port over Java embedded in Oracle except to rewrite it or find someone with the time to embed the JVM into Postgres ala the Tcl and Perl embedded languages - any takers? Webmail's being implemented by aD in this way, tying them even more closely to Oracle (and it's being implemented by Jin).

Jin's comments should also be considered in light of the fact that aD is apparently working on an Apache/mod_perl version (or at least is considering it). The use of AOLserver shuts them out of a fair number of potential customer sites, since Apache and Perl are both objects of religious significance to a fairly large slice of the Unix world.

This will slow aD down. Maintaining AOLserver/Tcl and Apache/mod_perl versions will be a far greater maintenance headache than trying to support a couple different dialects of SQL. More evidence in my mind that it is the relationship with Oracle, not technical considerations, that causes aD to refuse to help make the toolkit more portable in regard to the underlying RDBMS.

5: Response to A Template-Based ACS (response to 1)

Posted by Roberto Mello on 04/11/00 04:13 PM

I agree with Don's remarks. aD has its vision and will keep going with it. I just want to add a comment about the upcoming version of ACS for Apache. What I heard from an ArsDigitan is that Robert Thau is creating a mod_aolserver for Apache that will recreate the Tcl API inside apache. I think Thau is using mod_tcl (which has been available for quite some time) as base for his endeavour.

I don't know to what degree mod_aolserver will be compatible with AOLserver but I doubt it will have something like ns_schedule_ will be available.

This will bring many more religious Apache fans to the pool of aD's customers.

6: Response to A Template-Based ACS (response to 1)

Posted by Brent Fulgham on 04/11/00 10:17 PM

I can understand the ACS-core team's feelings that supporting multiple databases are a waste of time. However, you don't have to be in the "real world" long to realize that clients are not always rational or unbiased about how things should work and what tools to use. In other words, you don't always get to choose your own database and may loose contracts if you can't meet the client's requirements.

Even Phillip Greenspun himself has spoken of porting the ACS to *gag* WindowsNT and *retch* Microsoft SQL-server. And these would certainly be far more involved and labor-intensive than (as Don points out) modifications for different SQL dialects.

So one question that immediately springs to mind, is if Don's conjecture is true (that a database-agnostic system would not be of interest to aD, and they would be unlikely to fold such changes back into the source tree), then where does that leave us? When Ben suggested that ACS/pg might someday just be a feature-equivalent port of the ACS, rather than an architecturally-identical port I was somewhat shocked, but after learning more I wonder if this isn't perhaps the best option.

Perhaps I am just naive, but I really think it should be possible to migrate a website from one version of the ACS to another without having to hand-code every change. For example, at my current employer when we make a large change to a database construct that changes the way existing data should be stored, we have to write a "Rel" program (release program) that will walk through the database and update these values. These kinds of tasks should be possible without having to redo it for each and every site. However, I might not grasp the whole of the ACS's architecture (in fact I am sure I do not) and could be missing a key point.

7: Response to A Template-Based ACS (response to 1)

Posted by Don Baccus on 04/11/00 11:48 PM

Where does it leave us? At the moment, with a very useful though far from ideal open source web toolkit, with an uncertain future regarding how closely it tracks the Oracle version. It provides you with the nuts, bolts, wrenches etc that help you build your engine but does absolutely nothing to help you with the question as to whether or not that engine will fit next year's chassis...

I'm OK with that, for now. It's far from ideal, but it's still far better than starting from scratch when you build a site.

I think part of the problem is that Philip is such a good writer, and among other things talks about bringing software engineering to the web, etc, that expectations are raised to the point where the reality is a bit depressing.

The thing that annoys me the most is that apparently the CS curriculum at MIT includes a course entitled "testing is a sin". If it's not broke, it's not software. I'm about to go off to fix the 3.2.1 administration page for news, which is totally hosed (I'm working with the Oracle version, today, too) and has obviously not been tested with a fresh installation...grrr...puts me in a bad mood.

8: Response to A Template-Based ACS (response to 1)

Posted by Ben Adida on 04/12/00 12:26 AM

Wait, I have never heard of that class! That would probably have been quite a bit of fun :)

9: Response to A Template-Based ACS (response to 1)

Posted by Brent Fulgham on 04/12/00 01:03 AM

One interesting thing that Jin said, and I believe it is probably true, is that most of the ACS is the SQL -- the Tcl framework is just plumbing that connects it to the web page.

If we assume for the moment that this is true, then it seems like a logical plan would be to develop our own "modular" framework (perhaps based on aD's templating effort) that would abstract the DB interface, and provide a bug-compatible ACS. When we "port" from one version to the other, we would mainly be concerned with "porting" the SQL used by the forms. We would then need to update our templates or "plumbing" to support it.

This is similar to the task of porting a Motif-based program to GTK, for example. The underlying model is the same, but you might have to implement slightly different routines to match the "API" of the displaying system.

10: Response to A Template-Based ACS (response to 1)

Posted by Ben Adida on 04/12/00 01:17 AM

I have a lot of respect for Jin and Philip, but I have to disagree that the ACS is just the data model and SQL. I understand the desire here, because that gives it an SAP'ish sense, and SAP is worth a lot of money. The ACS also defines a number of processes, a flow through the data model and SQL. These processes are implemented in Tcl, and while one can surely call that "plumbing," it's pretty vital plumbing.

Also, if the Tcl really is just plumbing, then I sure hope that Arsdigita will start abstracting out the *real* ACS from this "plumbing" ... such is not even close to the case right now....

11: Response to A Template-Based ACS (response to 1)

Posted by Titus Brown on 04/13/00 01:46 AM

I have to throw my own oar in here ;).

It's clear that Tcl is not just the plumbing, as you say (Ben). I'm a little bit dismayed at the Oracle-centric aD stuff, and I have to wonder if aD is coming clean with us. Just what sort of business deals are aD and Oracle cooking up between them??
I'm beginning to not trust aD people's responses on the Web/db forum...

FWIW, it's become even more clear to me that Tcl is important as I've worked on the Python module. There's a large user base out there for ACS/Tcl -- not least of them being me -- and it's important to keep them mentally comfortable. Frankly, SQL is SQL, and soon PgSQL will be as good as Oracle SQL or better; and the transition from Tcl to Python (well, C to Tcl - Python's pretty easy) was much more painful than the transition from Oracle SQL to PgSQL. So, switching away from Tcl would probably deeply disturb aD/ACS using people far more than switching to PG would.

Some offline discussion with Brent leads me to believe that a templating system could become part of the PyWX project (or perhaps a spinoff, once we get the Tcl linkage up to par). Part of the requirement for templating IMO is extensibility of the underlying system, and Python is MUCH better at that than Tcl. (Not to mention, integrating Tcl code into Python is easy; vice versa is not.)

Does anyone have any references to object-oriented templating systems? OR is this a ludicrous idea? I haven't thought deeply about it, beyond having skimmed a fair bit of text on the aD templating system and others; but it seems like using an OO framework to organize a hierarchy of SQL/Tcl/HTML/whatever statements, and provide a standard interface by which the HTML section could e.g. access the SQL output,
could work out well.

Just to continue rambling, you could imagine defining a page with
a "loop" section in it: the "loop" section would have a "loopdata" construct that would be an SQL statement, and a "loopprint" construct that would be HTML wrapped around variables, and a "looptranslate" construct in which the Tcl people could define a set of functions for producing various modifications of the data that SQL didn't provide.

Or am I being silly and uneducated here?

--titus

12: Response to A Template-Based ACS (response to 1)

Posted by Daniel Wickstrom on 04/13/00 08:59 AM

For a comprehensive example of templating check out www.zope.com. Zope among other things is an object-oriented web developement framework implemented in python. It also provides it's own http listener (Zserver) and an object-oriented database (ZODB). Yes all of the acronyms associated with zope do start with Z. I looked at this seriously for awhile before I switched over to porting the ACS, but I found that it was overly complicated, and I didn't particularly care for the dtml language that was used for building dynamic pages. Alot of the things that happen in the Zope system seem like black magic, and by most accounts it has large learning curve. It is flexible enough that you could build an ACS equivalent written in python. It can also be configured to run on top of apache instead of using Zerver.

13: Response to A Template-Based ACS (response to 1)

Posted by Roberto Mello on 04/13/00 03:59 PM

When I last looked at Zope a couple months ago it seemed too fuzzy and complicated for me too. There are so many z-parts the the whole z-thing and each part is z-something. I never used it but their whole architecture seemed like the other application servers lying around (e.g. ColdFusion, etc.)

The nice thing, as Daniel noted, is that it is all object-oriented.

14: Response to A Template-Based ACS (response to 1)

Posted by Don Baccus on 04/13/00 07:26 PM

The view that the Tcl is just ACS "plumbing" is ridiculous, and IMO is a major reason why the ACS is such a hodge-podge. Time-pressure resulting from customer contracts are also and important reason, but if aD paid as much attention to the design of the Tcl infrastructure as is paid to the datamodel, the ACS would be a heck of a lot easier to understand, would be more maintainable, and less error-prone to begin with. The notion that the Tcl is "plumbing" is mirrored by what seems to be an almost total neglect of any effort to design a rational architecture for the set of scripts that make up the toolkit.

On the other hand, muddled code is pretty common in fast-growing projects where time is of the essence and deadlines tight. The ACS isn't the only software project to suffer from such constraints.

As far as whether or not an object-oriented approach to separation of queries, programs, and HTML makes sense, I've not really thought about it. I do believe the object-oriented paradigm would be useful for designing a reasonable model for module add-ins, and a natural way to think about providing easy-to-configure ways to define permissions, etc, for various modules and parts thereof.

As far as the templating problem goes, the problem lies in mapping the object-oriented paradigm to the hyperlinked, page-oriented reality of the web. I've not looked at Zope but it's not hard to imagine that such a scheme could get complicated in a hurry.

15: Response to A Template-Based ACS (response to 1)

Posted by Ben Adida on 04/13/00 07:42 PM

Okay, a few points.

Object-Oriented is great for a number of things, but you have to be careful about a few gotchas. For example, if you use an RDBMS, plopping objects on top of that is usually a nightmare, no matter how good your object-relational mapping is. I can go into extensive detail about this if anyone is truly interested, but I'll spare you for now :). Now, an OO model for certain levels of the code, like the modules, may not be a bad idea, but it's questionable that it's the only way to go. Defining a module callback interface with module registration will work just as well for our purposes here. All I'm saying is that we should think carefully about architectural changes that broad: we don't want to get stuck with a massive porting/rearchitecturing effort on our hands that not enough people want to support.

Another important point: making things modular, abstracted, and extensible, is perfectly possible in Tcl. I'm sure Python is a great language, but let's think here: how hard is it already to find
Tcl/SQL hackers? Do you want to add Python to the equation? I don't. I think it's great to have people explore different ways of using AOLserver, but let's not get into language wars for ACS purposes. Tcl, Python, C, Java, Perl, Lisp are all Turing-complete, you'll be able to do anything you want in any of these languages. Yes, you may love one language more than another, but we have big issues on our hands here (modularity, content abstraction, DB compatibility), and introducing another unnecessary piece to the equation is a suboptimal approach.

16: Response to A Template-Based ACS (response to 1)

Posted by Titus Brown on 04/13/00 09:13 PM

Don -- yes, "unfolding" an object hierarchy into a page sounds like it could get nasty. I'll have to take a look at Zope; I don't want anything complicated to use, and it doesn't seem (intuitively) that you'd have to *make* it complicated. Q: Are there any templating systems out there that people *like* and have used?

Ben -- Tcl is great: it works well for its current purpose, and it's extraordinarily easy to embed. I agree that requiring another language is a bad idea -- but I think that "OO Tcl" would end up being very much like C++, a hodgepodged mishmash of ugly code that only recently (after how many years?) has compilers that fully support it ;).

But if Python allows embedded Tcl (which it does, with very little effort) then there is a good reason to build an OO framework in it and have Tcl code working within that framework, instead of trying to build the framework with a tool that's ill suited to it (Tcl) or with an extension to that tool ([incr Tcl]) that's grafted on. Moreover, a well designed templating system shouldn't require anything but very serious infrastructure-style hacking -- and people interested in doing such usually don't mind learning new languages.

As an end to these language wars: I'm working on a few projects with PyWX, and I'll post my results here when I'm solidly on my way. It may be that working with Python isn't a great boon and merely complicates matters (which, in some cases, it does); whether or not these things can be overcome is a matter for the developers of PyWX. It's "our job" to bring it to you, not your job to come to us ;). But if you're going to argue that Turing completeness is enough for a language to possess, let's talk APL, PL/SQL, Fortran, x86 assembly language...

Cheers, --titus

17: Response to A Template-Based ACS (response to 1)

Posted by Don Baccus on 04/13/00 09:16 PM

Let me clarify (by repetition) my thinking - I stated that the object-oriented paradigm's a natural way of thinking about problems like the design of a uniform interface for modules. That's not a call for using an object-oriented language, nor to try to implement an object-oriented model in Tcl. The last thing on my mind is to suggest using anything other than Tcl for the ACS. I have an abstract curiousity in [incr Tcl], as I mentioned in an earlier thread, and a similar curiousity about Python, but that's it.

And I'm heartily in the camp that dismisses the notion that the object-oriented paradigm fits all problems - just as I dismiss the equally narrow view (rarely heard, these days) that it fits no problem, or the "it's only good for GUIs" variant of that argument (also rarely heard, these days).

I suspect that templating is a problem area where the object-oriented paradigm makes no sense, which is one reason I've not thought about it much. Studying Zope might make the problems obvious, I don't know, might be a good reason to take a look!

And I agree with Ben's comments regarding plopping objects on top of the relational model, though I've never personally tried to do such a thing (and doubt I will).

So I actually suspect that object oriented paradigm might help with our thinking about a uniform module interface, and that's about it. In reality, any implementation scheme we come up with has to plug-n-play with the existing ACS anyway, IMO - I'm not up for rewriting the ACS, if we try to do so I'll never get to finish my personal site work!

As far as language wars, screw it in my mind. The toolkit should be written in a single language as much as possible, given the fact that we're already stuck writing in two languages (SQL and Tcl)! And generating HTML, XML, and even some Javascript, and the Practical Map Server (which we should integrate at some point, at least in the sense of building some examples using the Postgres JDBC interface) is written in Java. Hell, there's even a little Perl, now.

See ... it's already a mess! I suppose one could argue that adding code in one more language (Python) won't significantly add to that tower of Babel, but I'd be hard to convince.

18: Response to A Template-Based ACS (response to 1)

Posted by Ben Adida on 04/13/00 09:24 PM

Titus,

Check my comment again. I recommended *both* against OO and against Python for the ACS. We have 1 million lines of Tcl code or so, and plenty of problems already. I do not believe that a "more extensible language" or that OO is going to solve our problems. Our problems need plain and simple software engineering work, and in that respect I like the KISS principle. Adding a new language, not matter how amazing and beautiful, will only decrease the number of potential users and increase the complexity.

I recommend you take a closer look at Tcl. It's a lot more flexible than you seem to imply, and it might be faster for you to learn more about Tcl (http://photo.net/tcl) than for you to fine tune your performance and thread-safety in Python...

19: Response to A Template-Based ACS (response to 1)

Posted by Ben Adida on 04/13/00 09:35 PM

If I seem hard-set on my principles here, it's because I am. Since 1995 I've heard many people tell me how much better their language would be at web scripting than Tcl. I've heard Scheme, Perl, C, Java, Javascript, and now Python as proposed web solutions.

I've always waited to see how these solutions turn out. I experimented with everything. I wrote NSAPI crud, I listened to MIT PostDoc Scheme lovers explain the beauty of the language to me (and I do love Scheme), I wrote some Netscape Server-Side Javascript, I wrote more Perl scripts than I wish to remember, and I implemented entire web sites in Java Servlets and JSP.

At the end of the day, I'm happy with a solution that is proven and works. Nothing comes close to AOLserver/Tcl because it was built by monster hackers like Jim Davidson who understood the issues of web programming years before everyone else.

Regardless, I think your exploring other avenues is a good thing. Try Python. Try other things. But give AOLserver/Tcl the chance and attention it deserves (e.g. just as much as you gave Python). I'm happy to check out anything you guys put together, and I will happily eat the web page I wrote this message on if I turn out to be wrong.

20: Response to A Template-Based ACS (response to 1)

Posted by Brent Fulgham on 04/13/00 10:08 PM

I don't think either Titus or I ever suggested that we wanted to force Python on anyone. We have a goal of extending the AOLserver with Python (now partially completed), and it seemed that Python had some features and techniques that might make it easier to resolve some of the shortcomings with the ACS's architecture.

Ben -- I absolutely will spend more time learning Tcl. However, increasing the performance and thread safety of the Python engine is (to me) a more interesting problem, and one that I started independently of the ACS. This work will continue.

However, I recognize that the ACS is the most powerful Open Source toolset available for developing web sites. Perhaps Zope has some useful features, perhaps not. I do know that the only "large scale" websites that I know of using it have only a fraction of the functionality of even the ACS/pg sites (which don't fully implement the ACS stuff yet).

I am reminded of the "UNIX way", which has always been to use lots of small tools that do a particular job very well. Perhaps Tcl is that tool for the ACS. Perhaps Python might be a very useful tool as well. Perhaps the ACS would be most useful if we used both. I can't say. I do know that your average UNIX Sysadmin must know lots of little tools. No one says "I'm only going to use shell scripting. No AWK or SED for me". It would be laughable. So I'm not sure it would be as crazy for a website architect or administrator to need to know Python as well as Tcl.

Furthermore, we can draw a distinction between some sort of mythical Object Oriented framework written in Python that uses Tcl, which organizes and coordinates the various Tcl/adp pages, and the Tcl/ADP themselves. We don't complain that you have to know C to modify the AOLserver engine, do we? Why complain if another loadable AOLserver module is written in Python?

At any rate, I just want everyone to know that we don't want to cause a language war and that was never our intent in starting PyWX. I would prefer to talk about what we *can* do to make the ACS more modular. Since this thread is getting quite long, maybe I'll post under a new topic.

21: Response to A Template-Based ACS (response to 1)

Posted by Lamar Owen on 04/13/00 10:38 PM

RE: # of lines of ACS/pg tcl code....
CWD=acs3-pg...
find -name '*.tcl' -exec cat {} ; |wc
257561 843409 7604576

250K lines of code is not small.

22: Coming here from Zope land. (response to 1)

Posted by Kevin Dangoor on 04/14/00 12:36 AM

I find it interesting to have stumbled across this thread. We've been using Zope for about a year (http://www.byproducts.com). It did take a little time, but now that I grok Zope, I think it is a very powerful tool.

Why am I here? Without spiralling too far from the point of this thread: 1) Zope is still relatively immature in terms of prepackaged solutions (like commerce, sessions and user profiles), 2) though benchmarks show relatively good performance, and we've had fine performance so far, I do wonder about scalability, and 3) I've always got an eye out for other tools. One comment about scalability: their going to release their replication engine (ZEO) as Open Source in May. Getting bogged down? Throw another machine at it! Granted, this in and of itself does not solve issues with python's thread safety, or the fact that the TCL interpreter is likely to be faster than python's.

The existing templating capability in Zope is pretty powerful. There has been discussion of new extensions lately to try to make it play better with page editing tools like Dreamweaver.

One powerful feature of Zope is called "acquisition". Everything in Zope is an object, stored in the ZODB. (Currently, ZODB is a single file that gets appended to with each change. It is possible to store the ZODB in a relational database, but no one has done this yet.) Every object in Zope can be traversed to directly via a URL, and methods can be called on that object by simply adding the method to the end of the URL. The ZODB itself is hierarchical. Acquisition lets objects that are farther down in the hierarchy access objects that belong to its parents.

What does all of this mean, practically? Most documents in Zope do a <dtml-var standard_html_header>, which will call the method standard_html_header. You can define one of these at the top of your site, and everything underneath will access it. If you feel like having a different design in a different part of your site, just define a new standard_html_header there.

Another thing that I really like about Zope that has content management implications is that objects can contain other objects. So, the catalog item objects on my site can contain any number of images, and I can easily iterate over the images to display them.

Though the ZODB is hierarchical, Zope has an indexing system called ZCatalog. This allows you to index data on whatever properties you choose and do (relatively simplistic) searches on that data. This is a way that you can get at data regardless of where it lives in the hierachy. On Zope.org, each member can author how-tos in their own space. After review, their how-to becomes visible in the main documentation area of the site (and to site-wide searches, etc).

DTML has its shortfalls, but it also has pluses... like <dtml-in>, which can iterate over objects easily. It can iterate over subobjects, SQL results from an RDB, ZCatalog search results, lists from python, etc.

I'm not trying to do Zope advocacy here or anything. I just thought I would toss out a few bits about how Zope does its thing. I do agree that Zope has a bit of a learning curve... if there's anything you're curious about, feel free to ask.

Kevin

23: Constructive Module Systems (response to 1)

Posted by Noel Welsh on 04/14/00 02:48 PM

I think a constructive module system would solve most of the current problems, and I believe this is what Ben and Don have in mind. Such systems, which I've used in PLT Scheme, specify the imports of modules as parameters that can be varied.

To take a simple example, consider a geometry package that relies upon a drawing package. Instead of hard-coding that linkage into the geometry package, that geometry package specifies that it imports certain functions (say, draw-circle, draw-line) but doesn't say where these functions come from. Only when the package is instantiated do the imports of the module have to be made concrete. So, for example, linking the geometry package to the drawing package creates a geometry package that draw on-screen. Linking the geometry package to the printer package creates a geometry package that draws to the printer. This is a simple and powerful system that doesn't involve state explicitly, but gives a lot of the benefit of 00 techniques.

Creating such a system can be done with classes. The module's imports become arguments to the constructor, and its exports are publicly visible functions (this requires first class functions, which TCL supports). I also believe this could be implemented with TCL's namespace, which are dynamic and may be a simpler solution than using [incr tcl].

24: Response to A Template-Based ACS (response to 1)

Posted by Titus Brown on 04/16/00 11:51 PM

In the interests of making some of my reservations more believeable (and I'm sure there's ego involved somewhere too;), I've been using Tcl on and off for five years. I've also written about 5000 lines of Tcl code for Web sites using the ACS in the last year -- I've run into several limitations, and part of the plan with PyWX/Python is to reduce or eliminate these, while making the transition so easy you don't notice... which isn't that hard. We'd also like to retain full Tcl functionality (which Brent was initially unhappy with, but I convinced him rewriting the ACS was a Bad Idea ;).

I'd also like to note that PyWX has gotten a nibble of interest from several ArsDigita people. The main objection I've heard is that they're too busy writing sites to worry about the language, which I suspect is going to be the dominant objection.

cheers,
--titus

PS the main limitation, for me, is reusable & extensible code. I find myself writing the same damn functions over and over again, with minor variations... ugh. It's like I'm writing in C again...

25: Response to A Template-Based ACS (response to 1)

Posted by Stephen . on 04/22/00 10:26 PM

Module Manager

...and lots of other interesting documents describing the direction of the next ACS... http://dev.arsdigita.com/acs40

26: Response to A Template-Based ACS (response to 1)

Posted by Don Baccus on 04/23/00 04:21 AM

Yes, it appears that ACS 4.0 will address some of the shortcoming of the current ACS. I hope so! And I hope it's done in a way that can be ported fairly easily to Postgres...

27: Response to A Template-Based ACS (response to 1)

Posted by Li-fan Chen on 04/23/00 11:33 AM

Regarding templating, n-tier, OOP, python, forking, scalability and well in general--anything associated with doing the right thing.... for what its worth I and a trillion other potential ACS/PG developers humbly request that the chief coders of ACS/PG utilize the worse is better philosophy. ;)

28: Response to A Template-Based ACS (response to 1)

Posted by Ben Adida on 04/23/00 05:25 PM

Hmmm, that link is broken by the way, although maybe that was a pun on "worse is better." Maybe you want this copy of it: http://www.ai.mit.edu/docs/articles/good-news/good-news.html.

I don't think we've done anything even close to the "right thing" so far, with Tcl as the programming language and with almost no complex work taken on by the ACS core (i.e. the developer interface to the ACS is not exactly simple). So, I think sprinkling in a bit of the "right thing" won't hurt too much. It's not like we're talking about using Lisp :) We just want to spend some amount of time thinking about how best to go about this which, granted, is a far cry from the barely-designed ACS 3.x ....

29: Response to A Template-Based ACS (response to 1)

Posted by Don Baccus on 04/24/00 12:54 AM

Proper design would simplify the implementation. I don't feel any need to assign any particular philosophical label to either the process, or the result.

If aD does a decent job with ACS 4.0 and appear to be heading on a more reasonable path in the future, we'll just port. I call this the "lazy-is-better" philosophy of implementing ACS/pg.

30: Response to A Template-Based ACS (response to 1)

Posted by Val Paussa on 05/09/00 06:55 PM

Hi,

I think what you're looking for here is an application generator with enough juice to generate the ACS from a specification. That way, you could port the ACS to any platform by altering the generator code. This would also have the benefit of standardizing the database access, page display etc. methodologies that have been cobbled together over time in the ACS.

I've gone over the work that Karl has done on the templating system for the ACS. It's very good. The purpose of that system is to allow a programmer to write a text document using an XML formatted schema. That document would describe the application in question. Looping, if and other constructs are then made available to the web designer that allow the web designer to fold the input and output fields of the application into the page in any way he or she wishes.

I looked at that and thought "Hmmmmm, if this XML structure was replicated as database tables, then we could throw some reusability in here." (Well, sort of, I'd already started on my design when I found Karl's stuff)

I believe that Karl and Philip have gone back and forth on whether the specification should be in the database or in XML. The XML approach is good because you can represent complex structures that are hard to represent in a database. The database approach is good because you can edit it online and have background processes that check your work.

I've got some sort of a database application generator running right now. It generates the pages but they aren't intelligently linked to each other. I'll have added the navigation components fairly soon. The basic idea behind it is that there aren't that many ways in which data is displayed. So, I write a display generator for each method. So far, I've written generators for List, View, Edit, and Add. Once I get those up and running, I think I'll stop and try to generate either Wimpy Point or the Ticket Tracker. (The ticket tracker is probably the candidate because Wimpy Point needs a linked list viewer and an archiver.)

The generator expects to find the description information it needs in the database when called. If the description information is there, it generates a page of code. If you want to add another field to the database, you describe and generate it. If you want to add a method to the application, you describe and generate. Anyway, that's the theory.