Forum OpenACS Q&A: Article on ACS x Zope

Collapse
Posted by Roberto Mello on
Hi all,

    Someone e-mailed me this article on zope.org that Kevin Dangoor (I
don't know what's his relation to the Zope Project) wrote.


    He tries to be nice, but he didn't know some of the ACS' features and
of course, his judgement was biased. But it is pretty interesting. He
emphasized Objects quite a bit, almost as if it was the only thing
that matters. I am going to reply to his article, and you are welcome
to as well (I think).

    http://www.zope.org/Members/tazzzzz/ACSandZope

    One last thing... he mentions that Tcl is about "an order of
magnitude" slower than Python. Is this true ?
Collapse
Posted by Li-fan Chen on
I thought Bertrand's write up that bashes RMS/Eric was bad. This article is written by the same ignorance. I can feel his intent to tease you to correct/flame him.

The choice items he chooses to shoot down are pathetic. Most ACS users knows dozens of work around for each Con he can discover.

The question is should I feel sorry for him and worry if he will anger Py folks, getting no help in the future for web authoring system he choices to use.

The ACS community is much larger than a "bunch of tcl kids" that he can push around with a few choice insults in the playground. I hope he give more systems a real try instead of writing advocacy pieces that even Python devotees won't want to be caught seeing their coworkers writing.

The ACS community is growing by leaps and bound--much like the Linux community--this may result in newbies who both advocate for and against ACS with ignorance. Ignorance is not bad as long as you sit down at the end of the day and really boot up and learn something. That's what I learn as a ACS newbie. To learn from the right people. Learn to filter out the crap though is difficult, I produce crap myself, but I am learning.

Give Kevin a break, the ACS system is not as hard as he thinks, and in time he will try to work his way through the problem sets and learn all that he needs to learn.

Perhaps he will never find merit in using a full RDBMS, but at least he will know exactly why in the far future. There's plenty of discussion that explains both side of the story at Web/DB forum (he should really READ--not just said he read them already)

Perhaps there's no merit to keeping Tcl, ACS, and web authoring so simple that although you can't drag and drop, you'll find all the things you normally can't drag and drop in the programming world are a breeze to understand--without more than a week long boot camp for a non CS student or adult. But at least he knows he can always refer people boggled with object relational databases, XML, and object oriented programming to ACS and really sink their teeth into the work flow. If he and a select few well trained CS kids are hacking away at a site, I have no problem with him being disgusted with Tcl and Perl. Each language in the world have their fans and "haters". That we can never change. But I don't find his advocacy methods worthy of praise. Detail discussion by both hot and cold headed people about the pros and cons of Python, Perl, and Tcl will not fit on a DVD drive and are sprinkled all over the net, I have bookmarks to each of these issues and I love all the inputs, but I dare not summarize them as recklessly as Kevin did, he is doing a disservice to people evaluating both Zope and ACS in my opinion.

In my opinion, you will do yourself a big favor to just install the damn thing (OpenACS or Zope) and try it out for yourself. And ask your manager who else might want to use the system. And when you find out so many people in your organization will want to use the system--ask yourself if you can teach all of them to get up to speed immediately. That's the true power of tcl--bar none. Since ACS is associated with Tcl now (instead of say ASP or Perl or Py) we'll have to learn to live with the warts and benefits of the language. And that universally easy to learn and live with syntax of tcl is a real plus.

Collapse
Posted by Li-fan Chen on
In reply to my previous post...I want to mention programmer efficiency and beginner's learning curve.

Most people pick their web toolkits at first reading to the language wars.. the db wars.. the web server wars.. the os wars.. etc.. at first.

Then they figure out that, well, they can't really rule anyone out--though that's the initial (being realistic) intent. If they are really reasonable, they keep tab on all systems and as Kevin discovered, which for the best of both worlds. It turns out that all those flames did was to point out each component's specific advantages and disadvantages, and whether it meets your business needs.

That's what I am trying to get at. You can use anything, just make it work with your current contract. That's the goal. Scripting languages at the end of the day are easy to pick up. So is SQL. The TCPIP and CGI and web server apis. And a bit of common unix/nt/mac wizardries you pick up over time from books and friends. They all are basically pick-up-able. They all work just about the same way. As a matter of fact they are so simple (for the designer to code) that they end up being scalable (for the user)--almost by accident.

The machines will run them no matter what and handle complexity rather well--the problem is programmer efficiency (that's fundamentally the debate between Zope and ACS Kevin is trying to decypher).

ACS tries to answer the problem by assuming everyone in your company will want to sit down with some chinese take-out during lunch break and log-in--some day--maybe NOW (I know this from past work experience). It seems that's what ACS is aiming for. Oracle was a minor ugly detour that sorta ended that little dream though but initially when Philip had the dream it was with Tcl, simple SQL on Solid (dirt simple to admin), and AOLserver (very simple and design done right). So ACS initially was aiming for easier learning curve for users who want to dig into a web programming work flow (just as Python made Highschool students express difficult ideas quickly--objected oriented or not...)

Zope again is a dream come true for many. I have no doubt that after a swat team of clear headed experienced object orientation savvy programmers have their way with the system: they'll come up with their homebrew tool kit like ACS and optimally users can go in and add content, extend classes, poke and fetch from object pools. Elegant and done right--despite planned complexity. But the programmers know they can't get anyone to touch the code completely--it's object oriented, there's a trillion acrynums and middle-layers involved, although it's object oriented and programmer friendly--and looks good on paper--it is hard to understand for those who haven't had community college C++ training. But it'll survive in a large company IT sweat shop. Just don't try to show it to the secretary. Zope had no intention to let the secretary dig in a write a script based on a complex custom tool kit. Unless all secretaries in the future comes OOP/XML/XSL/Object Database/N-Tier ready from MIT's Secretaries Media Lab.

So Zope (and Python), ACS and (Tcl) are both aiming for good things..


But all is not well in paradise...

ACS has to face maturity, just as Zope has to, we all see how complexity can drive any newbie insane. Solid to Oracle is like moving from the suburbia Idaho to NYC. So is simple Python CGIs to your modern day Zope. So we see that due to complexity. All that benefit we seek from both systems in terms of programmer efficiency or learn-ability may diminish as time passes.

The question is what to do?

Both are on a great track, both seek to try all sorts of things to keep things simple. If you don't like big RDBMS, the DB in Zope will do fine and minimize complexity, so will OpenACS's PostgreSQL be simpler for users of Oracle. There are other ways to help. Python is reaching into the classroom in high schools. ACS is carefully maintained in clear crystal tcl no matter how big it gets, and a proper license (GPL) welcomes all newbies to dig in and learn the basic design patterns (written by the best) one would want them to pick up. So things are good, both good.

Collapse
Posted by Titus Brown on
This all begs a couple of different responses ;). I'm developing sites with both Tcl and Python, both inside of AOLserver, and I'm also helping develop the Python interpreter for AOLserver (plug: PyWX.idyll.org/).

I've also developed sites with Apache/CGI/Perl, and I've read my share of Zope documentation. For a nonsensical and silly document on how Zope is the Antichrist, see Zope Is Evil. It DOES have some serious points in it, if you can struggle through the whole thing.

Note: Please don't pass on the link to large mailing lists unless you want to offer me a machine to host it on -- that machine is running three separate Web servers + two databases on 64 MB of RAM and ~5-10 active users. I'm not sure Zope can handle it, although AOLserver does quite easily ;).

I've also set up several (5 or 6) systems with Classic ACS/Oracle, and now four with OpenACS/PostgreSQL.

So, now, a couple of points:

  • I've extended both Python and Tcl; the interpreter for Tcl is leaner, and both are written quite cleanly. Based solely on this I can't believe that Tcl would be slower than Python. If anything, the abstraction involved in Python's (well-integrated) object system would probably tend to make Python slower... (I would be happy to offer people accounts on a vanilla system to run benchmarks.)

  • Python doesn't support efficient multithreading. (I can point people to some fairly recent posts on the AOLserver mailing list that explain this in more detail.) In fact, this is the reason for the ZEO.

    Why go to all that work when AOLserver will handle it all with no trouble?

  • AOLserver has been around much longer than Zope and is therefore demonstrably a solid product. Zope cannot claim this longevity. My fuzzy little ears perked up (skeptically) at all the claims of new features in Zope... Only time will tell if the new functionality works well.

  • Content management in ACS is lousy. Content management in Zope is obscure beyond belief.

  • Several very intelligent people (who aren't me) have had trouble with Zope. See PyWX advocacy for anecdotal evidence that is born out in my experience.

  • Zope is easier to install. ACS is easier to learn, in the medium and long run.

If I had to install Oracle to check out the ACS and didn't have a pretty serious reason to do so, I would have gone with Zope. Luckily someone who already knew how to do it paid me the first time, and then aD ran a bootcamp course at Caltech. So here I am... AOLserver is absolutely fantastic, and the ACS has a maturity that is hard to match. Integrating ACS and Zope functionality (through a PyWX class) is a long-term goal of mine, because I think Zope is pretty cool, but I would prefer to work with ACS/AOLserver.

I also hate Tcl, even though I've worked in it over a 6 year period. Python is wonderful. Hence, PyWX. (Note that IMO a significant part of PyWX will be providing a nice interface to (parts of) the ACS data model.)

One final note: objects are important. Stability is more important, when you're producing a big site. PostgreSQL is unhappily not perfect, but is getting better. ZODB? I can't understand the damn thing, and it's home grown. Pfawgh.

I could probably go on longer, but everyone is probably sick of PyWX stuff already ;).

Cheers,
--titus

Collapse
Posted by Don Baccus on
The article's not all that bad.  He points out that the ACS clearly is  more mature at the moment, listing e-commerce and other modules as being clear wins for the ACS.  After chalking up the pros and cons he states that Zope's better - it's the conclusion, not the data that's bizarre.

Also, he admits that his opinion regarding Tcl's being slow, etc, is based on hearsay and guessing.  He weakens his piece by then treating such unsubstantiated opinion as fact.  An objective and intellegent reader won't miss this less-than-subtle source of bias.  A Zope zealot (Zopealot?) won't care, having already made up their mind.

In my mind, a piece like this brings forth a couple of thoughts:

When it comes to toolkit acceptance, aD's dependency on Oracle is a liability.  It would be nice to see aD take a more proactive approach to supporting our efforts (via query abstraction, etc) because it will  be OpenACS, not ACS Classic, that wins the hearts and minds of the masses if there is any chance of this happening.  On the other hand, aD has a very clear business mission and founding a religion isn't one  of them.  I can't disagree with this decision.

From the technical side, the unstructured mess that is today's toolkit  is indeed a weakness, and even though the critic has only read about and not used it, he's managed to wander close to the truth (if he only  knew how buggy it is...).  We should code name the current release "mashed potato", because that's an accurate description of the formlessness of the toolkit as it exists today.

However, I don't believe a switch to an object-oriented paradigm or to  a language other than Tcl is necessary to greatly improve the toolkit  in this regard.  aD is (finally) adopting (at least some of them) the  attitude that there really should be a toolkit architecture rather than a bazillion pieces that aimlessly interact via brownian motion or  some similarly unordered process.  We'll see if they make a significant move in 4.0.  It's crucial that order be built out of the current chaos if the tookit's really going to mature into a truly engineered piece of work.

His praise of Zope in part stems from his belief that the object-oriented model followed by the team has led to a cleaner design.  My spin's a bit different - the design might well be cleaner,  as the ACS toolkit (as opposed to individual components of the toolkit) hasn't been designed but simply grew.  Again, I don't think a  paradigm shift to the OO world is necessary to generate a clean design, but a shift to SOME paradigm is...

Collapse
Posted by Don Baccus on
Check out byproducts.com. This is the author's company's magnum opus to date, according to the data he's provided Zope. Not much of a site...certainly simple using ACS pieces and presumably simple using Zope (at least I hope so!). I don't think the guy has much experience.

Also, his comments about the use of Intermedia being a pain in the ass, as gleaned by him from web/db, is certainly on the money. aD has for now given up on indexing photo.net automatically because it has turned out to be a nightmare of sorts.

Collapse
Posted by Kevin Dangoor on

Greetings...

Perhaps I should have made it boldface when I said "this comparison is based on things that matter to me". Different people will clearly have different criteria (like ease of learning).

When you get beyond basic document management, Zope is hard to learn. This isn't helped by the fact that the documentation still needs work. The docs I saw for ACS were much better, but I have no idea how good the docs are when you are trying to do really customize things. I didn't put this in my comparison, because I've been using Zope for more than a year. I've gotten past the learning curve.

When I started working on this, the OpenACS folks were still in the process of porting a lot of the pieces of ACS. I remember that the spam module wasn't there yet. At the time, I was worried that OpenACS would always lag behind ACS Classic quite a bit, though it appears that the OpenACS people have worked quite fast. If I had seen more elements that really compelled me toward ACS, I would certainly have put in the effort to install OpenACS or ACS Classic if my concerns about OpenACS had continued. But, I have the same 24 hour days and too much to do that everyone else does. In the end, I didn't see enough to compell me away from the system I already knew.

The original poster here mentioned that I talk about Objects a lot. Objects themselves are not important, but for what we're doing having reusable, easily customizable components is. And, I think Zope excels in this and there are some other things coming into existence now that make it better.

With regards to the "Zope is Evil" article, I'd have to agree with that. I like working on top of the Zope architecture, but I don't think I'd really want to be working on the architecture itself. Getting Zope running within a special Python interpreter inside of ACS sounds like a very big job to me.

If there are factual errors in my document, please let me know. I had initially thought that TCL would be faster than Python, but later heard that TCL is actually much slower due to the data structures used. If someone has done good benchmarks of the two, I'd be curious to see that.

I am a Zope user, but I still believe there is plenty to be learned by looking at how ACS does things. If anyone has any questions about how Zope does things, I'd be happy to shed whatever light I can.

Kevin

Collapse
Posted by Titus Brown on
I have to agree with everyone who says that objects aren't the whole
solution, or even a necessary part of the solution.  OTOH, objects can be pretty useful.  As an example of something that's relatively simple and straightforward, consider:

One very nice feature of Zope is what is called "Acquisition":

http://www.zope.org/Members/Amos/WhatIsAcquisition

and

http://www.zope.org/Members/jim/Info/IPC8/AcquisitionAlgebra/index.html

(thanks to Jimmie Houchin for the links, which were posted to the PyWX mailing list)

The basic idea seems to be that a generic Zope object will take properties from its position in a hierarchy.  That is, if you defined a particular security requirement for /admin/, and not for /bboard/, objects that know how to talk to Zope will be able to pull that requirement out.  This is a bit more general than filters (albeit more mysterious), because you can e.g. re-root trees, or have different display properties (coloring etc.) depending on where things are, and it will all just work.

On the other hand, I suspect that much of the complexity of Zope comes from requiring people to fit the Zope paradigm, which is something that the ACS does not do.  So, I think the ACS is in general going to be less complex to extend, esp. for small extensions.  And I think that Zope, by focusing on a (badly documented) complex hierarchy of objects, is going the wrong way, certainly for me.

A final note on ACS vs. AOLserver -- Zope serves as a Web server and an application server, although it's primarily built to be an application server as I understand it.  AOLserver is a Web server, and ACS combines data and presentation, while PyWX is just a language extension.  So, Zope/ACS interaccessibility in PyWX would just be through standard language mechanisms (i.e. a set of data-model wrapper classes for Zope to access ACS data).  Since the ACS has quite a bit more functionality than Zope, it seems likely that most of the interconnection would go that way.

But I don't know for sure, and won't until I understand Zope, which may take some time ;).

cheers, titus

P.S.  Yes, I know you can do something similar to acquisition very
easily with 30 lines of Tcl code -- but that 30 lines would have to be explicitly called from every Tcl file, which has its own benefits and problems.  It's a nice abstraction, the way it's done in Zope.  That's all.

Collapse
Posted by Don Baccus on
Well, actually the ACS is doing something similar with virtual URLs for groups, where part of the URL shows up as part of the set of values passed into the Tcl page.  It wouldn't be that difficult to extend the concept to something more general.

Have no fear, I'm familiar with OO and believe it to be a useful paradigm - though not a UNIVERSALLY useful paradigm as the most serious religionists try to argue.

I personally believe that for a large chunk of a web toolkit like the ACS, too much structure just gets in the way.  After all, a big chunk of the toolkit consists of glorified forms handlers and report generators.

The OO paradigm could be useful in the definition of how the core technology shared by modules, i.e. permissions and similar things, is used to build those modules.

But the toolkit's pretty simple, and a uniform and orthogonal API for modules based on any reasonable underlying paradigm  would be so much better than we have today that it doesn't really matter if an OO approach is used.

Collapse
Posted by Jimmie Houchin on
I am in the process of building my personal (at home) development machine. I will be installing both Zope and OpenACS. Yes, and PyWX. :)

I currently have most of my intellectual investment in learning Zope. So most of the designs for my site are from a Zope worldview. However, I want a high quality solution which lends it's self to ease of development, management and extensibility. Now in this context ease is a relative term.

I have studied Zope primarily as a user and have not yet delved into it's source. Zope is an incredibly complex animal. I think most likely that ACS isn't quite as complex. As I develop my site I will be developing both head to head to determine which in my mind and skills is the most viable solution for me. Each have their pros and cons.

One thing that Zope brings to the table now that I don't believe is ready yet in ACS is users, groups and delegation of capabilities and access. I know there is some code in there for this but I am not aware of how it compares to Zope yet. One of Zope's foundational philosophies is customers who have customers who have customers.

This philosophy being built into Zope provides users, roles and permissions as a native part of Zope. This provides great support for building communities and delegating responsibilities with limited and appropriate priviledges.

As Titus mentions Acquisition is a very powerful part of Zope. It allows for interesting object traversal via url. This is a great blessing when used right. It can also bite your rear. :)

One of Zope's strengths is it relatively database agnostic. If you have a legacy database or have database preference due to having purchased it or having an intellectual investment in it. Then you can use it. The ZODB is always available to store your data but it is not required that you store your data in it.

Coming soon to Zope is mountable databases. The ZODB upto this point was limited to a single file. Soon a Zope app can create multiple ZODBs. This will allow for you design you app in better way. The ZODB is also reasonably storage agnostic. There is coming soon a RelationalStorage for the ZODB. This will allow the ZODB to use a RDBMS of your choice to store it's data.

The ZODB isn't really that scary. It has also proven to be stable and safe. All of it's writes are appends. So if something dies mid transaction it should be safe up to that point. Also prior to packing the ZODB it creates a backup. It has it's pros and cons, but no one should be afraid of it.

This does increase Zope's data storage options. It does not change it's complexity. To a certain extent you have to use and see if the pleasure is worth the pain. Ouch! Did I realy say that. :)

From what I've seen about ACS 4.0, it will be adding some features which are in Zope's arena. I haven't spent much time looking at it yet so I'll have to get back on it. But hopefully aD will be able to incorporate these features and improve ACS without increasing complexity. And it looks as if that one thing they are trying to reduce.

Enough for now.

Jimmie Houchin

Collapse
Posted by Don Baccus on
Jimmie raises some interesting points.  He's the second person to hint  at the complexity of Zope being an issue.  The fact that notions such  as user roles and permissions are built into the core is very good, but the provision of such things shouldn't lead to great complexity.  In fact, this is one of the more frustrating things about the ACS in its current form - rationalization of such things would simplify the ACS, not add to its complexity, at least from the POV of a module author.

My impression is that the Zope framework was designed and implemented before a great deal in the way of modules were hung on it.  That's a great approach if you get the design right.  I have to wonder if perhaps it's not overdesigned, though, given comments about its complexity and also the relative immaturity of a bunch of the modules.  Both of these observations are based on comments by users, I've never used it myself.

Also, the basic AOLserver/Tcl/RDBMS paradigm is a very simple one.  There's virtue in keeping things as simple as possible, isolating complexity to necessary areas, like the API between modules and the core technology.  I think the ACS will grow to the point where there's a core which provides services to modules, a well-defined module interface to that core, and a much higher degree of separation of the generation of the UI/presentation from computation.

Do folks in the Zope world talk at all about simplifying their model?

Collapse
Posted by Jimmie Houchin on
Let me clear up one thing. The provision of users, groups and priviledges are not anything that clutters up or complicates Zope. That aspect is relatively easy to understand and use.

In my mind what complicates Zope is the unclear line between what is core and what is added or built upon the core. Or in this context what is the framework or what is built on it. There is not a clear delineation of what source (code) or knowledge one has to understand or have in order to fix, modify or extend certain parts of Zope. This also extends into simple usage of Zope.

For example ZServer. Most people will see ZServer as a Python implementation of a Web/FTP server. To a certain extent that is accurate. Now when you go to build your website you may be tempted to say I use Apache, AOLserver, etc. so I don't need ZServer. From my understanding of the mailing lists that is not so. ZServer is also the ORB for Zope. This also is not stated as such in the install docs. So it isn't clear if you can do without ZServer. If Zope can be used without ZServer what are the pros and cons. Once again it isn't clear.

The options for Zope are bewildering. They are a blessing but also bring a higher level of understanding to be fully used by the developer.

The developers at Digital Creations are great. They are fully aware of the complexity of Zope. They are working at creating and documenting Zope APIs. They are also developing a help system for Zope. I haven't installed the latest yet, which contains the help system. From what I understand there is a help tab or button which will provide context sensitive help.

I don't know if what DC is doing could be called simplifying there model or not. On one hand I would say yes, in that it looks as if they are refactoring their API, cleaning it up and documenting it.

Personally I hope for the best from these 2 fine tools. It is nice to have high quality options. I believe there are people who will enjoy one and not the other. I believe there are people who will understand one better than the other. They will be able to plug their mind in easier. It is also nice to watch innovatios from either and see them incorporated in the other.

ACSes simplicity and robustness are some it nicer features. I look forward to seeing 4.0 implemented and seeing how aD cleans up it's house. Hopefully it will make things easier for OpenACS. It would sure be nice if they would distill the esential of ACS from the db. Abstracting out the db doesn't require them switching from Oracle. And it would potentially make it easier for them to keep up with any changes in Oracle.

Jimmie Houchin

Collapse
Posted by Don Baccus on
"In my mind what complicates Zope is the unclear line between what is core and what is added or built upon the
    core. Or in this context what is the framework or what is built on it. There is not a clear delineation of what source
    (code) or knowledge one has to understand or have in order to fix, modify or extend certain parts of Zope. This
    also extends into simple usage of Zope. "

Are you sure you're not describing the ACS? :)

I'm serious, this is a big problem with the ACS, as well.  I suspect in part this is due to the relative newness of the web and the relative immaturity of tools available for site building.  Not to mention the fact that everyone wants their site up YESTERDAY.

Collapse
Posted by Jimmie Houchin on
For those of you not following Zope (and that will be me soon), Zope has just added a, hmmm (thinks a minute) feature.

DC has decided add Perl as an available language to Zope.

Perl people are being invited to the dance but only in limited capacity. For now Perl will be limited to "Perl Methods" which are editable thru the web. All else will require that they use Python. Or else the Perl community will have to hack in other capabilities into Zope. Which they can, Zope is OS after all.

DC plans on opening up Zope to multiple languages. This is just the beginning.

I prefer the model I see with ACS/OpenACS. I like the idea of a single unifying language. I agree with Philip on this. I also like the direction I see aD going on 4.0. Clean up the model and code, add features.

I have decided against my parallel development and will be going with OpenACS.

Go OpenACS.

Collapse
Posted by Dan Wickstrom on
This news seems hardly surprising.  My initial impression of zope was that somebody took the simple elegant expressiveness of python and turned it into something more obscure than perl, yet without the same power.  They always talk about the zen of zope, which to my mind, has always clashed with the zen of python.  I guess zope can now adopt perl's motto of TIMTOWTDI.
Collapse
Posted by Chris McDonough on
It's sort of refreshing to see "anti-Zope" postings. I haven't used ACS or OpenACS (I work for Digital Creations, I use Zope 😊), but its clear that Zope could learn a thing or two from them. It's also clear that we need better intro documentation.

I worked with Zope pretty extensively before I started at DC. During that period, I was pretty much on the same learning slope as Titus when he wrote the "Why Zope Sucks" article. Working with Zope at that time, well, it was hard. After I worked for DC a little while, I wrote a "HowTo" about my experiences named "Gaining Zope Enlightenment by Grokking Object Orientation" that's available at http://www.zope.org/Members/mcdonc/HowTos/gainenlightenment.

One of the things that makes Zope so hard to get a hold of is that you can't really leverage most of your existing skillset against it and still do super-customized sites. You need to learn about the ZODB and acquisition, DTML syntax, object orientation, the difference between a "document" and a "method", ZClasses, etc. I have the feeling that this isn't as true of ACS, and that ACS makes it a little bit easier for the newbie by providing things in a context in which he or she is already familiar. Is this pretty accurate?
Collapse
Posted by Roberto Mello on
Yes. Although the ACS needs improvements (what doesn't?), once you install it and you are familiar with AOLserver (which is pretty easy actually), then you can do a lot of stuff by just visiting the admin pages.

    It is efficient in the way that it uses the file system or the database according to what's best. Static pages, for example, are kept in the file system, and are stuffed in the DB just so they can make use of dynamically generated things and for searching (which is not working under PG yet).

    I'd say that the ACS is very nice. The main drawback was the lack of true modularity, but that will be done with ACS 4 which should be out in June if I am not mistaken. It will be even more butt-kicking than it is.

    I haven't done anything with Zope, but I read a bit of the documentation and I have to admit that I was overwhelmed. I think both systems can learn from each other as you said and I hope that happens.
Collapse
Posted by Don Baccus on
"One of the things that makes Zope so hard to get a hold of is that you can't really leverage most of your existing
    skillset against it and still do super-customized sites. You need to learn about the ZODB and acquisition, DTML
    syntax, object orientation, the difference between a "document" and a "method", ZClasses, etc."

You know ... web sites just aren't that hard.  I think that might be Zope's major problem.

Now, writing the systems and applications that underly websites IS hard.  AOLserver's fairly complex.  Postgres is a lot more complex.  GIS engines are complex.  Etc.

The ACS has lots of parts but is simple.  If Zope is as complex and hard to understand as you indicate, perhaps the creators, not potential users, should be seeking enlightenment...

Collapse
Posted by Chris McDonough on
<blockquote> You know ... web sites just aren't that hard. I think that might be
</blockquote>

<blockquote> Zope's major problem.
</blockquote>

I would agree... for someone who just wants a simple website, Zope is probably not for them.  But I don't think that's out target audience any more than it is ACS'.  HTML designers don't care too much about what's on the back end.

<blockquote> The ACS has lots of parts but is simple. If Zope is as complex and
</blockquote>

<blockquote> hard to understand as you indicate, perhaps the creators, not
</blockquote>

<blockquote> potential users, should be seeking enlightenment...
</blockquote>

I guess the hardest part to get over the hump with is understanding object orientation and some of the finer points of Zope like acquisition.  We're still building the tools that allow people to not have to understand these things.  We are trying to simplify, document and focus on less general problems (ala content management).  I suppose doing these things would be considered "seeking enlightenment."  I'm sort of glad, however, that a lot of time has been spent on the architecture thus far as it sure makes it a lot easier when you have a lot of interfaces you can write to reliably.  Now we just need to expose them.  😊

For the mean time, however, until good, comprehensive documentation is produced, the best I can do is offer my thoughts about how to begin to understand the basics of the architecture, and hope people get interested; thus the article.  It's not really an article that takes a high-handed tone, despite the title.  I'd recommend you read it if you're interested...

Collapse
Posted by Don Baccus on
"I guess the hardest part to get over the hump with is understanding object orientation"

Why?  Lots of people have experience with OO.

I just keep getting this feeling that you folks are making a mountain out of a molehill.  ACS-backed websites are by no means "simple websites", the toolkit's relatively simple because the problem of creating complex websites is relatively simple.  It's the pieces you glue together that are complex.  The datamodel's the most complex piece of the ACS itself.  That's the way it should be.

Why invent persistent store (ZODB) when there are proven database engines - object-oriented as well as relational - available?  It's this "we need to build it all" mentality that I think leads to a lot of the complexity.

Collapse
Posted by Chris McDonough on
<blockquote> Why? Lots of people have experience with OO.
</blockquote>

This is true.  I've also seen that there are a lot of programmers out there who don't, especially in the field of web development.  I had many responses to that article from people who know Perl, VBScript, and PHP, and never use to a great degree the OO constructs of any of these languages.  Zope makes really heavy (some might say absurd) use of inheritance; not to mention our homebrewed 'acquisition', which is something like inheritance but is containment-based.  This stuff is pretty central to Zope, which takes a little getting used to for non-OO types, and even sometimes for OO types.  The use of these features combined with our hideous documentation is why Zope is hard for people to get cranking on quickly.

In our defense, bringing an end-to-end object model to the web is something of a new and foreign thing, and AFAICT, Zope and Apple's WebObjects are the only two products that honestly try to do so.  I know most of people just want to get something done and not have to worry about what is going on inside.  But I do think that building web apps in terms of objects is a good thing, if only because it frees you from needing to think about webdev stuff in terms of 'pages', and instead lets you think about your site in terms of how pieces of code can interact with each other, how you can build interfaces to applications that let folks use them not from just within the application UI, but via XML-RPC/SOAP/CORBA/RSS, etc without needing to strain yourself by building all the machinery to talk to these things from scratch.

This is something of a religious point.  I certainly do not mean to say that large-scale applications can't be written in terms of dynamic pages or in terms of writing procedural code that goes and grabs stuff out of SQL and displays it to the user.  They are two very different approaches that share parts of the same goal.

<blockquote>      I just keep getting this feeling that you folks are making a
</blockquote>

<blockquote> mountain out of a molehill. ACS-backed websites are by no means
</blockquote>

<blockquote>      "simple websites",
</blockquote>

I didn't claim they were, on the contrary!  My apologies if you thought so, please read the sentence again.

<blockquote> the toolkit's relatively simple because the problem of creating
</blockquote>

<blockquote> complex websites is relatively simple. It's the
</blockquote>

<blockquote> pieces you glue together that are complex. The datamodel's the most
</blockquote>

<blockquote> complex piece of the ACS itself. That's the way it should
</blockquote>

<blockquote> be.
</blockquote>

This seems to be the major point of divide between ACS and Zope... we don't have a datamodel.  Instead we have objects that have interfaces.  You have data that sits in an RDB, then you have code that operates on it.  We have objects that contain both code and data, by way of the ZODB.  Even stuff that's in an RDB external to Zope can be represented by Zope in terms of instances of a class ("pluggable brains").

A little dig, here, my pre-proposed apologies.  I hope I don't get flamed too hard.  You guys have been working like mad to port ACS over to Postgres.  This is a lot of work.  It's a lot of work because there seems to be only a thin layer of abstraction between the code that makes up ACS and the database table structures.  There's a lot of dependencies there, I presume, which is what makes it hard.

OTOH, Zope has a Storage interface that you can back up with whatever serves your purpose.  The "default" storage is a FileStorage, that is one big honking file on the Zope server's HDD.  But one of the other uses of the Storage interface is our failover stuff, ZEO.  It lets us frontend a single "storage server" with multiple Zope clients which serve as webservers.  Despite the complexity of the task, the code that lets us do such a thing is less than 100k (total, with docs).  This is pretty amazing to me.  Solutions like, for instance, Allaire Spectra, have megabytes of code to go do this, and then you need to go make sure your RMBMS is partitioned and synced properly across a number of machines besides.  I'm somewhat attracted to the elegance of the Zope model when it comes to stuff like this.  It just feels right, whereas other solutions feel.. well.. clunky.  This isn't to say they don't work, I'm sure they work fine.  But I like the fact that the haystack is potentially partitioned into 100k, when the needle can be a single byte.

Another point:  Jimmie Houchin referred to a RelationalStorage in the works for Zope.  I am one of the folks writing this beast.  This code will let us store all the ZODB information inside a relational database.  It doesn't much matter which kind, because the whole implementation is less than 8k, and I can rewrite it in about 12 hours.

These sorts of flexibilies are possible because we're using an object model inside Zope and because we're trying desperately (and sometimes succeeding) to write cohesive interfaces to those objects.  The next step is to make more apparent that power to end users of the product by writing applications against Zope and documentation for Zope that exposes it.

We know have a *long* way to go, and it's clear that we can take some pages out of the ACS book when it comes to packaging something that makes people productive quickly.  Parts of Zope are still somewhat of a framework that DC uses for consulting engagements, though most of it has quickly grown into an application server or content management system in its own right.  We need now to concentrate on hardening the interfaces to the less clear parts of the framework and documenting them.  Hopefully, at that time, the time we put into the architecture will pay off.  Maybe not, in that case I'll be out of a job.  😊

<blockquote>      Why invent persistent store (ZODB) when there are proven
</blockquote>

<blockquote> database engines - object-oriented as well as relational -
</blockquote>

<blockquote> available? It's this "we need to build it all" mentality that I
</blockquote>

<blockquote> think leads to a lot of the complexity.
</blockquote>

DC was (and is) a Python company at heart.  AFAIK, at the time it was determined that there needed to be a persistent object database to front end Bobo (that was Zope's name at the time), there were no affortable OODB alternatives that had Python interfaces.  Jim Fulton is something of an OO "zealot", so using an RDBMS to store object data was not an option in his mind.  So... he made his own OODB  😊  Mine is not to question why.  He did it, it's solid, and it's pretty damn useful.

In more general terms than the ZODB, I think you're right, however.  Zope needs to cater more to people's existing skillsets.  This is where we think that applications like the Portal Toolkit, further community contribution, and better documentation will help.

Collapse
Posted by Jimmie Houchin on
Since my previous post I've been doing a lot of thinking about OpenACS vs Zope. As I've just built my development machine it's time to get started. I've been studying Python and Zope in my spare time for the last couple of years and have significant time and intellectual property investments. My ACS experience is purely via Philip's writings and these boards.

The only issue I really questioned with Zope currently was it's complexity in it's source code. No clear programmers documentation. A certain level of difficulty mastering it's code base.

On the surface it seemed as if ACS might deliver similar features but a simpler code base to master. On further review and reading consequent messages, I now longer believe that is necessarily the case.

OpenACS has invested significant time rewriting parts of ACSes code in order to use an alternative db. However regardless of OpenACS or ACS your still pretty well wired to a db not of your choice. aD is working on ACS 4.0. This is a major rewrite in order to as Philip puts it "to correct the sins of the past" (paraphrased). This rewrite will not necessarily abstract out any of the db parts. If it doesn't then the work is still required to 'port' ACS 4.0 to PostgreSQL. It will still be very db restricted regardless.

This somewhat demonstrates that ACS is different from Zope but not necessarily simpler.

ACS is a tremendous tool. If it delivers something you need, it can be a tremendous benefit. However, most websites contain much which is not touched upon by ACS. At that point you leave the benefit of ACS. Yes, you will still be in AOLserver, wait or is that Apache, no wait, it's Oracle. Oh well.

A lot of what Philip spoke of early has changed. AOLserver is no longer the platform. Tcl is not necessarily the language. There is no single language which defines ACS any longer. A little SQL, PL/SQL, Tcl, Java, adp, ... Then comes the new features with templates, etc. The knowledge and the skill set is broadening. Maybe the complexity too. Who knows? Who knows what's next?

Zope is not necessarily less complex. Digital Creations is working very hard to as Paul Everitt puts it "reduce the coefficient of complexity". Zope is probably somewhat more complex behind the scenes because it attempts to make the job easier for the 'user' of the Zope web development tool, not necessarily the 'developer'. Zope is great for managing content and users.

When I posted previously, I was concerned that adding Perl to what was previously perceived to be a single language tool was going to increase the complexity of the Zope machinery. From messages to the list and conversations with developers I know longer believe this. Factoring out the api necessary to be able make Zope elegantly handle multiple languages is requiring Digital Creations to improve the code. While doing so they are improving the developer api's and documenting them. :)

In the end, regardless of one's view of adding Perl to Zope, the machinery of Zope will be cleaner and better documented. This will no doubt improve the ability of DC to extend and add new features without exponentially increasing complexity.

A nice feature of Zope is it's transparent use of storage. One can create objects which model their data very easily. Zope will store them transparently in the ZODB. The objects can be modified, changed, extended, etc. and it doesn't affect the storage. This a major benefit to people who either want to quickly prototype a site, don't have the skills to use a RDBMS, or just don't want to use a RDBMS. As Ben Adida said, converting objects to RDBMS sucks (major paraphrase).

I happen to tremendously like ZODB. Coming soon is ZODB's ability to create multiple files, allowing one to partition their data as they see fit. Oh but RDBMSes already can do that you say. Yes, they can. But they don't dynamically alter the attributes/columns in their tables well. I can take a class or even just an object and add or subtract properties. ZODB won't care, it'll just go about it's business. ZODB is just as dynamic as Python. RDBMSes are not as dynamic as Tcl. ZODB is also storage agnostic. ZODB can use filestorage, Berkeley db, coming soon RelationalStorage (thanks Chris), and others as initiative and desire combine. ZODB has an api in which one can create a 'storage'.

If you prefer RDBMSes and SQL to ZODB, Zope can most elegantly accommodate you. Zope doesn't care if your using Oracle, PostgreSQL, InterBase, Sybase, Solid, ... You pick. You have legacy data and dbs, fine. Bring one, bring all.

Zope also leverages greatly inheritance and acquisition. These can provide tremendous opportunity to leverage data and code existing in various places in Zope. At any point in your development cycle you can see the need to do X and then remember code/data that you can inherit/acquire to fulfill your needs. This can often be done via an url without any coding. It's awesome.

In the end you can create a quality website with either tool. Delivering a high quality experience to the web sites visitor. The difference is in the philosophy of the developer and the tool. Each will deliver a different developer experience. If everywhere you look, you see objects. Choose Zope. If the world is a database, especially RDBMS, ACS may be your cup of tea.

I see objects everywhere. I've decided not try to fit my round world into a flat square world. Having said that, I don't think anyone will go wrong with either tool. Pick the tool which models your development philosophy. Enjoy your decision. It's a great day when you have fine choices. :)

Collapse
Posted by Don Baccus on
"You guys have been working like
    mad to port ACS over to Postgres. This is a lot of work. It's a lot of work because there seems to be only a thin
    layer of abstraction between the code that makes up ACS and the database table structures. "

This isn't an intrinsic problem in the ACS, and even less so of the thin-layer paradigm used in its (cough) design.  aD has no motive to abstract out the database queries.  We do.  It will happen.

So I view this as something of a red herring when considering the overall problem.  Just as Zope's documentation problem is a bit of a red herring - there's nothing about the paradigm or approach Zope takes that necessitates poor documentation, it simply hasn't been a priority.

"Jim Fulton is something of an OO "zealot", so using an RDBMS to store
    object data was not an option in his mind. So... he made his own OODB 😊 Mine is not to question why. He did it,
    it's solid, and it's pretty damn useful. "

Zealotry is a sin in engineering.  Your comment nicely illustrates my point that the Zope folks seem to have a need to build mountains out of molehills.

The ACS wasn't designed, it simply grew out of client-driven work.  You might say that aD has had a tendency to ignore the existence of molehills.  I prefer this sin to the sin of inventing needless complexity.

Collapse
Posted by Chris McDonough on
Well, as I said, if we're wrong, I'll be out of a job sooner or later.  😊

I commend your efforts and I wish you guys luck!

Collapse
Posted by Don Baccus on
Oh, you might well have a job whether or not you're right or wrong.

Success in the marketplace has little to do with technical merit.  That's what makes our profession so rewarding, right? :)

Collapse
Posted by Chris McDonough on
I won't take the bait.  😊

As I said, I commend your efforts and wish you guys luck.

Collapse
Posted by Jimmie Houchin on
Actually much of Zope's issues are derived from contract/consulting work. DC creates Zope products for clients, and if of general interest and client permitting releases them open source.

This creates a large body of work which previously had only it's developers working on it so it is understood by them but undocumented. It just takes time for the docs and community understanding to catch up. But we would rather have it available and undocumented than to not. This enables the community to use the product and help document. This creates a mutually beneficial relationship.

DC is creating a very nice api and help system for Zope. This will speed up learning Zope. As the api's are cleaned up, published and as what is 'core' Zope becomes more clear it will be easier to build a foundation knowledge of Zope. Once having attained a foundation knowledge the rest will be much easier.

I just don't see any mountains being created out of molehills myself. I see an excellent product which like many in software development the understanding and documentation lag behind the development.

It is apparent that Zope and ACS are approached by different philosophies. One who believes that either is the right way will likely not agree with the views of the other. Such is life.

The fact still remains that a developer who is like-minded with his tools can be very productive with them. If he tries to use tools which diverge significantly from his views and philosophy will be battling his tools and consequently less productive.

Both can be good tools.

Collapse
Posted by Don Baccus on
"The fact still remains that a developer who is like-minded with his tools can be very productive with them. If he
    tries to use tools which diverge significantly from his views and philosophy will be battling his tools and
    consequently less productive. "

Oh, bullpucky.  If you're any good, you'll be productive with any reasonable tool.  The question boils down to whether or not Zope's complexity gets in the way.  OO is a powerful paradigm, but this doesn't mean that Zope's well designed (or not).  The issue's not about religion, it's about execution.

A good engineer understands when the OO paradigm works - and when to leave it behind.  Not to mention when to use existing, proven tools rather than re-invent them simply because the vendor prays to the wrong god.

Collapse
Posted by Chris McDonough on
I agree with your points, not your conclusions...

For the most part, when you're selecting a tool to make a web application, you should pick what you know best, like best, and maybe even what is most popular and best supported, or even handiest.  I can't really fault a risk-averse Fortune 500 manager for choosing Vignette over Zope or ACS.  Why would he choose them?  He's never heard of them, it's a lot of work to do the research right, and his butt is on the line.

However, when creating the actual framework that allows other folks to create web applications, I don't think these constraints apply.  We know our butts are on the line, and we are consciously taking a path we think suits us and allows us to produce the best software we can.

There is lots of technology in Zope that we created, yes.  Most of these technologies are based on ideas that aren't really new, but that haven't been widely applied to web application development.  Desipte this, we also interface with most popular and commonly-used web technologies (Oracle, Postgres, other SQL DBs, Apache, IIS, soon Perl, DOM, and some various XML technologies including XSLT).  So I think we *do* support a lot of technologies that we didn't create.

In defense of the stuff we did create from scratch, I would also offer that if nobody ever created anything new, we'd still be using rocks to bash our grain to make flat bread.  The folks here are without reserve the smartest people I've ever had the pleasure of working with, and I'm very happy to have the pleasure of seeing the stuff they've created establish a life of its own.

I respect your decision to stand on the shoulders of giants by using proven third-party technology to back ACS.  We've done the same with lots of things in Zope.  Other things we've created ourselves, and FWIW, I think they're fantastic.

Collapse
Posted by Don Baccus on
"In defense of the stuff we did create from scratch, I would also offer that if nobody ever created anything new, we'd still
    be using rocks to bash our grain to make flat bread."

Known as the "it's new, therefore it must be better" argument.  Sorry,  not a sound engineering argument.

"The folks here are without reserve the smartest people I've ever had
    the pleasure of working with,"

Again, says nothing about the quality of their work.  Might simply mean you're socially impoverished.  And even smart people have built amazingly stupid software at times.

These are hand-waving arguments.  And you wonder why I'm suspicious that nothing other than religious conviction underlies your devotion to this approach?

"I respect your decision to stand on the shoulders of giants"

Newton was being sarcastic when he made his oft-misunderstood statement...

Collapse
Posted by Chris McDonough on
Uh, am I supposed to be defending Jim building the ZODB in a response to you?  I can't.  It's finished, functional, and stable.  There's not a whole heck of a lot I can do about it, even if I wanted to.  You can download it yourself as part of Zope from http://www.zope.org, there's a UML model at http://www.zope.org/Documentation/Developer/Models/ZODB, or you can read a draft article half-devoted to it (written very recently) at http://starship.python.net/crew/amk/python/writing/zodb-zeo.html

Is there something else specific I am supposed to be defending?  OO?  Dude, the *name of the product* is the Zope *Object* Publishing Environment.  It'd be a little tough for me (assuming I like our product) to not defend an OO approach to web development with a name like that.  😊

Is there anything else I should be defending?  My skills?  Perhaps my co-workers' skills?  The paucity of my social interactions?  My handwaving?  (BTW, Your suspicions about Zope being overengineered and over-complex are curious, as you've never used it, but I trust you've heard it from "smart people" on "good faith" and they're grounded in solid reason.)

This exchange is becoming pointless, but its been a lot of fun so far.  I have a strong conviction that we need more guys like you working for/on competing companies and projects.

Collapse
Posted by Don Baccus on
"Dude, the *name of the product* is the Zope
    *Object* Publishing Environment. It'd be a little tough for me (assuming I like our product) to not defend an OO
    approach to web development with a name like that. 😊 "

In other words, you're not capable of providing an objective evaluation, because you're paid to be religious.  Sort of like listening to a right-wing fundamentalist talk about evolution.

Fair enough - but, why not keep quiet except when in Church?  You're making it clear that you're committed to a religious defense.  Cool - but I'm more interested in an objective evaluation.  If you have nothing more to offer, there's no need to waste your time.  Spend it on your knees, lighting candles and praying to invented saints.

"Is there anything else I should be defending? My skills? Perhaps my co-workers' skills?"

No need to defend your co-workers' skills - after all, they're the smartest folks you've ever worked with!

Now, you might consider defending why I should be impressed by this.  Does it speak of your co-workers, or of yourself?  I don't see how it speaks to me at all.

Not that it matters - the important point is that it doesn't speak at all to the quality of the work they do.  Why do you keep answering in ways that attempt to focus attention on the quality of the people involved, rather than on their work?

Curious.

Collapse
Posted by Chris McDonough on
" Not that it matters - the important point is that it doesn't speak at all to the quality of the work they do. Why do you keep answering in ways that attempt to focus attention on the quality of the people involved, rather than on their work?"

Because the work is available for all to see.  Download it and find out for yourself what its quality is, read the ZODB UML model and read the article links I gave.  Why should I go through any effort solely for your sake when you're unwilling to expend any energy or thought on it?  You claim to want objectivity, but from your tone and your lack of initiative, I'd be more apt to guess you just want to look like the biggest fish in the smallest pond.

Collapse
Posted by Ben Adida on
Now I'm entertained! We need at least one good flame war on these forums, and now we have it.

What bothers me is that no real arguments have been presented here to make a point about Zope. If you're posting on the OpenACS web site (which you're more than welcome to do!), it behooves you to make a solid point about why you think Zope is good before we go out and take you seriously.

In particular, the idea that OpenACS / ACS is not as good as Zope because it includes "too many" differently named elements (Tcl, SQL, ADP) is laughable. So if we called it Qtcl, QDB, Qweb, would it be less confusing? Look, I'm perfectly open to the idea that we can learn something from Zope. But let's see some *real* arguments here, objective advantages, not warm-and-fuzzy arguments about what "feels more consistent" or how smart Zope developers are. Note that OpenACS is *not* willing to compromise on certain things: ACID compliance of the data store. Efficient calculation of aggregate information (this is why we like SQL). Scalability to millions of hits per day in practice, not just in theory. Quick learning curve and rapid development.

I'm sure Zope has a few things to teach us. I invite you to take into account these important restrictions and submit a real proposal for how we can benefit from Zope's approach. If you think Zope is simply better on every level, you should, by all means, use it!

Collapse
Posted by Chris McDonough on
""""
Now I'm entertained! We need at least one good flame war on these forums, and now we have it.
"""

He started it.  😊

"""
What bothers me is that no real arguments have been presented here to make a point about Zope. If you're posting on the OpenACS web site (which you're more than welcome to do!), it behooves you to make a solid point about why you think Zope is good before we go out and take you seriously.
"""

Actually, my original post was in response to Titus' "Why Zope Is Evil" article referenced in the discussion... I am interested in ACS, it's reputed to be very good, and I had hoped to get an answer to a specific question that I asked about ACS' accesibility to new users which was answered (very well) by Robert Mello.  I have no genuine desire for you to take me seriously beyond that, though I've been having quite a good time bickering with Don since then.

"""
In particular, the idea that OpenACS / ACS is not as good as Zope because it includes "too many" differently named elements (Tcl, SQL, ADP) is laughable. So if we called it Qtcl, QDB, Qweb, would it be less confusing?
"""

If someone said this, it wasn't me.  Believe me, we've got our own acronym problems.  😊

"""
Look, I'm perfectly open to the idea that we can learn something from Zope. But let's see some *real* arguments here, objective advantages, not warm-and-fuzzy arguments about what "feels more consistent" or how smart Zope developers are.
"""

OK.  I like the fact you've brought up specific points that I can actually address instead of insulting my mom or something.  😊

"""
Note that OpenACS is *not* willing to compromise on certain things: ACID compliance of the data store.
"""

Nor is Zope.  See http://www.zope.org/Members/jim/ZODB/TransactionsAreImportant.  It's actually just Jim referencing your website (wait, your article personally!).  We know transactions are important.  Every request to a Zope method is conducted within transaction boundaries.  The ZODB does ACID, and SQL transactions are by default encapsulated within requests to the ZODB.  This means that you can modify objects via the same method call within the ZODB and within a transaction SQL datastore (Oracle, Postgres, Sybase, and Solid currently, IIRC) in the same transaction and retain the ACID behavior of both.  Both datastores will be rolled back in step with each other if the containing transaction fails.  Writes to the default storage implementation (FileStorage) are all appends to a single file, and we're very careful about the bits.  I am currently developing a Zope storage that uses Interbase instead of the filesystem (see http://www.zope.org/Members/jim/ZODB/RelationalStorage).  You can see more at the ZODB UML model or you can read the article linked at starship.python.net that I gave earlier.  You can also read Jim Fulton's ZODB3 presentation for the yearly Python conference at
http://www.python.org/workshops/2000-01/proceedings/papers/fulton/zodb3.html

"Efficient calculation of aggregate information"

We interface to SQL if that's your bag, and we have additional cataloging machinery for stuff stored the ZODB that's somewhat more flexible than many SQL engines for full-text searching (I have a rough time with LIKE, I'm no RDB person myself, how does one efficiently full-text search an Oracle or Postgres DB?).  It stores references (path locations) to objects as well as metadata (cached attribute values and method return values of arbitrary objects), and provides more traditional indexing capabilities.  We also have the concept of "pluggable brains" which allows us to wrap return values of SQL data into a class instance to give it additional functionality.  We are hoping to extend the cataloging machinery of the ZODB to handle SQL datastores RSN.

"""
Scalability to millions of hits per day in practice, not just in theory.
"""

I concede this point.  The biggest Zope website I know the actual stats on (Zope.org) gets about a hundred thousand hits a day on a single server.  It's not taxed, but I have no available empirical evidence to present to show that it would go any higher.  We've addressed this considerably with ZEO (Zope Enterprise Option), which used to be a commercially available product packaged along with service but has been open-sourced and is being packaged for general consumption and will be released hopefully within the next few weeks.  It allows you to use mutliple Zope clients (no theoretical maximum limit) on arbitrary machines against a single 'storage server'.  Each client keeps an object cache, so reads are relatively cheap.  Writes are more expensive.  It's well suited to web applications in which most hits are reads.  The storage server invalidates objects in each distributed client's cache when it notices an object has changed in storage.  Invalidating client caches is what makes writes expensive.  More info on ZEO is available on the zope.org website.

"""Quick learning curve"""

I definitely concede this one.  We really fall down here.  We *really* fall down here.

"""and rapid development. """

Once you get past the learning curve, I think we've got a good rapid development story (if that's not an oxymoron 😊.  We obviously use Python, which is a great interpreted scripting language, that makes things a little easier than having a compiled language like Java (I'm sure the same is true of TCL).  We've got ZClasses which are classes that you can create through the web, whose methods can be traditional Python code, DTML methods (a templating and presentation language), SQL methods (methods that return values from SQL datastores), and various other things.  I'm working on a project right now with the development team of a customer using Java to write a web client app, and we're constantly ahead of them, even though IMHO they are very smart, it's just faster to implement in our environment.  I'm sure Don will jump in here and claim that this is not meaningful because I haven't presented their resumes to him nor mine.  Nyaaah.

"""
I'm sure Zope has a few things to teach us. I invite you to take into account these important restrictions and submit a real proposal for how we can benefit from Zope's approach.
"""

This is sort of a heavy task.  It would help if you read some of the articles and documents I've linked, after which I'd be happy to answer any questions you have.  I'm not here to convert anybody.  I've heard very good things about ACS and I hope to find the time to install it and maybe steal a couple ideas from you guys (I think actually I'm going to try to steal this bboard idea from you, it's really very well done, except I'm going to try to make email work both in and out, have you thought about trying that?).  I think that against my better judgment I just let myself be trolled, this explains my extended stay.  😉

An aside:  have you noticed that the email URL generated by bboard doesn't actually point to the discussion forum that the email references, but that it points to the container for the discussions?

Collapse
Posted by Don Baccus on
"I'm sure Don will jump in here and claim that this is not meaningful because I haven't
    presented their resumes to him nor mine. Nyaaah."

No, just the opposite.  I don't care about their resumes, nor how smart you think they are.  That's the point of more than one of my previous posts. Just show me the code, baby.

Reading through the pieces you describe - which correspond pretty much to my understanding (your claims that I've not looked at Zope are incorrect) - I'm still left feeling that the solution is far more complex than the problem warrants.  Perhaps I've been ruined by frittering away my long-distant youth writing optimizing compilers designed to wean those who write the code to fly airplanes and the like from assembly code - I have a great tendency to favor lean and mean solutions to problems that are easy to read and learn.

I'm just naturally suspicious of software development systems that have a steep learning curve.  While at times such complexity is justified, generally it isn't.

So ... exactly what's so tough about writing a large website that justifies Zope's complexity and resulting steep learning curve?  If I invest the time to overcome that steep learning curve, what benefits will I see that I won't get from the ACS approach of building a relatively thin layer on top of a relatively complex datamodel?  Why is a complex and hard-to-learn framework needed to solve such relatively simple programming problems?

That initial time investment is costly, and can only be justified if there's a measurable benefit compared to approaches that are, well, more approachable.  Answers like "well, you'll be well versed in the Zope philosophy" won't cut it.

Did I really see the initials "UML" up there?  I've contracted for Rational off-and-on for the past several years and my experience there  has only strengthened my prejudice against unecessary complexity.  (actually with Verdix, far more rational than Rational, at least until  they were bought by Rational).

Of course, Rational calls itself the "e-development company" these days.  I shouldn't pick on Zope when folks like this are claiming to have the One True Answer to web development...

Collapse
Posted by Michael Feldstein on
OK, I'm not a programmer, so some of this conversation is frankly beyond my competence, but it seems to me that a key religious difference was brought up by Jimmie:

"Zope is probably somewhat more complex behind the scenes because it attempts to make the job easier for the 'user' of the Zope web development tool, not necessarily the 'developer'. Zope is great for managing content and users."

Distinguishing between developers of the code and users of the code seems like an odd one to make in the world of Open Source. Not only is ACS (relatively) easy to use; it's also (relatively) easy to see how it's wired together. To my mind, this is a huge benefit for those users who don't like the way a piece of the damned thing works. I haven't worked with Zope, but I have worked with closed-source application servers that are based on a similar idea that the users of the tool didn't need to understand what was going on under the hood. It was a nightmare.

The other side of this argument might be that if you have a good OO system with encapsulation, most developers don't really need to understand how each object works. Perhaps this is different than not knowing how your application server works; I'm not in a position to judge that. Or maybe the argument is that once you understand Zope's OO, then understanding it's source code will be easy. I'm not in a position to judge that either. Regardless, neither of these answers satisfies the underlying aesthetic of ACS. Making the system itself "rat-simple" (in Philip's words) is a central philosophical tenet of ACS, where it is clearly an ancillary (if important) goal for the Zope folks.

In my mind, then, the question is what I get in return for giving up that simplicity. The one argument I have heard so far is superior portability, which the OpenACS team believes can be achieved without sacrificing the simplicity of the system. I won't evaluate this point since I don't have the expertise; I'll just say that it's a point of contention between the two groups. OK, next argument. One of the claims that I've heard in favor of using Python is that it's easy for one programmer to read and maintain another programmer's code. But again, this advantage seems to be nullified by the simplicity (or lack thereof) in the two systems. (And the advantage is usually touted in comparison to Perl, not tcl.) We don't know how development time or scalability compare because Zope apparently doesn't have enough of a track record to provide data. If there are other pros and cons to be considered, I haven't heard them (or understood them) yet.

It may be that you Zope folks believe that OO in general and the Zope implementation in particular will be superior in some or all of these areas (i.e., scalability, reduced development time, portability, ease of maintenance, and simplicity) in the long-run but that it's a theory you're still testing. There's nothing wrong with that. At this point, though, it sounds like your argument is just that--a theory.

Collapse
Posted by Chris McDonough on
"""Just show me the code, baby."""

http://www.zope.org/Products/Zope/2.1.6

"""Reading through the pieces you describe - which correspond pretty much to my understanding (your claims that I've not looked at Zope are incorrect)"""

You contradict an earlier post in this thread in which you say "My impression is that the Zope framework was designed and implemented before a great deal in the way of modules were hung on it... <snip>... Both of these observations are based on comments by users, I've never used it myself."  Which is it?  Have you used it or not?

"""So ... exactly what's so tough about writing a large website that justifies Zope's complexity and resulting steep learning curve?"""

I think the complexity issue is a red herring.  We need to document the model to shorten the learning curve, this is not necessarily a flaw in design nor does it indicate overcomplexity.

"""If I invest the time to overcome that steep learning curve, what benefits will I see"""

You personally?  None at all.  I like you right where you are.  😊  Seriously, though, I have no desire to evangelize Zope here, it's next to pointless (you have your own application server) and it's starting to become time-consuming.  I will be happy to answer or address specific technical questions or points raised about it (ala Ben's post) as opposed to participating in your current chest-beating session.  I can offer no further arguments for Zope here than the availability of the product, its documentation, articles about it I've linked to, and its source code.  If you're that interested in objectivity, do some work, I won't do it for you.

"""Did I really see the initials "UML" up there? I've contracted for Rational off-and-on..."""

I don't particularly like UML-style models that also serve as conclusive documentation.  It's way too much work for the average person to ingest.  This is an example of the kind of thing we need to fix.

Collapse
Posted by Don Baccus on
"Which is it? Have you used it or not? "

I've not used it.  Never said I have, therefore have not contradicted myself.  That doesn't mean I've not explored the Zope site, haven't looked at examples, or am totally clueless about how Zope works.

I've never used mySQL, either, but I have a pretty good understanding of how it works.  Better than many of its users, judging by the arguments those who live in that religious space make to justify the lack of ACIDity, etc, that make it so weak.

"I don't particularly like UML-style models that also serve as conclusive documentation. It's way too much work for
    the average person to ingest. This is an example of the kind of thing we need to fix. "

One thing we can agree on, at least.

As far as the rest of your answer, it's a non-answer.  Your attitude's  strange, I have no problem being objectively critical about the ACS.  A tool's strengths and weaknesses are generally fairly obvious to those who work with them.  I have a difficult time understanding why you can't spit forth a paragraph or two which objectively describes the strengths and/or weaknesses of the Zope approach.  You've found plenty of time to make statements that are irrelevant ("these guys I work with are really smart!") or religious.  You really can't take ten minutes to make a clear and concise statement of Zope's benefits?  Perhaps the philosopher wears no clothes?

There are undoubtably areas in which Zope does a better job than the ACS (since the ACS undeniably sucks in some areas).  One Zope user has deflated one of my expectations in this regard, though:

"In my mind what complicates Zope is the unclear line between what is core and what is added or built upon the
    core. Or in this context what is the framework or what is built on it. There is not a clear delineation of what source
    (code) or knowledge one has to understand or have in order to fix, modify or extend certain parts of Zope. This
    also extends into simple usage of Zope. "

Quoted from an earlier post.  I have to admit I read this with great disappointment, because this is precisely the area where I'd expect Zope to provide an example worth studying.  It is an area where the ACS truly sucks, and where repair work is needed.

So, if Zope doesn't provide a good example in this area, where does Zope provide good examples?  What shines in the design?

Collapse
Posted by Jimmie Houchin on
-----
OK, I'm not a programmer, so some of this conversation is frankly beyond my competence, but it seems to me that a key religious difference was brought up by Jimmie:

"Zope is probably somewhat more complex behind the scenes because it attempts to make the job easier for the 'user' of the Zope web development tool, not necessarily the 'developer'. Zope is great for managing content and users."
------

Yes I have stated and maintain that there are philosophical differences.

However, you just provided the reason beyond separation of 'user' and 'developer' with the "I am not a programmer..." statement.

There are many people using Zope to build websites who are not programmers. They don't know OO from procedural from functional from multi-paradigm, ... Nor should they need to. They don't know Python from Perl from Tcl. They don't know SQL. They just want to build a website.

Within website development firms you have seperation between people who do the programming and people who do the design. There are real seperations in real places. However, they are those who do it all.
This is actually not an uncommon model.
Real life, real things, do not generally require the user to know how it works. When you operate a real machine, you know how to interact with the machine. You have a basic 'contract' with the machine in that when you do a particular action it performs a particular function. It should be consistent and repeatable. There are users of the machine and there are developers/maintainers of the machine.

As far as open source goes, I am very pro open source. I use many open source products and never have the desire to look at the code. I feel better knowing it is available with appropriate licensing.

I am at work writing this message from a proprietary platform that has bugs I would love to see fixed. If I was an expert in this area I could do nothing due to the lack of source and licensing.

I advocate open source to friends who have absolutely no desire or intention in programming. It is still of tremendous benefit. Open source does not just benefit programmers it benefits everyone.

Collapse
Posted by Roberto Mello on
The point being discussed here is more on design, approach, advantages and disadvantages of each. ACS and Zope both have pros and cons, with some engineering issues being discussed so we can educate ourselves on both sides.

I am enjoying this thread a lot and hope it goes on.

About the issue of people not knowing how to program to build websites, this is clear. I can "build" an entire website using my Netscape Composer, or Dreamweaver (brr) or whatever. This is not what's being discussed.

The thing is that it has been noted and dicussed in many places (including this thread) that Zope is difficult to grasp, is difficult to start getting results with. The thread went on to try to find out why from people that know Zope. Same with the ACS approach.

Collapse
Posted by Don Baccus on
"There are many people using Zope to build websites who are not programmers. They don't know OO from
    procedural from functional from multi-paradigm, ... Nor should they need to. They don't know Python from Perl
    from Tcl. They don't know SQL. They just want to build a website. "

You can build significant websites with the ACS without doing any programming whatsoever, that's one of the toolkit goals.

Now ... there are some significant problems with the current implementation.  Presentation isn't split out from the guts for the most part.  But a primitive templating system using ADP pages (which allow the embedding of Tcl variables and code snippets in a way not all that different than done in DTML) exists and is being adopted for more of the toolkit in the upcoming ACS 4.0.  Admin pages used for customization tend to lag the data model.  New features like templating aren't backfitted to legacy modules in a timely fashion.

This, though, is an arsDigita resource problem, not a design methodology problem per se.  New features tend to fall out of client sites and are integrated just well enough meet contract needs - that's  my impression from the outside, at least.  Recently, aD has hired toolkit engineers whose job is to address this (and other) problems.  They won't be tied to client development contracts but will be free to  concentrate on core technology.  Three cheers!

Despite these flaws, if you're willing to stick with the vanilla presentation style, much can be done from the admin pages in a matter of an afternoon or so.

Customization's still primitive, but growing in usability from this point of view.  Certainly the goal is to make it increasingly possible for non-programmers to build ACS-based sites.  And to increase the level of customization possible without digging into and hacking the code.

So I don't quite see your point in regard to the Zope vs. ACS debate.  Does Zope provide a greater level of out-of-the-box customization than the ACS?  I'm curious.

Whether or not, it's a matter of design goals and implementation completeness, not Zope's OO design vs. the ACS thin layer atop an RDBMS approach.  Either paradigm can lead to a similar level of customization.

Collapse
Posted by Jimmie Houchin on
I was simply responding to Michael Feldsteins comments. He quoted a comment I was making about Zope's complexity being in developer space not user space.

I have stated many times that I think great sites can be developed with either tool. I think both tools provide significant support for a full featured website. They have different flavors. They taste different. Is that better? I didn't mention philosophy. Oops, it slipped. :)

At some point your website will generally need to do something not built into ACS. It may be very specific to your site. At that point your not in ACS but you are still in AOLserver and SQL and the Oracle/PostgreSQL. Your skill set requirements just increased.

With Zope that is not the case unless you choose for it to be. Zope provides a web server and transparent persistent storage (which meets ACID).

The scope of Zope, is the scope of AOLserver and ACS and SQL/RDBMS. This is where the difference in complexity lies.

I have read numerous times of Don banging his head against pg source. Using PG or Oracle or maintainance of either is a simple as ZODB.

So if you include AOLserver/ACS/SQL/RDBMS vs Zope. AOLserver/ACS/SQL/RDBMS isn't as simple or non-complex as only ACS. And that's not saying ACS is without issues. ACS has issues, and aD is addressing them in 4.0.

Basically I'm saying that the scope of learning in Zope is somewhat more than just ACS. It is also a documentation issue of howto and how things interact with each other. But like aD, DC is addressing these issues.

Also when Zope's learning curve is brought up, DTML is a part of that. DTML is integral to Zope. So the assessment of Zope's learning curve and complexity is based on integral capabilities that aD is adding to ACS and that are not in the current assessment of ACS.

None of this in Zope is poorly designed. It isn't the design which makes it complex. It is the scope of functionality and grasping it all without sufficient documentation. Also for me, I don't do this full time. I am learning in my spare time, so the process is slower/longer.

When we each talk, I talk about all Zope is and likewise you speak on what ACS is. So is somewhat an Apple and Oranges issue and can be difficult to do accurately.

Just for grins on this, hmmm, Win95 machine I have at work I looked at a default install of Zope and a download of ACS.

Zope had 474 *.py files and 221 *.dtml
ACS had 2361 *.tcl files and 105 *.adp

This can mean whatever you want. I'm not implying anything. It is just interesting data. And not all of the *.py files are Zope per se, some are a part of the included Python interpreter.

Collapse
Posted by Albert Langer on
"Just for grins on this, hmmm, Win95 machine I have at work I looked at a
default install of Zope and a download of ACS.

Zope had 474 *.py files and 221 *.dtml
ACS had 2361 *.tcl files and 105 *.adp

This can mean whatever you want. I'm not implying anything. It is just
interesting data. And not all of the *.py files are Zope per se, some are a
part of the included Python interpreter."

Hmm, I'm currently reviewing OpenAccess and untarred 3,326 files in 226 folders.

As a result of this thread I'm also reviewing Zope (thanks!) and just installed the binary on Win98. 1252 files, 69 folders. Loved the way I could just start playing with Zope immediately. But am deterred by the complexity of both as am not a competent programmer at all. (Not deterred by UML diagrams - wish they were included with the download).

I'm strongly inclined to prefer the steep learning curve of a sound OO architecture to a quick fix that ends up hard to maintain (eg the current necessity to create a minimal separation between presentation, data access and other code in *ACS just to be able to do upgrades and ports without forking).

But perhaps that's just because I don't expect to be actually working much on the "internals" anyway and just want to easily add minor enhancements/customizations without having to keep redoing them as things move along.

Two things worry me about Zope which I'd be very interested in comments on.

1. RDBMs are a well proven technology for reliably doing fairly simple things on a large scale. I suspect Zope will adapt better and do more complex things better, but will it run into a performance barrier that it just can't get past? (I like the potential of the ORDBMs approach in Postgresql, even though it is not yet being adequately used for what it is, MUCH more than just a true ACID RDBMS, but one that also has OO aspects, while still being sufficiently close to the normal RDBMS paradigm to be capable of achieving reasonable performance).

2. How come Zope doesn't have a working ecommerce or even minimal payments system built in (if I have understood correctly)? If its really so easy to build things with it, surely that would be there already? With my level of skills I don't like the idea of having to figure out the ACS ecommerce implementation, or even the Shoppe. But it's there, so it must be doable and from my quick look so far it will at least be possible to adapt it to different requirements, however difficult that may turn out to be. What worries me is that if the same was true for Zope, there ought to be something there now, but I can't find it. (Haven't searched thoroughly yet 😉

(Sorry, should shut up until I've finished studying both thoroughly, but this thread is already running anyway and I'd really appreciate specific comments on the above two issues from both perspectives).

Collapse
Posted by Don Baccus on
"At some point your website will generally need to do something not built into ACS. It may be very specific to your
    site. At that point your not in ACS but you are still in AOLserver and SQL and the Oracle/PostgreSQL. Your skill
    set requirements just increased. "

This is untrue.  You can reuse existing elements of the ACS, including  utility modules whose sole purpose is to ease the task of extending a site's functionality beyond that provided "in the box" by the toolkit.

Is this as easy as extending Zope?  It will depend on how closely the desired functionality matches existing functionality in either case.

"  With Zope that is not the case unless you choose for it to be. Zope provides a web server and transparent
    persistent storage (which meets ACID). "

Once you've learned how to use Zope objects which implement persistent storage, sure, you don't have to learn how  to use them.  This statement can be made about any software tool, though.

You imply that learning to use these Zope objects is easier than learning to use SQL.  Given that the SQL needed for most web sites is fairly trivial, do you have any empirical evidence that this is true?

Or is it just another item of faith we're expected to agree with.

"I have read numerous times of Don banging his head against pg source. Using PG or Oracle or maintainance of
    either is a simple as ZODB. "

Something else that's untrue.  If I want to develop objects in Zope that support both Postgres and Oracle, I'll have to deal with exactly the same query differences.  If I knew I wanted to do this in the beginning, I'd take care to build abstractions to isolate differences as much as possible.  I'd use unions instead of outer joins from the beginning.  Etc.

Just as I would do in the AOLserver environment if supporting two DBs was my goal.  In fact, that's one of the appeals of AOLserver's Tcl API.

And even more to the point - the same bugs in Postgres will exist, and  ZODB and Zope, though undoubtably fiendishly clever, aren't capable of dipping into the PG source and fixing bugs.

Now, as it happens, aD chose not to structure the ACS in a way that makes it easy to support multiple databases.  That's a choice that can  (and eventually will) be undone.  If people providing Zope objects transparently support a wide suite of persistent storage engines, my hat's off to them.  I have to wonder if this is really true in practice or merely in theory, though.  If true in practice, be assured that someone worked hard at making it real.

"Also when Zope's learning curve is brought up, DTML is a part of that. DTML is integral to Zope. So the
    assessment of Zope's learning curve and complexity is based on integral capabilities that aD is adding to ACS and
    that are not in the current assessment of ACS. "

I honestly don't understand the above.  If you're speaking of templating via ADP pages as being something that's not in the current assessment of ACS, then the reality is that for many users this will flatten, not steepen, the learning curve.  If you mean something else,  feel free to explain.

"Zope had 474 *.py files and 221 *.dtml ACS had 2361 *.tcl files and 105 *.adp

    This can mean whatever you want."

Not really.  It can only mean one thing: nothing.

Collapse
Posted by Don Baccus on
Well, to answer Albert ...

The AOLserver/Tcl/RDBMS solution has proven to be immensely scalable, whether or not the ACS is involved.  That's one of the appeals.  This is largely due to AOLserver being fiendishly efficient and due to the database drivers (for Oracle and PG, at least) and the Tcl interpreter  being built-in.  And with the fact that the folks trying to keep AOL up and running have put in many, many hours making sure that the threaded Tcl interpreter works very well.

Zope appears to be untested at the high end.  I doubt if there's anything intrinsic in the architecture that would require it to be inefficient, though.  If there are scalability problems I'd guess it would have to do with the fact that the AOLserver folks have HAD to make certain the server can scale, because AOL and Digital City are two of the busiest sites in the world, while the Zope folks haven't dealt with sites of this size.  (100,000 hits a day ain't nothin').

If there are scalability problems with Zope, my guess is they'll fix 'em before you really run into them, unless you're planning to roll out Amazon The Next Generation or something in the next few weeks.

Since you mention e-commerce, that particular module in the ACS is apparently pretty easy to modify.  In particular, it's one that's been templated so you can modify the look without digging into acres of Tcl  code.  And it's designed to be largely configurable from the web admin pages.  I've not worked with it myself, though I shortly will be  setting it up for a friend's business.  But I've talked to others who  have worked with it, and apparently they feel it stacks up with commercially available e-commerce kits quite well.

Of course, that might speak more about the competition than anything!

Collapse
Posted by Jimmie Houchin on
Albert: I did a search for specific file types. I may have missed some significant files in ACS that I'm not aware of. I only did *.tcl and *.adp. I am not familiar with ACS sources.

The difference in our Zope descriptions was also that I limited it to *.py and *.dtml. In Zope the dtml files are used for management screens and anything you want rendered into html. The py files are the Zope source code. The reason I searched for *.py is because Python bytecode compiles it's source files. Their will be a corresponding *.pyc file for each *.py file.

With Zope you can use the database of your choice. I like the ZODB which comes with Zope. Its most common storage is called FileStorage. ZODB can use various storages to hold the data including PostgreSQL via a new RelationalStorage. Regardless of which backend storage one uses for ZODB it is relatively transparent to usage. The user will not have to create tables/column etc.

Feel free to use any RDBMS of your choice with Zope. ZSQL Methods are an elegant way to interface Zope with any RDBMS. Zope will also cache the RDBMS items as objects. Their are many users of PostgreSQL on the Zope list. Or conversly their are many Zope users on the PostgreSQL list. :)

With the coming Zope 2.2, Zope will be including a new Help System. The Help system will help explain various parts of Zope and it's API.

A couple of different e-commerce products are being written for Zope via some users. Their is even a ZCommerce mailing list. This is an area which will definitely be improved and is as we speak.

There are many areas which Zope's OO framework is paying off nice benefits. There have been some very recent improvements that have been relatively easily implemented because of this.

The Zope community is very friendly and most helpful. Feel free to subscribe to any of the mailing lists and fire away any questions you have.

Collapse
Posted by Don Baccus on
"Regardless of which backend storage one uses for ZODB it is relatively transparent to usage."

RELATIVELY?  Care to define that term?  If it's not 100% transparent to usage, then the user's left with the same problem we have with the ACS.  When we've discussed database abstraction within the context of OpenACS, our goal to hide SQL differences from users 100%.  Of course,  that goal might be impractical for us, too.

Your comment that Zope users don't have to create tables is well-taken.  ACS users don't have to create objects...

Any reason for me to believe that creating an object in the persistent  store is any easier than creating a table?

I will admit that there's an appeal to the notion that you only code in Python and don't need SQL.  From my point of view, I don't see that  as an advantage to the average user, though, since the goal is to  provide a comprehensive toolkit so that users shouldn't have to program at all.

For instance, your users are working on Zcommerce tools.  The ACS simply provides you a comprehensive e-commerce module off-the-shelf.  Not programming at all is easier than programming, even if the programming environment leads to wet dreams...

Collapse
Posted by Roberto Mello on
"Zope had 474 *.py files and 221 *.dtml ACS had 2361 *.tcl files and 105 *.adp

This can mean whatever you want. I'm not implying anything. It is just interesting data. And not all of the *.py files are Zope per se, some are a part of the included Python interpreter."

Hmm, I'm currently reviewing OpenAccess and untarred 3,326 files in 226 folders.

Really ? So you judge one tool's technical aspects by the number of files in it ? Pathetic. And what the heck is OpenAccess ? Let's see how much it means... Here's a list of the ACS modules that are ready off-the-shelf withouth ANY programming, creating of tables, objects or whatever. Please post here which of these modules come with Zope so we can use the number of files in an aspect where it matters.

Adserver, Banner Ideas, Content Sections, Custom Sections, Content Tagging, Display, Dynamic Publishing System (integrated with ACS release 4.0 as of summer 2000), FAQ, General Comments, General Permissions, Graphing, Modules, New Stuff, News, Permissions, Poll, Press, Prototype Builder, Redirects, Robot Detection, Spam, Tools, Curriculum, Customer Relationship Management, Email Handler, Member Value, Portals, User Administration, User Groups, User Profiling and Content Categorization, User Registration and Access Control, User Session Tracking, Address Book, BBoards, Bookmarks, Calendar, Chat, Calendar Widget, Contact Manager, File Storage, Intranet, Ticket, WimpyPoint, Classifieds, E-commerce, Neighbor to Neighbor, Audit, Data Warehouse, Directory, Download, Site-Wide Search, Static.

Collapse
Posted by Chris McDonough on
test, maybe a short message works...
Collapse
Posted by Chris McDonough on
yep... I'll bust my message up into a few pieces, then. """ RELATIVELY? Care to define that term? """

OK, now we're getting somewhere. Here's a snippet of Python that sits in a configuration file on disk that creates a FileStorage instance (from memory, the spelling might be wrong, the thrust is not):

import ZODB
db = ZODB.FileStorage('/tmp/Data.fs')

Here's a snippet of Python that does the same with a relational storage:

import InterbaseStorage
db = InterbaseStorage('/tmp/zope_storage.gdb', 'localhost', 'user', 'password')

These are the only configuration differences, it is only configured once, and the remainder of the code in Zope is the same and is 100% transparent to the end user. The same operational rules apply when using either storage. One storage is hosted on the filesystem, the other is hosted in an Interbase database. A particular storage format/table structure is defined for either.

The rest is common between storages. For the most part, any valid Python statement can act against the ZODB (doesn't matter which storage type its hosted by). Objects that are not Python builtins must subclass from a Persistence class to participate in the ZODB. There is one other unobvious deviation from the norm that Python programmers need to understand, which is that they must treat mutable attributes of builtin objects immutably (ie. you need to notify the storage separately if you use methods of built-in objects (arrays, hashes, etc) that modify them "in-place", as opposed to using assignment).

Most storages support "versions" and "undo". Undo works on a per-transaction basis, and all transactions are undoable unless objects referenced in the transaction are modified by a subsequent transaction. Versions are long-running transactions used mostly for site "staging", where you can modify objects without having your changes show up in production until an explicit commit (handy for HTML developers) reducing the need for a separate staging server.

Recently, we've added support for Mountable Storages, which means that a single application namespace can host different storages simultaneously. This is desirable in cases that you want to make use of BerekelyStorage (a storage using BerkelelyDB) in situations where undo and versions are not necessary and the "all-writes-are-appends" properties of undoable storages lead to unwanted file growth. You can think of mountable storages somewhat like Unix mounted filesystems, where the application namespace is the root of the filesystem.

Users create objects, whether through the web interface, through DTML, or through Python within either storage. You interact with the ZODB in a manner consistent across them. Transactions begun by a web request or via DTML are automatically committed or aborted at the end of a request. Transactions begun within Python code must be committed explicitly.

Collapse
Posted by Chris McDonough on
(cont'd)

Here's an example of creating an object in the ZODB from the Python command line (any storage type will work, you needn't worry about it, it's configured at a lower level):

import Persistence.Persistent
import Zope
app = Zope.app()

class Foo(Persistence.Persistent):
   def __init__(self, bar):
       self.bar = bar

app.my_fooinstance_one = Foo(1)
app.my_fooinstance_two = Foo(2)

app.my_sequence_one = ['a', 'b', 'c']
app.my_sequence_two = [1, 2, 3]

app.my_mapping_one = {'a': 1, 'b':2}
get_transaction().commit()

For DTML, I could make a constructor method in Python named "manage_addFoo" for this class and would call it like this:

<dtml-call "manage_addFoo(1)">

The prior was just an explanation of how it works, to use Zope you don't need to know any of this. You install it, visit the "Windows Explorer-like" management interface through any web browser, log in, hit a dropdown box, and pick something you want to add (a DTML method, a Foo instance, a Portal, a mailhost, a Python method, whatever).

All management of Zope can be performed through the web aside from building disk-based Products (though you can build ZClasses, which can be part of through-the-web products), and things that could potentially circumvent the security interface, like arbitrary Python in an external method (though Evan Simpson has come up with a way to add restricted Python code through the web that doesn't circumvent the security system. It has the unfortunate name "Python Methods"). Most Zope content stored in the ZODB can also be accessed via FTP and WebDAV as well.

To get started, most application developers need to start by understanding DTML (this will take them a long way alone, but needs better documentation badly), then they should probably get to know Python (depending on their need to customize existing packages), then they should get to know ZClasses and other methods for building Zope Products.

Some of the more nonobvious things about Zope stem from the fact that instances (like DTML methods, which are just really HTML with fairly simple presentation logic, like for loops and if statements) aren't files on the filesystem, and people seem to have problems adapting to that. Jonothan Farr has created a product named LocalFS that allows people to use files on the filesystem as sort of pseudo Zope objects.

On an even higher level, aplplication developers can abstract out the Zope management interface entirely, allowing users to interact with the system in any way that the app developers define while still making use of the security system, DTML, and member-contributed products like "Squishdot" (a forum product not unlike this one but with a lot of extra fluff that I'm not really all that fond of), WorldPilot (an IMAP-to-web email web gateway), Banner Ad Product, The Wampum Generator (a credit card validation system), the Portal Toolkit (a community contribution and content management system), and other stuff. A full list of Zope Products is available at http://www.zope.rg/Products. I just looked and it looks like there are about 150 prebuilt products for Zope, most member-contributed.

BTW, none of this stuff has any lower-level dependencies, it can happen on Windows with FileStorage, on Linux with a RelationalStorage, on Solaris with ZEO, or any combination you choose.

Anyway, I'll shut up now, hopefully you get the idea.

Collapse
Posted by Chris McDonough on
(cont'd)

""For instance, your users are working on Zcommerce tools. The ACS simply provides you a comprehensive e-commerce module off-the-shelf. Not programming at all is easier than programming, even if the programming environment leads to wet dreams... "

This is true, point conceded. This is an area for improvement, and is one of the things we can learn from the ACS.

One thing that seems clear is that the ACS doesn't seem to delineate as much as Zope does between the developer and the end user. We have catered more historically to developers who want to use the platform to develop customized applications. We're starting now to try to cater more towards the end user that just wants to put something up quickly through things like the Portal Toolkit, which is a huge simplification of the Zope management interface and doesn't require you to think in terms of app dev, while providing community features like news, member-contributed content, workflow, email notifications of updated content, self-cataloging objects, and the like.

The Zope community (which consists primarily of developers, not end users) has beaten us up over and over again over documentation of Zope internals and our templating language. This is the number one thing we need to fix to allow people to contribute meaningful products. Once we do this, hopefully folks will be able to create things more efficiently, and will start to feed their creations back into the Zope community, after which we'll begin to see more stuff that "just works out of the box" ala ACS' ecommerce support. We also have contracts with customers that give us some leeway to open-source and generalize creations they've funded out of consulting, which is clearly our bread and butter at the moment.

Collapse
Posted by Albert Langer on
Thanks for the responses to my two questions.

Re Ecommerce support. I'm glad to hear the ACS module is likely to be easy to configure and will try it out. Will also follow up the suggestion to checkout Zcommerce etc and ask any further questions about Zope ecommerce in the Zope community. Question resolved.

Re scalability. Question still open and seems appropriate to explore here. I'm not convinced by Don's view either that ACS owes its scalability to the efficiency of AOLserver and the database drivers, or that while Zope is untested in this area there is no reason to expect that anything intrinsic to its architecture is likely to run into problems at the high end.

It appears to me that a really major part of ACS is a solid underlying data model that reflects a lot of work put into thinking about *how* to integrate web servers with databases. In porting the syntax from Oracle to Postgresql and in adding modules to ACS this data model can to some extent be taken for granted, but I suspect its vital to scalability.

My reason for concern about Zope's scalability (as opposed to any claim that it won't) is strengthened by the very helpful description of Zope's complete storage independence given by Chris. This has many obvious advantages, but the trade off may well be scalability. If data has to be persistently stored on disk and read from disk instead of all being cached in RAM, the effective limits to scalability can often become the physical seek times of disk heads. The very flexibility of objects relating to other objects through links as required, can then translate into multiple disk seeks as you traverse a path from one to another. SQL partially shields the platform developer from DBA/sysadmin work optimizing disks and indexes, and an abstraction layer further shields add on developers. But this is only partial. The flexibility of an RDBMs application platform and its add ons is in fact limited by the data model. If it wasn't it couldn't scale.

So far OODBMs have proved their advantages in maintainability, adaptability, flexibility etc in applications where relatively small numbers of users are simultaneously accessing highly complex structured objects. Yet RDBMs have remained dominant, not just for "historical" reasons, in large scale On Line Transaction Processing. Large "community" web sites may turn out to have characteristics more like OLTP.

I suspect the future belongs to peer to peer distributed networking between clients where the user is participating in many different communities and needs powerful adaptable software on the client (or a local server shared among various clients but not related to any particular "community"). A distributed version of the Zope approach could be very applicable to that future, making it very different from the now "kludgy" current distributed networking of net news and email list communities, and superior to large scale community web sites.

But I don't see how simply declaring complete independence from storage can change the fact that with current hardware, large scale web applications cannot hold their persistent data in battery backed RAM and therefore ARE dependent on storage structures.

Collapse
Posted by Chris McDonough on
Without ZEO, Zope maintains separate connections to a single "DB" object (which is the mediator between the physical storage and the rest of the system).  These connections are available to a pool of threads, each which loads its own copy of the application space.  When objects are read by the DB out of storage, each thread gets its own copy of the object and keeps it in an in-mem cache that is tunable and transparent.  If the object becomes stale, it is invalidated in each thread by the DB instance, and needs to be reread from disk.  An object hangs around in cache and reads need not hit the disk until the object is invalidated (by being written to in another thread) or by the cache being "garbage collected".

ZEO extends this model by reusing it and providing a similar invalidation scheme between distributed processes via IP.

Sorry, I can't spend much more time on this.... For more info, I'd suggest you visit zope.org or any of the links I've given in this thread.

HTH,

Chris

Collapse
Posted by Michael Feldstein on
Whew! I just stepped away from this thread for about half a day...

At any rate, I'd like to come back to the issue of non-programmers. There's a difference, I think, between the users of your web-based application and the developers of it (although the distinction can often be very fuzzy, I admit). If, for example, your application is a relatively powerful content management system, e.g., Vignette StoryServer or Karl Goldstein's new system for ACS, then you can have users who do quite a bit of customization without any programming.

Fine. Those are not the people I'm concerned about, since both systems seem to be able to address their needs relatively well. (I would argue that ACS currently has a fair amount of work to do in this regard, since the admin pages are a mess. I have no idea about Zope on this front, though I've heard good things.) I'm more concerned about the people halfway between those users and serious, heavy-duty programmers. I'm talking about the people inside organizations who usually end up maintaining the systems, which often means extending them as well. They probably won't need to write huge new modules on their own, but they might need to make a few changes to the functionality here and there (or fix something when it breaks).

These people need what Philip calls a "gentle slope". They need to be able to learn more about how the system works as needed. They need to not have to learn a whole new skill set (like OO programming) in order to make the relatively minor changes that need to be made. I don't know if Zope requires this kind of education, but so far it sounds like it does. ACS, on the other hand, does not. Tcl and SQL are both pretty easy, and ADPs are even easier. Even without the problem sets, boot camp, and all the rest, I feel pretty confident that I could learn how to make at least moderate customizations to ACS if I really wanted to without having any particular talent for hacking. ACS is designed to be open in that particular sense; in the sense that it practically invites you to play with it. This will be even more the case once some if the code mess is cleaned up.

I didn't mean to imply that Zope isn't truly Open Source. I just find it to be an odd (although perhaps not so unusual) kind of Open Source. Odd in the sense that the team seems to put its sense of architectural aesthetics ahead of accessibility of the code. It may be open, but it feels a bit elitist (if unintentionally so). Now, sometimes, when a problem is especially hard, that's the right tradeoff to make. I wouldn't expect, for example, to be able to learn to hack my own encryption software as an untalented hobbyist, and I wouldn't trust any encryption software that I *could* hack myself. But this web stuff just doesn't seem to be that sort of a problem.

At any rate, differences aside, I'd like to personally thank the Zope people for spending so much time and effort here trying to communicate their perspective. *That* represents the true spirit of Open Source, in my mind.

Collapse
Posted by Chris McDonough on
"""I'm more concerned about the people halfway between those users and serious, heavy-duty programmers. I'm talking about the people inside organizations who usually end up maintaining the systems, which often means extending them as well. """

I had a discussion today about this, and I think we concluded that we would partition Zope users into five categories.  In this model, we start with a core group.  The second group consumes the work of the core group.  The third consumes the work of the second and the first, etc.  Here's what I think they are:

1. Core hackers - people who actually evolve Zope internals and provide interfaces to low level stuff

2. Product builders - people who produce generalized reusable modular products for Zope but who don't necessarily hack internals

3. Content managers - people who organize and produce presentation logic for site structure, featured content, and syndication.

4. HTML designers - people who give dynamically-generated things a consistent and attractive "look and feel" using routines created by a content manager

5. Content consumers - people who consume the services created by the prior four groups (this may include contributing content, buy not meaningfyl formatting, layout, or logic)

Historically, we've catered to groups 1 (mostly internal employees), 2 (members of the Zope community like Butch Landingin, creator of Squishdot), and 5 (the customers of our consulting customers).  We've also got a good start in the catering to 3.  Our through the web management story appeals to 3, this is a target market for us, and we're hoping to make it more attractive for these folks though the Portal Toolkit effort and other "content-management-esque" features being added to the Zope core (like historying and revision control).

4 doesn't care what hosts the HTML, and 5 doesn't know nor care what lies underneath.  These are folks that (unless we're doing consulting), are *not* our target market.

We often run across peple trying to be in many of these roles simultaneously who resultingly need to absorb quite a bit of information to become proficient at all of it when using Zope.  I think these are the kinds of people you're talking about.  Unfortunately, our docs cater to *none* of these groups, we don't offer an end-to-end documentation story, and that needs to be fixed badly.  Zope is a very attractive product, and its initial simplicity in installation and intro documentation I think sort of lulls people into a feel-good state.  Then they try to do something that's not in the docs.  Look out!  Anger!

How does the OpenACS community see their userbase?  Would you divide it this finely or would you paint broader strokes and consolidate?  Would you include #4 and #5 as members of your userbase?

"""Odd in the sense that the team seems to put its sense of architectural aesthetics ahead of accessibility of the code. It may be open, but it feels a bit elitist (if unintentionally so)."""

The Zope core definitely does not have as relatively broad a contributors list as something like Linux.  I'd compare Zope to a *BSD-type project in this respect (as opposed to a Linux-type project).  Personally I'd like to see a better system set up so that more people were encouraged to submit core enhancements.

Collapse
Posted by Don Baccus on
"How does the OpenACS community see their userbase? Would you divide it this finely or would you paint
    broader strokes and consolidate? Would you include #4 and #5 as members of your userbase? "

You're much further along in this kind of thinking than we are - after all we've not made our first non-beta release, yet!

As an outsider to aD, I'd say they've mostly catered to groups #2 and #5.  In one sense, #1 doesn't really exist because much of what is core  technology in Zope is called "AOLserver" and "the RDBMS" in the ACS world.  We (OpenACS) have talked about the need to identify portions of the ACS which really are more core-technologyish, and to develop an API that will allow modules to cleanly register themselves  with the system, etc.  aD is starting to pay attention to such issues, too, so at the moment we're in a wait-and-see attitude to see what they do for their summer release.

Collapse
Posted by Richard Moon on
I'm a 'level three user' I would guess acording to the description above, though I've a long background in application development and rdbms. I'm unlikely to be developing products from scratch, but I am developing applications of my own. I had to laugh at the community college C++ graduates reference, since they hadn't invented C++ when I graduated.

I'm intrigued that people are talking about the complexity of Zope at the level at which I use it (building apps) I don't find it complex. Stumbling across a reference to it in "Learning Python" I downloaded it in my lunch hour and had it talking to my Informix database and performing useful queries by the time I went home. I'm not a Python hacker and I didn't need to be. I gave it to a colleague who worked through the document management docs and we set up some folders for others to manage. In a few days we had mailback forms, data entry, Squishdot (like Slashdot) running as an IT help desk, then we found the Zwiki and set up some wiki webs, then we put up Worldmail for time management and email.

All this without a line of Python and very little complex dtml. Since then I've got more involved in dtml and Python as I've needed to validate and hack around with user input etc using regular expressions but I was impressed that Zope solves many problems I didn't know existed (like the <dtml-sqlvar> which deals with sql quoting). I've implemented a secure login against user tables on the Informix database, I'm storing session cookie data in there as well. It works, I like it.

Cons ? Documentation (as we all know), some difficult syntax in dtml, lots of products still to be developed (Portal toolkit, e-commerce, education systems etc), lack of native Informix drivers - but there's a lot of potential IMHO.

It may be behind ACS in terms of ready available product (I don't know ACS though I'm going to try it out) but from my perspective I think Zope has a huge amount to offer.

Collapse
Posted by Michael Feldstein on
Interestingly, I think ACS (in general, including both Classic and Open flavors) has historically had a diametrically opposed approach to Zope's in terms of the priority they place on the needs of the different classes of users. I think ACS has mainly catered to 1 (core developers), 2 (power users), and 5 (end users), in that order. You'd have to say that (historically) the core group has been the one most catered to, since aD built ACS for themselves, primarily, and since the whole focus on tcl and sql is to keep development simple. There is also a gentle slope from being a 2 to becoming a 1. In fact, one of the main purposes aD had for GPLing ACS was to find more 2s, convert them to 1s, and convince them to work for aD And, of course,5 is big because ACS was built to be used by a community (and by aD's clients). Next comes the 3 (content managers) group, although I think that 3 is a bit hard to measure against Zope since we don't yet have a good content management system. ACS users can control the "presentation and the logic" of the site somewhat through the admin pages, but that's not the same as being a "content manager". The 4 group is way down at the bottom of the totem poll, although both aD and OpenACS teams are working on solutions that will at least make life at the bottom a bit more tolerable.

The differences between the ACS and Zope camps around user priorities may help explain why the two groups don't always seem to be speaking the same language. Although I think both sides have been implicitly talking about the group that Chris has labeled 2s and that I have been calling "power users," it may turn out that the OpenACS community has been talking about the needs of an entirely different group of people (with different aspirations) than the Zope community has been talking about, even though we think we're talking about the same people.

Most core ACSers are either 1s who used to be 2s or 2s aspiring to become 1s. I think there's a certain fix-your-own-car or assemble-your-own-PC pride among the core ACS community. The ACS power users lke being able to dig into the guts of the system. It gives them a sense of control.

In the Zope community, on the other hand, the 1s are a very different group from the 2s and, in many cases, never the twain shall meet. They're the OSS Web Toolkit for the Rest of Us; their power users don't necessarily want to know what the guts of the beast look like as long as the damned thing works.

If I'm right, then the issue of the learning curve (which we've talked about quite a bit) changes entirely. Whose learning curve are we talking about and what to they think is at the top of the curve? To a certain degree, different kinds of power users (looking for different kinds of power) go to Zope or to ACS for similar reasons that different kinds of power users choose PCs or Macs (or Linux, or whatever) as their primary platform.

It would also mean that, in order to get a meaningful comparison of tradeoffs, we'd have to lay a whole bunch of tasks to be performed by the different user types side-by-side to get a complete comparison. It may turn out that there's an "elegance" factor in Zope--for one or more of the 5 groups--that we're not seeing because the ACS crowd is more focused on simplicity (and therefore elegance) of an entirely different sort.  That doesn't negate the objective issues, but it does mean we have to be sure we're always comparing...er...apples to apples.

Hope I haven't offended anyone here. These are very rough characterizations, and I'm not yet sure that they're true (or interesting to anyone but me). But I do think we're onto something  here. I think the differences in software engineering aesthetics (AKA "religion") may be a bit broader than we've talked about so far.

Collapse
Posted by Albert Langer on
Hi,

I've just spent a few days careful reading in Zopeland after a few days careful reading in OpenACS. As a complete newcomer to both, still not having DONE anything in either, my understanding is very limited. Still, most people who are busy doing things in one or the other don't have time to carefully read what's going on elsewhere, and are also more likely to have preconceptions about it.

For what it's worth here are some observations.

1. Zope has LOTS of documentation. Just very hard to find your way through it. Still immature but likely to be a great platform.

2. For access to RDBMs databases via SQL, Zope provides pretty much exactly the sort of data abstraction facilities that Ben was talking about in:

https://openacs.org/doc/sql-abstraction.html

This keeps getting confused (at least to me) with the fact that Zope can ALSO use an RDBMs as the "storage" for its OODBMS instead of using an append only flat file. The Zope sql-abstraction facilities are EASIER to use than Tcl and DO NOT require an understanding of ZODB and the core internals of Zope.

Note: This does not imply that porting SQL code from Oracle to Postgresql would itself be any easier, just that it would not be entangled with presentation.

3. I still haven't found out how Zope handles connection pooling with external SQL queries.

4. Using an OODBMs for persistence has MAJOR advantages over using an RDBMs for MANY things and DOES include full support for transactions.
ZODB is "aggressively optimized" for many reads, few writes. That is exactly what is required for MANY web applications.

5. I haven't come across anything that indicates an appreciation that RDBMs also have MAJOR advantages for MANY OTHER things. I remain convinced that for OnLine Transaction Processing an RDBMs is inherently more scalable and deals with concurrent writes better (as well as being inherently less flexible, adaptable and harder to develop for and maintain than Zope).

6. As Kevin pointed out in the article that started this thread:

"Ecommerce:

ACS: ArsDigita has developed an e-commerce module that does many of the things that Amazon.com does! Professional and customer reviews, gift certificates, discounts for certain groups of users, discounts for users with a coupon code and more. Plus, they have customer service management (an email handler that makes it easy to track email), and order management. The e-commerce module does not handle payment by check, but this should be relatively straightforward to code.

Zope: There is a product called EMarket, but it is not fully featured. We have our own home grown code. This code does, IMHO, a good job of product display. Zope doesn't (yet) have good, standardized support for members/users. This is coming very soon, though.

Advantage: ACS. There's no comparison in this category. ACS has great commerce features out of the box."

7. Ecommerce happens to be one of the areas where an RDBMs is a REQUIREMENT. I see nothing in in the Zope architecture that would prevent somebody competent in both RDBMs design and Zope from re-implementing the ACS ecommerce module in Zope (the same may also be true of other modules but I haven't looked).

8. Unfortunately there seems to be little overlap and little appreciation of the importance of an RDBMs for ecommerce apps in Zopeland so I am doubtful that this will happen there soon.

9. Since I'm neither interested in nor competent at building such a substantial application, but just need some of those facilities to be there, it looks like I have to use the ACS ecommerce, despite preferring the Zope platform.

10. Since ACS classic apparantly has little interest in an sql abstraction layer and OpenACS is primarily focussed on the porting of SQL code from Oracle to Postgresql, it occurs to me that OpenACS ought to be very interested in carefully studying the data abstraction layer in Zope and considering how to make that SQL code available to both platforms.

As I lack the skills to do that myself, but have an urgent need for a small part of the ACS ecommerce module and a future need for various things that Zope offers, I'm seriously contemplating trying to just run the two together, much like shunting users off to another site to handle the credit card processing.

Is that as brain damaged as it is inelegant?

Anyway, I have an immediate question which people here familiar with AOLserver should be able to answer easily. Will there be any hassle running Zope behind it on the same http port and IP address?

Zopeserver, like AOLserver is a single process. To allow flexibility, it is commonly installed behind Apache with a Persistent CGI or Fast CGI or a proxy pass through, using Apache for SSL support and URL re-write rules as described in the links below.

Is it reasonable to assume one can do pretty much the same with running Zopeserver behind AOLserver instead of Apache? (In particular a pointer to the AOLserver module for rewriting URLs would be helpful, I had a quick look but couldn't find it. Ditto for Persistent or Fast CGI support, presumably less common in AOLserver than in Apache since CGI just isn't the way it would otherwise be normally used).

http://www.zope.org/Members/guy_davis/install_routes

http://www.zope.org/Members/cavnit/apachevirt

http://www.zope.org/Members/anser/apache_zserver

http://www.zope.org/Members/kedai/apache_zope_fcgi

Collapse
Posted by Pedro Lima on
Titus Brown wrote a long time ago...

"I've extended both Python and Tcl; the interpreter for Tcl is leaner, and both are written quite cleanly. Based
solely on this I can't believe that Tcl would be slower than Python. If anything, the abstraction involved in
Python's (well-integrated) object system would probably tend to make Python slower... (I would be happy to
offer people accounts on a vanilla system to run benchmarks.) "

I just saw a report comparing several languages (tcl, python, perl, java, C, C++ and rexx). You can find it
at http://wwwipd.ira.uka.de/%7Eprechelt/documents/jccpp_tr.pdf

It shows that perl is faster than python which is faster that tcl. Although this could hardly be important enough to make it a decision
criterium, IMHO.

Collapse
Posted by Richard Moon on
Very perceptive comments about acs vs zope, (and I quote)
"
7. Ecommerce happens to be one of the areas where an RDBMs is a REQUIREMENT. I see nothing in in the Zope architecture that would prevent somebody competent in both RDBMs design and Zope from re-implementing the ACS ecommerce module in Zope (the same may also be true of other modules but I haven't looked). "

It has struck me many times and I have commented on this in zope mailing lists - Zope's interface to RDBMS is so good, yet so incomplete. You have a wonderful abstraction layer which easily allows you to move a zope application from Oracle to PostgreSQL (or write support for both in the same applciation) - so good, yet support for different RDBMS is so bad, Zope provides 'official' support for Oracle, Sybase and ODBC (Win32 only), relying on third party products for PostgreSQL, MySQL. No Linux ODBC, no Informix, no DB2, no Interbase ... It seems to me that Zope could distinguish itself by supporting 'all' the RDBMS's likely to be used. Then you could write a Zope app which was portable across any environment.

" 8. Unfortunately there seems to be little overlap and little appreciation of the importance of an RDBMs for ecommerce apps in Zopeland so I am doubtful that this will happen there soon."

There are Zopers with RDBMS background (like me) but we are in a minority. It is hard to believe that people would seriously try to build an ecommerce application using the Zope ODBMS but they have. Whether the idea of mapping the Zope ODBMS onto an RDBMS ( someone has just releases this with the ODBMS running in Interbase) is the best or worst of all worlds I'm not sure.

I have been working with a client who has gone for ACS over Zope mainly because of the lack of ecommerce on RDBMS in Zope. Yet it seems to me that Zope has so much potential, and the Zope products which don't need RDBMS are so attractive, that it would be an ideal platform to incorporate both.

I have done some work with Zope and RDBMS and I've found it powerful and effective.

Perhaps after I have worked with the ACS ecommerce product I may be able to comment more about implementing it in Zope.

Collapse
Posted by Albert Langer on
Richard Moon (mailto:richard.moon@nec.ac.uk) responded to a message you
requested notification for in the OpenACS compared to other toolkits and technologies bboard:

Subject:  Response to Article on ACS x Zope

[RM]
Very perceptive comments about acs vs zope, (and I quote)

[AL]
"
7. Ecommerce happens to be one of the areas where an RDBMs is a REQUIREMENT. I
see nothing in in the Zope architecture that would prevent somebody competent
in both RDBMs design and Zope from re-implementing the ACS ecommerce module in
Zope (the same may also be true of other modules but I haven't looked). "

[RM]
It has struck me many times and I have commented on this in zope mailing lists
- Zope's interface to RDBMS is so good, yet so incomplete. You have a wonderful
abstraction layer which easily allows you to move a zope application from
Oracle to PostgreSQL (or write support for both in the same applciation) - so
good, yet support for different RDBMS is so bad, Zope provides 'official'
support for Oracle, Sybase and ODBC (Win32 only), relying on third party
products for PostgreSQL, MySQL. No Linux ODBC, no Informix, no DB2, no
Interbase ... It seems to me that Zope could distinguish itself by supporting
'all' the RDBMS's likely to be used. Then you could write a Zope app which was
portable across any environment.

[AL] I gather OpenACS has concentrated on porting ACS from Oracle to Postgresql and is interested in also supporting other RDBMs but is hindered by the lack of ACS interest in a data abstraction layer. So each port will be a major development and subsequent maintenance undertaking to keep in sync with ongoing ACS development. Its interesting that the ACS interbase port seems to be an entirely separate project. I would have thought that adding the client drivers for each RDBMs to Zope (and likewise to AOLserver) can reasonably be left to those interested in supporting each particular RDBMs and neither will have great difficulty obtaining that when actually needed.

The real issue is the data abstraction layer. There's a LOT more to porting between different RDBMs than just the drivers.

BTW a couple of specific questions.

1. Is there any problem with the third party Zope Postgresql client driver that would make it more difficult to use the OpenACS Postgresql code with Zope? (I just assumed it "works" and will be more suitable for specific Postgresql facilities than going via ODBC, but have no knowledge).

2. Is there some difficulty implementing Linux (and FreeBSD etc) ODBC client drivers for Zope that explains why there is only a Win32 ODBC driver? I just assumed that since Zope itself is mainly python with no OS dependencies, all the Zope work would be on "talking to ODBC" which has already been done for Win32 ODBC and the remaining (much harder) work would be on the RDBM ODBC driver side, not on the Zope (or AOLserver) ODBC client side at all, which has already been done (and equally applicable for ODBC on AOLserver) for all the various OSes. I assumed that if Zope can talk to an ODBC client driver on one OS it could easily talk to an ODBC client driver on another OS and Linux, FreeBSD etc ODBC drivers are readily available. Likewise I assumed that when an ODBC client driver is available for any OS that AOLserver is implemented on, it would not be a major separate job to interface it to AOLserver on that OS. Again, I have no knowledge.

[AL]
" 8. Unfortunately there seems to be little overlap and little appreciation of
the importance of an RDBMs for ecommerce apps in Zopeland so I am doubtful that
this will happen there soon."

[RM]
There are Zopers with RDBMS background (like me) but we are in a minority. It
is hard to believe that people would seriously try to build an ecommerce
application using the Zope ODBMS but they have. Whether the idea of mapping the
Zope ODBMS onto an RDBMS ( someone has just releases this with the ODBMS
running in Interbase) is the best or worst of all worlds I'm not sure.

[AL] Well I can't be sure either as I'm no expert on any of RDBMs, ACS or Zope, but I very strongly suspect that any belief that this would be the best of all possible worlds simply arises from people with RDBMs background being a minority among Zopers. It just doesn't make sense to me because Ecommerce requires:

1) Concurrent shared access with entirely separate internal systems for accounting and fulfilment (and often for the product catalog), which can be be interfaced directly to a web RDBMS via ODBC or native RDBMs drivers if available, but will not benefit from either Zope or ACS web related stuff themselves. I don't see how the ZODB using an RDBMs for its "storage" helps with this at all. How would they interface to what the ZODB assumes is its private storage? If they have to interface via Zope as I assume, then the fact that Zope happens to be using the same RDBMs for storage is simply irrelevant.

[continued in next message due to 4K limit]

Collapse
Posted by Albert Langer on
[continued from previous message due to 4K limit]

2) A significant volume of transactions that have a relatively high ratio of writes to reads. Any OODBMs is better at a high ratio of reads to writes and the ZODB is "aggressively optimized" for that situation. This makes Zope very good for combining "dynamic" (but read only) web pages with decentralized administration and maintenance of the site, including product catalogs unless they are volatile and large. The writes required for such ongoing administration and maintenance are relatively low volume and the transactional support provided by ZODB should be adequate, with much easier and more flexible customization through Zope. However I can't see it working for any significant volume of "shopping cart" and payment transactions, or even a reasonably volatile user base, let alone the individual page tracking and correlations that ACS ecommerce provides. These are fairly static applications so ease of development and customization is less important, but the high volume of actual writes is more important. Given that ACS has already done the main development and OpenACS has ported it to a viable open source RDBMS, there should be no need to duplicate those major efforts. As far as I can see the ZODB would still be navigating via pointers as an OODBMs and the fact that its "storage" happens to be within an RDBMs would not change the fact that each navigation would still require at least one full disk seek for every write. How would any benefit be obtained from RDBM indexing, clustering etc if the ZODB is just appending each object to a single RDBM table?

[RM]
I have been working with a client who has gone for ACS over Zope mainly because
of the lack of ecommerce on RDBMS in Zope. Yet it seems to me that Zope has so
much potential, and the Zope products which don't need RDBMS are so attractive,
that it would be an ideal platform to incorporate both.

[AL]
This sounds similar to my situation. I'm actually recommending use of OpenACS ecommerce module in parallel with separate development of other aspects of a site in Zope. Still not sure whether THAT will be the best or worst of both worlds 😉

Glad to find at least one other person caught between the two worlds who might be interested in figuring out how to minimize the pain in the bum that one gets from trying to ride two horses or sit astride a sharp fence!

Seems to me that OpenACS faces related problems just from needing to keep in sync with ACS while also needing a data abstraction layer, but not being particular interested in Zope as such.

The way I see it:

ACS has a body of expert knowledge of the problems of RDBMs based web sites but is not interested in any RDBMs platform except Oracle. It also adopted a sound architecture getting away from CGI by using the web server as the RDBMS client. It is already running into the limitations of non-OO large scale development but has no real understanding of OO design. I have the impression most RDBM people think the OO stuff is just an overcomplication for web related stuff that can be done more easily without it. (The fact that they actually did build ACS without it and Zope has nothing comparable in important areas such as ecommerce confirms that view, but says nothing about what happens when the much larger and more complex systems made possible by what they have already achieved just keep on growing and becoming less and less maintainable).

OpenACS understands the importance of that body of expert knowledge and needs to stay in sync with ACS to continue benefiting from and contributing to it, but also needs at least a data abstraction layer while not having much interest in anything more OO than that. OpenACS is likely to have the greatest appreciation of the need for a solid data abstraction layer.

Zope has a similar architecture to ACS as regards getting away from CGI and using the web server as the database client. It has a thorough understanding of OO design but only a minority that would appreciate the importance of the body of expertize ACS has made available on RDBMs based web sites. (In particular the very specific issues for ecommerce which ACS achieved an understanding of through hard experience are not yet understood). I have the impression most Zopers think that just because the OO stuff completely abstracts the database from the viewpoint of application developers and content managers, the body of expertize ACS has provided is simply irrelevant. (In particular ecommerce looks like just another app). Their concept of a data abstraction layer seems to be limited to a data abstraction layer for talking to Zope.

I'm hoping that there might be some interest among OpenACS people in working out how to decrease the coupling between the RDBMs based modules of ACS by a separate data abstraction layer that would ALSO make it easier to use those modules with Zope (or any other application platform) AS WELL AS their primary concern of making it easier to use them with Postgresql and other RDBMs as well as Oracle.

If that is possible, it strikes me as a better solution than simply migrating the modules to Zope or any other particular platform. As "stand alone" modules they could be kept in sync with their original development as part of ACS instead of forking off and duplicating effort.

Something like an XML-RPC interface to ecommerce application domain objects stored in the RDBMS?

[RM]
I have done some work with Zope and RDBMS and I've found it powerful and
effective.

Perhaps after I have worked with the ACS ecommerce product I may be able to
comment more about implementing it in Zope.

[AL]
Sounds like you could be ideally placed to take the initiative in getting people interested in making it "stand alone" so that it could keep in sync with ACS as well as being available to Zope. Any interest in that cf "forking"?

I am cross posting this to the Zcommerce list:

http://lists.codeit.com/mailman/listinfo/zcommerce

as well as to the OpenACS bboard below.

Will also be posting correspondence with Paul Everitt, ChrisMcDonough and Kevin Dangoor to Zcommerce for discussion there as promised.

Will be tracking any discussion in either place but without any further cross-posting, so anyone else interested in both would need to be subscribed to the list above and also use the "notify" facility on the bboard below.

[robot]-----------------

To post a response, come back to the bulletin board at

https://openacs.org/bboard/q-and-a.tcl?topic_id=8&topic=OpenACS%20compared%20to%20other%20toolkits%20and%20technologies

Collapse
Posted by Ben Adida on
Everyone keeps talking about DB abstraction like it's not big deal, as if it were something we are simply choosing not to work on.

Let's put it very simply: Oracle has outer joins. Postgres does not. If you have some sort of DB abstraction where you supposedly have nothing to change when you switch from Oracle to Postgres, it means one of three things:

  • your DB abstraction never uses outer joins, so you're less efficient on Oracle, and in fact you're stuck with the lowest common denominator of SQL.
  • your Postgres query breaks because it doesn't know outer joins.
  • you've written a complete SQL replacement that encompasses all the concepts of SQL (tables, joins, references, aggregates) that gets automagically transformed to the correct underlying RDBMS-specific version of SQL.
I can't say for sure that I know how Zope handles it, but my bet is on the first solution: that you're left with common-denominator SQL power. Sure, it's functional, but I'd be interested to see the actual queries performed and the resulting performance.

There is a reason for using an RDBMS, and that is the efficient storage and retrieval of data. Adding abstraction on top of that can only make things less efficient. It's not an easy compromise to make to simply give up efficiency for compatibility. OpenACS is trying to see how best to approach it, but the answer is far from simple.

Collapse
Posted by Jimmie Houchin on
Ben:
I can't speak for anyone else here, but what you write on SQL abstraction here, https://openacs.org/doc/sql-abstraction.html pretty well somes up what I would like to see in ACS/OpenACS.

I wish you had a link to this article on the OpenACS. At least I didn't find one. I had to search all over the aD site to find it. I couldn't remember where I saw it. But here is where I found it on the aD site. http://acs40.arsdigita.com/doc/acs40/

It would be nice for the Tcl to be able to call the query appropriate to the db in use. The SQL could still be optimized for each db, using the features if available and working around where not.

This would help open up the potential db pool and potential user pool. This would allow other issues which determine database choice to permit using ACS/OpenACS. I know aD believes in Oracle but other organizations have made other choices based their criteria. aD opened up the server choice some while keeping the SQL hardcoded to Oracle. I would have preferred the opposite. What was the purpose of opening up the server choice? Was it so that clients could choose the server based on their criteria. I think the db choice can be opened up without major compromise. Hopefully aD will listen to Ben and open up to making a change. If not hopefully it can happen here in OpenACS.

It seems this would be the easiest way to open up to InterBase and other future OpenSource databases. Who knows who will open up what in the future.

I also believe this could help make the code less fragile to changes in the database's version of SQL as they add or change features or syntax.

I am sure if I've made in errors in thought someone will help me out here. :)

Collapse
Posted by Albert Langer on
[BA]
Everyone keeps talking about DB abstraction like it's not big deal, as if it
were something we are simply choosing not to work on.

[AL]
Not everyone. In the first part of the message you are responding to I said:

"The real issue is the data abstraction layer. There's a LOT more to porting
between different RDBMs than just the drivers."

[BA]
Let's put it very simply: Oracle has outer joins. Postgres does not. If you
have some sort of DB abstraction where you supposedly have nothing to change
when you switch from Oracle to Postgres, it means one of three things:

your DB abstraction never uses outer joins, so you're less efficient on
Oracle, and in fact you're stuck with the lowest common denominator of SQL.

your Postgres query breaks because it doesn't know outer joins.

you've written a complete SQL replacement that encompasses all the concepts of
SQL (tables, joins, references, aggregates) that gets automagically transformed
to the correct underlying RDBMS-specific version of SQL.

[AL]
Let's add some further complications:

a) Postgresql is not just an "almost SQL92" RDBM but also an ORDBM which has inheritance, can create some amazing writeable views using "Rules" and can dynamically load data types and functions, aggregates, indexes and operators that can move a lot of stuff normally done in middleware right into the database engine where it can be done more efficiently. Actually using that stuff could eliminate a lot of middleware business rules and result in a far simpler set of queries and updates much closer to the application domain model than is possible using Oracle. Actually using those features would require a significant (although simplifying) restructure of the application itself. This would break easy upgrading to new releases of ACS classic.

b) Anyone optimizing a general web ecommerce package for a particular site will not be able to do much using just DBA techniques with indexing and tablespaces because there may not be many disk spindles to play with available. Small sites that cannot follow Oracle recommendations for more than a dozen disk spindles will still need to do some efficiency optimization based on the actual characteristics of their sites (actual local ratios of browsing, products, categories, subcategories, sessions, users, lookups, discounts, purchases etc). This will require local SQL tuning using "Explain" AFTER the tuning done by the module developers.

Thus when going for optimum efficiency, there is NO WAY to avoid using different SQL code, not just for different RDBMs, but even for different sites using the same RDBM.

On the other hand, many simple queries can even use "lowest common denominator SQL" independently of particular dialects, without much impact on efficiency. (Especially if they are really working on "views" created in DDL and stored procedures or rules optimized for each RDBM).

The issue then is what can a data abstraction layer do to make the maintenance of multiple versions of SQL DDL and DML as painless as possible for SQL developers and make it as invisible as possible to non-SQL developers working on the UI etc. It certainly cannot cannot completely eliminate the need for multiple SQL versions. But perhaps it could dramatically simplify the development and maintenance of those versions and reuse of their (considerable) commonalities?

[BA]
I can't say for sure that I know how Zope handles it, but my bet is on the
first solution: that you're left with common-denominator SQL power. Sure, it's
functional, but I'd be interested to see the actual queries performed and the
resulting performance.

[AL]
I don't fully understand how Zope handles it yet either, but I can confirm that you would lose that bet.

As I mentioned a week ago in this thread:

"2. For access to RDBMs databases via SQL, Zope provides pretty much exactly the
sort of data abstraction facilities that Ben was talking about in:

https://openacs.org/doc/sql-abstraction.html
"

The simple version just provides convenience for plugging parameters into SQL dialect specific templates. See the Z SQL documentation:

http://www.zope.org/Documentation/Guides/ZSQL

This is however a big advance on what ACS will be able to do, even with the proposed sql-abstraction, because people needing to edit SQL scripts and people needing to edit UI interfaces can be easily given fine grained permissions to do so from their web browsers without being able to screw everything else up easily.

The next level currently requires some vOOdOO to understand "ZPatterns". See especially the section on "Dynamic Data Objects" in the contents map at:

http://www.zope.org/Members/pje/Wikis/ZPatterns/HomePage/map

Suffice to say that it is NOT one of the three options that are possible without "very OO database OO" but a fourth option made possible by OO. Yes, you still need RDBM specific (or site specific) SQL for optimization, because there is NO WAY that can be avoided. But it hides this behind umpteen layers of framework code so that it is completely insulated from the application domain queries and updates and UI templates written by non-SQL developers. This makes life much harder for people implementing the framework (ie core Zope developers), just as life is hard for people writing the internals of Postgresql. But it makes life much easier for people customizing UIs, people customizing queries and updates based on application domain models and for people optimizing RDBMs engine access, because they can all keep out of each other's way. (If it's the same bunny doing them all, at least they can think about just one thing at a time).

[BA]
There is a reason for using an RDBMS, and that is the efficient storage and
retrieval of data. Adding abstraction on top of that can only make things less
efficient. It's not an easy compromise to make to simply give up efficiency for
compatibility.

[AL]
If "abstraction" meant least common denominator SQL, that would be true. If it just means that you are passing through multiple layers of OO framework on the way to the RDBM specific SQL, the overhead is insignificant compared with the benefits in portability and ease of development. (Especially when the framework is running in a single process and not being marshalled and unmarshalled continually like COM).

[BA]
OpenACS is trying to see how best to approach it, but the answer
is far from simple.

[AL] The answer is so far from simple that only open source projects could tackle it 😉

Data abstraction isn't an OpenACS problem, OpenACS is just encountering that common problem.

OpenACS (and ACS) people know far more about RDBMs than most Zopers ever will. (The discussion of "Indexing Agents" in the reference above strikes even me with my very limited RDBM understanding as somewhat naive).

Zopers and other OO experts know far more about abstraction than most RDBM experts ever will.

I think you should get together on this one. It's HARD.

Collapse
Posted by Jimmie Houchin on
Albert Langer, wrote:
2. For access to RDBMs databases via SQL, Zope provides pretty much exactly the sort of data abstraction facilities that Ben was talking about in: https://openacs.org/doc/sql-abstraction.html

  This keeps getting confused (at least to me) with the fact that Zope can ALSO use an RDBMs as the "storage" for its OODBMS instead of using an append only flat file. The Zope sql-abstraction facilities are EASIER to use than Tcl and DO NOT require an understanding of ZODB and the core internals of Zope.
[snip]

3. I still haven't found out how Zope handles connection pooling with external SQL queries.

Jimmie writes:

I believe here is where some of the confusion is. In number 2 above it is mentioned that Zope abstracts out SQL like Ben talks in his article. From my understanding this is not quite accurate. Zope has no SQL. Let me repeat, Zope has no SQL. There is no relational data model in Zope. All SQL and RDBMS development is done solely by the app writer or the user of Zope. All SQL used in Zope is custom for the specific app, for the specific db, by the specific user. Zope has what is called ZSQL methods which are method bindings for queries which integrate the Queries into Zopes object structure and ZODB. It provides some syntactic sugar to make incorporation of a RDBMS into Zope's object world easier. It is very nice, however, I think were talking apples and oranges.

Concerning ZODB's capability of using 'RelationalStorage'. ZODB is Zope's object storage model. It can use a variety of containers to store it's object data. Each storage has different characteristics.

Up to very recently the only usable storage available was FileStorage. It is a single monolithic file. Because of Zope's versioning and undo capabilities a ZODB can growing very rapidly. This is due to store multiple versions of a changed object until the ZODB is packed removing extra versions. On certain filesystems this created a problem due to the 2gb file size limitation. Thus your entire dataset could not exceed 2gb unless you went with a RDBMS to store your data.

A RelationalStorage would help with this, as most RDBMSes get around the 2gb problem. The RelationalStorage is easily abstractable do to the fact that it is creating a single table and storing rows of 8 byte binary data. I could be wrong on this, but this is what I remember. So there is no complex SQL to worry about.

Zope's connection pooling isn't probably any different than AOLservers. You just configure Zope to talk to your db and it'll handle the rest. I believe you can configure threads or handles, but there is not a whole lot to it.

Hopefully this can clear this up some. Zope SQL abstraction and ACS SQL abstraction are two different beast. Zope itself has no SQL to abstract.

Collapse
Posted by Chris McDonough on
[BA] I can't say for sure that I know how Zope handles it, but my bet is on the first solution: that you're left with common-denominator SQL power. Sure, it's functional, but I'd be interested to see the actual queries performed and the resulting performance.

[AL] I don't fully understand how Zope handles it yet either, but I can confirm that you would lose that bet.


There is no SQL abstraction layer in Zope like Ben's talking about. You still need to write DB-specific SQL to gain results out of a query.

To my knowledge, the only system that abstracts this layer out well is Apple's WebObjects which I believe can use Oracle, Sybase and Informix as its object persistence layer without requring that you write any SQL. We don't do this. We instead use the ZODB as our object persistence layer, and we let people connect to RDB's and use that RDB's specific set of SQL commands. To get anything useful out of the RDB's, you need to write system-specific SQL.

Zope does abstract out the ability to process results of SQL queries. If you've got a Zope system with connections to different databases (for example if you're grabbing stuff out of an Oracle DB and a Postgres DB within the same Zope instance), you create database connections to the two DB's, you create "SQL methods" for each query for each DB, and then in Python or DTML you pass arguments in to the "SQL method" that houses each query, and you can process its results generically. You needn't worry about where the data came from. Here's an example:

SQL method "getstatus" query:

SELECT status
FROM customer
WHERE id = <dtml-sqlvar custid>

DTML method to process the results from this query:


Customer status:  <dtml-var "getstatus(custid=1)">

The DTML method passes the value of custid in to the SQL method, gets the result back, and displays it.

This is the extent of Zope's SQL abstraction. You can process query results independent of the data source by creating a "database connection" to a specific DB (Oracle, Postgres, MySQL, Solid, ODBC, and Sybase) then creating "SQL methods" against these database connections that accept arguments and return results. The content of the SQL methods is necessarily database specific.

Collapse
Posted by Don Baccus on
Well, this last example is no win over the AOLserver/Tcl/Adp environment.  The  level of abstraction's the same.  As it happens, the ACS doesn't abstract out the SQL, but that's a design/business decision not at all dependent on the environment the toolkit runs in.

So regarding which environment provides a superior database abstraction, the answer is "neither".  But the ACS per se flunks in this regard, something the OpenACS folk bitch and moan about daily.

Ideally, in the Zope world, you deal with Zope objects, in which case you needn't concern yourself with how stuff's stored.  That's good.

The direction the ACS is going is to split presentation and computation issues in such a way that for a large percentage of sites programming won't be needed at all.  At this surface level, I suspect DHTML and Adp pages are going to look a lot more alike, than different.  Most users won't need to poke "under the hood".

It's different for us OpenACS porting folk, of course.  As it would be for a Zope implementor porting Zope for use with a brand-new RDBMS.

Is the OO paradigm and the set of available Zope objects better for the "under the hood" design that should mostly be the realm of the developer, rather than user of the toolkit?

Well ... the RDBMS/thin-layer approach taken by the ACS has certainly led to the development of a large set of modules, many quite comprehensive and all of the useful.  I'm in a results-oriented mood at the moment ...

I'm kinda tired of comments like this:

"OpenACS (and ACS) people know far more about RDBMs than most Zopers ever will...[SNIP]...
    Zopers and other OO experts know far more about abstraction than most RDBM experts ever will. "

There's an implication here that perhaps OpenACS folks don't know much about the OO paradigm because we're RDBMS experts.

For starters, I'm no RDBMS expert.  Picked up "SQL for Smarties"
about 15 months ago, that's pretty much the extent of my expertise though I must say I've gotten rather good at writing queries.  For seconds, I'm hardly unaware of  OO principles and of the value of this paradigm for decomposing a large set of problems.  The fact that I'm working on a toolkit which wasn't designed using OO principles doesn't  mean that I'm ignorant of, or unskilled in, that paradigm.

This broad hinting that those who don't necessarily buy the notion that the OO paradigm is the best paradigm for web site development do so out of ignorance is a cheap substitute for substance.

Collapse
Posted by Albert Langer on
[CM]
There is no SQL abstraction layer in Zope like Ben's talking about.  You still
need to write DB-specific SQL to gain results out of a query.
To my knowledge, the only system that abstracts this layer out well is Apple's
WebObjects which I believe can use Oracle, Sybase and Informix as its object
persistence layer without requring that you  write any SQL.  We don't do this.
We instead use the ZODB as our object persistence layer, and we let people
connect to RDB's and use that RDB's specific set of SQL commands.  To get
anything useful out of the RDB's, you need to write system-specific SQL.

[AL]
My understanding is that Ben is talking about something quite different from WebObjects.

To clarify that, I have posted some notes about Web Objects in a separate topic, "Apple WebObjects", on this bboard:

https://openacs.org/forums/message-view?message_id=15107

Can we all agree that something like WebObjects is beyond the scope of what OpenACS is after for data abstraction and beyond the scope of what Zope intends to provide?

From my perspective the use of ZODB as an object persistent layer is fine for many things but completely unacceptable for Ecommerce OLTP transactions. For those you need not only to get things out of RDBMs, but also put them into RDBMs and the application is really an RDBM application.

I don't buy the idea that Ecommerce based on ZODB alone without an RDBM is viable, for reasons explained in 1) and 2) of:

http://lists.codeit.com/pipermail/zcommerce/2000-June/000257.html

BTW If you are aware of anybody else attempting Ecommerce OLTP with an OODBMs please provide a reference. Otherwise please confirm that Zcommerce efforts to do so using ZODB without an RDBM should be considered "experimental".

Despite that, I still believe Zope could be useful together with an RDBM based Ecommerce module and I am interested in whether it is possible for the existing ACS Ecommerce module to be made more "stand alone" and used with both OpenACS and Zope, or whether development would inevitably need to "fork".

I claimed that:

For access to RDBMs databases via SQL, Zope provides pretty much exactly the
sort of data abstraction facilities that Ben was talking about in:
https://openacs.org/doc/sql-abstraction.html

Based on that view I am proposing that OpenACS people and Zope people should get together in work on data abstraction.

Your comments below, and those from Jimmie Houchin, seem to indicate that I'm wrong about that.

Given that you know far more about Zope than I do, I'll have to accept your view. But before abandoning the idea I'd like to get it clear whether we are really talking about the same thing.

[CM]
Zope does abstract out the ability to process results of SQL queries.  If
you've got a Zope system with connections to different databases (for example
if you're grabbing stuff out of an Oracle DB and a Postgres DB within the same
Zope instance), you create database connections to the two DB's, you create
"SQL methods" for each query for each DB, and then in Python or DTML you pass
arguments in to the "SQL method" that houses each query, and you can process
its results generically.  You needn't worry about where the data came from.
Here's an example:

SQL method "getstatus" query:

SELECT status
FROM customer
WHERE id = <dtml-sqlvar custid>

DTML method to process the results from this query:

Customer status:  <dtml-var "getstatus(custid=1)">

The DTML method passes the value of custid in to the SQL method, gets the
result back, and displays it.

This is the extent of Zope's SQL abstraction.  You can process query results
independent of the data source by creating a "database connection" to a
specific DB (Oracle, Postgres, MySQL, Solid, ODBC, and Sybase) then creating
"SQL methods" against these database connections that accept arguments and
return results.  The content of the SQL methods is necessarily database
specific.

[AL]
Above is my understanding of what "Z SQL" offers.

But I thought the "Dynamic Data Objects" were intended to provide rather more.

Could you please comment more specifically on the following excerpt from my previous posting and explain what I have misunderstood:

**************
The next level currently requires some vOOdOO to understand "ZPatterns". See
especially the section on "Dynamic Data Objects" in the contents map at:

http://www.zope.org/Members/pje/Wikis/ZPatterns/HomePage/map

Suffice to say that it is NOT one of the three options that are possible
without "very OO database OO" but a fourth option made possible by OO. Yes, you
still need RDBM specific (or site specific) SQL for optimization, because there
is NO WAY that can be avoided. But it hides this behind umpteen layers of
framework code so that it is completely insulated from the application domain
queries and updates and UI templates written by non-SQL developers. This makes
life much harder for people implementing the framework (ie core Zope
developers), just as life is hard for people writing the internals of
Postgresql. But it makes life much easier for people customizing UIs, people
customizing queries and updates based on application domain models and for
people optimizing RDBMs engine access, because they can all keep out of each
other's way. (If it's the same bunny doing them all, at least they can think
about just one thing at a time).

***************

What I had in mind is that Ben's document refers to a simple template for inserting parameters, into SQL much like what "Z SQL" provides, but with tags like {Oracle, Postgresql} which indicate whether a particular SQL statement is suitable for use with a particular RDBM. This would simplify maintenance of multiple ports by reusing whatever can be reused and providing sufficient isolation from the rest of the application for SQL development and Tcl development to be less tightly coupled.

It seemed to me that although the "ZPatterns" framework is currently used only for multiple login authentication (eg from LDAP directory or RDBM instead of Zope folders in the ZODB), it is intended to be far more widely applicable and could be used to provide data abstraction of the sort Ben is after, by people who understand the problems of porting between different RDBMs as well as for providing application Domain Models, Roles and facades for people using the ZODB.

Collapse
Posted by Albert Langer on
[JH ...comments re data abstraction and SQL omitted as covered by response to CM]

Concerning ZODB's capability of using 'RelationalStorage'. ZODB is Zope's
object storage model. It can use a variety of containers to store it's object
data. Each storage has different characteristics.

Up to very recently the only usable storage available was FileStorage. It is a
single monolithic file. Because of Zope's versioning and undo capabilities a
ZODB can growing very rapidly. This is due to store multiple versions of a
changed object until the ZODB is packed removing extra versions. On certain
filesystems this created a problem due to the 2gb file size limitation. Thus
your entire dataset could not exceed 2gb unless you went with a RDBMS to store
your data.

A RelationalStorage would help with this, as most RDBMSes get around the 2gb
problem. The RelationalStorage is easily abstractable do to the fact that it is
creating a single table and storing rows of 8 byte binary data. I could be
wrong on this, but this is what I remember. So there is no complex SQL to worry
about.

[AL]
Using an RDBM is a rather odd way to get around a 2GB file system file system limit.
I suspect there is more to it than that. But anyway, your account confirms my impression that the use of an RDBM for ZODB storage has no relevance whatever to the question of integrating Zope with RDBM SQL based OLTP applications like Ecommerce.

[JH]
Zope's connection pooling isn't probably any different than AOLservers. You
just configure Zope to talk to your db and it'll handle the rest. I believe you
can configure threads or handles, but there is not a whole lot to it.

[...]

[AL] I understand that Zope application developers can just use db handles in much the way that ACS application developers can just use those provided by AOLserver. My concern is whether Zope  really is pooling them properly like AOLserver or whether there is some interaction between the ZODB handles used by Zope and the RDBM handles used by Zope that could result in some limitation in the performance of an RDBM application with Zope cf a similar application with AOLserver.
I guess this is a matter of "Zope internals" that I'll have to pursue in Zopeland.

Collapse
Posted by Albert Langer on
[DB]

[...]So regarding which environment provides a superior database abstraction, the
answer is "neither".  But the ACS per se flunks in this regard, something the
OpenACS folk bitch and moan about daily.[...]

I'm kinda tired of comments like this:

"OpenACS (and ACS) people know far more about RDBMs than most Zopers ever
will...[SNIP]...
    Zopers and other OO experts know far more about abstraction than most RDBM
experts ever will. "

There's an implication here that perhaps OpenACS folks don't know much about
the OO paradigm because we're RDBMS experts.

[AL]
There was no "implication". There was a straight forward third sentence:

"I think you should get together on this one. It's HARD."

If I'm wrong about there being complementary skillsets that could benefit from working together on RDBM SQL data abstraction, then I'm wrong.

Taking offence at the suggestion is being "tiresome", not being "tired".

[DB]
For starters, I'm no RDBMS expert.  Picked up "SQL for Smarties"
about 15 months ago, that's pretty much the extent of my expertise though I
must say I've gotten rather good at writing queries.  For seconds, I'm hardly
unaware of  OO principles and of the value of this paradigm for decomposing a
large set of problems.  The fact that I'm working on a toolkit which wasn't
designed using OO principles doesn't  mean that I'm ignorant of, or unskilled
in, that paradigm.

[AL]
I also picked up Celko's book about that long ago, but haven't got good at writing queries.
Likewise I'm aware of the value of OO principles, but not a competent OO developer.
If I had both skill sets instead of neither, I would put forward a concrete proposal for a data abstraction layer. Perhaps you could do so?

Instead, all I have is sufficient appreciation of both to believe that there could be some value in projects whose primary orientation is towards one or the other, but which are both concerned with database driven web applications, working together on the specific problem of RDBMS SQL data abstraction, which seems to me to involve both, (plus sufficient arrogance to say so and not be worried about being proved wrong).

[DB]
This broad hinting that those who don't necessarily buy the notion that the OO
paradigm is the best paradigm for web site development do so out of ignorance
is a cheap substitute for substance.

[AL] Bitching and moaning "daily" is not even a cheap substitute for improved database abstraction. Taking offence at other proposals for getting there has no substance.

Ben's document on data abstraction did not strike me as "bitching and moaning". But he indicated that actually implementing it is "far from simple". If you have a substantive proposal for implementing Ben's document, or some other alternative to bitching and moaning, please point me to it.

Collapse
Posted by Chris McDonough on
To answer some of Albert's q's...

The ZPatterns stuff is the brainchild of Phillip Eby, a developer at Verio (the hosting provider).  It (among other things) is a mechanism to provide medium-independent "propertysheet" support.  "Propertysheets" in Zope are rectangular sets of typed data that can be attached to Zope objects arbitrarily, and as such lend themselves very well to storage in an RDB table.  And I think this was probably what Philip had in mind in providing Dynamic Data Objects.  Although to be honest, I might be totally wrong... I can't really tell what the intent is now.  I've heard it's a good set of ideas, but personally I'm stumped most of the time when I read his stuff because he names things very opaquely and seems to explain things in such a way that you *do* need to community college C++ degree to begin to grasp it.  I think it's one of those projects that needs to gain some traction and obtain a critical mass of real-world examples in order to make its ultimate intent clear to mortals like me.  Until then, I can't speak much for it, although I do recommend you continue looking at it.  (Maybe you can write some code using it that would explain it to me!)

ZPatterns is not part of the Zope distribution.

I think Ben had in mind at least in his most recent posting an abstraction layer that allowed RDB-savvy people to write SQL-like queries that would automagically be translated into their equivalent optimized Oracle- or Postgres-specific SQL dialect at runtime.  I don't think anybody in Zopeland has the intent of writing such a beast.

You're probably right in assuming that the ZPatterns stuff is about as close as it gets to wanting to do something like that.  But Zope propertysheets are generally by nature pretty small fixed sets of data (maybe 5 - 12 properties per object).  You would need to declare some sort of table structure for a particular object type, create the tables to store the propertysheet data for those kinds of objects, then do the right voodoo to tell the objects to obtain their properties out of SQL instead of out of the ZODB.  This could probably be abstracted in such a way that people don't need to write any SQL, but it would never approach the sort of complexity that I think Ben is talking about because it would probably practically necessitate an approach something along the lines of defining a table to hold propertysheet data for each different object type.  If you wanted to do ad hoc queries against an existing set of data in multiple tables (ones that might require unions, outer joins, or insert selects) would need to be (just as painfully) manually constructed in the SQL dialect of your choice.  Personally, I'd probably try to keep it simple and create views on existing tables that would let me operate as if I were doing so on a single table.

Somehow I get the feeling that in this conversation we're all talking an entirely different language than one another.  I assume that Ben's concerned about complicated SQL abstraction between Oracle and Postgres because he wants to get rid of Oracle dependencies in ACS.  I think Albert's talking about being able to abstract the SQL out enough to not have to think about it at all when constructing an application.  I'm probably just plain not thinking about it at all.

I looked at Ben's sql-abstraction page... I don't know any tcl, but I think I get the gist.

It would seem to me to make sense to abstract out both 1) the access mechanism for querying and the 2) resultset that comes back from the query.  In Zope we do the first part by creating a "method" that contains the actual query (which can be as DB-specific or generic as you like), connecting it to a database connection (a pool of database handles for a specific DB type).  You can then pass in arguments to the method that have counterpart template definitions within the query itself (ala CUSTID = <dtml-sqlvar abc>), and the method passes the query in to the DB and obtains its results.

The second part (returning the results of the query to the caller) is handled in Zope by wrapping the resultset in a class instance that lets us process those results generically.  It would probably be good to do something like this in ACS by defining a datatype that could hold arbitrary common elements that could be processed in tcl code generically.

Though it's not a trivial engineering task, it seems like the biggest problem isn't implementing it, but encouraging people to use it.  From what I understand there's a large body of existing code that does not use any sort of abstraction like this.  Convincing folks of the usefulness of writing against your interface instead of yanking the stuff right out of the tables in their code is probably the hardest task.

Collapse
Posted by Albert Langer on
[CM]
Somehow I get the feeling that in this conversation we're all talking an
entirely different language than one another.  I assume that Ben's concerned
about complicated SQL abstraction between Oracle and Postgres because he wants
to get rid of Oracle dependencies in ACS.  I think Albert's talking about being
able to abstract the SQL out enough to not have to think about it at all when
constructing an application.  I'm probably just plain not thinking about it at
all.

[AL]
Thanks for the detailed exposition in the remainder of your posting. I'll study it carefully and check with Phillip Eby. Meanwhile, I've got the same feeling about talking different languages and will refrain from continuing to cause confusion.

Quickly though, if by "application" you mean something like an Ecommerce module, I certainly don't believe anyone writing one should attempt to not think about the SQL at all while writing it. On the contrary I think the most important work is the SQL. On the other hand somebody customizing a web site and using the ecommerce module in conjunction with other applications should not need to think about the SQL. (That has more to do with abstraction between the domain model and the UI than with database abstraction and I gather ACS classic is addressing this for version 4).

I also have a different interpretation of Ben's position. My understanding is that he and other OpenACS people have ALREADY ported from Oracle to Postgresql, thus removing all Oracle dependencies. The problem as I see it is how to remain in sync with future upgrades to ACS classic (Oracle) version and extend to supporting other RDBMs with less pain. My impression was that he does not want "complicated SQL abstraction between Oracle and Postgres" but believes (as I do) that without RDBM specific SQL the result would be unacceptably inefficient.

The question then is how one can handle regular updates from ACS classic (Oracle) and ports to multiple other RDBMs (or even just Postgresql) with minimum pain while using RDBM specific SQL wherever necessary for efficiency.

I believe "ZPatterns" may be very relevant to that, but will obviously have to study it further before asking anyone at OpenACS to take that idea seriously, in view of your comments.

Collapse
Posted by Richard Moon on
Albert Langer asked

1. Is there any problem with the third party Zope Postgresql client driver that would make it more difficult to use the OpenACS Postgresql code with Zope? (I just assumed it "works" and will be more suitable for specific Postgresql facilities than going via ODBC, but have no knowledge).

I've used it on a small scale project and it worked fine. It survived the upgrade to PostgreSQL 7.02 as well. I don't know of any specific problems with it - I'd just like it to be part of the standard Zope distribution and treated as a core part of Zope.

2. Is there some difficulty implementing Linux (and FreeBSD etc) ODBC client drivers for Zope that explains why there is only a Win32 ODBC driver?

I really don't know. I asked on zope lists about why the lack of database drivers and was told that DC were supporting Oracle, Sybase and ODBC only. I asked why ODBC was only WIN32 but I didn't get any reply. This was before Chris McDonough joined them 😊 Chris - any news ?

Albert also said -
[AL] Sounds like you could be ideally placed to take the initiative in getting people interested in making it "stand alone" so that it could keep in sync with ACS as well as being available to Zope. Any interest in that cf "forking"?

Like others on this list I'm proceeding slowly down the maze of web development, hoping to avoid dead ends and sometimes walking on top of the hedges to have a better view. I will have hands on ACS (not Open ACS) soon and will be able to see what the capabilities of the system are (in the areas of ecommerce and education mainly). I also intend to keep on with Zope if I can because I think its going to be really, really good ...

Chris McDonough has outlined the constructs in Zope which let you treat the data source in abstraction from the presentation layer. It actually goes further than he mentions - the data returned by a <dtml> tag could be coming from anywhere. The fact that it is coming from a PostgreSQL database (or a Python list) has no impact at all on the presentation of that data.

The clever thing about Zope is that using this approach is really easy and doing it any other way would be more difficult so any Zope application has this abstraction built in.

For example the General User Folder product which is a simple login manager is used to validate user logins and passwords. As supplied these are contained in lists - so the software calls a dtml method called "UserAuthenticate" which just contains the list of valid users
Fred
Joe
Sarah

To change this so that it validate this against an SQL database all you do is to replace the physical list of users with a dtml tag which refers to an SQL method which returns a list of users.

The software which calls "UserAuthenticate" neither knows nor cares whence the data cometh.

So as far as abstraction is concerned as you know I wasn't suggesting that Zope can automagcally optimise SQL requests for various databases. However the application itself needs no knowledge of how to get the data, that can be contained in the method which does the retrieving.

Moving an app from Oracle to PostgreSQL would involve, for example, spoofing OUTER joins wherever they occur, but it would then be easy to have an attribute which you set to say which database was in use, the SQL method would generate the correct code based on the value of the attribute.

Thinking about it you could have a set of attributes for example "outer join supported", "scroll cursors supported","row level locking supported". The correct code could then be generated depending on how those attributes were set. Perhaps 😊

There may be more you can do to make the abstraction more complete, but that would still have no impact on the presentation layer (which only ever needs to call the same method to get, insert, update or delete the data it needs). Therefore you could add better abstraction "underneath" at any time in the future, once the technology is available. In the meantime, hand-optimised SQL code would work well enough.

As to whether I can be of any use, well once I get hands on ACS I'll let you know 😊

Collapse
Posted by Don Baccus on
"I also have a different interpretation of Ben's position. My understanding is that he and other OpenACS people
    have ALREADY ported from Oracle to Postgresql, thus removing all Oracle dependencies. The problem as I see
    it is how to remain in sync with future upgrades to ACS classic (Oracle) version and extend to supporting other
    RDBMs with less pain. My impression was that he does not want "complicated SQL abstraction between Oracle
    and Postgres" but believes (as I do) that without RDBM specific SQL the result would be unacceptably inefficient."

OK, you seem to understand our situation fairly well.

There's no need for an overly complex SQL abstraction mechanism, but we certainly would like to see some level of abstraction implemented.

There seems to be a perception that this is a technical problem.  It's  not, really.  There's a bit of a paradox:

If we fork from aD's sources, we could implement a reasonable approach  to abstraction ourselves.  At worst, we'd just pull the SQL out rather than leave it inline in the scripts.  Doing this would lead to the application layer being database agnostic in that they'd call module-specific utility routines to deal with the database.  Conceptually equivalent to calling a method in Zope which hides the SQL from the application.  This really isn't a big deal (other than there being a lot of application code to visit).

BUT - if we fork from aD's sources, we lose all the major advantage of  abstracting out the SQL from the application scripts.

In other words, decoupling from aD's sources doesn't really help us keep in synch with aD's sources ... SQL abstraction only helps us if aD agrees to adopt it.

You see, it's really not a technical problem, at least not in the way some of the newcomers posting here think ...

Collapse
Posted by Chris McDonough on
Re:  ODBC.  I don't know much about the low-level database adapter stuff in Zope, but AFAIK the problem with ODBC on Linux is one of reliable middleware... I think there is mxODBC or something like that that can talk via... err... CLI?.. to an ODBC datasource on UNIX.  But I think it's not really ready for primetime.  I might be wrong, maybe it has matured since I last heard of it.

There are reliable solutions to this problem.  We had floated the idea of using middleware from Openlink Software (http://www.openlinksw.com) for all our relational database adapters, writing to an interface they provide that abstracts out the connection-layer differences.  But it's not open source, and using more than five connections against any particular database requires a license from Openlink.  I don't exactly know what happened to this plan.  It just sort of fizzled.  We don't want to be in the business of maintaining our existing database adapters every time Oracle or Sybase spurts out a new release.  It's just not a good use of time here.  It would be much better for us to throw this job over the wall to Openlink and write to their interface.  But for now, we're stuck maintaining the ones we've written.

In the meantime (assuming that plan still exists), writing an ODBC DA for Zope on Linux is not high on DC's to-do list, and nobody from the community has stepped up to the plate to make it available.  This is an excellent opportunity for somebody from the community to contribute a piece of software that people ask for from time to time.  It's probably not going to come from DC if somebody else doesn't provide it.

Collapse
Posted by Dan Wickstrom on
"There are reliable solutions to this problem. We had floated the idea of using middleware from Openlink Software (http://www.openlinksw.com) for all our relational
      database adapters, writing to an interface they provide that abstracts out the connection-layer differences. But it's not open source, and using more than five
      connections against any particular database requires a license from Openlink. I don't exactly know what happened to this plan. It just sort of fizzled. We don't want
      to be in the business of maintaining our existing database adapters every time Oracle or Sybase spurts out a new release. It's just not a good use of time here. It
      would be much better for us to throw this job over the wall to Openlink and write to their interface. But for now, we're stuck maintaining the ones we've written. "

Do the drivers really need to change that much every time oracle or sybase releases a new version?  I wouldn't think so.  Besides, if you do have prolems with the driver, do you really want to be dealing with a third-party closed source solution?

Collapse
Posted by Chris McDonough on
"""
Do the drivers really need to change that much every time oracle or sybase releases a new version? I wouldn't think so.
      Besides, if you do have prolems with the driver, do you really want to be dealing with a third-party closed source solution?
"""

When they don't work with the new version, yes (eg. Oracle 8 --> Oracle 8i).

As far as dealing with a third-party closed-source solution, it's either that or continue to write and maintain lots of them ourselves.  As these sorts of things are driven by consulting engagements, it's only once in a while that we write a new one.  Maintaining existing ones takes time that we don't make any money from.  And as much as we champion open source, we are in business to make money.

As far as depending on open source goes, people from the community don't seem to be flocking to Zope to write quality third-party database adapters (although many folks have already ala MySQL and Postgres, there are some serious holes like DB/2 and Informix).  And when folks like Richard Moon ask me when an ODBC adapter for Linux would be available, if we were using Openlink, I could say "right now" and point him to the Openlink adapter instead of saying "as soon as you write one".

Collapse
Posted by Albert Langer on
Thanks to Richard Moon, Chris McDonough and Dan Wickstrom for info and comments re ODBC drivers.

Following from the Postgresql docs it appears that there are available open source client ODBC drivers for FreeBSD and Linux as well as for Windows. See:

ftp://ftp.postgresql.org/pub/odbc/psqlodbc_faq.html

http://users.ids.net/~bjepson/freeODBC/

http://www.iodbc.org/index.htm

The main open source one appears to be maintained by the same openlinks people that distribute the commercial ODBC drivers Chris mentioned, so perhaps same code base?

There was also a mention of ApplixWare, which I don't know anything about.

Based on the other comments, I gather that Zope just needs to be able to use an OS specific ODBC client driver and not a Zope specific one, so it seems there is no actual barrier to using Zope on any platform with any RDBM on any platform, apart from the usual non-commercial support hassles (and corresponding benefits).

If that's not a good bet, please correct me.

PS This of course has nothing to do with data abstraction etc. Will try to follow up on that and other issues after done some more reading. Just wanted to provide those ODBC links immediately.

Collapse
Posted by Chris McDonough on
Based on the other comments, I gather that Zope just needs to be able to use an OS specific ODBC client driver and not a Zope specific one, so it seems there is no actual barrier to using Zope on any platform with any RDBM on any platform, apart from the usual non-commercial support hassles (and corresponding benefits).

This isn't quite true. Zope-specific database adapters (DAs) still need to be written for each DB you want to use. For example, you're not going to be able to use SQL methods to grab stuff out of Interbase any time soon unless you write the Zope database adapter yourself. There is an article on how to write a Zope DA at http://www.zope.org/Members/petrilli/WritingADA. This might give you an idea at what's involved in interfacing Zope to a new relational database.

In the case of ODBC (which for all intents and purposes is one type of database, although writing to it allows you to communicate with many) if we were able to use the open source ODBC stuff, we would probably need to do some rigging of the existing Zope ODBC DA to allow it to be used on UNIX. If this happened, it would probably be part of a larger effort to use Openlink adapters for many databases. I'd be really glad to see someone outside of DC pick up the task of interfacing ODBC to Zope on UNIX, because it seems like the DC Openlink initiative has died on the vine.
Collapse
Posted by Don Baccus on
This isn't quite true. Zope-specific database adapters (DAs) still need to be written for each DB you want to use. For example, you're not going to be able to use SQL methods to grab stuff out of Interbase any time soon unless you write the Zope database adapter yourself. There is an article on how to write a Zope DA at http://www.zope.org/Members/petrilli/WritingADA. This might give you an idea at what's involved in interfacing Zope to a new relational database.
No magic bullet. Ok, I'm shocked, astonished, and otherwise horrified to learn this. After all, I've got so much to learn from Zope's OO approach!

Look, folks, the nitty-gritty hiding of RDBMS differences is a major pain in the ass, due largely to vendor (and open source provider, including PostgreSQL) slackness in regard to implementing SQL92.

It's adherence to the existing SQL standard that's the problem, not the paradigm used to build various applications that sit upon them.

Porting, as a problem, shouldn't even exist. The intersection of various RDBMS' compliance with the SQL92 standard should be sufficient that we shouldn't even be worrying about this. That's the real problem. Postgres doesn't have outer joins. Oracle thinks the empty string ('') is equivalent to NULL (boo, hiss, total rape) and has a very non-standard (albeit useful) way of dealing with dates.

On and on.

The problem does exist, and does because the intersection of features with common syntax isn't up to snuff. That's not aD's fault, nor the Zope folks fault. It's just the way of the world.

Collapse
Posted by Chris McDonough on
It's adherence to the existing SQL standard that's the problem, not the paradigm used to build various applications that sit upon them.

Yes, I think you've expounded on this theme before and you've made it clear that folks considering an OO approach to web development irritate you. While I don't think I want to address the invectives you've heaped on this particular marriage of technologies, I don't disagree with you when you say that SQL standards compliance is lacking across the board. It will likely never reach a stage total cross-platform uniformity, and folks within the procedural and object worlds both need to deal with it as best possible.

Zope provides a weak level of abstraction by strongly encouraging the developer to consistently compartmentalize SQL queries into methods that take arguments and return generic values. The benefit attained by refraining from hard-coding needle-like SQL statements into your haystack of app scripts is, as we all know, not completely ideal, because you *still* have to change your SQL logic when you change databases. This sucks, yes. Writing database adapters sucks, we know this. There is no magic bullet, we know this too..

But always semi-centrally compartmentalizing and defining your SQL logic as functions apart from the presentation imposed by templates is a step in the right direction in light of the fact that it *can't be any other way* due to the market realities you spoke of. Waiting for all the RDBMS vendors to "come to Jesus" on SQL92 is a waste of time. This will not happen, we all know this.

With this in mind, practically, I'd hope you guys can work out with aD some sort of abstraction convention to which you can both write that lets you maintain parity with their code without needing to do a hairy seek-and-destroy on all the places within the app scripts that the standard differs between Oracle and Postgres every time they make a release. My impression based on Phillip Greenspun's disdain in The Guide for consistently imposing this sort of abstraction is that it seems a tough political task.
Collapse
Posted by Don Baccus on
Yes, I think you've expounded on this theme before and you've made it clear that folks considering an OO approach to web development irritate you.
I have nothing against the OO paradigm, I'm simply not one of the Faithful who believe it is the One True Paradigm. It's a great approach for decomposing certain types of problems. If the ACS had been designed in an OO manner, I'd certainly still be porting it over to Postgres.

My irritation has been with those who somewhat naively suggest that OO is the answer to the problems we face in trying to establish and more importantly maintain a true Open Source solution for building websites based on aD's ACS.

I'd hope you guys can work out with aD some sort of abstraction convention to which you can both write that lets you maintain parity with their code without needing to do a hairy seek-and-destroy on all the places within the app scripts that the standard differs between Oracle and Postgres every time they make a release. My impression based on Phillip Greenspun's disdain in The Guide for consistently imposing this sort of abstraction is that it seems a tough political task.
This, not the fact that the ACS wasn't developed using OO methodology, is the problem. Right. You've got it.

On a practical note, aD's moving to a centralized database API, which will pass all SQL through Tcl procs. They're doing it in order to rationalize the handling of database handles and to make it more difficult for their SQL weenies to break transaction semantics by calling nested procs that turn off transactions in the midst of a bunch of DML statements executed by the caller.

They're not doing it to help us with our porting problem. However, we'll be able to massage most of the more common Oracle-isms that have required manual porting in the past, so it helps us nonetheless.

Collapse
Posted by Richard Moon on
Chris Withers said "As far as depending on open source goes, people from the community don't seem to be flocking to Zope to write quality third-party database adapters (although many folks have already ala MySQL and Postgres, there are some serious holes like DB/2 and Informix). And when folks like Richard Moon ask me when an ODBC adapter for Linux would be available, if we were using Openlink, I could say "right now" and point him to the Openlink adapter instead of saying "as soon as you write one".

I can see the problem you have but please don't give up on it.

Openlink works well and those of us who need to interface to a range of commerical databases can hardly quibble about using commerical ODBC adaptors as well. In fact there's an argument that says if our clients think its worth having a commercial database then we should have a commercial database adaptor.

I'm using Openlink now to connect Zope to Informix and it works but restricts me to running Zope on NT which would not be my first choice of platform.

An ODBC DA for Linux/Unix which worked with Openlink would at least get people like me out of your hair Chris 😊

We do seem to be getting off-topic a bit though - but its nice of those OpenACS guys to host this Zope discussion for us 😊
Collapse
Posted by Chris McDonough on
Well, I can't promise anything in the way of ODBC connectivity for Linux anytime soon... sorry! It's just not on the roadmap right now. You mail want to mail petrilli@digicool.com (Chris Petrilli, RDBMS adapter product mgr.) and ask him about the future of Openlink support on UNIX.

On a totally unrelated note, ZEO was released to open source today. If anybody is interested in a transparent distributed transactional object database, you may want to check it out... http://www.zope.org/Products/ZEO
Collapse
Posted by Albert Langer on
[AL]
Just thought I'd post item below with detailed technical explanation of ZODB write performance issues from someone who, unlike me, actually KNOWS. I believe it totally confirms my previously unsubstantiated strong belief that an RDBM approach (whether ACS or Z SQL) is absolutely essential for certain write intensive applications (including Ecommerce where it is also essential for other reasons relating to audit trails, interface with internal systems via ODBC etc etc).

The article also supports my belief that "Z Patterns" is highly relevant both in the Zope ZODB/Z SQL context and in the context of data abstraction for ports of an RDBM based application platform such as ACS between different SQL dialects. I can't imagine how Z Patterns could possibly provide a useful isolation of application issues from fundamentally different persistent storage mechanisms such as ZODB, and a *specific* SQL RDBM (and also LDAP etc) without *also* being useful for managing multiple SQL dialects. (Note the reference to easy porting from ZODB to "Oracle or Sybase").

This of course does not resolve issues about whether "Z patterns" is ready for prime time yet, but whether it is or not, a project such as OpenACS, where multiple SQL dialects is a *fundamental* issue, could only benefit from whatever effort it takes to *very* closely study Z Patterns - if only in the expectation of learning from their mistakes.

BTW the references to "relational storage" below are about use of ZODB stored in an RDBM instead of a flat file or simple dbm - often confused with the entirely different approach of using SQL to a separate RDBM for persistence without ZODB (the article is advocating the latter and explaining how Z Patterns allows easy common application development for either or both).

Seeya, Albert

[reply context snipped]

[Ty Sarna mailto:tsarna@endicor.com ]

Unfortunately, this doesn't deal with cases where the conflicting state
is contained in many objects (see note by PJE in the ZODB Wiki).

Also, there is a whole other area of difficulty for high-write-volume
ZODBs, which is the ammount of IO that needs to be done.  First, by
nature ZODB can't rewrite a single attribute of an object, it has to
rewrite the entire thing.

Indexing is also a bear from an IO perspective.  First, BTrees currently
keep a count at each level, so every change to a btree changes a node at
each level of the BTree.  For a ZCatalog, there are a lot of btrees
(something like 2n+4 for n indexes, I think -- don't quote me on that,
it's been a while), and each one changes (last I looked, every index was
updated even if the value indexed in a particular one hadn't changed.
This may have been improved since).  Not only is this bad from a hotspot
point of view (always a conflict on the root node of the tree), but you
end up doing a *lot* of IO.  During my experiments that led to
BerkeleyStorage, I was watching the Data.fs grow by 47K per transaction
for adding indexed objects of ~1K in size.  Watching this with
tranalyzer, this turns out to be 1K of object, and 46K of updated btree
pages :).  Note that BerkeleyStorage only prevents the file from growing
that much -- it still has to do all that IO (in fact, it has to do ~2-3
times that much IO, due to the nature of BerkeleyDB.  A relational
storage would have similar issues.  For ammount of IO done, FileStorage
is about as efficient as you can possibly be -- it's just that it trades
that off against space reclamation).

Also, with any kind of Berkeley or Relational storage, there is a second
hidden IO and storage penalty: you're storing a btree inside a btree. In
other words, the lower-level DB uses btrees to store your objects,
including interior nodes of the higher-level ZODB btree. Every interior
node of the ZODB Btree needs a leaf node (and supporting interior nodes)
in the DB's btrees. so you get taxed twice, on both I/O and storage
space used.

Not to discourage anyone from using ZODB, necessarily.  There are a lot
of things it's fantastic for, and without a doubt ZODB is getting better
at handling higher write ratios. Over time there will be more and
more applications that previously would have required an external SQL or
other kind of database that can be done in ZODB instead.  However, there
will also IMHO always be applications that ZODB just isn't as suitable
for. You have to thing long and hard before committing to one or ther
other. And then there's the worry of what happens if you chose wrong.

We were faced with exactly these issues, and the extremes of them, to
boot.  We have a *large*, *very* high write ratio, lots of indexes type
of application based on ZPublisher/DTML that we'd like to port
to/replace with something Zope based.  Yet we might need to make another
instance of this same type of application used by only a few people with
a small ammount of data -- it would really suck to have to have to have
another instance of the same expensive database system to support a
miniscule ammount of data, because everything was coded only with SQL in
mind).

This is what led ultimately to ZPatterns -- you can write applications
and not have to decide up front on ZODB or SQL.  And you can change your
mind later (Seen that TV commercial? suddenly your online store is
selling a zillion items per month instead of the 1000 you planned for.
oops!).  You can even decide on an instance by instance basis.  You
configure with ZODB for a small department or client, and Oracle or
Sybase for a huge one -- and the small guy doesn't have to pay for the
DB license and DBA!). Since then, we've discovered a number of other
benefits to the model.

Hmmm... I didn't intend to write a ZPatterns advertisement when I
started, honest! But this seems to have turned into one nonetheless :^)

_______________________________________________
Zope maillist  -  mailto:Zope@zope.org
http://lists.zope.org/mailman/listinfo/zope
**  No cross posts or HTML encoding!  **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope-dev )

Collapse
Posted by Don Baccus on
Sorry, I don't see the relevance.

The ACS lacks SQL abstraction due to a business decision made by aD.  They're only interested in Oracle, and don't want to put in the effort  (spend the money) to support an RDBMS-agnostic toolkit.

That's all there is to it.  If they'd made a different business decision, the technical task wouldn't overwhelmingly difficult.  In a very real sense, the porting problem we face is an artificial one, not  in any way intrinsic to the design and implementation of a toolkit like the ACS.

This whole discussion to some degree is based on a strawman argument, i.e. that SQL abstraction is lacking in the ACS for technical reasons.

Now, how exactly will ZPatterns help me to convince aD to abstract out  SQL?  Or to convince the Postgres development group to up the priority of the implementation of outer joins?  Or support me while I do it myself?

We've got some tricks up our sleeves for ACS 4.0 that will ease the porting problem greatly even if aD doesn't take a single step to intentionally help us (they've taken a step in their design that unintentionally helps us).  Once outer joins exist in Postgres, porting won't be much of a hassle at all even if aD continues to lace their toolkit with Oracleisms.  We'll be translating a bunch of them automatically by hooking into the new database abstraction level they're implementing to solve an entirely different problem.

Collapse
Posted by Paul Gresham on

Wow ... an interesting thread ... We are building a collaborative website for an engineering firm, and have completed many features already in Zope. However we are now considering trashing everything we've done, for "Other Options" and this week we are reviewing them in depth. I think Zope is an amazing piece of work, but here at least are the reasons so far why I like AOLServer and aD.

  • A framework with a security model we can understand without drinking diesel oil and nitroglycerine cocktails.
  • A security model that doesn't break so easily (even if we do sacrifice the depth of functionality provided by Zope)
  • My files are (and can be without effort!) in folders on a disk
  • Everything else is in a ORDBMS which is a.Documented well and b.Already pretty well understood within our team.
  • I can still use Python (My favourite language)
  • Many ready built modules already in use (i.e. higher version numbers that 0.0.0.0.1 beta alpha zeta z.z release)
  • I can use java servlets with Tomcat

Again I think Zope is fantastic but we have just spent so much time trying to figure it all out. Believe me we have have our own custom file system, an online file manager and are also playing with distributing stock prices through it ... Its a big decision coming up.

Our other options under review are Enhydra, Locomotive, hand coded in python or java. I can post the results if anyone is interested. Regards, Paul Gresham

Collapse
Posted by Albert Langer on
[PG]
Wow ... an interesting thread ... We are building a collaborative website for
an engineering firm, and have completed many features already in Zope. However
we are now considering trashing everything we've done, for "Other Options" and
this week we are reviewing them in depth. I think Zope is an amazing piece of
work, but here at least are the reasons so far why I like AOLServer and aD.

A framework with a security model we can understand without drinking diesel oil
and nitroglycerine cocktails.

[AL] Could you elaborate? My understanding was that Zope had a relatively easy security model for end users to understand but required some "learning curve" for developers to get to grips with how to use it. Since you have already completed many features in Zope you must have passed any reasonable learning curve. Was it just frustration at having had to do so, or still a major problem despite having already learned?

[PG]
A security model that doesn't break so easily (even if we do sacrifice the
depth of functionality provided by Zope)

[AL]
What exactly breaks? Is it problems getting the desired permissions correct (ie same as above), or that the mechanisms don't always work as they are supposed to?

[PG]
My files are (and can be without effort!) in folders on a disk

[AL] Why is that so important? If your data is in ORDBMS, python software in folders on disk, static pages in folders on disk (with some effort), semi-static pages in folders on disk (see ZMirror) and dynamic DTML pages in ZODB but accessible via ftp and webdav, what is the problem?

[PG]
Everything else is in a ORDBMS which is a.Documented well and b.Already pretty
well understood within our team.

[AL]
Sounds like the way to go for many apps instead of using ZODB to hold actual data (whether or not ZODB uses a DBMS for its "storage"). But why is that easier to do with ACS than with Zope SQL methods?

[PG]
I can still use Python (My favourite language)

[AL]
I need to use ACS too (for reasons below) and want to use Python with it. How exactly do you plan to do that? (Not a rhetorical question - I'm a newbie to both Python and Tcl and to AOLserver).

[PG]
Many ready built modules already in use (i.e. higher version numbers that
0.0.0.0.1 beta alpha zeta z.z release)

[AL]
That's why I need to use ACS too (mainly for DBMS based Ecommerce and CRM modules). But am thinking of trying to do anything not already pre-built using Zope and integrate the two via the DBMS accessed by both. Since you have already completed stuff in Zope the prospect of working with 2 different platforms might be less daunting for you than it is for me. What made you dismiss that possibility?

[PG]
I can use java servlets with Tomcat

[AL]
I don't understand. Isn't that really something outside ACS that you could just as well do as something outside Zope?

[PG]
Again I think Zope is fantastic but we have just spent so much time trying to
figure it all out. Believe me we have have our own custom file system, an
online file manager and are also playing with distributing stock prices through
it ... Its a big decision coming up.
Our other options under review are Enhydra, Locomotive, hand coded in python or
java. I can post the results if anyone is interested.

Regards, Paul Gresham

-----------------

[AL]
Thanks for your comments above and any further explanations.
Please do post the results of your other reviews - I'd be very interested.
(Perhaps should go into separate subject headings in this bboard, with a reference to them here so that people only being notified about this one thread can look for them).

Collapse
Posted by Michael Feldstein on
Now that we're nearing a first release for 4.x, I thought it might be
interesting to revive this thread. While some of the conversation
here is specific to Zope's architecture and philosophy, it seems
to me that some of these posts read equally well when you
substitute "OpenACS 4.2" for "Zope." I suppose one of the core
issues is the inevitable trade-offs that happen when you move to
a more robust, heavily *engineered* architecture, which is what's
happening in the move from 3.x to 4.x. I thought it might be
interesting to bring up this thread again as a sort of gut check.
Collapse
Posted by Fazal Majid on
This discussion is really interesting.

1. Abstraction vs. Performance

I have written for my company a transactional message queue implementation dumping data directly into Oracle via OCI rather than using the AOLserver driver, and I have also written Python extensions in C and built my own ZODB-style (but with much more limited scope) Python object to Oracle interface, so I have some experience with the internals in both systems.

I tend to agree with Ben's comments about how SQL abstration leads to an unacceptable loss in performance. I am not sure outer joins have their place in an OLTP application, but here is the example I would use: if I want to insert rows in a table (for instance an order) and use the primary key for that row as a foreign key in another table (for instance a line item in an order).

In Oracle, I can use the non-standard INSERT INTO ... RETURNING syntax and make a single round trip. In PostgreSQL, I would first need to select from a sequence, then make my insert, hence two network round trips. This makes for a big impact in concurrency and performance. In very scalable Internet-facing applications, it is very important to be able to squeeze every last drop of performance you can from the RDBMS, which SQL abstraction layers get in the way of.

In my example, the best of both worlds is to abstract an order business object and then fine-tune the corresponding SQL queries for each RDBMS, that way you can still have a common body of presentation code.

The fact ArsDigita has chosen as an implementation and business decision not to make this business object layer sufficiently modular does not mean the ACS architecture itself is flawed, as the product can be refactored. Of course, this poses challenges to the OpenACS team since they want to leverage as much as possible the aD work.

With aD's move towards Java buzzword-compliance, I'm not sure this is going to be sustainable in the long term anyway, so the whole point might become moot at some point where only the module specs and design will be common to ACS and OpenACS, with the actual Java vs. Tcl implementations radically different.

Conversely, there is no reason Zope applications cannot be fine-tuned to have hot spots written directly to a RDBMS using hand-tuned queries. It will still be difficult to have good decisional support from an object schema shoehorned into a relational database, but it is certainly possible using triggers on the ZODB storage tables.

2. Comparing the frameworks

I installed ACS 4.2 Beta (Tcl) last March to play around with the workflow module, and I am now tinkering with Zope. The way I see it, ACS is more scalable than Zope simply because of the quality of the underlying web server, whereas Zope would be more scalable in terms of building large complex applications (where the customized part is significantly larger than the standard module part) on top of the framework. This is an oversimplification, of course, but Zope might be a better candidate for Intranets while ACS would be better suited for consumer or large community web-sites.

Zope certainly has no monopoly on limited documentation - I tried long and hard to build my own modules on top of the ACS 3.4 workflow facility in spite of the limited documentation, and finally gave up and rolled my own in Python.

Collapse
Posted by Michael Feldstein on
I don't pretend to be able to follow Fazal's comments here, but I
am a little concerned in general about not being able to see the
forest through the trees. Since this thread was started, the
OpenACS project has clearly been evolved from an effort to take
an existing system built by ArsDigita, keep up-to-date with it, and
fix what's broken in it  to an effort that's about envisioning,
designing, and building the system that the community wants.
Now that the community is truly charting it's own destiny, I think
it's a good time to think back to earlier conversations (like this
thread) that forced us to articulate what we liked and what we
wanted at the time and see if we're still on-target, if we're
off-target, or if the target has moved.

One of the themes of this thread, as I understand it, is that some
of the ACS partisans felt (at the time) that it was better to have a
relatively simple core architecture that is easily understood by
newbies than a complex core architecture that may accomplish
more but is much harder to learn. At least on this specific
philosophical issue, OpenACS 4.x seems to take one or several
steps away from this philosophy and closer to the Zope
philosophy. So the question I'm raising now is not so much
about specific architectural differences between Zope and
OpenACS as it is about general trade-offs between learnability
and the benefits of a relatively complex core.

I am certainly the last person to effectively evaluate this question.
However it is an important one that is closely related to the
branding and marketing issues the community is currently
discussing. Who is this toolkit for? What would be the most
useful to them? Which parts of the architecture need to be easily
learnable, and which parts are OK to make more complex in
exchange for more power or flexibility? Is the best trade-off to
make the core architecture complex but keep the APIs necessary
to create modules simple? Are there parts of the core
architecture that need to be easier to learn than others? Are any
parts of the core architecture in danger of becoming too hard to
learn? These questions are critical to answer if we want to
increase the adoption rate by developers and convince clients
that OpenACS will make it cost-effective for them to train their
staff to maintain and improve a web app built with it.

Collapse
Posted by Don Baccus on
There are some complexities in the core architecture and I sympathize with Fazil's wrestling with the workflow package.

Part of the problem is the lack of good examples of code effectively using the various core tools.  This is largely because the existing packages were ported by aD into the 4.x framework while the framework was being constructed.  They faced cart-before-the-horse and learning curve issues.

I'm currently working on part of a new dotLRN package that makes use of workflow in what I think is an interesting way.  Workflow drives all event and registration creation and administration actions and once you understand how all the pieces work, it's very simple to use.

But it took me a while to figure out just how workflow could simplify, rather than complexify, my code.

As examples like this, more examples making effective use of the service contract model (such as the dotLRN portals system), workflow, the content repository, and the like are published  and made available I think they'll make excellent learning tools for newcomers.

Even better would be a simple example package tying together the concepts in a more trivial way, but that requires we find someone with time on their hands to do this.

The current exercises with workflow and service-contract based developement show us a few things.  Neophytos needs to write a service contract editor much like the workflow editor (but simpler because contracts are simpler).  It's on his list for "round two".  Workflow needs some enhancements, including the central UI (one advantage of a "workflow-centric" package is that the user can visit a central "task todo" page as well as the package-specific pages to do work.  Tasks appear in both places as though by magic!  Really!)

And the new portals package should make writing portlets very easy.  In the dotLRN package I'm working on we're experimenting with writing a single pagelet to serve as portlet and package index page, rather than follow the typical "write it twice" paradigm we've seen in the past.

Good examples and good documentation will make the toolkit more approachable.  The learning curve will certainly be higher than with OpenACS 3, there's no doubt of this.

But when you learn the tools, you'll be able to cobble stuff together *much* faster than in OpenACS 3.  That's the trade-off.  For a one-off written by someone who never expects to write another website and wants to keep life as simple as possible, OpenACS 3 may always be more appropriate.  But for those who expect to be implementing a lot of complex websites the tools we've inherited, along with our enhancements and the addition of new tools like service contract are a real win.

Collapse
97: take a look at webware (response to 1)
Posted by good bye on
For those interested in investigating a Python-based "web application framework" that isn't Zope, I suggest taking a look at WebWare.

http:// webware.sourceforge.net

Webware has some nice features, particularly:

  • java servlet-esque appserver
  • python server pages (copy of .jsp, much like .adp)
  • Task Scheduler ( much like ns_schedule_daily)
It comes with a handy utility class for creating a threaded database pool. I've combined this with the postgres driver pyPgSQL (http:// pypgsql.souceforge.net), and a simple user table, and have found it to be a great system for quickly building DB based web apps.

In the true spirit of open source software, the documentation is not very helpful. If anyone runs into any problems setting this up, feel free to email me about it.

Collapse
Posted by Michael Feldstein on
Don, I'm getting a couple of things from your post. First,
OpenACS 4.x is probably not suited for a hobbyist who wants to
create one database-backed web site for personal use. Nor is it
necessarily the best choice for commercial users who want to
create and maintain one simple to medium-complexity site
themselves. For both of these goals, 3.x may be as good as or
better than 4.x. In contrast, 4.x is intended to be an improvement
over 3.x for people who expect build either a bunch of moderately
complex web apps over time or one fairly large and complex app
that could be better accomplished through pre-built and
pre-integrated building blocks. Does that sound right?

I'm also hearing that, for this intended user base, it will be crucial
to make service contracts, the content repository, and workflow
relatively easy to learn how to use. (Do they also have to be
relatively easy to learn how to extend or modify?) I would imagine
that we would want to add permissions and templating to this
list. It's possible that portals will eventually make this list as well,
particularly if the module merges with the templating system.

From the perspective of the user described above, are there any
other features that need to be easy to learn how to use? Are
there any others that need to be easy to learn how to extend and
modify?

Interestingly, the list we've just outlined here looks an awful lot
like the crucial pieces of a real content management system.

Collapse
99: ACS vs Zope--Books (response to 1)
Posted by David Cohen on
I recently saw this and I've been looking for an opportune time to mention it. At this writing, the ACS--in all its incarnations--has a total of one, high-level (and, at this point, outdated book), namely "Philip and Alex's Guide to Web Publishing".

Meanwhile, according to this page at Amazon, Zope has 4 books in print and another 2 expected in 2002 (one of which in Feb).
In a related vein, "Slash", the code behind Slashdot, has a book coming out from O'Reilly--read about it here

Collapse
Posted by Don Baccus on
Slash?  Really?  Have you ever looked at the code?  It more or less fits on the back of an envelope ... it doesn't need a book.  Maybe a pamphlet.

But we should investigate getting a book out on OpenACS 4 ... once we have a better idea as to what it is ...

Michael - part of my observation is based on the fact that many folks are eager to just start hacking without investing much time in acquiring  in-depth knowledge of the tools at hand.  Sadly many professionals are guilty of this, too.  OpenACS 4 will not reward the casual hacker nearly as much as it will the person willing to make that investment.

For a *large* one-off project I think OpenACS 4 would save time over a development OpenACS 3 project.  This assumes, again, appropriate examples, documentation, and more package pieces.

But for simple projects, OpenACS 3 is certainly more approachable for the newcomer.

Of course, for *really* simple projects, where the pre-supplied packages are sufficient, OpenACS 4 is really easy to use.  The ability to build subsites, mount those packages you want while leaving out those you don't want, "skin" the site with a master template - you can do a lot more without doing any programming whatsover in the OpenACS 4 world than OpenACS 3 world.  *if* you can live within the pre-defined package space.

Which is one reason we need more packages ...

Collapse
Posted by Todd Gillespie on
Don & David: another way to consider the "Slash" book is that the /. guys will now be bringing their creative interpretations of spelling and grammar to the dead-tree print world.
Collapse
Posted by mark dalrymple on
of course, that should read "from the speling and grammer dpartment"
Collapse
Posted by Senthil Kumar on
Zope is more open and can quickly integrate new technology to it. This is a product that has everything for web development that is proven and production ready. How many products in the market has  database, search engine, webserver, ftp, Perl/php/webdev/xml/xslt/html support for free.

I agree learning zope is tough. Not for weak heart, if you are a good python programmer you will master it quickly and appreciate the product.

Zope is envy to other scripting language products. Like any good products zope too is getting lot of criticism from people who don't use it.

-- Yoma

Collapse
Posted by Jon Griffin on
Ummm, let me see.

I can dump AOLServer and use the Zope internal web server, then I can dump Oracle/Postgres and use thier internal DB, all of course much more scalable in the process. And, still not have the features I can get with OpenACS, FOR FREE.

Sounds like a great deal, I think I will convert all my clients starting now. Since this is a complete flame bait I won't even bother with more of a reply.