Forum OpenACS Development: FYI: My Vision and priorities for OpenACS

Request notifications

We've had this item on our OCT meeting agenda a couple of times, but nothing's really happened so far. At the same time, I have given this a great deal of thought over the past few months.

So I think it's a good time for me to give my perspective on where I think should go, how what we've been doing over the past year or so fits into this picture, and what we're willing to commit to move this even further along in the future.

The Goal

My overall goal with what I do with OpenACS, apart from the everyday need to stay in business, is to expand the use of OpenACS.

The more people use OpenACS, the more resources we'll have to continue to develop OpenACS, and the more fun we'll have. So growth of the OpenACS user base i job #1.

The Analysis

OpenACS has features in quite a broad range of areas, including:
  • Groupware, including course management (.LRN), and project-management.
  • Traditioanl CMS
  • Ecommerce
  • Toolkit for building web apps
  • Portal/systems integration: Using OpenACS to sync data with other systems
I think the strengths of OpenACS are in the groupware space. I really see .LRN as groupware with specific naming and applications, such as portfolio and homework. .WRK I'd see as groupware with project-management tools (thanks, Jade). When I say groupware, I mean this in a broad sense that could very well include CRM features, membership/constituent relationship management, knowledge management.

I do no think that traditional CMS and ecommerce are core strengths of OpenACS, and I think there's too long to go to make OpenACS a compelling player in this market. Besides, I don't think the pure CMS market is going to be around for all that long - organizations will want highly interactive web sites, applications, rather than brochureware static pages. Which will bring the CMS market over in our arms, if only we stay put and stick it out.

On the other hand, I do see content management in a broader sense as core to OpenACS. All community applications are about inputting, filtering, distributing, and managing content, and our features in this area need to be sharp as ever. So for OpenACS, CMS is a basic horizontal feature, not a vertical app.

The toolkit features of OpenACS are very important, because it lets us efficiently build applications and features that attract users, and it makes OpenACS perfect for slapping a good web UI on top of older back-end systems (systems integration above), I do not see toolkit as a goal in itself. In the web toolkit space, we're competing agains Java, .NET, and LAMP, and I just don't think we're going to make OpenACS a strong player in that market.

A core strength of OpenACS over most of the competition is that everything is integrated. This is also a weakness, because it does lead to not-invented-here syndrome and reinventing things that other open source solutions already exist for. But it's so integral to OpenACS, that I think we are best off sticking with this approach and play it to our strength. Integration can be very powerful. If the parts that are integrated are themselves powerful, and the integration is done well, you effectively get the cross-product of all features.

Our technology platform -- AOLserver, Tcl, PostgreSQL -- is never going to be particularly sexy. Neither for developers (toolkit market), nor for organizations. So our option is to provide great value to people who care more about what the software does, what it costs, and the services that are offered for extending and customizing it, than which particular brand of technology sits behind it.

The bottom line is that we should make OpenACS as valueable and attractive out of the box, as a very well-integrated and fully-featured groupware platform, with specializations in education and project-management, which also integrates well with third-party software, targeted at people who aren't religius about technology.

What That Takes

To do this requires user interfaces with navigation and usability optimized for gruopware applications. It requires more features. It requires better integration between the applications. And it requires that we make systems integration easy.

Improving the user interfaces with an eye towards groupware use is what we've been pursuing over the last year or more. And it has gotten a good chunk of the way already, to the extent that OpenACS 5.0.0 does let you do pretty much everything you need to do for a basic gropuware site without requiring a Ph.D. from MIT.

Concretely, we've revamed the acs-subsite UI so you can actually drop in some applications, invite some people, and off you go. We've improved installation, so once you have OpenACS core running, you can easily install new packages from the repository, and start using it in your subsite within minutes.

For integration within OpenACS, if we want to play the integrated nature of OpenACS to our advantage, the key is integrating the site-wide services, such as categories, notifications, RSS feeds, workflow, and WebDAV, to cover all applications.

For example, this would mean that the project-manager is integrated with logger which is integrated with your calendar, or that notifications are integrated with webmail. Or that you can tell the system your interests once, and choose to get notified or subscribe to an RSS feed of all objects that match your interests across the system. That you can access all objects over WebDAV.

Initially we had just permissions. We've added notifications in most places. We're currently adding categories and RSS feeds. So we're making steady progres, but as long as we need to go application by application it will remain a tedious task. More about this later.

In terms of systems integration, we need to do import, export, and two-way syncing. We've done this one-off for message keys to do internationalization, we've done it for users to do external authentication. I think we're ready to get a coherent scheme going for this, before we add one-off solutions for more IMS, SCORM, etc.

Finally, we're hurting from the duplication between OpenACS and .LRN. It was necessary initially, in order for MIT and OpenForce to get a system ready for shipping early, but it's costing us extra time in development, testing, and bug fixing. It's causing the user experience in .LRN to be clumsier than desired. And it's limiting OpenACS' and .LRN's ability to benefit from development in the other pieces. So we need to get .LRN and OpenACS merged. The approach should be to move things from .LRN over to OpenACS, making it more OpenACS-compatible in the process, and then replacing the .LRN code with the corresponding single code base, which will be in OpenACS.

Current Priorities (The Roadmap)

Looking ahead a few releases, which I'll just label 5.1, 5.2 and 5.3 for the sake of this document, let me lay out my priorities.

For OpenACS 5.1 (code freeze in a few weeks):

  1. Improve user interfaces, usability, navigation.
  2. Improve services integration: Categories, RSS feeds, etc.
I think navigation is the key sticking point right now. For example, I think notifications and "Change locale" belong under "Your Account", not just in terms of links, but also in the context bar and other cues to the logical structure of a site. Similarly, we need to find out where to place categories, general-comments, etc.

Fixing these things will allow .LRN to come one step closer to OpenACS.

For OpenACS 5.2, on the top of my list are things like:

  1. Merge .LRN and OpenACS - will require portals and other things to get integrated. We won't get everything, but we'll get as close as we can with the resources we have.
  2. Graphic design improvements, including a skinning feature, so local installations can easily customize their design, and a respository of skins, so people can share designs. Looks are very important, but navigation must be fixed first.
  3. Getting started on making OpenACS data-centric instead of application-centric, also denoted "Data Soup" (see various blog posts at http://collaboraid.biz/developer/blog). This will enable us to even more rapidly build applications, and, even better, whenever there's a new service like categories, you just have to integrate it into the data soup once, and all data soup-based applications will immediately have UI for it.
The data soup thing is major. A realistic plan is probably to get a prototype version into 5.2, and then have the polished version ready for 5.3.

We will probably want to put out a .LRN v3 release after OpenACS 5.2, and a .LRN v4 release after 5.3, depending on calendar time and demand for new features in .LRN.

These are my thoughts about OpenACS. I value your comments, and I'm looking forward to taking this journey with you all.

Collapse
Posted by Ola Hansson on
Lars,

This seems like a highly worthwhile journey, indeed. (When does it begin?)

I'm not going to comment on your plan/vision more than saying I agree a hundred percent but I'd like to add one thing that I think you'll agree is rather important as well ... Namely, "personalization". It is important IMO to have parameters that can be set per package instance (like we have today) but also per package key and, not least, individual user settings.

I'm sure there may be more paths we'll want to persue, as well - this is just one example that came to mind.

/Ola

Collapse
Posted by Lars Pind on
Actually, one important piece I forgot to mention is "social software" features, like network of people, searching through your network's objects and friends, etc. All part of the groupware umbrella, though.

/Lars

Hello Lars,
the plan sounds good ... and here are my 2 cents...

The integration stuff is pretty important, and of course is a major thing to do. Also, about notifications, we have the plan to add notifications to all the possible objects that still need on .LRN in the short term, since we should provide a lightweight communication with the entire system through simple emails / SMS. So basically, the 5.1 vision for me is the correct one. UI improvements are a must.

IMHO, .LRN v3.0 should come with SCORM / IMS compliant, plus new packages like evaluation / grade book (Galileo is doing), assessment (IMS QTI compliant in a first phase, e-lane will also work on this + Heilderberg), Enterprise Integration which is something achieved with IMS Enterprise integration, but on top of that should go webservices or the like (something Galileo will work on as we need it soon) ...

User-tracking-behavior, small object rating, and an improved searching tool is part of the road-map of e-lane as well. Also, is needed an easy way, more less automated to applying patches / upgrades, and I'm pretty sure that it'll come soon.

I don't intend to give the impression that we'll do everything, rather than that, we want to coordinate our efforts and limited human resources with all the community in order to enhance openacs / .LRN.

About the paradigm of being a non sexy technology, personally got rid of it and try do as most evangelism as I can, might be a good place to start with the technology communities, probably a good way will be presenting papers and keep all the community posted about that, there was one time where the community of ACS was growing by the enthusiasm... But in the end user / buyer side is about marketing, and that's another thing. The very first place to do marketing is in openacs, our toolkit (easy installation and you have covered the most of the things) and of course, the .LRN site (which BTW, has the potential to make the needs for openacs / .LRN consultants to grow exponentially), I'll stop here and keep the eye on the MK.

Collapse
Posted by Jade Rubick on
A few things:

> this would mean that the project-manager is integrated
> with logger which is integrated with your calendar

This is a very high priority for me. Lars, let's talk about how to best do this.

I think one of the biggest barriers to getting OpenACS into more people's hands in the installation. It has improved A LOT in the last year or so, but it would be really great if someone could step forward and commit to doing Debian packages, or an install script, or something. Or a series of scripts.

And then we should advertise our existence on places like FreshMeat as well.

Anyway, this is just something that I'd like to see happen. I don't really have the resources to do it myself, and unfortunately, I think a lot of us are in that boat. But if we did, it would make a big difference for the community. It might be more important than anything else we're doing, actually.

Collapse
Posted by Lars Pind on
Jade, I know this is as important to you as it is to me. I haven't contacted you yet about this, because I aren't sure myself. I'll dig into this after the 5.1 release, however.

On installation, I'm personally of the opinion that everything we can do to make OpenACS install a simple matter of .deb or .rpm packages would be a huge step, but given that I don't have the resources to make it happen, offering hosted solutions would be a viable solution for now.

Freshmeat and other forms of advertising are indeed important. We've also talked about getting on sourceforge in the past. We have a pretty active community, and making it to top 5/10/20 on sourceforge could very well drive some new interest in the platform. Heck, it might even motivate some of us to put in some extra effort to move one more nudge up the activity ranking.

/Lars

Collapse
Posted by Malte Sussdorff on
First of all: Excellent points Lars.

What does it take to get us to sourceforge and up their ranking? How could we transfer our activity into their ranking?

If we want to adopt more developers, we'll most likely need the .deb or .rpm packages even if the installation got considerably easier.

Hosting solutions will mainly be attractive to new users unless you talk about virtual servers with preinstalled OpenACS. But will a *new* developer pay for this kind of hosting when only wanting to try and play with OpenACS in his/her sparetime ?

Collapse
Posted by Malte Sussdorff on
Regarding hosting solutions and ASP services. I know of a couple of people considering to start their own ASP services. And most likely there are even more out there. I wondered if there would be an incentive to build at least one ASP solution (done by one or more people building a partnership) up on behalf of OpenACS, which would allow to pay for core enhancements.
Collapse
Posted by Benjamin Bytheway on
Lars Pind on Jan 30 2004 11:33:35--
> The toolkit features of OpenACS are very important, because
> it lets us efficiently build applications and features that
> attract users, and it makes OpenACS perfect for slapping a
> good web UI on top of older back-end systems (systems
>integration above), I do not see toolkit as a goal in itself.

In principle, I agree with this, but I do want to emphasise the importance of OpenACS as a toolkit. As you said, we're not sexy enough to attract developers to the toolkit itself, but I think that once those developers come and begin to discover the inner workings of openacs, they must find that as a toolkit we are as good as anybody out there. I think that in many ways, we are already better than most.

I think that along with the roadmap outlined in this thread, we need to embark on an "API cleanup" of sorts. Much of this cleanup work has already been done, but it's time to take it one step further. There are a lot of deprecated procs around that have been carried through multiple versions now. There are also some procs that have been replaced with new procs, but little indication is made which of the procs the developer is supposed to use. The API should be modified, where needed, to provide a clear and consistent interface to core functionality. API's should also make liberal use of namespaces, and tcl interfaces should be provided wherever possible.

There is also much that was written to work around the limitations of tcl 7.x, where now we have the ability to write much better code with 8.x constructs. Code that is obviously guilty of this should be rewritten, preserving backward compatibility whenever possible (multirows are a good example of something that could be reworked). Use of upvar and uplevel should also be examined to ensure we aren't making things hard for the developer when they have to read our source.

Documentation is probably better now than it has ever been in any *ACS project, but again I think we can do better. Much of the design docs are original aD, and not necessarily relevant anymore. A "Developers Guide" has been somewhat elusive because the recommended best practices and much of the API has been in a state of flux. I think the API cleanup would go a long way toward making such a document possible.

I don't pretend to think that what I've said is anything new, but I do think we've reached a point where OpenACS needs to have more community support holding it up, or it's liable to collapse under it's own weight. I think improving the toolkit portion of openACS is critical to building the needed developer community. Not only does the barrier for users need to be lower, but the learning curve for developers needs to be eased as well.

OpenACS is a project with rough edges, but it still holds a lot of potential, which is why this "silent complainer" wants to help make it the best.

Collapse
Posted by Barry Books on
I think OpenACS is generally going in the right direction. The integration and installation is getting better. As part of 2 I'd like to see OpenACS move to xhtml and css stylesheets. WebDav could be huge combined with import/export.

As far as getting more people to adopt OpenACS I think the big sticking point is AOLServer. While it works great it's not Apache. I think many more people could run OpenACS if they could say we're going to run Apache, Postgres and some open source toolkit. TCL also seems to start religious wars, I'm not sure why I guess it's because it's better than Perl or PHP.

So without trying to start a fight would porting to Apache/mod_tcl help to increase awareness or be a great amount of work with no benefit?

Collapse
Posted by Andrew Piskorski on
Barry, the perceived shortcomings of AOLserver vs. Apache have been discussed for years here in the Forums, and AFAIK the informed consensus has always been that AOLserver is here to stay - and for good reason.

Most importantly, while Lars didn't get into explicity detail about this in his vision and priority outline above, it should clearly follow from what he did say about what he thinks is important - which I agree with - that the whole AOLserver vs. Apache thing should not be a priority.

You really think AOLserver is "the" sticking point? You already touched on the Tcl vs. PHP issue... Let me tell you what I think: Replace AOLserver with Apache (lots of work for no technical gain at all), and will OpenACS get many more users and developers? No. In reality, what would happen is that yet another insurmountable "the" sticking point will arise. PHP vs. Tcl vs. Python vs. Perl, PostgreSQL vs. MySQL, etc. And in the end, maybe even ultimately the plain old "I don't care how good it is, it's not what XYZ already uses" sticking point.

What it all boils down to in the end is that OpenACS isn't LAMP, nor is it .Net, Java/J2EE, nor whatever other language/environment du-jour will next spark the fancy of the IT trade magazines and programming masses. And as long as it isn't one of those things, there will always be yet one more "the" sticking point "preventing" OpenACS adoption. Going down that road is playing to OpenACS's weaknesses, fighting a losing battle, and I don't see any sense to it.

Better to stick with the high-quality underlying tools OpenACS has now (AOLserver, Tcl, PostgreSQL, Oracle), and - more importantly - the sound technical and practical reasons for picking those tools in the first place.

Now, yes, there are a minority of people who really genuinely want to run OpenACS on Apache, or MS SQL Server, or whatever else. And the work John Sequeira and a few others have done towards those ends is laudable and good - and may even help in some of the external sysytems integration goals Lars touched on above - but it can't be the central focus of any OpenACS strategy - not any successful strategy anyway. Or so it all seems to me.

Of course, to partially contradict myself, if good support for Apache, PHP, Python, FastCGI, and whatever else just dropped into our laps, that would be great and maybe very valuable! (In fact in my Copious Free Time I've even thought a bit about working on AOLserver Tcl/PHP integration, just because it's interesting and potentially useful. Not quite so interesting and useful as integrating both AOLserver's database support C code and OpenACS's db_api into a Tcl library useable from outside AOLserver, but still...)

I just don't think any of that makes sense as a central focus, goal, or plan for the OpenACS project as whole. Make this stuff a subordinate goal, sure maybe, but please let's not convince ourselves that focusing principally on any one of these Apache vs. AOLserver religious-type issues would be a good strategy.

Collapse
Posted by Nima Mazloumi on
I fully support your thoughts.

I personally think that there are not enough resources to have different projects like OpenACS, dotLRN, dotWRK and so on. The problem should be solved by

1. Configuration of the installation. Which packages are required, how the site structure is. OpenACS should become completely portlet based.

2. Using ifferent ontologies for different purpose. And for this purpose internationalization was the best thing that happened to OpenACS. Thus we not only have support for different langauages but also within a language for different purposes. We should have different catalogs for different setting.

You are doing a great job,
Nima

Collapse
Posted by Jun Yamog on
Hi Barry,

John is already working on this.

http://www.jsequeira.com/projects/portable.nsd/

Collapse
Posted by Andrew Piskorski on
Lars, your vision statement above sounds pretty good overall to me. It's also very nice to see a community leader both thinking carefully about this, and writing it all up so we can think about it too. The more the better. :)

Emphasizing OpenACS's strength in integration makes sense. That concept and core strength has been there all along in some way or other starting at least as far back as Philip's "the important thing is the integrated data model" spiel. I do sort of worry though about too much emphasis being placed on "integration" and losing OpenACS's other historical emphasis on rapid development, fast turn-around, and a toolkit that just plain gets out of your way when you need it to.

These two "integration" and "rapid development toolkit" goals are not contradictory, in fact they can and should be mutually supporting, but there is tension between them. It's easy to imagine and fear so-called "integration" getting exclusive control and wandering off into the same wasteland that swallowed ACS/Java.

Anyway, on balance emphasizing integration still makes sense to me, so how to do it? Increasing the use of various existing tools (Workflow, List Builder, etc.), the dotLRN/OpenACS refactoring and integration work, and other existing efforts (rpm/deb installs, etc.) should all obviously help there.

But past those, in practice, further integration improvements are going to have to mean adding much more powerful tools to let OpenACS package authors more simply and powerfully achieve integration, consistent behavior across packages, etc. (Or so it seems to me. Any contrary opinions?) And I think that's going to be the case regardless of whether OpenACS receives a huge influx of new users and developers or not.

With current resource levels, new tools will be needed to leverage the same developers' time into faster improvement. With lots more developers, probably much the same new tools would be needed to help channel all that extra effort into the most useful paths, rather than diverging off into all sorts of contrary directions that end up hurting integration.

This data-centric "data soup" direction seems to be major new integration idea so far. But, I don't really understand what that means yet. Are there any concrete examples of that sort of thing around yet?

Hm, also, I'm not sure why Ecommerce is considered a non-strength of OpenACS. I thought that historically, quite a few ecommerce systems had been built with OpenACS, and Ecommerce functionality often blends fairly naturally into other groupware/community aspects of a site. But I never had any personal experience doing Ecommerce stuff at all, so are my assumptions here misplaced?

Collapse
Posted by Joel Aufrecht on
I see two different, complementary paths for integration, first internal integration and second external:

[E]mphasizing integration still makes sense to me, so how to do it? When I started writing OpenACS documentation in 2002, the old docs recommended making new records in your new app "ACS Objects," but the only tangible benefits to an extra 200 lines of nasty pg/sql were permissions and general comments. Now you can make your records cr_items (actually you could do that all along) in < 10 lines of code and the list of benefits is: permissions, version control, mime typing, comments, notifications, workflow, categories, and rss (?). The key is urging everyone to use acs_objects/cr_items as their basic building blocks, and to write features based around acs_objects.

For external integration, the idea is that an OpenACS site is a neutral "meeting place" for different web services data. If we can get basic feeds of external data into OpenACS, we can treat them as local objects and - whammo - get all the OpenACS features.

Example: You have customers with data on one system, and you want to use that data on a second system, and you also want email notifications when the customers do stuff, and you want to categorize your users and have the same categories be in sync on both systems. In order to get the two systems to talk at all, you will have to get each to input and output lists of records. Instead of reconciling the lists in one or the other of the systems, you use tools in OpenACS to reconcile the lists. When you add a third system, maybe LDAP authentication for your users, you just link it in to OpenACS and it's trivially linked to the other systems.

The technologies we need to be a good central integration server are simply importing and exporting data and synchronizing data. We've done each of these in isolation, with acs-authentication (batch import and sync of user lists), authentication service, two-way syncing (new feature in acs-lang 5.0.0), RSS feeds. The next steps to compete here are: generalize the solutions in these packages into core tools, start talking more standards (IMS, etc), and start using one or more methods of globally unique naming (guids, unique names via email addresses/server names, etc)

Collapse
Posted by Barry Books on
Andrew,

Just to let you know what side I'm on I don't think I would run an Apache based OpenACS. I've got no legacy code and I think Apache 2.0/mod_tcl has maybe 80% of the features of AOLServer 4.0, plus AOLServer has 10 years in the field. I do think Apache has better support for virtual hosting which is one reason why Apache/PHP is so popular.

I know there are big differences between Postgres and Mysql. From a business standpoint Postgres is easier to distribute.

In some ways I see this like AOLServer on Windows, I would not use it for production but it's nice that it's there. I do think that OpenACS would get much more exposure if it ran under Apache and could perhaps become an Apache project, but there are very large hurdles to getting there. I do agree there is no technical reason to even try.

Collapse
Posted by Talli Somekh on
This is a good first stab Lars, and thanks for bringing up the point. Certainly these were profoundly addressed at each of the Linux World's I've been at as well as each time I write a proposal.

I think we should explicitly avoid being labeled as "groupware", "CMS" or whatever because we will always be behind the ball. This is because in a buzzword laden world there's not choice but to differentiate an application every few years, and we'll never keep up.

For example, "groupware" was adopted as the industry term after "community" ran its course. Now, "social networking" is replacing "groupware". Whatever they mean, it's all crap because the software stayed the same - just the VC money went different places.

Furthermore, the OACS is being used by many different people than just groupware/community/whatever developers. For instance, Andrew's use of the OACS is for a hedge fund application. Bart is working for a company using it for a semiconductor modeling application.

The reason they chose OACS is because it provides a foundation for generic services like user registration, notification, permissioning, etc. In other words, they are using it as a platform for web-enabled database applications.

Furthermore, the development tools are quickly moving in that direction. The OACS is finally fulfilling its promise (thanks to all of our hard work) as a services based platform in which a sophisticated application can be quickly cobbled together because of tools like ad_form and listbuilder and APIs for the CR and portals.

What this suggests to me is that we ought to market the OACS as a platform and use examples of vertical applications (another promise that was long ago made and now being fulfilled.) We can point to things like .LRN and Project/Open as examples.

Ultimately, we should have things like this:

OpenACS - Platform
  Groupware
      .LRN
      Project/Open
  CMS
      XCMS
  CRM
      Cyber-advocacy systems

This is what the J2EE and the .NET folks are trying to do = build the infrastructure and let specific markets and developers decide what to do.

I think ultimately this will be a more beneficial approach as the OpenACS community can be more honest with itself. We are not marketeers, but a community of software geeks trying to do as good a job as possible developing the OpeACS.

Which brings me to Apache... I think this is important to make our next big step. I disagree with Barry that mod_tcl will do the trick, although I could be wrong. I think John's approach will ultimately be easier because it models AOLserver a lot closer and because it uses FastCGI, once it works we get every other FastCGI enabled server for free (namely IIS).

I also think that Apache+Portable.nsd will never be equal in stature to AOLserver, but it lower the barrier to entry massively. People (and I am less interested in the geek who disregards AOLserver out of hand than I am in the IT department with policies about which apps can be deployed) are not too afraid of Tcl once they understand the OpenACS. Almost all servers in large institutions or enterprises are running Tcl in one form or another so that really isn't too big of a deal.

Sysadmins are a big fan of downloading and giving things a run, though, and if they can do so by setting up FastCGI, Portable.NSD and PG, they would be happy to do so. I know this from at least three conversations I had with IT staff from universities at Linux World last week.

After they run OACS, and they are inevitably impressed, they will go the extra distance to try out AOLserver. This I'm quite sure of. Particularly if they are considering deploying the application for 40K users.

The other benefit of course, is in integration. Providing an Apache front end (either with Portable.nsd or even some kind of interface to AOLserver using mod_proxy or native FastCGI tool) would be a big help as we would have access to the full range of apache mod_*.

Anyway, those are a couple of extra thoughts.

talli

Collapse
Posted by Jade Rubick on
Here's my vision for where OpenACS will be in a couple of years:

First of all, it will be extremely easy to start up and test it out. Perhaps Debian packages, or a series of scripts that can be run. Someone should be able to install it in 5 minutes or less, or that should be our goal.

Second, I imagine it will be much easier to create packages in the future. The way I imagine it is:

You create the datamodel. In the data model, you lay out enough meta-data that you can get these pages for free, by just copying a standard package template:

- a list page (based on list-builder), with pagination.
- a view page, which simply lists each item, and has links to edit and delete pages. It also optionally allows you to view all the revisions of that item in a simple format.
- a delete page, which deletes that item.
- an edit page, based on ad_form, which automatically generated the form for you based on the meta-data of the data model.

Really, this vision is not that far off. It just takes another 5% to put everything together. I think that's the direction we're going, too. :-)

I also would like to see some work on making ad_form work better with multiple items.

A couple years further down the road, we could use the content repository to create the data model through the UI, and do all of the above as well. You would really have to program anything at all, except for customization, and optimization of queries and more complex applications.

Collapse
Posted by Mark Aufflick on
Kudos to everyone who has taken the time to post here (and of course Lars for taking the time to build and explain his vision).

There are lots of great thoughts here about making OpenACS more attractive to all different people, but something needs to be seen for it's beauty to be appreciated!

I think that as well as working on the tech side (which is essential) we need to be working on active and passive promotion, and that can take some creative angles.

I like Lars' idea of shifting to sourceforge so we could get into the top X rated projexts on sf.net - sure to turn some heads. We should work at building and collating cool stats for the openacs.org homepage - I think one cool stat is how many google searches turn up an openacs or adj page in the first 5 results - dunno how you would graph that though!

Speaking of the openacs.org homepage, I think it's time for a redesign - just a visual redisign. It looks kinda, well, early 90's.

We should all also write some articles for journals - my experience writing an article on ecommerce with Rafael shows that it's not too hard to get a spot in a commercial journal like SysAdmin. Getting some books on the shelves of Barnes & Noble (go Jade!) is a good idea too.

Other promotion is sites. We need some really good promo sites - even if we have to group together to build some free sites for non-profits who can't afford development - and make sure that press releases get to every IT trade press in the world (insert maniacle laugh here) - you would be really surprised how easy it is to get stuff like that in newspapers.

Collapse
20: OpenACS sucks on Google! (response to 1)
Posted by Andrew Piskorski on
Last I checked, Google does an incredibly lousy job of indexing openacs.org. AFAICT none of the Forums threads show up at all - not one! And no, I'm not necessarily volunteering to fix this, but I suggest that making openacs.org Google-friendly would be vastly more productive than re-writing the home page.

Eric Wolfram has some good info on that, and he posted more of it here in the Forums. And note the link at the top of that post to the previous big Google discussion, too.

If I remember correctly, back when openacs.org was running OpenaCS 3.x, it was much better indexed by Google. Maybe the big "no forums threads" snafu happened when cutting over from the 3.x BBoard to the 4.x Forums package (which of course was a long time ago), but I don't really know.

Collapse
Posted by Bart Teeuwisse on
I wouldn't be suprised if the poor ranking of openacs.org -the forums in particular- is due to the use pages that require query parameters. Forums for example uses forum-view and message-view both of which take an ID to locate the forum/message to display.

As Eric Wolfram observes in http://wolfram.org/writing/howto/3_1.html directory index pages rank better and are more frequently indexed by Google. I can attest to that too. About 2 weeks ago I added a 'GNU Arch' section to 'the Code Mill' (http://www.thecodemill.biz/services/arch/). At the same time I started hosting the #arch IRC logs (http://www.thecodemill.biz/services/arch/irc/).

A fortnight later these pages rank within the top 10 Google search for 'GNU Arch IRC' (http://www.google.com/search?&q=gnu+arch+irc). At the same time the latest daily log (not an index page) that google appears to know is over week old.

Changing the forums package to use .vuh files instead of forum-view or message view would probably make quite a difference.

/Bart

Collapse
Posted by Joel Aufrecht on
Hmm - the number one link at the moment on Google for http://www.google.com/search?&q=gnu+arch+irc leads to is a page with a query parameter in the URL:

www.thecodemill.biz/publications/blog/one-entry?entry_id=9769

So either something's changing or the query parameter is not a 100% fatal problem.

Also noteworthy is that Bart's page returns a 200, not a 301 or 302, and that it requires a host: tag. Can someone explain to me why a host: tag is required and how my browser knows to use it (I had to use "view HTTP Headers" in the Mozilla Web Development Toolbar to find out that difference between a sucessful mozilla page view and a failed telnet :80 page view.

joel@joel-desktop joel: telnet www.thecodemill.biz 80
Trying 66.92.28.174...
Connected to dsl092-028-174.sfo4.dsl.speakeasy.net.
Escape character is '^]'.
GET /publications/blog/one-entry?entry_id=9769 HTTP/1.0

HTTP/1.0 503 Service Unavailable
Content-Type: text/html
Content-Length: 169

<html><head><title>503 Service Unavailable</title></head><body><h1>503 Service Unavailable</h1><p>The service is not available. Please try again later.</p></body></html>Connection closed by foreign host.

joel@joel-desktop joel.import: telnet www.thecodemill.biz 80
Trying 66.92.28.174...
Connected to dsl092-028-174.sfo4.dsl.speakeasy.net.
Escape character is '^]'.
GET /publications/blog/one-entry?entry_id=9769 HTTP/1.0
Host: www.thecodemill.biz

HTTP/1.0 200 OK
Set-Cookie: ad_session_id=6485003%2c0 %7b934 1075884795 4C0E03284109752C174B64754F47C1B355011360%7d; Path=/; Max-Age=1200
MIME-Version: 1.0
Date: Wed, 04 Feb 2004 08:33:15 GMT
Server: AOLserver/4.1
Content-Type: text/html
Content-Length: 14286
Connection: close

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
 <html>
  <head>
    <meta http-equiv="content-type" content="text/html; charset=iso-8859-1">
    <title>the Code Mill | GNU Arch IRC logs on-line</title>
    <link href="/style/sheets/thecodemill.css" rel="stylesheet" type="text/css">
    <style type="text/css" media="screen">@import "/publications/photos/photo.css";</style>
    <script src="/scripts/searchhi.js" type="text/javascript"></script>

  </head>

(rest of page omitted)

Collapse
Posted by Bart Teeuwisse on
Interesting Joel,

when I googled for 'gnu arch irc' a few hours prior to your search the first 2 hits in thecodemill.biz domain were /services/arch and /services/arch/irc, NOT the blog entry that tops the search result set now.

The reason that you have to include the host header is because thecodemill.biz is behind a hostname based proxy: pound. In other words thecodemill.biz is a virtual server.

/Bart

Collapse
Posted by Jeff Davis on
Joel, Bart's site is vhosted (I think he uses Pound but I am sure he will pipe in). When something is vhosted if you don't send the Host header it has no way to tell which server to talk to.

As for indexing pages with query variables, google definitely does, although it limits the number of pages it gets this way according to their information for webmasters:

1. Reasons your site may not be included.
Your pages are dynamically generated. We are able to index dynamically generated pages. However, because our web crawler can easily overwhelm and crash sites serving dynamic content, we limit the amount of dynamic pages we index.

Collapse
Posted by Steve Manning on
>Can someone explain to me why a host: tag is required and how my browser knows to use it.

Isn't it the case that your browser always has to send it if it is sending a HTTP/1.1 request: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.23

"A client MUST include a Host header field in all HTTP/1.1 request messages . If the requested URI does not include an Internet host name for the service being requested, then the Host header field MUST be given with an empty value. An HTTP/1.1 proxy MUST ensure that any request message it forwards does contain an appropriate Host header field that identifies the service being requested by the proxy. All Internet-based HTTP/1.1 servers MUST respond with a 400 (Bad Request) status code to any HTTP/1.1 request message which lacks a Host header field."

    Steve

Collapse
Posted by Ben Koot on
I just googled for OACS, and was surpised to find my playground page to be nr 2. Is there any code in the edit this page application that triggers this reult by any chance? http://www.google.nl/search?q=oacs&hl=nl&ie=UTF-8&oe=UTF-8
Collapse
Posted by Deds Castillo on
Ben,

It's probably because the directory itself is named oacs and it's 1 level deep into your site.  That site structure will most likely score high points in google.

Considering searching for "ben".  The top results will have either ben in the domain name or have ben as the name of a directory 1-level deep into the site.  Searching for "ben koot", however, will give you desired results.

Deds

Collapse
Posted by Lars Pind on
I've thought some more, and have to agree with the amendments of Talli, Nima, and others.

In order for OpenACS to be successful for a variety of sites, including intranet, extranet, e-learning, project-management, social software sites, etc., the user experience must be completely customizable.

Hence, it should be possible to completely reorganize the user interface, the pages, of an OpenACS installation.

I'm currently prototyping this with Branimir. Here's what we have in mind:

The user interface consists of a number of "places" or "subsites". The subsite defines the master template, which in turn defines how navigation is rendered.

A place consists of a number of pages, organized into page folders for navigation.

A page consists of one or more "page elements" or include-templates, which are arranged on the page using a layout template (like the portal's 2-column layout, 3-column layout, etc.).

Navigation, including context bars, are automatically generated from the user interface specification, so that moving a page somewhere else, or changing its label (what the page is called in the navigation) will change it everywhere instantly - no more having to go through all the files to change the "set title ..." and "set context [list ...]" lines if you want to change a page title.

For example, in .LRN, "My Space" and each "Class" and "Community" would be places. The tabs in these portal-based pages would correspond to page folders. And the application pages would be fully integrated in this tab-based local navigation, so we avoid the confusion when you're on the "My  Calendar" tab of "My Space" versus inside the calendar application mounted at "My Space".

That's one dimension.

Another dimension is that for each page element or master template, the tcl/adp pairs of today, we need to separate it into 3 layers, with clear APIs between them:

- Script (TCL)
- Rendering (ADP)
- Stylesheet (CSS)

When placing a "page element" on a page, you can choose to override the ADP or the CSS or both, thus reusing the code, but completely changing the way that it looks.

In order to make this work with completely redefinable page flows, the "Script" part needs to have three types of API:

- Parameters: For example the package_id (except I want to replace package_id with parent_id, more below).

- Links: Since pages can be shuffled around at will, any outgoing links in the script part will have a corresponding parameter saying which page in the site map it will link to.

- Properties for rendering: Which variables are available to the template author.

Similarly, the template author should document which classes his template uses, so a CSS author can write a different stylesheet.

Another dimention is to unify our object system. All objects should have a parent_id, not package_id. Ideally, I'd do away with package_id altogether, as it only serves the purpose of acting as a 'container' of objects, and with parent_id, any object can be the container of other objects.

We also need to augment objects with title (including i18n) and with presentation_data (object-specific data, which a callback proc can use to render the object nicely, like Dirk has suggested before), and all the metadata that we need to present auto-generated forms and lists.

What this will do is give OpenACS a back-end, which you use to define objects and page flows, much like the old CMS package did. The front-end and the back-end will be separate user interfaces, which makes sense, because they will have different users - end users for the front-end vs. admins/developers for the back-end. It'll make OpenACS the "CMS for web applications" or "CMS for social software", depending on the VC hype of the day :)

Why all this? Because today if you want to have a user experience that's not a straight OpenACS out of the box user interface, then you'll have to fork the pages in the packages. And just changing the template for the pages wholesale is not going to be enough. Client sites need the flexibility to change the entire pageflow, and if they can do so while still making upgrades possible, everybody benefits.

So this will make .LRN a distribution of OpenACS, namely a collection of packages and specialized page flows, just like you'd expect it to be.

We'll try to make the prototype available as soon as possible. If nothing else, we'll demo it at the Copenhagen Bash on March 25-26. And we'll be writing some more detailed docs and putting together some diagrams to make it more clear. And then if other people agree on the direction, we should create a branch in OpenACS CVS tree to all work on this together.

/Lars

Collapse
Posted by Tom Jackson on

Good ideas:

  • Removing title/context bar from the tcl script, it either needs to be in a database, or constructed on the adp side. Not sure if it works or not, but when you use the property tag, is that variable then available elsewhere on the same page (like using the set command?) If so, any text 'created' in the tcl script via a set command should be replaced by a property tag.
  • Doing away with special hierarchies hard coded in to various tables: like package_id. This one I consider especially bad as it makes a display function (package) owner of data...not good. Everywhere package_id has crept into core models will need to be fixed for this removal to work. However, all hierarchies are special purpose, and if you don't need set of semantics, the trend it to utilize an already existing hierarchy with new semantics. Thus we have the parent_id, context_id confusion. What is needed to handle this once and for all is to create a special table (acs_object_structure), which creates a series of hierarchies. By simply adding a tag to a mapping you can use one table and one set of (tree_sortkey/connect by) functions to maintain things. This will end the confusion, and allow users to create hierarchies without the overhead of building all the maintainance functions needed. For mapping objects to packages:
    acs_object_structure__create(:parent_obj, child_obj, :tag);
    
  • Objects should be handled more generally. My Big complaint with the CR/CMS is it is too removed from acs_objects. All the basic functions of CR should just apply to any object. Objects are all data anyway. How you display the data is a different story, and many folks will disagree on how it should be done. If there were more similarities between object types and content types a lot of things would be much easier. For instance, what if we had a message type object which just contained fields for title and body. The same message type could be used for forum messages, email messages and blog messages. The difference would be that you would map the message to different containers using acs_object_structure. You could easily create composite structures as well. For instance a web page could be composed of a series of messages, photos and other html fragments (each being objects themselves). However this composition would still need a method of arranging the data: a templating language which would specify what parts of the object to display, which subset to display, and in what arrangement....

Collapse
Posted by Dave Bauer on
Lars,

Sounds great. This is the exact kind of thing I have been thinking about for a while now. I look forward to seeing it in action. The ability to change the look of the site without forking the distributed code has been an oft-requested featue for some time now.

Collapse
Posted by Barry Books on
I would agree with the last statement. I've been trying to understand CMS and it seems to be built on acs_objects sort of. Perhaps I'm oversimplifying things but I don't see why you can't have an acs_object_type storage type of versioned and build CMS on top of that. If you call get_attribute without a version id you get the latest. If you call with a version id you get that version. Set attribute just resets the current value. If you want a new version call new_version. Now I don't have to care if an object is a CMS object or just an object.
Collapse
Posted by Tom Jackson on

That would work for one object, but if you want to select multiple objects, you need to know what table the data is stored in so you can write the select statement.

Collapse
Posted by Barry Books on
I'm not sure I understand your statement. Does CRM support storing content across multiple objects? I've read the docs a few times and I see it supports linking objects together which I guess is the same thing.

or

Did you mean multiple attributes instead of multiple objects?

Collapse
Posted by Tom Jackson on

Barry, what I meant was that a function like get_attribute works great for selecting a single object attribute. Assume a message object with attributes title, body. If you wanted a summary of messages, a list of titles, it would be more efficient to know where the object attribute is stored...so you can write an efficient query to return a list, instead of writing a loop of get_attribute...Versioned storage is the most complicated to work with. Having non-versioned storage would be easier, generic storage, easier still, although less efficient. With generic storage you wouldn't need to know anything about how storage was handled since object metadata would be used to return results. Versioning could also be folded into generic storage. Issues with generic storage are inefficiency and the difficulty of data integrety, since data typing and references would be difficult to establish.

Collapse
Posted by Don Baccus on
"The difference would be that you would map the message to different containers using acs_object_structure. You could easily create composite structures as well. For instance a web page could be composed of a series of messages, photos and other html fragments (each being objects themselves). However this composition would still need a method of arranging the data: a templating language which would specify what parts of the object to display, which subset to display, and in what arrangement...."

This sounds suspiciously like the current CR datamodel supported by the CMS application.  We've talked about generalizing this stuff to work with objects for a long time.  There's nothing novel about the idea.  Perhaps we're getting to the point where we can realistically talk about devoting some resource to this now that the toolkit's become relatively useful.  I actually have this on my hope-to-find-time-to-do list by summer.

Collapse
Posted by Bruce Spear on
The only thing I might add to the design scheme is an account of the social and productive relations: how to build into the design a permanent invitation for developers to contribute to the toolkit.  We might think of this -- and I write as more or less an amateur -- in "extreme programming" terms: of building in a system of rewards for volunteers that would accrue on an immediate and regular basis.  This is no news, really: I think we are heading in this direction already, and all I'm suggesting is that we might talk about it a little bit more along the way.  I'm thinking about how wonderful it has been for me to have had a number of excellent programmers give me all sorts of help -- I am new to this business -- and to have witnessed them do simply tons of work on the things that interest them: I am completely impressed by the variety of motivations contributing to such voluntarism: working an a good cause is fine, but abstract; working on a tool that I can use is more direct; building something that is state-of-the-art appeals to our professionalism; and building things togther is fun and the kind of work I, and many of my users, would like to do -- stuff like that.  I am learning that such voluntarism is among the defining features of our community, and I think it is among the defining features of the application itself.  That is, for my interest in the development of the user-friendly and user-modifiable interfact, that as we make room for the user's creativity we would increase not only the system's efficiency, but also, contribute to the kinds of cooperative social relations so many of us prize.
Collapse
Posted by Lars Pind on
Bruce, I like the way you're thinking.

/Lars