Forum OpenACS Development: Current best practises: Dynamic Attributes, Package Linking, Callbacks

I was wondering what is going to be the best practise in 5.2 for dynamic attributes (dynamic types vs. ams). We want to extend Project Manager to support additional attributes for the project as well as the task.

In addition to attributes, we'd like to link certain other objects (e.g. files) to tasks. Would we use cr-rels for this or create a dynamic attribute that references the CR ?

Under what circumstances should an attribute be added and when would it be enough to add a catagory? I assume free text would be an attribute whereas choice would be category? Or is a category too much overhead and we should stick to dynamic attributes whenever we do not need to browse the object (e.g. task / project).

What is the best way to link e.g. a file-storage instance to a project manager instance? Assume each client should have access to a project manager, where the various projects are handled. In addition there should be a file-storage to store all the files along with a forums instance to discuss things. Currently we are using a subsite where each package is mounted and look with "closest_anchestor_package" for the package_id e.g. of the file-storage to use. Has this changed with TIP #76 (as you could now mount the file-storage under the project manager). Or should we stick with the current approach between logger and project-manager, that you have a mapping table?

While we are at it, currently the logger and the project manager have code that checks if the other package is installed and if yes, offers more functionality. Would this be made easier with a callback? Actually, how would you add more functionality to a package X if package Y is installed and makes use of an instance of package X? Is there any example in the code?

If it would be possible to enhance other packages functionality in an easy way by a third package, a lot of custom code could be taken into it's own package instead of modifying the code in the original package (e.g. if we make some adjustments to project manager we have to make sure to strip out custom code that adds additional features to PM before committing. If the additional features could just "plug in" that would be useful).

Malte,

I recommend using subsites to groups related packages. It makes good sense, and seems much better than creating a special mapping for each pair of package types. Subsites are a generic way to group any package without adding any new code. I think that using the tabs or other subsite-based navigation and the related-items package is more flexible and less error prone than trying to predict every possible combination of useful packages.

To add functionality you need to add a callback interface in project manager, for example, and then implement that callback in your custom function package. That seems like the best way to go about it with the existing toolkit. Much better than having special project manager specific code that replicates callback type functionality.

My biggest question is how scalable subsites are. If I have 1000 projects, witha subsite for each project, does that make sense? 1000 instances of file-storage?

That sounds like a recipe for disaster, but I don't know the internals that well.

Jade,

I don't think there are any scalability issues. A subsite and a package instance is just one object. Having 1000 objects of one type should be possible, and if there is an issue, its a bug. I know there are .LRN installs with 1000s of site nodes and dotlrn instances, so subsites shouldn't have any additional overhead beyond that.

The wins for arranging packages and assigning applications groups with subsites are good. Also it makes alot of sense to group packages using the same tools and make the tools to manage them consistent.

Due to the fact that one instance of project manager handles multiple projects and one instance of forums handles multiple fora and one instance of file-storage handles multiple folders we "only" have to come up with a way to link the project with the forum and the folder.

Reason is that it does not make sense for a user to have the option to create multiple forums for a single project, especially as you can work with subprojects (my estimation is, the moment a project becomes large enough to warrant a new forum it is time for a subproject).

Any idea how best to link them (acs-rels)? Furthermore, how should the user interface look like (I don't like the way I have to manually mount and add a logger instance to PM).

Though, in the end, I do agree that the *easiest* solution is to create an acs-subsite for each project and make sure that the user interface does not provide all the functionality. I think we have this already and it's called .LRN communities 😊.

Well, we will bang our head against this once we come to it. On a related note though, has anyone actually used / installed .LRN on 5.2?

Malte

The best benefit would be to improve acs-subsite where you felt it needed to be improved, but a general solution to combining packages makes more sense to me than to custom code every time a new package comes out and you want to relate it to project manager.

I don't see .LRN communities as anything except a special case of acs-subsite. There has been a great amount of work done to make subsite creation with packages setup and .LRN should move to use these facilities. That way a site could take advantage of the same type of community features with or without .LRN.

Hmmm now I see the problem!

If you want to use your enhanced project manager within .LRN it won't work cleanly.

I recommend integrating the new feature that allows any package to "stand-in" for acs-subsite and work in a similar way for grouping packages, with .LRN. This is on my list because otherwise we are going to end up with more and more .LRN specific checks for community grouping features and it has to stop :)

I talked to Timo a little bit about it and we came up with the idea, as we "only" need a forum and a folder for each project, to use one instance of project manager, forums and filestorage and to create folders / forums whenever a new project is created. As the forum and the filestorage folder would be generated through the project manager we might be able to have the context_id set to the project_id of the project we just created.

This has the benefit of reducing the of site node entries and does not take with it the "overhead" of acs-subsites. Additionally it does not even need callbacks as the "associated" forums_id and file-storage-folder ID can be stored within project manager.

If we then have a functionality like /o/$object_id working, then we could arbitrarily add objects to a package and when you call the view page e.g. for a project it would check which objects are linked to it and call the /portlet/o/$object_id functionality that returns the default include for the display of the object.

I guess it is a good time we meet in Madrid and I hope we can sit down over a couple of glasses of wine and think a little bit more as I assume this might be of general interest, especially as the goal is not that much different from what .LRN tries to achieve.

I just want to add my five cents about projects and subsites, based in my experience at Competitiveness.
Excuse me if this is off topic but I don't have much experience with dotLRN and not even
had time to play with the new project manager package.

In the clustermanagers system we decided to create projects just as subsites because we saw them
more like project-rooms where you can add/remove members, link any package you want (etp, forum, file-storage,
chat, log hours, events, calendar, etc) and even change the look and feel per project. The possible combinations
are amazing and you just need to create the rigth callbacks procedures in each package to link them all.

For that reason I really like the idea of creating a subsite for each project. However I remember
a discussion about "spaces" and "subsites" in the dotWRK project by Lars:


(Sub)sites and spaces

Brainstorming about OpenACS site-map/navigation/structure with Branimir ...

We discussed the distinction between a (sub)site and a "space". A subsite is something that has a specific
appearance, it has its own login page, one "your account", etc ... A space is a place where specific people go
to work on specific things -- a project, a special-interest group, etc. We currently use the 'subsite'
application for this purpose, too, but logically, I don't think there should be a "Your Account" page,
for example, per space. Just one per site.

The reason we came to this is, we were playing around with the host-node map to map http://foo.test.com to a the
subsite at http://test.com/foo. This broke a bunch of things, such as the site-wide admin link, which will now
require to be a fully qualified URL that goes to http://test.com/acs-admin/, not just the current /acs-admin.
(There are other things broken about host-node-map; for example, it cuts off parts of the URL, so if you
say http://foo.test.com/foo2 it does a redirect to http://foo.test.com2 - go figure!)

Anyway, we want to do multiple sites on one OpenACS instance, where those sites are completely isolated from
each other, completely different design, no admins on the one site should ever have to know that the other
sites even exist. So when you go to "Your Workspace", it's on the (sub)site.

But in this scenario, I wonder if you even want to have the user database be shared. Just because I have
an account on one site that shouldn't mean I have an account on the other sites.

Also, I'm not sure there's any point in mounting a site inside of another site. If the sites are completely
isolated, what would the implications be?

Things that would remain site-wide:

- software installation/upgrade
- internationalization/localization
- package manager, and other developer features

Things that are currently site-wide but would belong to the "site":

- users
- authentication

So it seems like OpenACS' current subsite construct is half-way between "space" and "site".

What do people currently use subsites for? To what extent can we make changes to this (it's pretty core...)?

Does this still make sense? Some advance in this direction was made?

Thinking about scalability, currently we have about 1000 users, 50 subsites, an average of four project
per subsite and a heavy use of permissions, in a single instance of oacs (in a not dedicated Athlon
2600 machine), and performance has not yet been a problem.

About the /portlet/o/$object_id functionality I don't really know what are you talking about.

Hope it helps.

Malte, Dave,

I didn't realise that this subject had been discussed already - I was thinking through this exact issue just the other week because I too want to permit the user to have a file storage folder for each project.

The solution that I came up with was almost identical to the one Malte and Timo propose above. That is to say that project manager adds a folder (parameter driven) and within that a folder can be added (same parameter driven) at project creation time - we can just add the necessary fields to the add-edit-project page. The datamodel would be amended so that the associated file storage folder is an attribute of the project object.

I don't like to idea of creating a subsite for each project for various reasons. I can see the similarity in the requirements but this risks going too far done the 'object philosophy' route rather than the sql 'dynamic types' route. Lets keep it simple!

Rehards
Richard

Sounds very similar to some of the things I am doing for dotfolio (e-portfolio application). Essentially I am creating a subsite for each e-portfolio owner, and each subsite contains file-storage and web-blogger packages.

I too was unsure of scalability, so I wrote a script that created 1000 e-portfolios, or 1000 subsites. Although it took a long time to create the 1000 subsites, there seemed to be no detrimental effect to performance when hitting a particular e-portfolio subsite. Mind you I didn't test scalability when a number of users hit the application simultaneously. But I doubt acs-subsites will contribute to a drop in performance, if any.

The linking of project tasks and file-storage items sounds very similar to how I'm linking blog entries to file-storage items. I am using the relate and clipboard packages for this. You might want to investigate these packages in COP and dotFOLIO.

I think my main objection to using subsites functionality for stuff like this is that it represents a form of feature creep in so far as the subsites functionality is conceived (and named) to support literally subordinate sites. In the purest sense therefore this could be as simple as each country within a pan-national organisation, or each branch within a national organisation, or each state within a US organisation.

The subsite itself is an acs-object and inherits permissions. There are groups created automatically (I believe) to help manage the permissioning. The whole thing is geared up to integrating multiple similar, though functionally seperate, sites under one umbrella whilst benefitting from the fully integrated central services of the OpenACS. If not careful we end up with thousands of obscurely named parties in the system, and with thousands of superfluous and confusing site nodes. We become confused as to which hierarchy serves which purpose!

Subsites also follows an object model philosophy and permits limitless layering. Flexible indeed but the arithmetic will always beat us in the end.

With issues such as the one under discussion (of having a file-storage folder connected with a project), It just 'feels' wrong to use the subsites functionality to solve these kinds of problems - hugely flexible though it is. I really believe that the best solution is to keep it simple and have the folder_id as a property of the project.

Project manager could make a request of the File-Storage module that it create a folder at project creation time and set the context_id of the folder to inherit from the project's object_id. Any sub folders created in the File-Storage module would then inherit correctly. Aggregations of projects (multi-layering of projects and sub-projects which is supported in the datamodel and will be implemented fully soon) will then take care automatically of the greater hierarchy without complexity or overhead.

This sceme could also be made to work for creating a project specific forum. We surely wouldn't want a whole new set of subsites - we'll have hierarchy overload!! :-o

Also, who knows what other services might need integrating in the future. Dynamic types potentially provides us with a method of dynamically extending the project manager data model (or that of any other package) to support links with objects created by other modules. This is in line with the core thrust of OpenACS which is for 'scaleable' web applications. That means erring on the side of a sql based philosophy rather than going too far down the object orientated route.

Regards
Richard

After some discussion with the OCT I think the best way to add a file storage folder or any other object to a project is as follows:
  1. In the parameter section of project manager define what object types instances (e.g. a forum) should be generated along with the project. You could use the object type and the package_id where the object should be generated at (or just use the closes anchestor). Preferably we could do this in a general way that would allow the generic linking of packages and is not specific to project manager (e.g. by adding a "object-link" package that also stores the sort order for display)
  2. After adding the project, create all other object type instances (using callbacks ?) and
    • Relate the new object_id to the project_id using acs-rels (and not by an extension using dynamic types).
    • Set the context_id to the project_id for the new objects
  3. When you display the default page for the project manager include the other objects by displaying their default include that displays the object (something like th /o functionality). This needs the following things:
    • Each package we want to include needs to have a default include that is able to display the object type.
    • The default include's name needs to be registered with the object type (additional column ?). In the long run this should be changeable, so you could define to use a different default template in the install.xml.
    • In the project view page there needs to be a "hook" to display all related objects using their default include.
As we are under some time constraints, we will make our life easier though and only deal with forums and file-storage as objects to add to the PM and add them fixed to the PM pages (instead of using the hook method).

This approach also has the beauty of the ability to use the portal package instead of the current page with includes, if I only knew how to create a page with the portal package in the first place. It would also be cool if we could get the portal package to accept a parameter that this page can not be changed by the user. But this is a different topic.