Forum OpenACS Development: danger: HEAD changed for TIP#42,43 - could be unstable

Just to let everybody know: I just changed the HEAD for TIP #42,43. Had to change acs-kernel, acs-authentication, acs-events, acs-reference, acs-subsite, acs-service-contracts, acs-tcl, acs-content-repository, acs-mail and acs-messaging to add package_id and title to the table acs_objects.

Most important changes are in the interface of acs_objects.new (added title and package_id, but backwards compatibility is ensured) and in the content-repository.

Due to the mass of changes the kernel could now be unstable, so please do not update from head for the next few days if not necessary and if you need your site running.

Timo,

Are these changes supported for both Oracle and Postgresql?

Collapse
Posted by Don Baccus on
The postgres version doesn't even install - Timo, I fixed one error, am tracking others, shouldn't basic install be tested before committing something like this?

Is this going in 5.1?

Collapse
Posted by Don Baccus on
It's dying now because acs_objects and the content_repository both have "title" columns, and the composite view built for a content type dies in PG, at least, because of this.  If it's not dying in Oracle ... which of the two titles is being returned in the view?

Timo - I'm not sure what you intend, here.  Are we carrying this around as duplicate information for the time being in CR revision types and objects?  We need to bite the bullet at some point and fix (change) CR-using code to use the object title and remove the revision title but ... when?

I'd rather us get this right even if it causes short-term pain for those with custom packages and even if it slows the release of 5.1 ... those that use content_revision.new() and the generated views should be OK after the change so I'm not sure how extensive the impact will be.

Anyway ... what's your intent here?  I didn't want to just start munging the code without discussion, first.

Shit. I only tried upgrading a system since i thought i'm using the same scripts for that anyway, but did not look at views joining with acs_objects so that there're ambiously defined columns now. will try to fix this as soon as possible.
My intent has been to maintain a duplicate of the title column of package-specific columns, at least for a while. the mid-term goal would indeed be to drop the title column from those table so that only the title in acs_objects is used.
Collapse
Posted by Lars Pind on
My inclination would be to roll back these changes and postpone to 5.2.

I think the changes implement some very important and desired functionality, but it comes at a great risk to our 5.1 release.

Here's why:

- We need to ship frequently, so people can take advantage of the rapid pace of improvements to our software (software is useless unless you ship it).

- We have decided to cut scope when required to meet release targets, due to our limited resources (can't just throw more bodies at the problem).

- This commit is huge, destabilizing, is 5 days late, it breaks installation, and is likely to introduce many more subtle bugs down the road.

- It's my impression that at the point right before this commit, 5.1/HEAD was fairly stable, and could be tested, stabilized, and released relatively quickly.

- This has to do with the fact that, aside from this particular change, 5.1 was mostly a simple, incremental upgrade, with no radical changes. I would like to keep it that way.

Therefore, I'm going to suggest at the OCT meeting tonight that we roll back this commit, and focus all efforts (hopefully including Timo's) on stabilizing and releasing a 5.1 release as quickly as absolutely possible.

Ideally, we want all community effort to go into shipping 5.1, rather than fixing and playing with Timo's new changes. So perhaps we could ask Timo to work privately for another week or so, until install, upgrade, test cases, and cursory browsing works again, then commit.

This would also help break our expensive (in time, resources, frustration) habit of committing broken functionality.

Then this much desired change can go in as soon as we have branched, and Timo and all effort can go towards stabilizing and taking advantage of this change.

I do want to stress that I'm grateful to Timo for doing this important piece of work. I see these changes as critical to OpenACS going forward, and it's great to see the enthusiasm and confidence and ability to getting it done. Thanks, Timo!

/Lars

Collapse
Posted by Don Baccus on
I agree and am willing to help get our standard and contrib CR packages working with Timo's changes for 5.2.

Thinking this over, I'd rather us schedule a time to bite the bullet and adopt the change wholesale rather than try to sneak into it with duplicated columns etc ... and it is far too late to do this for 5.1.

Collapse
Posted by Dirk Gomez on
It is also a good opportunity to tackle I18N properly. It would be a big mistake to break and fix up OpenACS now to add name and package_id to central and then break and fix up OpenACS to make name and package_id multilingual. (The odds of us never doing this are *very* high)

Here's an idea based on a post by Branimir, based on an idea by the venerable Drazen Kacar. We add a column name_presentation which holds the instruction on how to format the object's name. Here's an example for calendar.

In a search result the name of a calendar object should be:

"Listening to MP3s while drinking beer on 18-March-2004 21:00 - 23:00"

Time and date are not part of the name the user gave, but are added by OpenACS. Based on your locale the output may have to be formatted like this:

"18. März 2004, von 21:00 bis 23:00: Listening to MP3s while drinking beer"

The object's "real" name remains unaltered but the system-added information can be transformed. In this particular case the acs_objects.name column could hold a list and acs_objects.name_presentation would hold a proc that transforms dates, localizes msg_keys and reorders the output.

Gee - I don't know that I'm comfortable with that understanding of a "name". That looks more like a "title" to me.

I think if we make something as fundamental as an object's name so complicated, it will really increase the learning curve.

Certainly it needs much discussion!

oh - and has there been any further decision about rolling back Timo's commits? I'd really like head to install, and like Lars said we really want to ship 5.1 asap.
Mark, my example is real-worldish not hypothetical.

Presentation_name should probably not go into acs_objects but rather acs_object_types or a package-dependant acs_object_types_styling table.

I think back then we stuffed the column into a table called acs_objects_description which is very much alike cr_items. Branimir, do you remember where we put the presentation column?

Regarding i18n, multiple translation support, we came up with the solution to assessment (as none seems to be feasible) of adding a locale to our tables (for the default locale) and have an additional table that stores the content for other locales.

This looks like this:

as_items (item_id, locale, title, description, ...)
as_item_locales (item_id, locale, title, description, ...).

Reason for not storing all information in the locales table: Performance, as we do not forsee the need to query the as_item_locales very often (if a users locale matches the one in as_items, don't bother to make a lookup to as_item_locales at all).

Not sure if this is the best solution to tackle internationalization, but as it does not come out of the (CR) box we have to cludge around it on our own.

Collapse
Posted by Dirk Gomez on
I intentionally picked an example where you can't just a couple of different text blurbs.

I don't see where your solution would be more performant. Are you planning/expecting that as_items is full-table scanned on almost every access?

Collapse
Posted by Malte Sussdorff on
as_items as a table has 5 attributes that need to be internationlized. If I understand you correctly in our case you'd suggest to add 5 columns with procedures for each of the attributes ?

Don't get me wrong, I do like your solution of storing a procedure. Just some understanding problems. So acs_name would hold a list of all the names (or a list of list, with the name and the locale). Or how does the procedure in name_presentation know which list item to choose.

Are you planning on executing the procedure for each acs_object.name display? (assuming you have a list of lists for the name this might be pretty time consuming or am I mistaken)?

as_items will be queried on every single question that is displayed in the system. I'd like to limit the joins done on this table as much as possible. Same goes for as_item_attributes (dynamic attributes added to the item depending on the item_type). We might have to internationlize these attributes as well. Which would be another (expensive ?) join.

I don't think a full-table scan will be executed, but even without a full table scan I think we should try to avoid joins as much as possible.

Last but not least, why can't we store an adp-snippet in the localized content, which allows you to call a procedure if necessary.

One more thing I don't understand. The name of an object contains dynamic data (as your example suggests), is this possible? Or is the name of your object "Listening to MP3s while drinking beer".

As you can see I'm utterly confused what you want to do and how you want to do it. Maybe you can give an example of what you would store in acs_objects.name and acs_object.name_presentation for your example, using a Germany tranlation of "Listening to MP3s while drinking beer".

If you are mainly concerned with presentation, I think we might tackle this using adp-snippets? Or, if we assume we have a presentation of the name (e.g. for the quick categories display), we could have a procedure registered for storing a presentation name dependent on locale *on insert*. This would make sure the language of the locale and the presentation type match (because you only get the german presentation if you have a german locale content for it). For obvious reasons this should go in a seperate column for name_presentation (as the name might be different from the name_presentation string which you want to present as per your example).

Sorry for thinking out loud here:

What do you think of generating an acs_objects_locale table (as described for as_items, including overlaying), add the locale and a name_presentation to acs_objects (and the locale table :)) and insert the name presentation on creation of the object, handled by the package. This way the calendar package would store the name "Listening to MP3s while drinking beer" in acs_objects.name, "Listening to MP3s while drinking beer on 18-March-2004 21:00 - 23:00" in acs_objects.name_presentation, "en_GB" in acs_objects.locale and "18. März 2004, von 21:00 bis 23:00: MP3 Musik hören und Bier trinken" in acs_object_locales.name_presentation and "de_DE" in acs_object_locales.locale. For sure it would  insert the german locale *only* if someone translated the the calendar entry name.

As I said, my reasoning behind this is that I don't think it is a good idea to switch the format of a presentation depending on the locale, while keeping the content of the presentation in the original language. But this is just my thinking, please proove me wrong.

I'm really keen on getting more ideas on how to I18N content and I have to admit that I never thought (til reading Dirk's comment) about I18n the *presentation* of the content. That's what happens if you are not dealing with calendar :).

Last but not least, if we could come up with a solution to internationalize names and attributes of objects in a consistent manner that would be useable by all packages, this would be great.

Collapse
Posted by Dirk Gomez on
I'm not proposing a solution to I18N content, I am proposing a solution to I18N acs_objects.name.

We already have objects whose name is a message_key: in file-storage. How would a site-wide search or a linking package know how to deal different types of object names but with a instruction?

In my example the user-supplied object name never gets touched, only the rendering changes and is locale-dependant here.

The procedure in presentation knows how to deal with the content in name by "magic": name has #foobar#, presentation has lang::util::localize. If presentation is empty, nothing happens. If name is a list, the proc in presentation needs to know the semantics of the list elements.

We should move out I18N content to a different thread.

Timo didn't really agree to do internationalization, and IIRC, the TIP explicitly avoided I18 until later.

Perhaps I18 can be added in after his work is complete, stable, and added in?

Yes, I know. And I thought it was not a good idea to not even try to tackle it and I still think it is not a good idea: what good is a solution that ignores one of the fundamental design decisions.

Now that this stuff has been postponed we may at least try to solve I18N.

I will not do i18n for these TIPs. If somebody does have the spare time and energy (Dirk?) please go ahead!
My solution is to add a field to the same table that holds title or name. The effort for this should be near-zero for the person that adds the other fields.

However there doesn't seem to be much interest in discussing i18n, we don't have a solution yet, so why discuss who will do it?

Dirk, I totally agree this is important. I just think that process-wise, Timo only agreed to do the first step. Later steps should probably go through the same process (i.e. TIP etc)

I18N is very important -- it's part of what makes OpenACS so strong. But you're right, nobody is really stepping forward to do it.