Forum OpenACS Development: OpenACS: the core and the packages/plugins

Dear all,
first of all my best wishes for the new year; I wish all you endurance, strength, health, and as much happiness as these times can allow.

I was about to continue the discussion on https://openacs.org/forums/message-view?message_id=5508475, but eventually I decided to start a new one.

Imagine a new potential user is interested in adopting OpenACS as base for the development of its applications. Should she use version 5.9 and then change everything when OpenACS goes to version 5.10; or should she use version 5.10 and take on her own the burden to verify that all packages in which she's interested work as expected?

I got from Fisheye, from the branch oacs-5-10 (https://fisheye.openacs.org/browse/~br=oacs-5-10/OpenACS/openacs-4/packages) the OpenACS core, and yet packages "search" and "tsearch2-driver" as they are do not work (on Windows only or also on Linux?). I made them work on my Windows distro and I will post my fixes on "Bug Tracker".
On top of that as soon as I added an "external" package, e.g. Contacts , it didn't work either because of the "acs_rel_type__create_type" issue, or because it uses some deprecated functions or because its SQL calls do not work any longer on PostreSQL 13.1.

Anyhow, I'm currently taking some selected (by a customer) packages and making them work on my Windows distribution on top of Version 5.10 and I'll make all of this available as my "standard" Windows distribution.

Hopefully this effort will also be useful to some other people/organizations.

Again my best wishes to all of you,
Maurizio

Collapse
Posted by Antonio Pisano on
Dear Maurizio,

I have answered on the bug-tracker about tsearch. Regarding the "Contacts" package, this and some of its many dependencies are not among the packages we actually maintain. I am courious on whether you just tried this one package randomly or if you need it in your project.

However, this commit https://cvs.openacs.org/changelog/OpenACS?cs=MAIN%3Aantoniop%3A20210103153317 should improve the situation. There is still a lot of deprecated api in there and I am not really sure about the many "US-oriented" packages it requires, but better than just breaking at install time.

As a rule of thumb, is always better to provide detailed information about the problem at hand, rather than just dropping the "It's broken!" bomb. All the packages currently in the oacs-5-10 branch are maintained and do work to their capacity (save the occasional bug), so I suspect there to be an explanation for your issues that has something to do with the specifics of a Windows setup.

Please keep sharing your findings with us so that we can massage any fix into the codebase. A working Windows installation is a valuable asset to the project and I am glad that you are investing time into it.

Ciao

Antonio

Collapse
Posted by Carl Lemp on
Hi Antonio,
I was trying to get a working set of packages on Windows and Maurizio was debugging/fixing the things I was running into. I am new to OpenACS and am trying to work my way through the maze of historical packages to find the functions I need. In the case of Contacts, I just need a place to store information (phone numbers, emails, roles, etc) about people related a community and Contacts looked like a likely candidate. If that is not the package that is kept up to date, which package(s) do OpenACS communities use to store basic contact information?

Thank you in advance.
Carl

Collapse
Posted by Antonio Pisano on
Hi Carl,

I am personally not aware of a package that does exactly that, and I do not advocate against the Contacts package if it serves your purpose, just expect some maintenance to be needed. I will be happy to port any fix or improvement upstream, if you will be so kind to share it with us.

Collapse
Posted by Carl Lemp on
Hi Antonio,
What I'm looking for is just basic contact information storage. It would be similar to the data that is stored in pretty much every email program and every mobile phone these days so I was expecting it would be data that most online communities needed to store. I'd be happy to share any fixes that I fund and those fixes would come through Maurizio since I'm still too new to OpenACS to attempt any change myself.
Regards,
Carl
Collapse
Posted by Brian Fenton on
Hi Carl

another option for you might be the Attribute Management System (or Dynfields as it is called in Project Open) https://openacs.org/xowiki/ams which you could use to add whatever attributes you need (phone numbers, emails etc) to the PARTY object. I haven't personally used it, but from what I know it should be a good fit. Hopefully someone can correct me if there is a better tool.

best wishes
Brian

Collapse
Posted by Maurizio Martignano on
Dear Brian,
the Contacts package relies on AMS.

Thank you,
Maurizio

Collapse
Posted by Brian Fenton on
Ciao Maurizio!

good to know - thanks.

un saluto
Brian

Collapse
Posted by Maurizio Martignano on
Dear Antonio,
my replies here below.

I have answered on the bug-tracker about tsearch.
[MM] More on this later, where you talk about "rules of tumb".

Regarding the "Contacts" package, this and some of its many dependencies are not among the packages we actually maintain. I am courious on whether you just tried this one package randomly or if you need it in your project.
[MM] Carl, my customer, already answered. We're looking for a Contacts package, and it seemed to me that "contacts" was the best available option,

However, this commit https://cvs.openacs.org/changelog/OpenACS?cs=MAIN%3Aantoniop%3A20210103153317 should improve the situation. There is still a lot of deprecated api in there and I am not really sure about the many "US-oriented" packages it requires, but better than just breaking at install time.
[MM] These are exactly the very same fixes I applied to the package, to make it work.

As a rule of thumb, is always better to provide detailed information about the problem at hand, rather than just dropping the "It's broken!" bomb. All the packages currently in the oacs-5-10 branch are maintained and do work to their capacity (save the occasional bug), so I suspect there to be an explanation for your issues that has something to do with the specifics of a Windows setup.
[MM] Well... well...
Given this piece of code
if {[callback::impl_exists -impl $driver -callback search::driver_info]} {
array set info [lindex [callback -impl $driver search::driver_info] 0]
# array set info [list package_key intermedia-driver version 1 automatic_and_queries_p 1 stopwords_p 1]
} else {
array set info [acs_sc::invoke -contract FtsEngineDriver -operation info -call_args [list] -impl $driver]
}

if { [array get info] eq "" } {
ns_return 200 text/html [_ search.lt_FtsEngineDriver_not_a]
ad_script_abort
}
The initial if passed OK, but then the following statement did not work, so I got the error displayed by
ns_return 200 text/html [_ search.lt_FtsEngineDriver_not_a]

If you check my fix, I forced the execution of
array set info [acs_sc::invoke -contract FtsEngineDriver -operation info -call_args [list] -impl $driver]
and then that piece of code "worked".

Exactly the same behaviour and consequent reasoning for the following piece of code:

if {[callback::impl_exists -impl $driver -callback search::search]} {
# DAVEB TODO Add subsite to the callback def?
# FIXME do this in the intermedia driver!
# set final_query_string [db_string final_query_select "select site_wide_search.im_convert(:q) from dual"]

array set result [lindex [callback -impl $driver search::search -query $q -offset $offset -limit $limit \
-user_id $user_id -df $df \
-extra_args [list package_ids $search_package_id object_type $object_type]] 0]
} else {
array set result [acs_sc::invoke -contract FtsEngineDriver -operation search \
-call_args $params -impl $driver]
}

When "search" got fixed this way, the system was looking for a tsearch2::search implementation
which as you saw I provided into "tsearch2-driver"

I'm not sure this is a specific Windows problem, it looks to me as code that needs some restructuring.
Frank and Malte stumbled on the very same problems and they are not working on Windows
https://openacs.org/forums/message-view?message_id=5465077

Collapse
Posted by Antonio Pisano on
Thanks for the pointers.

So it seems that the callback implementation is detected as existing, but then returns null when it is actually invoked.

If one looks at: https://openacs.org/api-doc/proc-view?proc=callback&source_p=1

There are basically two ways this can happen: no command is found in the prefix ::callback::${callback}::impl, or the command returns the empty string.

There is little chance that the empty string is returned, as this particular callback implementation returns a constant:

https://openacs.org/api-doc/proc-view?proc=callback::search::driver_info::impl::tsearch2-driver&source_p=1

https://openacs.org/api-doc/proc-view?proc=tsearch2::driver_info&source_p=1

hence I suspect that for some reason we do not enter the foreach that says: "foreach procname [lsort [info commands ${base}::$impl]] {" in the call to "callback".

Would you be so kind to check whether this is actually happening? Just insert a debug in the loop and see if the tsearch-driver callback implementation of the search::driver_info contract is found.

If this happens to be the issue you have much more than a broken search on your hands, basically the whole callback machinery would be broken. Let's just see if this is really the case and we can work from there.

Thanks in advance

Antonio

Collapse
Posted by Jon Griffin on
I have a contacts package that I wrote long ago for a client. It should be fairly international, but it is from 2000 or so right after I worked for ArsDigita. If you are interested in looking at it, let me know. You will need to update it, but it should be fairly easy.

I also wrote the Reference Data packages originally and believe that I used them on contacts-lite (what I called the package).

I don't use OACS at all anymore except for one client who is moving away after 21 years to another system.

Collapse
Posted by Carl Lemp on
Hi Jon,
Thank you for offering the package. I will start with the existing Contacts package and see how that goes. If I run into too many issues, then I may give up on it and fall back to contacts-lite. I don't think I need all the flexibility of AMS that underlies Contacts but I don't know that for sure yet so I can invest a little time and see if it provides any benefit for my application.

Thanks again,
Carl

Collapse
Posted by Gustaf Neumann on
Hi Jon!

Nice to hear from you! Do you have your package somewhere on a public repository? If not, please send it to me.... it would be nice to have your package on the list of OpenACS packages in the public repository.

We have currently just 100 packages maintained. Would you have the resources to maintain your package further? But even if not, having this package would be a useful resource.

all the best
-g

Collapse
Posted by Jon Griffin on
Hi Gustaf,

I can upload, but let me look through it first to make sure nothing proprietary is in it. I'll also write some docs up.

I don't have any current OACS sites, so maintaining would be difficult. I guess I could spin up a site for testing, but hopefully not.

Collapse
Posted by Carl Lemp on
Hi Jon,
If the offer to provide the contacts-lite package is still open I would like to evaluate it for my project. It did not take me long to figure out that using the Contacts package will be a burden I can't afford.

I'll document the issues I found with the Contacts package here for anyone else who might be evaluating it in the future:

Contacts Package Issues in 2021
Changes were required just to install it and those changes were made by others. But, after it was installed, I found:

1. It pulls in an additional 5 packages that are not on the 5.10 branch (AMS, Postal-Address, Telecom-Number, Organization, and Mail-Tracking). As Gustaf pointed out, Contacts and those packages will need to be maintained by the project using them. In my case 49 out of the 50 other packages I think I need are all on the 5.10 branch already so adding 6 unmaintained packages would be a burden.

2. There are a large number of configurable parameters and some of them point to folders and portlet files used to build the UI. As installed, the referenced folders do not exist so the data is not always displayed. The help text for the parameters is not helpful in determining what folders these parameters should reference now.

3. Going through the screens to figure it out, some screens of data can't be submitted because of missing data. Attempting to add the missing data through other screens can throw errors. These may be due to item 2 above but maybe not.

4. Adding a person through AMS breaks the Members package.

5. Everything is based on flexible lists of parameters in AMS but understanding the intent of the lists is not easy since language translation keys are missing so the names of the list are somewhat obfuscated.

Collapse
Posted by Jon Griffin on
I appear to have no access to CVS. The error is no smtp configured.

I can send these to someone by email if you want. I also found a few more packages that may be of interest. Meeting Minutes and module development (MDEV, I think it was part of what I did right after ArsDigita and when I was working at LAUSD).

There is also a mini-contacts which I have but doesn't seem to be something I used.

I don't have an easy way to setup a server since my servers all run apache and I don't want to add a whole other dev stack. Maybe I'll setup a local dev on my windows box, but again, I really don't use OACS anymore for any projects.

Collapse
Posted by Jon Griffin on
I'm guessing no one is interested in these. I will hold on the the tarballs for a bit, but no guarantees of when I may just do a cleanup.
Collapse
Posted by Carl Lemp on
Hi Jon,
I'm definitely interested in at least the mini-contacts package since I'm trying to put together a prototype and need to manage some contact information. I did not want to answer for OpenACS since I am not associated with the OpenACS project and have no way to add it to their public repository even if I work with Maurizio to bring it up to date.
I will however, gladly use it as a starting point if you can send it or upload to a shared location somewhere. Thank you.
Collapse
Posted by Gustaf Neumann on
Jon, i just missed your posting due to the stupid tree display. As mentioned above, please send the tar ball to me via mail, or put it one some public server where i can access it.

Many thanks!

Collapse
Posted by Gustaf Neumann on
Maurizio,

currently there are about 100 packages maintained (see: https://openacs.org/xowiki/packages), the "contracts" package is not included in this list. if there are volunteers to maintain this package, please step ahead. There might be different versions of the contacts package as well be around.

Concerning search: i did just now a fresh install of oacs-5-10.
* dropped the old DB, created a new one
* started OpenACS, did the usual OpenACS Installation for new databases (fill out the form), restart
* on the fresh instance, "install more packages" and selected tsearch2, xowiki, forums), mounted search, restart
* created a forum and a posting with "hello world"
* waited a little (search::indexer runs every 60s)
* go to /search, type in "hello" -> voilà, here it is.

This is with PostgreSQL 13.1 on macOS. i am very sure, i see the same results on Linux. In your entry in the bug-tracker, you seem to bypass the service-contract, so this seems to be the problem at hand (as Antonio indicated as well). Note, that service-contracts are used in OpenACS for multiple purposes. So, when these do not work under windows, similar problems will show up on other places as well.

Collapse
Posted by Maurizio Martignano on
Dear Gustaf,
I got the point of the criticality of this issue.
I'm currently investigating it and I will come back to you and Antonio as soon as I have something.

BTW: I don't manage to login in "fisheye" and whenever I try to download something that is larger than more or less 3.2 Mb the zip file I receive is corrupted. Could you please help me on this.

Thank you,
Maurizio

Collapse
Posted by Gustaf Neumann on
i can't login in fisheye either (probably only Stefan can do so) - but i think, this does not open more possibilities (aside of administering the fisheye instance). I was not aware that someone can download zip files from there. The normal way is to download packages via /acs-admin/install/...

When time permits, i will add as well download links on openacs.org for tar files (for users without CVS).

Collapse
Posted by Maurizio Martignano on
Dear Gustaf,
don't bother, I thought that by logging in you could have more features, less restrictions; I'll use CVS.

Maurizio

Collapse
Posted by Gustaf Neumann on
don't bother ...

too late. Via [1] one can now download directly the packages as .apm files (.apm files are tar.gz files including a manifest, an can be as well locally installed via the apm manager).

-gn
[1] https://openacs.org/repository/

Collapse
Posted by Maurizio Martignano on
Dear Gustaf and Antonio,
I finally found the issue that prevented the Search from working.
My bad, I had a misaligned configuration with some packages not working properly. Getting everything using CVS fixed the issue.
Now I have a further problem: if I add something like a comment, a post in a forum and the like, it appears in the search. If I create a new page in Xowiki, its contents do not appear in the search. What am I doing wrong?

Thank you in advance,
Maurizio

Collapse
Posted by Antonio Pisano on
Good that you could find the issue!

Regarding XoWiki, from the top of my head: is the page you have created published?

xowiki/admin/list, then check if the square beside the page name is green (published) or red (unpublished)

The indexer runs every 60 seconds by default, so you might need to wait a bit to see the result of your changes.

Collapse
Posted by Gustaf Neumann on
The pages are are added to the indexer queue each time these are updated. So you might have to edit the page. Otherwise on has to use "reindex", as Iuri mentioned, which produces for every published page an entry in the seach-queue.

After editing a page, the content is added to the search-queue, which is processed every minute. If there is a problem with indexing, you should see a message in the error.log from this step.

Collapse
Posted by Maurizio Martignano on
Dear Gustaf, Antonio and all the others,
thank you for all your suggestions.

I'm using subsistes and xowiki is in one of them.

I noticed this error:
https://www.spazioit.com/images/naviserver/subsites_error.jpg

As soon as I fixed it, and I'll report it in BT, the search started to work also inside xowiki.

Once again thank you to all of you,
Maurizio

Collapse
Posted by Gustaf Neumann on
This is not related. Maybe, you restarted the server, after activating search or similar.
Collapse
Posted by Iuri Sampaio on
if I add something like a comment, a post in a forum and the like, it appears in the search. If I create a new page in Xowiki, its contents do not appear in the search. What am I doing wrong?

Is "re-index" ad proc schedule working fine?

Collapse
Posted by Maurizio Martignano on
Dear all,
I would like to thank you all of you for your contributions.
Thanks to your help now my Windows-OpenACS distribution, available at https://spazioit.com/pages_en/sol_inf_en/windows-openacs_en/, contains a working oacs-5-10 OpenACS.
Thank you,
Maurizio
Collapse
Posted by Maurizio Martignano on
Dear all,
two simple questions:
1. does bug-tracker works on OCAS-10 branch?
2. which version of OpenACS this site is?

TIA,
Maurizio

Collapse
Posted by Gustaf Neumann on
The "bug-tracker" instannce on openacs.org is a package from the HEAD branch and it works with acs-core from oacs-5-10; Most of the package at openacs.org are from the oacs-5-10 branch.
Collapse
Posted by Maurizio Martignano on
Dear Gustaf,
thank you for having answered me. I asked the above question because I think I noticed a strange behaviour in the bug-tracker in my Windows distribution. I have to say I'm relieved to see that the very same behaviours occurs also in your standard installation. Perhaps I'm misusing the system. I'll try to explain here:

1. I entered a test bug as shown in here:
https://spazioit.com/images/naviserver/bug_error01.jpg

2. I edited the difficulty and priority of this bug and when you do so
the corresponding fields disappear when you click on the bug,
as shown in here:
https://spazioit.com/images/naviserver/bug_error02.jpg

3. I tried in any way to modify the difficulty and priority of this bug,
as shown in here:
https://spazioit.com/images/naviserver/bug_error03.jpg
but whatever I do the main list always shows me what we had
before:
https://spazioit.com/images/naviserver/bug_error01.jpg.

Am I doing anything wrong?

Thank you in advance,
Maurizio

Collapse
Posted by Maurizio Martignano on
Dear Gustaf,
It really seams related, thank you!

(I'm less worried now. Seeing that the issue is a general one and not a "Windows" one, has relaxed me quite a bit.)

Again thank you,
Maurizio

Collapse
Posted by Gustaf Neumann on
Concerning bug tracker workflows: The intended workflow is as follows:

Some user A opens an issue, many people can comment, the maintainer M should resolve the issue, then A should either close or reopen it. It can also be reopened later.

Design: https://openacs.org/doc/bug-tracker/bug-tracker-spec