Forum OpenACS Development: Announcement - contacts-heavy application being developed - accepting feature requests

Hello, I have been working on a very extensive contacts application. I have taken a look at contacts-lite (I think Jon Griffen wrote it), and it won't come close to meeting our needs. Thus I guess I am making contacts-heavy :) The uses we have are for contacts management for our entire school. It does not require contacts be be users on the system but also has provisions to deal with those scenerios. I have been working on one for a while now and would like to let you all know its in the works and get some feedback on some issues (unless lots of features are requested by the community I think I will have a stable and fully tested version of this software available by June 15th or so - but only working in postgresql if you are interested in porting to oracle please let me know):

Features

  • "backwards compatible" with openacs 4.6.2 contacts-lite requires 4.7
  • unlimited custom attributes/fields. Forms are generated using acs-templating and the standard subsite adp form template so it looks right entering and updating both work with the use of templates
  • restricted access to certain fields based on permissions - e.g. somebody in the recruitment department can't see the social security number of a student (only the financial aid and immigration (i.e. student visas) offices can see this), but they are allowed to see things like home address, or work phone. half way there
  • import/export from standard apps - I'm only going to make it work with Outlook CSV files, but it shouldn't be too hard to add other apps.
  • archiving of all changes to this data - i.e. you can change the home address, but it will keep the old one in the database - and this old one will only be accessible to high level administrators. Hopefully this would allow people that "disappear" to be contacted regarding things like defaulting loan payments etc, since we could say "hey, they lived with their parents 4 years ago so lets check and see if their parents know how to get to them" done and working
  • Extensive categorization. Again, since it needs to be compatible with the 4.6.x branch I won't be able to use the current categories package. Note that the categories "module" I am writing is hopefully being written in such a way that in the future people can drop contact-categories and go to a generic contacts package - though contact-categories is substantially different from the generic one so we might not want to do this.
  • Conflict resolution - i.e. with a database of 1200+ users and many different staff using it people will inevitably enter the same person twice - this will provide a way for people to find these conflicts and merge the contact data, based on last modified - or one department is more important than the other so its info takes precedence. remember that even after a merge all data is backed up - so if we got something wrong accidentally we can still find that address that staff member "X" entered 4 months ago.
  • Integration with users - i.e. if we say, Jane Doe (openacs user_id 9999) is the person we are referring to in regards to contact_id 10277, we can let Jane Doe enter as much of her private info as we want to via the pvt/home.
  • Fully integrated with bulk-mail
Features we don't need, but I would be happy to work on with some help - one of our goals is to have this app be useable and adapted by the community we are willing to do stuff the community asks for
    A little back story. I wrote many additions to survey that allowed you to "export" surveys so people can collect custom data in other applications. Anyways, my additions never took hold, in part because of an understandable lack of time on the part of certain people. Well, I've learned a lot since then and I think that this application would be very useful to a number of people for managing custom fields in other applications. Since I already have the ability to integrate with forms created by acs-templating I have also made it possible to enter custom fields inline in other pages. All you need to do is put these three commands inline with your secondary form (in an application such as events):
    Where form elements are defined by another app.
    
    contacts::forms::get_custom_fields -form "entry" -object_id "1234"
    
    in "is_valid"
    
    contacts::forms::save_custom_fields -form "entry" -object_id "1234"
    
    and if you want after is_valid to enter the default values (i.e. those values that already belong to the user submitting the info)
    
    contacts::forms::get_user_info -form "entry" -object_id "1234" -user_id [ad_conn user_id]
    
    This already works and assumes that object_id "1234" is the object that has a "service contract" with contacts - where the admin defined the fields e.g. "home_phone, home_address, work_phone" as those he/she wanted to collect. If no attributes have been defined for that object it simply doesn't give the form new fields. I think this could very easily become a service contract type thing - since the calling package doesn't need to know anything and its automatically saved for that app in my contacts app. So, here are my questions:
    • if I am to implement this as a service contract how should I have the contracting application choose which custom fields to use. Possible solutions:
      • the contracting app is given a link - and they need to have admin privileges on contacts to add these custom fields to their object this link then gives them a list of all available custom fields and checking the boxes next to those they want to include. They select it this way. I can do this myself and think I know how to set it up.
      • the contracting app has some form of "select custom fields" inline in their application - and contacts the service provider has to provide that application with the 100 or so possibilities and they are selected from within the contracting app. If you think this is a better solution I will need some pointers on how to implement this.
    • is it okay to "force" contracting applications to enter three lines of code on their pages? Most other service contracts can sum up the entire service contract in one small section. If it should be condensed in one section please give me some pointers on how to set it up so that it can only be referenced from one section.
As a small aside question, if have been referencing custom attributes by a attribute_name field that is unique, and have a small questions about performance optimizations. Does using a varchar as the unique key make queries slower than when using an integer as the key?

Also, please let me know if there are any features missing that you think are important. I have been given the go-ahead to make this as generically-useful as possible in order to support the community at large.

I guess my biggest question is, are you still using the underlying packages (i.e. telcom-number, postal-address)?

Contacts-lite was really just a veneer over those base services. Also, it was being used by a very large school district long before OACS had 4.x (running AD 4.x) so it was sort of specific to their needs. The original had no changes to the persons table and no way to link a contact to a registered user. Thus the current state of the package is in flux.

Much time has gone into the underlying services though and the data model is about as international as you are going to get, which is why I am asking about your plan.

i actually started working on it before i knew postal-address exists - so no i'm not using it. However, i did base my datamodel on hr-xml standards for addresses, so it would be trivial to use the postal-address package instead of the one i have built into contacts. I'm just using mine for the time being since i'm used to it, but once i get to a more publishable state i can easily convert this to postal address. one concern i have is that postal address lacks some of the procs i use to speed up data retrieval and storage. maybe when i get close to publishing i can show you what i have and see how we can integrate them with one another. I actually think that the integration of addresses in contacts that i have is much more straightforward than having a seperate package - since persons or parties can be represented by an individual contact it makes user attribute retrieval centralized throughout the system. We can cross that bridge when it gets close to it.

I took a look at telecom and decided we didn't need it, though it would be easy to add. ultimately i decided that a simple text box would be more user freidly for us instead of having multiple boxes just for a phone number.

i built the data model with specific to datatype based storage in mind. what i mean by this is that it is easy to add a new storage type for the storage on things like phone numbers, which can be done via the telecom package or whatever. i mean it trivially easy - so i guess i could say it makes use of both, but i'm currently only using those widgets built into ad_form, and we don't have a telecom widget in ad_form yet so i'm not using it - i though having a number of fields for area code, country code and the phone number would be too confusing for our users (who admittedly are primarily north americans - that is why i made it simple to add a new storage type).

Mathew, I think your contacts-heavy project sounds like a great addition to the community.

However, I must admit I am concerned about the possibility of branching. The project management software will be using Jon's packages, for example, and if you don't make your package build on his work, then the two will be incompatible.

When the time comes, please do talk with Jon about how to move the data models together. I hate seeing more branching and splitting, especially when it seems like both of you have spent so much time to carefully construct the data model.

i do intend to integrate with Jon's stuff at some point... as i  said addresses would easily be done, and telecom i just didn't do because we didn't need the complexity - but as i said, it would be easy (as in half an hour to set it up and 2 to test it)... i have tested my system so i don't want to spend those 5 hours right now - but it would be easy to do when doing clean ups at release time. The imperssion i am under is that Jon's and My work are not at "production" states yet (mine isn't for sure) and i don't want to build a package based on something that might change...
I wonder are there any news about this project? The "Address book" in the CVS is outdated and to-be-removed so we don't have any address book-like package.
Yes, i'm still working on it... i've made it way too customized for a particular project i am working on - it is in production on that already, and working fine. Hopefully I will be able to release a generic one soon, but that isn't a job priority - just a personal priority, and since personal priorities don't make money... anyways, i've started making it generic (i.e. a second re-write) and in the process i am re-doing a fair amount of the logic to make it cleaner (as is always the cases with a re-write). If i had a dedicated week to work on it i could have it cleaned up, but getting that amount of time is difficult. I have speculated about timeframes for completion in the past and have always been wrong, so i won't say anything other than that i've done most of the hard work already and that things have been slowing down at work lately.

If you are interested in helping to work on it let me know...

Why not put the code in /contrib/packages?
i'm not quite ready to put it in contrib/packages because of some naming inconsistencies that would just frustrate people (they frustrate me). once i have cleaned that up i'll gladly put it there...
Hi Matthew,

Are you still working on this?

Thanks

Collapse
11: Any news on this module? (response to 1)
Posted by Gary Roesler on
Can we get the status of this project?
yes i am working on it, it was for a specific application at first but i am now on my second version of a generic one. it will for sure be released by August 1st. I am on vacation right now so i do not have the code in front of me and i will return to work on June 1st... so i sadly will not be able to give access to the code until the first week in june. Since it is pretty specific to my school i am first programming it in a sepcific way (which is where it is at now) and will for sure have it be more generic my August 1st (its going into production for the entire student management system at the end of July so that august 1st deadline is solid). I will gladly respond more thoroughly when i return from vacation. By the way Istanbul and Kiev are lovely and i can highly recommend them to anybody. If you want a fanatastic hotel that is pretty small (15 rooms) in istanbul the "ibrahim pasa oteli" is great  :)
Mathew, I was about to start work on a package based on contacts-lite to do..........well it seems exactly what yours does!!

I was going to hook into the existing underlying stuff that Jon Griffin wrote for the reasons others have stated above.

Is there any possibility that you would accept some help in getting a generic version of your package ready to be put on CVS. I would be delighted to put time towards this objective - and to migrating the data model to Jon's underlying tables.

I have some time next week that I can devote to this.

Regards
Richard

Richard, I'm going to email you off list. I'd be happy to have the help...
I committed Matthew's code to the oacs-5-1 branch just now. It will now show up yet on the remote repository install, because it hasn't been tagged with the openacs-5-1-compat tag.
Hi All,

I downloaded contacts package from cvs ( -r oacs-5-1) and all requirements for it. Required packages and its versions

organizations version="0.3d" postal-address version="0.1d" ref-countries version="0.1d" ref-us-counties version="0.1d" ref-us-states version="0.1d" ref-us-zipcodes version="0.1d" telecom-number version="0.1d" ref-language version="0.1d" ref-itu version="0.1d"

were downloaded with -r openacs-5-1-1-final.

All packages ok in oacs-5-1-1-final. I could add a person contact, archive contacts, filters working ok.

When trying install on dotlrn (-r oacs-5-1) organizations package failed with this error:
Error: Failed to install Organization, version 0.3d. The following error was generated: psql:organizations-plsql-create.sql:28: ERROR: duplicate key violates unique constraint "acs_obj_types_pretty_name_un" CONTEXT: PL/pgSQL function "acs_object_type__create_type" line 37 at SQL statement PL/pgSQL function "inline_0" line 3 at perform
I think this error is because I have Ernie's lors package on my dotlrn install. In its datamodel lors package use same pretty_name "Organization" as organization package use to perform "acs_object_type__create_type"

see: http://cvs.openacs.org/cvs/openacs-4/packages/lors/sql/postgresql/lors-imscp-create.sql?rev=1.1.1.1&only_with_tag=HEAD&view=markup and look for "Organization"

and

http://cvs.openacs.org/cvs/openacs-4/packages/organizations/sql/postgresql/organizations-plsql-create.sql?rev=1.3&only_with_tag=openacs-5-1-1-final&view=markup and look for "Organization"

I do not know if contacts and required packages developers have dotln in mind but i think having contact management for classes and communities seems a good feature and of couse scorm/ims features in lors are good too.

Maybe a change in "Organization" pretty_name for one of these packages? Could you Jade or Jon Griffin take a look and evaluate this issue? If you want you can tell me how i could help.
Orzenil,

Thanks for this bug-report. Could you actually post this in the bug-tracker?

I personally don't use the lors package, so I'm not very motiviated to fix this. But it should be on bug-tracker. I wonder if Ernie would be willing to fix it on his end?

It should be a fairly easy fix.

If nobody else fixes it, I'll probably get to it eventually. But let's first get it in bug-tracker.

Hi Jade,

The bug report about it is https://openacs.org/bugtracker/openacs/bug?bug%5fnumber=2069.

Nima added same bug reporting in https://openacs.org/bugtracker/openacs/bug?bug%5fnumber=2074

Nima was in touch with Ernie who is in charge of LORS and he will modify the conflict about duplicate pretty_name in new version of LORS.

See: https://openacs.org/forums/message-view?message_id=198666

Thanks for your attention, Jade.

Here's where he announces it's available:

https://openacs.org/forums/message-view?message_id=197490

PS Sorry for the profusion of postings on this. I just want to make sure that people who search for contacts find the right spot :)

A breadcrumb for those who follow...

I tried to install Contacts on the HEAD version of OACS and it crashed while trying to install the data model.  I traced the problem to the final inline_1() function in views-create.sql. This function reads the package_id and attempts to initialize some package data with calls to contact__view_name_save.

The code fails because the package_id is not yet available at this point in the install.

A temporary solution, which allowed me to load the package, was to move the final function out of views-create.sql and execute it manually (from the plsql prompt) _after_ the package had been installed using the package manager.

Thanks for the posting Dave. I've created a section in the bug-tracker for contacts, so you can post bugs there in the future. I believe Matthew is working on this bug right now. Thanks for posting it.
Yes i was working on it, that bug is fixed.
Jade: I'll use the bug-tracker for future bug reports.  Thanks for getting that set up.

Matthew: I failed to thank you for making this important code contribution.  Thank you!

Can someone kindly describe the process on how to integrate an existing package to use contacts?