Forum OpenACS Development: Contacts 1.0d and AMS 1.0d Todo List

The purpose of this thread is to serve as a community accessible todo list for Contacts 1.0d and AMS 1.0d (attribute management system). Please add your comments and todo tasks here and indicate which of these tasks you will be working on.
Collapse
Posted by Matthew Geddert on

I will work on the following:

  • Create a way to have a standardized default attribute list mapped directly to a new contacts instance
  • Extend advanced search functionality to relationships and provide an mechanism for nested searches (i.e. find people where query 1 and query 2 apply where there is a relationship to an organization in query 3)
  • Enable reporting - this ties directly to advanced searches. I.E. percentage of total people that are in query 1, etc.

Other todo items:

  • Work on permissions and add checks, i.e. who can add people to what group, etc. who can see what attribute
  • Enable deleting contacts
  • Enable merging contacts that are duplicates of eachother
Collapse
Posted by Matthew Geddert on

I will also work on

  • Enabled functionality in Mail Merge that is demoed already in code - including saving canned letters and signatures
  • Enable bulk updating of attributes by admins as a bulk action on the index page - i.e. select all people in a query ( people in the states CA, OR, NV and WA) and map a relationship with a certain sales person, etc.)
Collapse
Posted by Malte Sussdorff on
My todo list for the weekend includes:
  • Setup of a default list for contacts (Hi Matthew :))
  • CSV Import of contact data
Further tasks until middle of June:
  • Mail forward support. If a mail is send to the party_id of a contact in the system, the mail will be forwarded by the system to the contact. Using the enhanced acs-mail-lite storage capabilities (ups, I guess I need to TIP them as this is core, darn....) the mail is stored in the contact history. This allows sending mails through Outlook.
Later Tasks:
  • Support for incoming emails to show up in the contact history as well.
  • Support for Clients of Clients (so you can track the actual end user of your services). Though I think this will just be another relationship type 😊.
Collapse
Posted by Malte Sussdorff on
I finished the default list for contacts using "after-instantiate" callbacks. Now there are two more things on my list:
  • Add a relationship Employee / Employer with additional attributes like
    • Work Phone
    • Work Fax
    • Work E-Mail
    • Position
  • Finish I18N of contacts (it is flaky at best)
Collapse
Posted by Malte Sussdorff on
We are adding a couple of groups that are of general use in a CRM system. The question is whether we add them to contacts or create a new package CRM that adds these groups to the contacts system.

Groups I envision:

- Customers (individual and organizations)
- Clients of Customers
- Suppliers

Relationships:

- Employee/Employer (person/orga)
- Key Account Manager / Key Account (person/orga)
- Holding / Subsidiary (orga/orga)

Collapse
Posted by Matthew Geddert on
I would think that it would make the most sense to hav a way for initializing a group structure much like initializing attributes - all of which can be saved in an init file or in a smiliar way to the way bug-tracker has a configuration for tickets and for bugs - where you choose the setup you want at configuration time. That way the same group names aren't needed for different types of orgs (we are a school and would need "donors", "alumni", etc.) and other org types would need other names. I also think that this simple group structure addition does not warrent a new package (CRM)... unless I'm missing some components that would also be part of CRM.

If the purpose of creating a CRM package is better marketing I'm all for changing the name of contacts to CRM.

Collapse
Posted by Matthew Geddert on
I commented out the callback procs on the packages/contacts/www/contact-add.tcl page since, I can't see why we would have those scripts there, and more importantly it breaks functionality with my install on OpenACS 5.1.5. If these procs provided some needed functionality please let me know what they do and provide a work around for them in OpenACS 5.1.5 before adding them back in to cvs.
Collapse
Posted by Malte Sussdorff on
Could you make a commit with this commented out to 5.1 and treat it as a backport? The reason for the callbacks is fairly simple. We need additional attributes for a contact depending on the installation of the project-manager package and project/open packages. Workaround would be to make contacts depend on the other packages and this is something you'll not want :).

What you can do though is create a dummy procedure "callback" that takes arguments as callback does, but has not code. This could be stored in contacts-init.tcl and wrapped around an if statement that checks if there is already a callback function and only initialize this particular callback function if there is none installed.

Regarding the setups, I will add seperate procedures to contacts in contacts-populate (?) like contact::populate::customers that will create prepopulated lists. We could then write a page in the admin interface that checks all procedures in the contact::populate namespace and have them available as an option to execute.

Collapse
Posted by Matthew Geddert on

I have just uploaded a significant update to cvs. You have to reload the sql/postgresql/contacts-search-create.sql functions and make sure you add the contacts_search_log table from that file and add deleted_p to contacts_searches (or drop and recreate that table). Here is what has changed:

  • The searching code has been significantly cleaned up and standardized throughout the app. We can now search with the proc contact::search_clause in the xql file much like listbuilder, pagination, etc clauses for sql
  • Moving contacts display to lib/contacts has been fixed to allow CSV exports and redirect the user if only one result is returned. This it retains the functionality that had been there before the separation
  • Mapped Groups are no longer displayed in the list of available groups on the index page. Instead from now on you only need to map a group if you need custom form information for that group. We should decide what the best way to deal with groups within contacts is. The heirarchy I had previouly programmed quickly breaks down on sites with complex group structures. We should also have ways of managing what type of membership. Does anybody have suggestions?
  • Users can view other users searches and administrators can make searches "public"
  • public searches replace the previous dropdown on the index page as the "default categories/groups". This has the advantage for an install that needs access to many different groups (such as .LRN) that the admin can sanitize the list. The admin can also put custom searches in there - which is often more helpful than a group by itself. This allows for people to great "all people" and "all organization" searches, but also get as detailed as they want.
  • Below the new public searches on the index page are the users 10 most recently used searches - much like popular office apps show recently opened files. That way if a users uses some other users search frequently they can easily access it.
  • Since the saved searches section lists the number of results from a search and they can be exported to CSV users can create simple reports. They simply created as many advanced searches as they want and they can then export the numbers to a spreadsheet program to make those pie charts everybody loves 😊
Collapse
Posted by Matthew Geddert on
Oh, and also note that I have added a ForceSearchBeforeAdd parameter to the contacts.info file. Please add this manually since as far as I can tell there isn't a way to load parameters from that file.
Collapse
Posted by Dave Bauer on
Mattew,

If you increase the version number, when the APM applies the upgrade it will add the parameter. Also its much easier to add an upgrade SQL script than to manually source all the changes.

Collapse
Posted by Malte Sussdorff on
The change to search and groups caught me off guard, but I managed to write an additional script that is called if there is more than one group mapped when adding a new user. This will allow you to fill in all the details about a user at creation time (and not after clicking on "edit" afterwards).

Furthermore tighter integration with existing packages is under way (e.g. the files section will show the FS Folder Portlet instead of the existing (nice working) file code if a content_folder is linked to the contact).

Furthermore the mail capabilities are on my todo list for today/tomorrow, which should enable you to send the mails along with attachements or a whole folder zipped into one file.

Last but not least I hope to get incoming email working.

Collapse
Posted by Matthew Geddert on
The next thing I am going to work on is enabling bulk updating of attribute by admins and relationship searches. I will likely get this done within a day or two. This will allow admins to do a bulk action on a list and modify an attribute based on "checked off" contacts. Most frequently one would do this by searching for everybody that have certain characteristics and then selecting everybody in the list and updating it. It will also allow advanced searches on how many relationships of a certain type a contact has, and wether or not the related organization is part of another advanced search.
Collapse
Posted by Matthew Geddert on
I bumped up contacts version number to 1.0d2 and added an sql upgrade script, thanks for the recommendation Dave. I also added the DefaultPageSize parameter and dependency on categories.
Collapse
Posted by Malte Sussdorff on
Isn't the version number ordering

1.0d1
1.0d2
1.0d
1.0a1
1.0a2
1.0a

Just curious as I'd have said that the next number would have to be 1.1d1 as the 1.0d "closes" the 1.0 development ordering.

Collapse
Posted by Matthew Geddert on
I know that the apm views 1.0d2 as an "upgrade" from 1.0d so I am assuming that is correct.

I have uploaded the new search api. This will allow people to create contact::search::condition_type::${type} procs that can be used via contacts advanced search functions. Potentially any package could create procs that interface with contacts. If a better way to do this is vai service contracts or callbacks we can - please take a look and see what i've done.

Some of the changes I have made to contacts lately require AMS updates, so please make sure you get that when updating.

Lastly, I am noticing that our site needs a number of customizations to the contacts-ui that would not be necessary for a generic install. Until now I have been checking if the package "tasks" is enabled, which is a site specific package. I think it might be time for us to create a "ContactsMasterTemplate" (not the "S") and "ContactMasterTemplate" parameters so that they can be rest on a site basis. Does this sound like the correct solution to this problem?

Collapse
Posted by Malte Sussdorff on
I learned something new (aka 1.0d is followed by 1.0d2 and so on.), cool.

You'd normally integrate any functionality from third party packages by writing a callback and using application links to mark that this instance of the tasks package feeds data into that instance of contacts. This callback could e.g. set the master-template variable accordingly. Furthermore, if you work with includes, you could create a multirow that is extended by callbacks. The multirow would contain the src of the include and any parameters to be passed along. Though it would be even better if callbacks would work directly in the ADP pages.

This works similar to the idea of Container Items that P/O is using, though the callback approach is more flexible and does not rely on DB statements. You could find a similar concept as well in the portal package(s), though the idea is different.

To sum it up, the method you suggest is the easiest one to implement, though not flexible enough to my liking. What you should do at least is create a callback to set the templates , but maybe I'm just too much into callbacks nowadays :).

Collapse
Posted by Matthew Geddert on
I've uploaded more search conditon types, search bug fixes and search caching. I also changed contacts to make the assumption that all contacts are part of the "Registered Users" (-2) group. Since the code is heavliy coded to depend on (-2), I figured this was fine. This means that the -2 group is auto added for all contacts via www/select-groups and www/contact-add and www/contact-edit and its got its own links on the admin page. This allowed me to add the ability to "delete" contacts (available on the groups page to admins). This marks thems as "deleted" on the '-2' group and means that they won't show up any more.

I have also added parameters for setting the template for a contacts instance. Callbacks might be better, but it doesn't work with openacs 5.1.5 (which I need it to do) so for now the parameters will have to do. Once I don't need to use 5.1.5 and I have the time I can possibly convert it to callbacks.

CSS formatted letter templates and saved form letters are coming in the next day or so... Once I add form letters cr_items I will add sql code, create an uppgrade script and bump up the version numbers a step (which will auto load the new parameters - which have placeholder defaults so they aren't needed right now)

Collapse
Posted by Carl Robert Blesius on
Assuming a "contact" is a registered user does not make sense to me (or maybe I am missing the whole point of this package...).

Maybe you could add a little blurb about what the package is actually for here? http://cvs.openacs.org/cvs/openacs-4/packages/contacts/www/doc/

Collapse
Posted by Richard Hamilton on
Matthew,

I have grave reservations about this assumption that all contacts are registered users. In fact I would go as far as to say that if we continue to develop down the road of ERP this will block progress.

Also, Claudio's existing work on ERP may prove to be incompatible with this assumption.

Can I please pursuade you to look for another convenient means of handling 'delete'.

If I can't pursuade you, and we use the 'groups' functionality for this, instead of using the existing registered users group would it not make sense to do one of the following instead:

Option1)
Have a group called 'deleted' and assign contacts to these. (Seems like a bit of a cludge though since the contact's state is an attribute of the contact).

Option2)
Set up a group per contacts instance and assign contacts to the group associated with the package_id.

Or why not go a different route and simply always add by default a deleted_p attribute to every contact. I don't know whether you can conveniently do this using any 'typed' attributes or whether it is easier to auto-adding deleted_p in the user defined ams 'record' definition for contacts.

It seems to me that the debate really needs to be expanded to cover how we should store object metadata in applications that use ams.

Regards
Richard

Collapse
Posted by Matthew Geddert on
I don't have much time right now... so I won't go into as much detail as I could... But the way contacts is dealing with groups is that all parties are part of the group registered users (-2), this can be a person who is not a user or an organization (who obviously cannot log in) or a user who can log in. When you add a person you can specifiy whether or not to create a user or just person object. So contacts treats the registered users group as a list of all registered contacts. I know this doesn't work great in theory, and in practice it doesn't allow for private contact lists (which we should allow in the long run) - both of which are not problems for Me or Malte (as I understand it). I'm under time constraints and this was the hack that worked for what we were doing. When I get the time (or the need to do it), or if somebody volunteers I would be happy to work out how to fix this. It shouldn't be too hard to fix.

My comment above simply stated that because of the way contacts has been set up currently the changes I made will not affect current contacts setups in practice. This issue will be resolved before contacts leaves the development stage (probably with an application group per contacts instance or something similar).

Collapse
Posted by Richard Hamilton on
Matthew,

Thanks for the explanation. I can see why this is not a trivial decision. I think my discomfort stems from something that is neatly summed up by a statement from the previous posting:

"all parties are part of the group registered users (-2), this can be a person who is not a user"

The idea that a party that is not a registered user can be part of the group 'registered users' seems undesirable to me.

In fact if you consider the case where a party is a group of groups, there may not be any registered users amongst them. I think that if contacts treats the party objects in this way then there is a real chance that in the future a single set of relationships may produce different behaviour in different packages.

Regards
Richard

Collapse
Posted by Matthew Geddert on
Please take a look at the following thread where I discuss deleting contacts... once thats resolved we can deal with the registerded users group issue:

https://openacs.org/forums/message-view?message_id=302396

I have updated contacts to version 1.0d3 please upgrade via apm (I have included the sql upgrade scripts). You can now save messages (letters and emails) and view/use other users saved messages and perform printed letter mail merges. Printed letter mail merging is done with CSS and only tested on Firefox on Windows. If you select letterhead it will indent the left side (since our schools letterhead has information on the left margin - if you need to change this you can edit www/resources/contacts-print.css). In the long run to print consistently no matter what browser is used we will likely need to convert html/xml/xhtml or whatever to pdf... any volunteers? 😊

What I'll be working on now:

  • General usability bug fixes, page flow, etc.
  • Allowing any attribute to be used in a mail merge (right now only first_names, last_name, and name work right).
  • Allowing person objects to be "upgraded" to user objects
  • Figuring out how to not make sure every contacts needs to be in the group -2 (registered users) and implementing that solution
Collapse
Posted by Malte Sussdorff on
I had a look at the new functionality. Where can I see the list of the logged messages? Furthermore, why are you using your own logging table instead of extending the mail-tracking package by content type? Now we have the messages stored in contacts and the mail-tracking package and we need to keep two different lists, one for all mails send using the mail-tracking package (which is everything using acs-mail-lite) and one for contacts, though it would be considerably easier to extend mail-tracking to track also "letters". Maybe we should have it renamed to "communication tracking". Unless there is a compelling reason to keep two different tracking mechanisms, we will change the contact::message::log function to store the logs in mail-tracking and get rid of the special tables in the contacts package.

Furthermore we will work on templates with header and footer for letters, storing the result as a PDF file. Christian is going to work on this from our side in the next week or so.

As for the default group, maybe we can just create a new "contacts" group for all contacts and add them to "-2" when upgrading a person to a user.

Last but not least, I think the "any attribute in mail merge" could be solved once we a procedure in AMS that allows us to retrieve the attribute value if given an attribute name. Then it should be a piece of cake (I'd assume).

Collapse
Posted by Matthew Geddert on

Our users wanted and I have a created a custom history tab for a contact. This page has all the comments, attribute changes, messages sent, tasks completed, etc in chronological order with a consistent time and users stamp. It looks much like the comments page but includes much more info and intersperses comments with other information - its one place to look to see how a contact has related to our organization. I will attempt to generalize this page and include it with the cvs.openacs.org install of contacts in the next few days. this page has the view of contact message log info.

I don't mind mail-tracking being called mail-tracking. The reason I added the table seperately is threefold:

  1. That table mimics the saved contact messages in cr_items which allows me to save the title, description (i.e. subject in an email) and content (i.e. body in the email). This lets me use a institutional name such as "Admissions: introductory letter for international students" that our staff uses in the history page and have a link to the actual message sent. A staff member knows that the contents of that form letter is and doesn't have to read the specfics unless they want to. So, if we were to use mail-tracking only I would want a title attribute that can be used for this purpose
  2. Mail-tracking logs all login messages for users, forum notifications, etc. i.e. way too much info for us where many of our contacts will also be users (and dotLRN users at that) all those messages being logged will significantly decrease the usefulness of mail tracking for us since there will be a lot of noise. All our staff cares about is the official notices sent to contacts and those will all be sent via the messages interface in contacts. I can see why some people would want everything logged, we just don't
  3. I was in a big hurry and didn't look into customizing mail-tracking to fit our needs since creating one table and one proc to add info to that table was much faster than negotiating how to make mail-tracking fit our needs when i was preparing to show my progress to some other collegues

I think there is a place for mail-tracking and a place for the contacts mail log, but not both simultaniously. So maybe we should have a parameter that sets how much logging of messages should be done and selecting the appropriate place? Or is there a way in which mail tracking could be used for our needs?

As far as default groups go... I agree that an application group for the contacts instance is the way to go for now. In the long run we will want to be able to allow users or groups to have their own contact lists that aren't shared part of an instances default list. But having an application group for now should do it. I guess well just need a sweeper proc that runs every 60 seconds or so and finds any new users/organizations and adds them to the ocntacts instance.

Lastly, the "any attribute mail merge" would work easily the way you suggest but not quickly when doing a 500 letter mailing (which our school needs to be able to do). My issue is in figuring out a way to do it quickly with large mailings... without requiring a trip to the database per attribute per party_id. Although I agree that it shouldn't be too hard... i should have that working within the next day or so.

Collapse
Posted by Malte Sussdorff on
Ah, now I see the difference and yes, your system makes a lot of sense. We just keep mail-tracking around as some email messages are generated through other packages and you might want to make sure you will find out which mail was send to what user.

Furthermore, you can make good use of mail tracking within contacts by limiting the mails that are displayed in the mail tracking system to only come from the contacts package_id (filter by package_id). I guess the functionality is not there yet, but I will write it then :). This way you won't have to store the mails in your system as well.

Collapse
Posted by Matthew Geddert on
I've uploaded contact 1.0d4 with sql upgrade scripts. This removes the dependence on the registered users group (-2) and instead uses an application group as its default group. Its a pretty tricky upgrade so i hope the scripts work for your installs (they worked on the two different data sets I use contacts with).

I also:

- modified lib/history to make use of callbacks so other packages can provide history information
- modified lib/contacts.tcl bulk actions to make use of callbacks (I for example use a callback to allow users to bulk add tasks to contacts, etc.)
- did some bug fixes