Forum OpenACS Q&A: Handling large numbers of users

Collapse
1: Handling large numbers of users
Posted by Reuven Lerner on 10/15/02 03:33 PM
We have a number of OpenACS sites with a large number of users.  For
example, we've bulk-loaded about 20,000 users into one of them.  (Yes,
this took a long time.  Thank goodness for transactions when things
went wrong!)

The problem, of course, is that the pages that grant permission -- or
any other page that asks me to select a user from the list of existing
users -- takes a long time to download.  Indeed, it sometimes takes so
long that my browser times out.  Even if it were to download
immediately, looking through more than 20,000 parties for the one
person to whom you want to grant permission is not my idea of a fun time.

There's already functionality (which is used in several places in
OpenACS) that is designed to prevent precisely this scenario, allowing
you to enter part of someone's name and then choose them from the
resulting list.

Are there any plans to make this (i.e., choosing a last name, rather
than getting a <select> list of all parties) the standard interface
when working with users (or large numbers of users) in the system?
This is an important enough issue for us that we can contribute it
toward 4.7, but I first want to be sure that no one else is working
toward this same goal.

Collapse
2: Response to Handling large numbers of users (response to 1)
Posted by Jeff Davis on 10/15/02 03:43 PM
Lars wrote a user selection widget (used in bug tracker) for doing this
and I think the intent is to address these scaling issues in the admin ui and elsewhere in 4.7 (and possibly backported to 4.6 since
given what;s on the plate for 4.7 the release cycle might be too long for most people).
Collapse
3: Response to Handling large numbers of users (response to 1)
Posted by Yonatan Feldman on 10/15/02 04:23 PM
i like the idea of scalable ui's. for dotlrn i built a scalable ui for user administration. when the number of users is small (less than 50 or so) it simply displays them in a list. between 50 and 500 i add a dimensional bar that allows you to browse by first letter of last name. if there are more than 500 a search box is presented.

of course this adds the overhead of counting all the users for each request to the page, but this can be easily cached.

i'd like to see this type of ui used more in openacs.

Collapse
4: Response to Handling large numbers of users (response to 1)
Posted by Louis Zirkel on 10/15/02 05:15 PM
We experienced this exact problem when we converted our system to ACS Classic 4.2 just over a year ago. The problem stems from the fact that, as you say, the page takes a long time to download. What's worse, the page that we were using presented the list of users in a drop down box. Now how is it that a drop down box makes any sense whatsoever when you're talking even a small number of items by comparison (say even 100 or so).

In the end, I ended up slightly modifying and inserting a new page into the process of granting permission. The page that I added allows the administrator to enter in text to search on from the list that gets generated. I believe it only allows searching in the email address the way I wrote it, but it could be extended to included name as well. Anyway, we've found it to be invaluable in granting permissions.

As a side note, the permission system in ACS Classic 4.2 just sucks, but that's just my personal (not so humble) opinion.

If anyone is interested I would be happy to go back and try and disect the changes that I made and make them available.

Collapse
5: Response to Handling large numbers of users (response to 1)
Posted by Vadim Makarov on 10/21/02 06:11 PM
I'd like to see this type of ui used more in openacs

I just want to voice my vote for moderation here. For people on fast connections, it is often more convenient to have the whole list downloaded and then scroll or local search to the desired item, than to jump through extra forms/pages/links every time just to get to the thing.

The way the content is split on most sites nowadays, makes me think the whole purpose is to show more banners, not make user's life easier. Oh yes, if their designer can't figure how to make a page which is not one big <table>, then it's about the only remedy to show ten posts per page. Don't copy them blindly please.

Collapse
6: Response to Handling large numbers of users (response to 1)
Posted by Reuven Lerner on 10/21/02 07:19 PM
Vadim,

Even when you have a really, really fast connection, having more than a few users in a pull-down list gets very frustrating very quickly.  It takes a heckuva long time to grab 20,000 users (in my particular case) from the database, to say nothing of the time that it takes to send them via HTTP and have the browser format them into a form.  I ran Lynx on a local machine and still had to wait a fair amount of time to get all of the results.  (Perhaps this says more about the overloaded PostgreSQL server at hub.org than anything else, but it wasn't fun regardless.)

So yeah, a fast connection is a big help overall -- but it's no substitute for a good UI.

I wouldn't mind a kernel parameter that lets you specify the maximum number of items that appear in a selection list before it turns into a two-stage selection.  That will allow individual sites to tune the UI to their needs.