Forum OpenACS Q&A: E-commerce 'category-browse-subcategory.tcl'

I have installed a clean e-commerce from HEAD and have added a  single category, several subcategories and no subsubcategories. I have added a single product to one of the subcategories which is 'In stock', 'available', and set to be displayed on user search.

If I do a product search, or view 'ALL' products as admin, I see the product as expected - no problem.

However, if browsing the subcategory, no product is displayed. The reason appears to be that the following query returns no rows. The reason that query returns no rows is that it left outer joins to ec_user_session_offer_codes which contains no rows.

I am not clear what causes the ec_user_session_offer_codes to be populated but I presume it to be connected with the process of putting things on sale.

The query is:

  <fullquery name="get_regular_product_list">
    <querytext>
      select p.product_id, p.dirname, p.product_name, p.one_line_description, p.sku, o.offer_code
      from $product_map($sub) m, ec_products_searchable p left outer join ec_user_session_offer_codes o on
    (p.product_id = o.product_id and user_session_id = :user_session_id)
      where p.product_id = m.product_id
      and m.${sub}category_id = :${sub}category_id
      $exclude_subproducts
      order by p.product_name limit :how_many offset :start_row
    </querytext>
  </fullquery>

Whilst waiting for comment I will compare with code from a working 4.6.3 system in search of enlightenment! 😊

Regards
Richard

I tried that first of all. Then I connected to the db using psql and issued the queries direct against the tables. With the left outer join to an empty table you get zip back! 😊

R.

However, I still have the problem.

So I have flushed the cache and I still have the problem. Product exists and can be searched and displayed, but the category browse function doesn't work. So this now must be caused by either the 'exclude subproducts' clause or the user session id.

R.

Brian

😉

Some of the category browse code caches other values. I'm guessing that a server restart would fix it..

The category browse pages were originally coded to handle all three category levels on one page.. and were then separated to add more level specific options.

The code on these pages needs to be simplified, new features added, and moved into /lib

I'm making this priority for the weekend.

cheers,

Torben

1) Different product managers could then have admin control over different products and categories of products. This would allow permissions to match organisational product management and sales accountabilities.

2) Categories of product and products themselves could be restricted to specific users and user groups. Perhaps this would be useful for example in a members' or purchasing club where certain products are only available to club members.

3) Built in notification capability through notifications in core. This would make notifying users for any reason (i.e items changing stock status or pre-ordered new products being made available for sale) very straightforward.

4) Better integration with core, better abstraction.

5) Easier to write content provider to site wide search module and would allow us to use sws to replace the ecommerce search.

Regards
Richard

The ecommerce code isn't yet up-to-date with openacs standards, but importantly it *does* work with much of it.

Regarding the general idea of moving code to OpenACS core features, one can apply these disadvantages:

The more core features are used, the less likely that someone without much experience will be able to maintain/fix it when there are problems, and the more likely a de-stabilized problem with another package will affect ecommerce.

A) Use acs-categories for the product categories instead of dedicated data structures.

i) Would deeper category structures degrade performance?

No degradation. My understanding is that the hierarchal representation is "shadowed" into a column for fast referencing etc.

ii) Would ecommerce performance be degraded in systems that grow to very large numbers of acs_objects?

ec_products is already linked to and an extension of acs_objects. Look at the table definitions in ecommerce/sql. Degradation shouldn't be significant, unless one starts adding per product permissions etc.

B) Use list-builder to generate product listings.

This may be fine, is certainly fine if can be completely defined in ecommerce/lib so that it's easy to switch to another includelet if needed.

Benefits:

1) Should be quick to code, consistent presentation and robust.
2) Provides pagination and other features already.
3) Better abstraction and code structure.
4) Benefits from code performance improvements to list builder and core rather than requiring separate optimisation.

Questions/disadvantages:

i) Would performance be acceptable with large datasets?

I believe so. listbuilder used to handle some db related functions in tcl (which was slow), but since about 5.1 uses the db properly to gain the most in performance.

ii) Is display layout flexible enough?

I'm not sure. if css references can be controlled, then yeah. If the css references are fixed, then any product list display changes would likely affect lists all over the affected website.

C) Use AMS for re-code of custom product fields.

Benefits:

1) Automatic form generation for use in re-coded product edit pages.
2) Better abstraction and code structure.
3) Tighter integration with core.

Questions/disadvantages:

i) Performance with large datasets and large numbers of custom fields?

Automatic form generation is fine until you don't want it automatic. I conceptually understand how ad_form works, but have never gotten one to work ..not even the example notes package. I believe that using ad_form could significantly decrease the qualified labor base, should a company want to hire help to make changes. ad_form has it's own meta language space different than tcl, sql, and adp/html.

ad_form has the limitation of requiring no more than one form per page. Ecommerce tends to have multiple forms per page.

====
Yes, the display code needs to be separated from data. These ec_ procs should be deprecated in lieu of using includelets and formatless procs.

I think the data model has reached it's limit, and needs radical changes, which is one of the reasons for starting ecommerce-g2 project ( http://dekka.com/p2/ ). One main limitation is that ec_items does not have a quantity column. ec is currently not suitable for shops that allow/promote bulk quantity sales.

I agree that ecommerce can benefit by using xowiki or implementing xowiki-like processes. See http://dekka.com/p2/xowiki/online-catalog Xowiki css needs to be standards compliant and the css more cross-browser stable. Try changing some of the DIV background colors to see the differences in display between browsers.
I haven't had a chance to play with adp includelets much in Xowiki, but believe it's going to be straight forward once the ecommerce/www pages have been deconstructed into ecommerce/lib.

There already exists some code in ecommerce to use (an early version of) site-wide search. Hopefully it wouldn't take too much to update it to work with the latest.

cheers,

Torben

# TCL PAGE BEGIN - remove me

# Demo of ad_form
ad_page_contract {
    @author Richard Hamilton (mailto:richard.hamilton@webteamworks.com)
    @creation-date 21-12-2008
    @cvs-id@ $Id$
    Demonstration of ad_form calls and multiple forms per page
} { 
} -properties {
    context:onevalue
}

set title [ad_quotehtml "Ricky's little ad_form demo"]
set context [list "Demo"]

# The following condition would tel you if the request is for a new form or if the form already exists - i.e. the
# user wants to edit the data already submitted. This is used when implementing an add/edit page.
#
#if {[template::form is_request form_one] } { }
#
# In contrast, the following condition would tell you if the request is for a valid form that is already defined to
# which one may want to add additional fields based upon conditional statements:
#
#if [template::form is_valid form_one] { }
#
# When any of the forms is submitted (and you can specify as many buttons as you like on each form), by default
# the values will be passed back to this tcl file for processing using the introspective facilities provides by
# template::form::get_button or template::form::get_action
#
# The structure of your add/edit page therefore is basically arranged around three modes controlled by conditional
# statements that direct execution according to the current state of the form accessible using the template::form api.


# OK, here we go....let's declare the forms
template::form create form_one
template::form create form_two

# By the way, if you add and -html {name value} to the end of the above you can specify any required form attributes
# such as javascript references and MIME specs as per oacs documentation on form-procs.tcl

# Check whether we arrived here from a previous form submission or if these are to be new forms
if {[template::form is_valid form_one]} {
    set button_clicked "Form One's '[template::form get_button form_one]' Button"
} elseif {[template::form is_valid form_two]} {
    set button_clicked "Form Two's '[template::form get_button form_two]' Button"
} else {
    
    # The forms have not previously been declared so now declare the form elements

    # Set this variable to avoid an error in the template
    set button_clicked "f"
    

    # Start of elements for form_one

    template::element create form_one form_one_id \
        -widget hidden \
        -datatype number \
        -value 1

    template::element create form_one form_one_field_one \
        -datatype text \
        -label "Field 1" \
        -html { size 10 maxlength 10} \
        -value ""

    template::element create form_one form_one_field_two \
        -widget textarea \
        -datatype text \
        -label "Field 2" \
        -html { rows 5 cols 80 wrap hard } \
        -value "You can specify default values"

    template::element create form_one form_one_field_three \
        -widget checkbox \
        -datatype boolean \
        -label "Field 3" \
        -options { { High t } } \
        -value "f"

    # End of elements for form_one


    # Start of elements for form_two

    template::element create form_two form_two_id \
        -widget hidden \
        -datatype number \
        -value 1

    template::element create form_two form_two_field_one \
        -datatype text \
        -label "Field 1" \
        -html { size 10 maxlength 10} \
        -value ""

     template::element create form_two form_two_field_two \
        -widget textarea \
        -datatype text \
        -label "Field 2" \
        -html { rows 8 cols 100 wrap hard } \
        -value "You can also specify style sheet classes or other html on a per element basis"

    template::element create form_two form_two_field_three \
        -widget checkbox \
        -datatype boolean \
        -label "Field 3" \
        -options { { High t } } \
        -value "f"
        
    # End of elements for form_two

}

ad_return_template

# TCL PAGE END - remove me

# ADP PAGE BEGIN - remove me

<master>
<property name="title">@title@</property>
<property name="context">@context@</property>

<h2>Ricky's little ad_form demo</h2>

<if @button_clicked@ eq "f">
  <h3>Form One</h1>
  <formtemplate id="form_one"></formtemplate>
  <h3>Form Two</h1>
  <formtemplate id="form_two"></formtemplate>
</if>
<else>
  <p>
    The following button was clicked:&nbsp;@button_clicked@
  </p>
  <p>
    To try me again click <a href="ad_form_demo">here</a>
  </p>
  <p>
    The documentation for template::form can be found <a href="/api-doc/proc-view?proc=template::form">here</a>
  </p>
</else>

# ADP PAGE END - remove me

The questions you raise are very good ones, and I wouldn't pretend to have all the answers. My first thoughts though are as follows:

1) My understanding was that one of the benefits of using the API layers is that protection against such basic possibilities would be provided. I am certainly not aware of having had any problems caused by that in the code I have used template::form for.

I can see that with a shop where there is the possibility of placing multiple identical orders, or taking multiple payments, that this is very important indeed.

My, probably naive and complacent, assumption has been that as long as every row that you write from a form to a db has a primary key of some sort, then your protection against this happens where it actually should be happening - inside the RDBMS.

There is an article on Jon Griffin's site that provides an example of ad_form use and mentions that double click protection is built-in by pre-constructing the primary key. Is this just another way of saying what I have just said? It would also be possible to put a select query in the form validation step to look in the target table for the relevant primary key before doing the insert.

http://jongriffin.com/articles/10/

It is quite old and talks about ad_form rather than template::form but may still be useful.

If there is a specific issue that I haven't grasped then please let me know so I can research it.

As far as having to process data in sequential files is concerned, I haven't checked yet, but I'd be astonished if there isn't an option to change the default action to point to another tcl file. Failing that you could do an internal re-direct once you establish that the form has been submitted. In the end, failing all else, for specific cases you could fall back to the old way!

2) Funnily enough, the same Jon Griffin article covers diong this:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
File upload widget

ad_form -name upload_test -html {enctype multipart/form-data} -form {
{upload_file:file {label "Enter a filename or use the browse button"}}
} -on_submit {
set filename [template::util::file::get_property filename $upload_file]
set tmp_filename [template::util::file::get_property tmp_filename
$upload_file]
set mime_type [template::util::file::get_property mime_type $upload_file]
}
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Clearly this needs to be de-constructed and re-coded using template::form semantics but at least the basics are there.

3) I am not sure why this is seen as a problem. I would expect the built-in date widget to serve the purpose. I am not aware of any restriction on the number of date widgets in a form, or on the number of forms on a page.

https://openacs.org/api-doc/proc-view?proc=template::widget::date

4) I should probably break cover here and own up to the fact that I loathe it when lists of things have editable and submittable form fields on them! 😊 YUK!

Dave's suggestion of an Ajax helper is a creative one but I still don't like it. I understand the benefits of minimising clicks, but really, clicking on an edit link in a list is no more effort (or clicks) than clicking in a form field to activate it and give it 'focus'. In any case, on a long list you can end up:

i) with a massive page that is very slow to load

ii) losing track of the fields you've changed as they disappear off the page, and having trouble finding the submit button.

or

iii) if the list is paginated, either having to keep track of pending edits whilst the user pages backwards and forward, or forcing the user to commit edits with one of those horrible 'update cart' links before he can continue paging through the data!

and

iv) where do you stop? Should all fields in all columns be editable? Like a spreadsheet? Over a stateless protocol? This gives your page immense destructive power and makes it very difficult for you to provide a user with a simple and safe way to verify accurately that their intended changes are correctly recorded prior to submission. Your options become quite limited and your user could easily become overloaded and make costly mistakes.

What is wrong with the current design standard where you display a list and provide edit and delete icons on each row for editing them? A simple form is presented which is intuitive, and once dealt with the user is returned to where he came from.

By the way (owning up to vested interest) - I have a 10 year old laptop and am frequently using it on mobile connections, and my browser absolutely hates trying to deal with those mammoth list/form nightmares that I encounter on lots of .asp based websites! 😊

Regards
Richard

If you don't like ad_form the acs-templating forms API is just as powerful just much more verbose.

In fact there are some things the ad_form apis can't do and when that happens you can just apply the acs-templating forms api where needed.

Re:3b

As I understand it, forms created using the forms API by default have their variables passed as form vars rather than url encoded so url surgery is not an issue. The validation and verification services usually provided by ad_page_contract are also taken care of by the forms API.

It should basically take the sweat out of it and reduce the risks, but you're right to want to test everything thoroughly.

Sorry for getting things muddled. I understood that template::form was the recommended way to access the API (either with or without the 'template::' prefix). I had mis-understood the ad_form/ template::form relationship.

I think that template::form code can be laid out to be really easy to read and understand and may therefore have the edge for maintainability. I certainly found it easier to get my head around initially. I'll have a play with the ad_form method though if I understand Dave correctly it can in certain situations be more limiting.

Thanks for the steer Dave.

R.

I am searching for my electronic copy of Roberto Mello's ad_form cheatsheet which I think you'll find very helpful too.

I'll sent it as soon as I find it.

R.

I meant to post a reply to your first point about how to 'pre-generate' a primary key.

In the conditional block:

if {[template::form is_request form_one] } {
**select nextval from your_sequence**
}

Then when you set out the form specification, include a hidden field that contains the key value as the unique identifier for that interaction.

Any double click of the form's buttons will not satisfy the above condition and so will not re-generate the key.

https://openacs.org/api-doc/proc-view?proc=template::form::is_request

Regards
Richard

I thought that ad_sign was to verify that:

1) The form has not been changed (as per DB)
2) That no url surgery has taken place

I didn't think that ad_sign had anything to do with double-click protection.

R.

https://openacs.org/forums/message-view?message_id=156815

Particularly:

"Creating your own layout within the templating system as Don said works like this:

<formtemplate id="myform">
some html stuff ...<br><br><br>
<formwidget id="mywidget1" size="20" ...>
...
</formtemplate>

As soon as the contents of the formtemplate tag are not empty it will use those to render the form instead of rendering it automatically."

I don't know if this is much use to you, but I find it useful to use Javascript to disable a submit/button after clicking. It's hard to double-click a button that's grayed out. Of course, the user may have Javascript disabled, but in this day and age, it's quite unlikely and anyway, they will probably have other problems too. 😊 Of course, it all depends on your accessibility requirements too.

Something like this should do it:

<input type="submit" name="pay" value=" Pay " onclick="this.disabled=true;document.myform.submit() >