Forum OpenACS Development: HEAD install.xml feature is broken

Collapse
Posted by Randy O'Meara on
File refs are relative to packages/acs-bootstrap-installer/installer. This report applies to HEAD CVS of 9/12/03, 5 PM Pacific Time.

For some reason the install.xml tDOM objects are not persistent. The in-memory tDOM objects (actually tcl commands) are not surviving between the time they are created (in index.tcl) and the time they are used (in packages-install.tcl. See the debug trace log entries below.

Is anyone aware of this problem? Has it been fixed but not applied to HEAD>

index.tcl: after parsing install.xml

RMO: root_node='domNode0x89ce220' (name='application')
RMO: xml_node_get_children_by_name: calling '['domNode0x89ce220' child all 'actions']
RMO: xml_node_get_children_by_name: return 'domNode0x8915430'
RMO: actions='domNode0x8915430'
packages-install.tcl: retrieving persistent DOM
Core packages instantiated and mounted
RMO: get install.xml actions
RMO: xml_node_get_children_by_name: calling '['domNode0x89ce220' child all 'actions']
Error: Error sourcing /devel/openacs5/packages/acs-bootstrap-installer/installer/packages-install.tcl:
invalid command name "domNode0x89ce220"
    while executing
"$parent_node child all $name"
    (procedure "xml_node_get_children_by_name" line 14)
    invoked from within
"xml_node_get_children_by_name [nsv_get acs_application node] actions"
    invoked from within
"if { ![ad_acs_admin_node] } {
    ns_write "  

  • Completing Install sequence by mounting the main site and other core packages.

    <blockquot..." (file "/devel/openacs5/packages/acs-bootstrap-installer/installer/packages-install.tcl" line 44) invoked from within "source $__file "

  • Collapse
    Posted by Bart Teeuwisse on
    Randy,

    without having had the chance to look at the creation of the DOM tree for install.xml, my guess is that tDOM didn't create a persistent DOM.

    If the tDOM parses xml like so:

        dom parse $xml doc
        $doc documentElement root

    then the DOM tree and the associated Tcl command object (document and all node objects) are freed automatically when doc gets freed. Which happens at the end of index.tcl.

    Use the following form instead.

        set doc [dom parse $xml]
        set root [$doc documentElement]

    parses the XML in the variable xml, creates the DOM tree in memory, make a reference to the document object, visible in Tcl as a document object command, and returns this new object name, which is then stored in doc. To free the underlying DOM tree and the associative Tcl object commands (document + nodes + fragment nodes) the document object command has to be explicitly deleted by:

      $doc delete

    or

      rename $doc ""

    Using this form you can pass doc around, but don't forget to delete doc or you'll create a memory leak.

    /Bart

    Collapse
    Posted by Peter Marklund on
    Randy,
    I've had that error message a couple of times too. The funny thing is that for me, sometimes it shows up, sometimes it doesn't. I haven't investigated the problem further.
    Collapse
    Posted by Randy O'Meara on
    Bart:

    Thanks for your insight. I just looked through the code again and I believe it's doing the right thing. Even so, I changed it to do *exactly* what you prescribed above. Same problem.

    Peter:

    Have you seen the tDOM processing of install.xml work properly on HEAD? If so, can you recall when it last worked? I'm wondering if it has ever worked since the port to tDOM. If so, then I can focus on changes in this area of code since it was last known to work. I don't see a problem with the way it's coded, but the objects are definitely not persisting between pages.

    I have verified that there is no aolserver restart happening between the time the objects are created and when they are used, so I see no reason for them to have disappeared.

    Could this have something to do with the version of tDOM? My version is 0.7.7.

    Randy

    Collapse
    Posted by Peter Alberer on
    Hi,

    i think you must keep in mind that tdom persistent objects are just available in the thread in that they were created. If the different install tcl files are served by different threads, the relevant tdom objects will not be available. The best way to go is to delete every doc at the end of one page and recreate it in the next tcl page from the underlying xml doc. Otherwise you have tdom objects lying around that you maybe cannot use in the "next" page and that just need memory.

    Collapse
    Posted by Randy O'Meara on
    I haven't really had reason to delve into aolserver cross-thread caching (or copying) as yet, so I'm not familiar with the techniques to do so.

    Is there a way to make the objects available across pages and threads? I see that it would be a fairly straightforward fix in this instance to just re-parse the install.xml file again in the referencing page. But how about in an instance where we really do need them to persist?

    Collapse
    Posted by Randy O'Meara on
    Peter Alberer hit the nail directly on its head. tDOM persistent objects are not visible across aolserver threads. This is a difference between the previous XML support and tDOM that probably has greater impact than this one instance.

    Anyhow, thanks to everyone's advice (particularly Peter A.), I've tracked the issue down and submitted a bug with corresponding patch.

    Would someone with commit rights please apply this patch right away?

    Thank you,

    Randy

    Collapse
    Posted by Peter Alberer on
    As far as aolserver cross-thread caching (or copying) for tdom objects is concerned, i must admit i cannot really image a case where this could really be useful. Parsing an xml-document with tdom is so fast, it usually takes only milliseconds, and performance-wise keeping the tdom document around simply does not pay.

    But i must admit that my use-cases for xml-processing were mostly based on xml import and export of certain resources, so that the passing of tdom objects between pages was not necessary.

    If you want to make up an xml document by integrating information from several html pages (wizard-style) and have an up-to-date tdom-document of the info around, i think the best way is to:
    -create the xml doc on the first page
    -for every user-input page
    --read user info
    --add user info to xml document
    --write document to xml [$root_node asXML]
    --cache this xml-source in memory
    -on the next page recreate the xml-document from memory
    -on the last page remove the xml-document from the cache

    In my opinion, the only thing you could do to make sure that tdom commands are available to any aolserver thread is to create those documents in a special thread, dedicated to this task and interact with this thread from the request threads to get the tdom commands.

    One thing i can recommed when dealing with complex xml-documents is to use xotcl. This object-oriented tcl add-on can be used within aolserver and makes processing of large xml-documents with deep hierarchies very easy. Objects can be aggregated (nested) and therefor you can keep the hierarchy of the document and deal with the data in memory.

    The page that is most heavily used in my system shows multiplechoice questions to students (self-assessment). For that page i have to parse the xml-representation of the question, do some calculations with the data in it and read info from the database as well. I can easily save all that information in an xotcl-object-hierarchy, serialize that to tcl code, cache the tcl-code and recreate the whole hierarchy for the next access to the same question. No need to keep the tdom objects around :)

    Collapse
    Posted by Randy O'Meara on
    Peter,

    Thank you for your excellent insight, thoughtful responses, and assistance in increasing my knowledge of tdom and its use.

    Randy

    Collapse
    Posted by Peter Marklund on
    Seems the dom nodes are procs, and as we know, procs loaded in a certain Tcl thread are only available within thread. I avoided the error during bootstrapping by simply loading the xml doc again on the package page. The fix is committed to CVS HEAD.
    Collapse
    Posted by Randy O'Meara on
    Thank you, Peter.
    Collapse
    Posted by Don Baccus on
    Thanks for the head's up and fix for this, folks, I was unaware that persistency of DOM objects was different for ns_xml and tDOM.

    I hadn't ran into the problem myself when install .LRN, just luck, I guess.