Forum OpenACS Q&A: Reactivating ACS-Workflow 4.5 (Petri Nets)

Hi,

just to let you know: We are going to reactivate Lars Pind's ACS-Workflow 4.5 package for use in ]project-open[. Anybody out there using it except for the Quest guys? (Hi Michael and Brian!)

We have already a version working on OpenACS 5.1.5 and we'll soon start to develop an integration package to wrap ACS-Workflow functionality and to integrate it with ]project-open[ (buttons to advance the workflow, the task list, ...).

We are going to keep the package independent from ]project-open[ and put all ]po[ specific functionality in extension modules, in order to facilitate sharing the code with other community members.

However, we're going to introduce a few small changes:

- We are going to modify the assignment functionality so that it works with groups instead of users. There are just too many users in our system.

- We are going to update the GUI of some pages

- We are probably going to extend the WF with form pages that allow to modify an object based on the "acs_attributes" SQL metadata system. We are using "FlexBase" alias "DynFields" alias "AMS_with_database_tables" to handle extensions of OpenACS tables at runtime. (Michael and Brian, this corresponds to your SmartForms package...)

I have read the statement from Lars that this workflow would be too complex for most of the tasks that you want to do in OpenACS (publish articles, receive payment, ...). I agree with him.

However, we are actually going to use the expressive power of Petri Nets in order to allow customers to define sequences of actions on projects, tasks, employees, invoices and other objects in ]project-open[.

If you're interested in the code: Our branch of ACS-Workflow is available via anonymous CVS via: (the password for "anonymous" is empty).

cvs -d :pserver:mailto:anonynous@berlin.dnsalias.com:/home/cvsroot checkout acs-workflow

Bests,
Frank

mailto:frank_dot_bergmann_at_project_dash_open_dot_com
http://www.project-open.com/

Collapse
Posted by Richard Hamilton on
Thank you for posting the link Frank. I have been wondering about the applicability of the workflow API to a number of desireable future developments in OpenACS. It is nice to see some development going into it that can be rolled back into the toolkit.

I don't know if anyone is actively working on workflow related things at the moment but I know that in the fullness of time I myself will be looking at how the workflow facilities might be able to help us with some new functionality.

Regards
Richard

Collapse
Posted by Frank Bergmann on
Hi Richard,

as Lars said: It's overkill for most of the issues you might encounter. And the Petri-Net-WF is largely based on PlPgSQL, so it's not following the current "callback" coding style at all.

BUT: It's a _real_ workflow.

The new "workflow" module is really a plain state machine, implementing the GoF "State Pattern". "Toolkit" instead of "Solution". Makes a lot of sense, because it's more lightweight, but sometimes you need a real WF, like when coordinating projects in large groups.

Bests,
Frank

Collapse
Posted by Malte Sussdorff on
We have had great success in using the "normal" Workflow system together with tasks generation in Project Manager, though it was painful and does not feature a GUI. So Frank's approach, once useable, might be the way to go in the future.
Collapse
Posted by Don Baccus on
A strictly correct petri net that always eats all tokens is often very, very difficult to construct. Your customers aren't going to be able to do so unless they're familiar with Petri nets, and very few people are. Getting to a clean termination state with no unconsumed tokens floating about the net is very difficult once you start considering exceptions and error handling in your workflow.

Do you have an example of a workflow that customer would want to define on a project, task or employee that an FSM won't handle?

I do miss the GUI and with the old workflow I actually figured out how to hack template pages that could either be displayed by the old workflow's central "tasks" pages as well as stand-alone in the client package's pages (which sorta amazed Lars) ... these are useful things that the current workflow misses. Also the new workflow has some performance issues.

Overall I kinda wished Lars had just written an FSM engine for the old workflow package rather than dump it for a less capable FSM-based workflow package.

In fact my original argument was for the OPTION to use a simpler FSM, not to toss Petri nets overboard entirely. In other words to give the workflow designer the choice.

But having said that, I've not run across a situation where I've actually need the more expressive power of a Petri net. I've seen contrived examples in the paper(s) Lars originally read, but nothing in the real world.

Collapse
Posted by Frank Bergmann on
Hi Don,

Yes, there are cases. Basicly, any split or join requires either "spawning subworkflows" or multiple tokens.

]project-translation[ is going to implement several workflows in the area of software localization where several tasks have to be processed in parallel.

And yes, it's going to be very important to gather information during the course of such a workflow. I don't know what you did with "template pages", but I know that Quest.ie has adapted the Simple Surveys to a kind of forms, and we're going to use "DynFields" (AMS integrated with acs_attributes, it's now working pretty nicely).

But you are right, normal customers won't ever be able to implement a new workflow themselves (unless they'd dive in very deeply into the architecture). But that's not an issue, it may actually be an interesting feature to use, because it profides us with consulting revenues.

Do you have any links/code for your "template page" hack?

Bests,
Frank

mailto:frank_dot_bergmann_at_project_dash_open_dot_com
http://www.project-open.com/

Collapse
Posted by Don Baccus on
OK, if you're developing workflows for customers, then petri nets are more reasonable. That wasn't clear from your first post.

I still think that having two engines, one FSM and the petri net engine, would be vastly more useful than having the petri net engine alone.

Parallel workflows that join later are fine ... until you try to incorporate exceptions into it. Something like a "cancel" operation can complicate the net to a great degree. Of course, you can code such things outside the workflow but then you're defeating the purpose of having the application unaware of the internals of the workflow being modelled ...

Assuming you want the workflow to be a proper petri net.

Collapse
Posted by Frank Bergmann on
Hi Don,

Assuming you want the workflow to be a proper petri net.

No, that doesn't really matter to us, except if there would be a real penalty of violating this condition...

Maybe you'd looking for the WF-GUI, put around the new "state machine" package (I don't want to call it workflow, even if that's the package name...).

I think we all should have a closer look at Lotus Notes and it's scripting capabilities and may Oracle WF or something similar. Ease of use if first, technology is second...

Frank

Collapse
Posted by Malte Sussdorff on
I totally second Frank here. If we could manage to get a decent UI around the Workflow package supporting both Petri Nets and a State machine this would be the optimum. Questions is, can this be done and who has got the time to do it?

Furthermore, we can't call the petri net based package workflow now, as we already have a workflow package upon which some packages rely.

Collapse
Posted by Don Baccus on
No, that doesn't really matter to us, except if there would be a real penalty of violating this condition...

In essence you can't tell if your Petri net is full of bugs or full of features ... :)

I think we all should have a closer look at Lotus Notes and it's scripting capabilities and may Oracle WF or something similar.

I'm not familiar with either.

Ease of use if first, technology is second...

Heh, which is why I like FSMs rather than Petri nets! But yes, I'm aware that Petri nets are more expressive.

Collapse
Posted by Ciaran De Buitlear on
Hi Frank,
As you know we have been using ACS-Workflow 4.5 (Petri Nets) with ticket tracker for the last few years with, for a time, your able assistance. Anyway to summarise some broad elements of workflow/ticket related functionality where were have done work which may be of benefit to others:

Hierarchical workflow
look and feel and usability of the main ticket list screen and the task handling screen /wf/task
Assignment
Relationships between tickets
Deadlines

Perhaps we could have a chat about some of the details and if and how some of these features could be of general use.

There's a few things we'd like to do in the fairly near future, mainly along the lines of making workflow configuration more user friendly and less error prone. This very much at concept stage at the moment but we expect to get moving in the coming weeks. Hopefully we can co-ordinate our efforts for the common good!

Collapse
Posted by Malte Sussdorff on
Ciaran, this sounds like a good approach. Before the chat it would be helpful if you could provide a demo of what you achieved with the petri based workflows. I can talk a little bit about what we did with the current workflow package and hopefully Frank and others can chip in with their ideas (e.g Jarkko with his knowledge of Simulation).

Once we have a broader understanding what the various needs are and where we are using the various technologies, hopefully we could end up with a work plan on how to integrate all these ideas into a core workflow system that supports both approaches along with guidelines on how to upgrade your existing packages to make use of this new workflow package.

This will also have the additional benefit that we will get a good description of the OpenACS Workflow capabiltities.

As Frank mentioned dynfields, are there any plans to commit this to the toolkit (I know we wanted to do it but at the time when I looked at it I did not have the courage to go through the code and check it out utterly as it proved to be a bigger hassle to integrate with Contacts instead of AMS than Frank first let on). Furthermore, as he is most knowledgable, he should commit this and maintain it out of the OpenACS tree (taking into account that this is GPL, something I'm not always sure of when dealing with P/O, i get confused from time to time).

Collapse
Posted by Frank Bergmann on
Hi,

Ciaran, I didn't know whether I would be able to mention Quest and my past involvment with you, so I've been holding back a bit...

I think it would be a very good idea to setup a common codebase with your enhacements and some future extensions. However, I see one important issue that the Quest WF contains some customizations that wouldn't be compatible with a general purpose WF.

Also, ]po[ would have to use the WF attached to several object types in addition to "tickets", including im_project, im_invoice and im_company.

So one of the most important next steps would be to identify the common functionality, "factoring it out", putting it into a clean package and converting all extensions into extension packages. I know that this can be quite a painful process once you've got some customers running the code in production use...

How do you see this?

Bests,
Frank

Collapse
Posted by Ciaran De Buitlear on
Hi,
Just talking internally here about how to proceed. Will respond soon. Sorry about the delay.
Collapse
Posted by Andrew Piskorski on
In addition to wftk which Lars found non-useful back in 2001 or so, there is also PGFlows, "an entreprise level generic workflow system written in plpgsql with users management(A PHP API is also provided)." PGFlows is listed as in "Planning", presumably meaning that nobody has written any actual code for it at all. But anyone interested in it might be even more interested in the much more mature OpenACS Workflow packages...
Collapse
Posted by Frank Bergmann on
Hi Andrew,

Thanks for the pointers. Just had a quick look at both projects. Both are announced as "developer components". I understand that non of them is integrated yet with OpenACS, is that right? (Atleast I didn't find any links to our beloved platform...) The SOAP interface of wftk sounds interesting though...

But let's put things into perspective. ACS-Workflow 4.5 is a mature and stable package, well integrated with the rest of OpenACS and proven in "industrial" environments. Quest.ie is running it in several _huge_ projects and we (at http://www.project-open.com/) have actually started to sell workflow configurations to our customers. Any new WF engine would have to compete with this.

ACS-Workflow's main disadvantage for our customers is that you can't (yet?) use Microsoft Visio to design the workflows and that scripting it is quite complicated. And these are the points that are going to become painful for us when the new MS workflow hits the street.

Any ideas on this end? Maybe we should look for open-source workflow design packages that could be integrated with either ACS-Workflow 4.5 or Workflow 5.x?

Bests,
Frank

Collapse
Posted by Andrew Piskorski on
I meant more like, "Heh, since OpenACS Workflow appears to be ahead of these other open-source workflow projects, maybe we can evangelize OpenACS Workflow to people involved or interested in them, and get more people contributing to OpenACS." :)

The SMC State Machine Compiler stuff might also be related.

Collapse
Posted by Frank Bergmann on
Hi Andrew,

Ok, ok, got it ... 😊

I fear that both OpenACS workflows are too integrated in the platform to be useful outside. It's an advantage and disadvantage... Did you seem my slides about component architecture and Library Design?
http://www.project-open.com/whitepapers/Project-Open-Component.Architecture.ppt
It's just tow completely different things to create a reusable component (desirable) and an integrated solution (also desirable) ...

Just checked out the state machine compiler. Nice. Converts a state machine definition into code, based on the GoF "state pattern". However, we'd need more an equivalent of http://ganttproject.sourceforge.net/. GanttProject is a Java frontend for a project scheduling that could (will!) be integrated (some day...) into ]project-open[ (and maybe also project-manager).

But actually the GUI may not be as much of a problem than providing SysAdmins with a seamless integration of WF, scripting language (=> Lotus Script, ...) and OpenACS/ ]project-open[ environment. I don't know if you've seen how routing decisions are programmed in the WF those days. Extremely cryptic and non-transparent: You need to introduce a new variable on the WF-object-table and then you can route depending on the value of that variable.

Instead of this we'd probably need a TCL interface providing the state variables of the WF as TCL variables, some type of debugging support etc.

Well, yes, and a graphical environment to manage all that. Maybe a bit utopical...

Frank

Collapse
Posted by Nima Mazloumi on
Hi,

have you guys checked this:

- Business Process Execution Language for Web Services provides a means to formally specify business processes and interaction protocols
BPEL4WS V1.1 specification ftp://www6.software.ibm.com/software/developer/library/ws-bpel.pdf
- Business Process Modelling Language and Notation by the Business Process Management Initiative (BPMI)
Specification: http://www.bpmi.org/bpml-spec.htm
Notation: http://www.bpmi.org/bpmn-spec.htm

These are the two de-facto Standards in modelling processes.

Greetings,
Nima

Collapse
Posted by Frank Bergmann on
Hi Nima,

Have you guys checked this:

- Business Process Execution Language [...]
- Business Process Modelling Language and Notation [...]

I've just given a workshop about workflow techniques in the "L10n World" conference in Seattle. In this tutorial workshop I have actually used the BPMN to explain all examples: http://www.project-open.com/whitepapers/L10nW-Workflow.051022.ppt

However, BPEL really is a language for synchronizing SOA type machine-to-machine workflow. It has not that much been designed to handle human interaction.

And what we're now considering the main issue isn't really touched by all of these languages: How to integrate the interaction between the Workflow and the SQL data model. To do this you need forms, business logic and (TCL or PlPgSQL) scripts, something which is not part of BPEL.

Bests,
Frank

Collapse
Posted by Nima Mazloumi on
I found some other activities that deal with workflow in web applications:

- WebML
- jBPM
- WfMC specification XPDL
- Spring Webflow (a finite state machine)

I looked at jBPM. I also provides user interfaces and database persistence.

Collapse
22: Pi-Calculus for Workflow? (response to 1)
Posted by Andrew Piskorski on
I just stumbled across a mildly interesting article, A Conversation with Steve Ross-Talbot "Merging the Worlds of Academics and Practitioners", ACM Queue vol. 4, no. 2 - March 2006. In it, among other things, Ross-Talbot claims that for practical use the Pi-Calculus (search on lambda-the-ultimate.org) is clearly superior to Petri net theory. Apparently, the Pi-Calculus allows callbacks to change the dependency graph on the fly, while Petri Nets must be static.

He seems to say that from that it follows (he doesn't say why) that the multiple thingees using the Pi-Calculus are composable while multiple Petri Nets are not. If accurate, that's interesting, as composability is a big advantage in practical robustness... (For comparison, mutex locks are not composable but transactions are, which is one of the many advantages of transactions.)

Apparently his startup released and maintains their pi4soa "Pi Calculus for Service Oriented Architectures" toolkit under an Open Source (Apache) license.

Collapse
Posted by Frank Bergmann on
Hi Andrew,

The most frequent critic of the Petri Net based WF 4.5 is that it's too complex and that the power of Petri Nets is not needed for most real-world applications. So I guess that the PI-Calculus would not really improve this situation...

My personal feeling is that the majority of the WF 4.5 complexity comes from all the things around the actual Petri Net such as roles & assignments, "panels" and the custom PlPgSQL code that you need to implement custom actions.

So the situation should actually get not much worse with PI-Calculus... 😊

Bests,
Frank

Collapse
Posted by Nima Mazloumi on
Hi,
I am just wondering what the current status on workflow is at OpenACS. Which packages exist and where can I learn more on how to use them.

Thanks,
Nima

Collapse
Posted by Frank Bergmann on
Hi Nima,

There is a - more or less - working version of the "acs-workflow" package (4.5 Petri nets) in our CVS. You can access the content using:

cvs -d :pserver:mailto:anonymous@berlin.dnsalias.com:/home/cvsroot checkout -r b3-0-0-patches acs-workflow

The differences to the stuff in the OpenACS repository are our fixed of the changed ADP conventions (adding ";noquote" to many fields). Also, I think I've fixed one or two real bugs. I believe there are no ]project-open[ stuff in it.

Our roadmap for acs-workflow is to keep it completely independent from ]project-open[. We are using the ]po[ component architecture to create extension screens in extension packages.

Cheers,
Frank

Collapse
Posted by Andrew Piskorski on
Hm, I was recently reading about object-capability systems and the like (e.g., Shapiro's Coyotos, Miller's E), and learned of the existence of the "Join Calculus". It is, reportedly, provably equivalent to the Pi Calculus, but perhaps (reputedly?) better for actual implementations, particularly those involving distributed processing (a la Erlang). (I don't actually understand it, though.)
Collapse
Posted by Frank Bergmann on
Hi Andrew,

Do you remember Lars' initial rambling against the Petri-Net workflow and why he went for a state machine, and actually reduced the workflow the "State Pattern" (GoF, Gang of Five)? He said that very few people would actually use the features of branching/joining and other fancy Petri-Net stuff that makes it touring-complete.

We're reactivated the "Petri-Net Workflow" because of all the support pages, defaults, transition panels, role assignments etc., but not because the underlying formalism is Petri Nets... We've already written some WFs for customers, but all WFs could have been implemented using State Machines (the least expressive type of all of these networks).

Btw., the WF is actually running and available now as part of Version V3.2.beta1. You can check our SourceForge site for download in the "Support Files" release set (not the main V3.1 set). The download also includes a database dump of a working demo system.

perhaps (reputedly?) better for actual implementations,
particularly those involving distributed processing
(a la Erlang). (I don't actually understand it, though.)

These networks are frequently used by academics to proove strange mathematical claims on areas ranging from parallel processing to Business Process Engineering. However, prooving stuff is really different from designing user-friendly workflows...

Actually, what we'd need is something like a Workflow Designer's Guide that explains the possibilities and difficulties with WFs, and allows people to see where using WFs is useful and where not. Quest.ie (Ciaran De Buitlear, Michael Hinds and others) have actually started with such a document for their customers, but I'm not sure if it available publicly.

I had some very interesting conversations with Mervyn Colton some time ago about how to map a complex process to a WF and why this is an art and not engineering. But yes, this is the right vocabulary. Lars' Petri-Net WF is about mapping business processes to IT systems, and you'd have to start by defining and drawing your processes.

Cheers,
Frank

Collapse
Posted by Andrew Piskorski on
Frank, yes I remember Lars' explanations. You're reasons for going back to the older Workflow package are interesting. It's always better to hear from people with actual practical hands-on experience. :)
Collapse
Posted by merv colton on
Hi Folks,

Frank is correct, We've had some good success with Petri Nets, including some of the powerful constructs that we needed to add (and many thanks to Frank Bergmann and to Christian Eva for some of these)

As also made some changes to the inbox, which shows the users their tasks.

We would be ok on sharing this, What would be useful? A case study of one of the more complex workflows? Put into a slide show with a few screens?

Merv.

Collapse
Posted by Frank Bergmann on
Hi,

Yes, funny, no reply so far. It seems that Quest.ie and project-open.com are the only ones really interested in this gem.

I think that's OK. The package has been designed by Lars for a particular type of customer needs, which may be different from the needs by dotLrn customers and the other community members.

Merv, we would be very interested in a slide show or something similar (PowerPoint???) by you. Also, such a presentation could be part of an OpenACS showcase that lists all the success stories of OpenACS?

Cheers,
Frank