Forum OpenACS Development: Re: GUI Redesign of OpenACS, ]po[ and Others?

Hi everybody,

It's great news that there is interest in the community on advancing the OpenACS GUI.
I was involved in creating the Bootstrap3-Theme for OpenACS (https://openacs.org/forums/message-view?message_id=5330385), which is currently in use on the OpenACS website and available for download/installation. Back then it was meant to provide a basic starting point for a responsive design, where developers could place individual styling on top.
This theme is a few years old by now, and new themes with new design ideas would certainly make sense. A Bootstrap v4 based template could be an option (https://getbootstrap.com/docs/4.5/getting-started/introduction/).

My personal experience is that with the current theming and templating options of OpenACS, designs can be brought quite far. One of the most recent designs we did here in Vienna, was the design for the MOOC platform LearnPublic: https://learnpublic.wu.ac.at/ (Design by FONDA, implementation by WU).

I'd be interested to learn from your experience and to further exchange on ideas, e.g. where do you see the most need in advancing the design, what patterns could be used for that, etc.

Cheers, Monika

I think it is useful to distinguish a few sub-tasks in this broader discussion about "redesign", which in my opinion would be:
1) upstream platform capability to implement a certain web paradigm in the first place
2) upstream platform support to make migration from one graphical design trend to the next one as easy as possible
3) downstream effort to adapt user's own design (markup, assets, fonts, layouts...)

An example of 1) is for me the set of tools needed to support Web 2.0 (database interaction, scriptable pages, validation and so on...) which the previous static Web 1.0 didn't need. There is to me no need for new OpenACS features to support the definition of Web 3.0 in [1], as this has only to do with graphics. This is not so clear with the definition in [2]... One feature for connectivity that we are discussing at WU that might fit in the 1) category is push notification support. Some work for this has been done recently by Gustaf on Naviserver. Semantic web boils down in the end to additional markup. 3D graphics has mostly to do with browser support and can also be just markup in the end (see e.g. [3]). It is unclear to me what a web platform can do to support "artifcial intelligence use cases", other than provide backend scripting capabilities for processing information...

2) is what we try to achieve via the templating system. As Monika pointed out, there is already quite much one can do to customize the look and feel of their applications. One shortcoming I see at the moment is the tcl duplication that right now is necessary when one overrides e.g. the vanilla login page with a new one in their own theme. In the future it might be possible to just let the adp file be customized. Apart from this, a sane application of best practices, such as the definition of the typical graphical elements of one's design in reusable components (includes, widgets, portlets, templates...) and the correct use of CSS styling, should help to migrate at least most of the design without touching every single program... even if I know from experience there will always be a few special pages which will need custom intervention.

to me 3) is the kind of effor that needs to be done every few years by everyone in order to maintain website sexyness... at some point a new theme package implementing a fresher graphic design might popup from upstream to be used as boilerplate for others as the current bootstrap theme has.

I would love to hear your take on the matter. What is Web 3.0 for you? What is that OpenACS is missing in your opinion that would enable this "new kind of web"? Is the templating system sufficient for your web design purposes?...

Cheers, Antonio

[1] https://nicepage.com/doc/article/20348/web-design-3-0-when-your-web-design-really-matters
[2] https://www.geeksforgeeks.org/web-1-0-web-2-0-and-web-3-0-with-their-difference/
[3] https://aframe.io/

Dear Maurizio,

About your "machinery" subject I agree with you that no much support for "modern UIs" is provided by OpenACS out of the box.

Take the case of template::list. The way it was designed is nowadays less suitable for a modern UI, as it assumes "data retrieval" to happen together with rendering/sorting/pagination... at each and every request, while UIs today would fetch data from e.g. a JSON endpoint and update the UI dynamically and/or progressively without the user noticing a new request.

However, I am not 100% sure such fancier kinds of UIs cannot be achieved with just existing tools and plain HTML/javascript or if we should go for a "new and improved" template::list. For instance, one could render an existing template::list as JSON by defining a new style and hack it into a paginated JSON datasource that one could query from e.g. from ajax or websockets... For many use cases, a plain multirow can be very powerful as well to produce not only HTML, but many kinds of output (csv, JSON, XML...)

One issue with platform support for fancy UIs is the adoption of some javascript framework vs rolling up our own code. This has proven challenging to maintain in the past (e.g. ExtJS, YUI...). Even frameworks which used to be very established in the past such as jQuery UI or jQuery Mobile have declined over time. It will always be some kind of moving target.

Worth thinking about anyway.

Antonio

Thanks for sharing Maurizio.

At cognovis we have gone through multiple iterations of technological change in the past few years which might point to a direction OpenACS & ]po[ might go towards.

We started off with OpenACS and intranet functionality. This was plain OpenACS doing enhancements in the templating system for clients. We enjoyed using Listbuilder and played around with XoTCL.

Then we started to build upon .LRN, still using the OpenACS framework but learned more about the need for custom packages and how to do proper abstraction of custom(er) code from the main packages. At that point our thought process was „have custom packages and use callbacks / procedure overload wherever possible“.

After a few years we got invested in ]project-open[, more specific with Translation clients. At that point we faced the decision, stay with OpenACS and include ]po[ modules on top or Fork off ]po[ and allow for later merging of our code changes back to the main repository. We took the latter route, even though that resulted in us having to merge a lot of ]project-open[ code changes back into our fork, as the intended back-merging never happened.

With the advent of more need for interactive designs we embarked on utilizing Sencha ExtJS as our JavaScript framework, still very much Web 1.0 Style. We generated portlets with JavaScript code running in them (could have used jquery for that and guess what... we did as well. Why not make our live more complex.....). This made the funcationality work nicely, but it felt out of place within the rest of |project-open[.

So once we were asked to build a customer portal we decided to jump on the opportunity to build a native APP which just works using the REST API, splitting our small team of two into frontend and backend developer. This has proven successful in 2019 for other apps as well and we were happy with that approach, did it allow us to build Web 2.0(ish) applications. Sadly what we did not do is follow good coding practices (test coverage.... 0(ish) % - API Documentation ... Kindoff existing - Security - Handled individually for each endpoint etc.). Still, we were productive and the customers were happy with the „snappiness“ of the application.

Then the 2020 renewal for the Sencha License hit us. And lets just say a considerable price hike with reduced capabilities did not go well with my understanding of customer support. So we went looking again for a (JavaScript) Framework which allows us to build standalone application that are easily customizable. But we went further and added a few more things to our list:
* Ability for customers to engage in the UI design. We wanted to make sure that customers can easily design the UI and us having a fast turnaround time to make it work for them, even in a rudemantary fashion, so we can still have 2 week śprints and learn from them quickly after each iteration.
* Change over to Typescript. Not sure why, but the other two developers said that this is the much better choice for the frontend, so ... agreed 😊.
* Support for a test driven development process. At least not make it painfully difficult.

In addition to this, we changed our development ways to:
* API Documentation first. During refinement (of product backlog items) we nowadays write the API documentation, which (using a custom enhancement on top of intranet-rest) provides a mock server as well as OpenAPI 3 documentation
* UI Design sessions instead of User Story writing. We actually schedule meetings with the customer and design the UI in rough sketches together before we do the actual coding work.
* Allow coding by firing up a docker container (actually multiple ones), to make it VERY easy for a frontend developer to have something to connect against in the backend (without running a windows installer or firing up a virtual machine). docker-compose up -d is awesome 😊.
* Use NPM to potentially make our live miserable by including Typescript stacks for additional functionality instead of writing that within ]project-open[ & OpenACS.
* Test out that people with NULL experience in OpenACS & ]project-open[ can generate nice applications for customer consumption. Aka... we started working with a Student who’s current goal in live is Frontend Development using Typescript - definitely not learn TCL.

So... where are we now:

* We still have our apps written in ExtJS which work nicely and provide a good starting point for running standalone apps with ]project-open[
* We have rebuild out APP for customers and freelancers using Webix UI, building it based on UI Designs done between the frontend developer and the customers using the Webix UI designer ( UI element designer with automated code generation ).
* As a (paying) customer of Webix, we are really happy to utilize out of the box functionality for things we usually use like Chat & Kanban Board & Filemanager. Why do I mention this? Because we did not have to build it ourselves. We „just“ had to connect the API endpoints and translate the Widget „logic“ into something which we could store in the backend.
* API first works using the cognovis-rest package ( I will write up what it does and how to develop using it once I get around to it. Distribution is not something I would do without Frank‘s permission though, as I build upon / reused some of his code). But it is a lot of fun to sit in Sprint Planning Or Refinement and hash out the first version of the API between the whole (albeit smallish) team.
* We use Swagger Editor / Postman with the automatically generated OpenAPI documentation to access the endpoints (aka write endpoint tests) before actually fleshing out the TCL code doing the work. Helps a lot in the actual development (as we can do „code until successful entry“).
* Automated testing is working... ish..... We used Wallaby.js (which supports typescript despite the name) which allows us to integrate our testing into Visual Studio Code and immediately get notices if something does not compute.

If I had three things to be grateful for in our new approach:

* Improved customer happiness and interaction. Their (customer) willingness to play around with UI designs and testing options, if only half backed, is really encouraging. Though it is burdensome that one of them knows how to use CSS and styles the hell out of the application before we actually write functionality.. but story for a different day.

* Ability to build upon the shoulders of others (Apart from OpenACS & ]project-open[ obviously). As much as we see mocking of JavaScript Framework hell, using Webix has saved us from part of it and provides a consistent design language. Feels like writing iOS apps with Swift libraries, where all ties nicely together.

* Mixing CRUD operations on object types (similar to ::xo::db classes) with proper type support in OpenAPI (more than JSON obviously provides) as well as extensibility with „custom“ endpoints makes it so much easier for frontend developers to „just use„ ]project-open[, Yes, documentation is hard and it is boring, but it helps us so much to reevaluate what we are doing and how someone with zero experience in the OpenACS datamodel can still provide a simple community application with just a few days work (if the know their UI framework) Is gratifying.

We are still on a path of learning, adopting and tweaking how we are developing code. But the current split between frontend development and OpenACS endpoints feels go to us now and limits the need for me personally to do actual backend coding, so I can enjoy my second happiness (and income) stream: Teaching Product Owners how to do Scrum and Developers about XP practices.