Forum OpenACS Development: GUI Redesign of OpenACS, ]po[ and Others?

Hi!

I'm starting a new thread after a great comment from Maurizio here:
https://openacs.org/forums/message-view?message_id=5466996

Here is a slightly shortened version:

[...]
But, apart from functions, the user interface also matters, and it matters a lot. And it is not just a question of actual user interaction, how the users actually use the system. Unfortunately it is also a matter of fashion. And here I'm afraid both OpenACS and ]project-open[ fall short. They are still kind of stuck to Web 1.0 style. But Web 2.0 has come, and it has gone too... Now it's time of Web 3.0, see for instance https://www.geeksforgeeks.org/web-1-0-web-2-0-and-web-3-0-with-their-difference/
or
https://nicepage.com/doc/article/20348/web-design-3-0-when-your-web-design-really-matters
Please stay with me, I know that the majority of this is only "fashion"; but it is "fashion" that matters a lot.
[...]
But if we want to make sure OpenACS and ]project-open[ will be still used in a future to come, and by new adopters, we need to work heavily on their user interfaces.

I'm 100% with Maurizio if terms if the uglyness of the GUI of ]project-open[ and OpenACS.

Brian Fenton recently talked to me about the same subject. He actually had a friend create a mock-up of a considerably more cool GUI at his/their own cost! It's based on a redesign of AIMS (the Quest grant management product he is working on). So it has been proven to work on OpenACS already. Here is a screenshot: http://www.project-open.net/en/file/brian-po-gui-white.jpg

At the moment we are busy with ]po[ V5.1 which is dedicated to security and includes an update to OpenACS 5.9.1.
After that we would be ready for ]po[ V5.2, which could include such a new GUI.

=> Maybe the OpenACS community could agree with such a design (or an amended version of it) so we could adapt such a GUI together?

From the ]po[ side there will apparently be a lot of work to add "class" and "id" tags to the existing GUI. I'm not sure about how much work would be required to apply such a GUI design to OpenACS. But I understand that once the basework is done, it should be easy to create new CSS "skins".

@Maurizio:
I understand that you would want to form part of such an initiative?

Cheers
Frank

Collapse
Posted by Monika Andergassen on
Hi everybody,

It's great news that there is interest in the community on advancing the OpenACS GUI.
I was involved in creating the Bootstrap3-Theme for OpenACS (https://openacs.org/forums/message-view?message_id=5330385), which is currently in use on the OpenACS website and available for download/installation. Back then it was meant to provide a basic starting point for a responsive design, where developers could place individual styling on top.
This theme is a few years old by now, and new themes with new design ideas would certainly make sense. A Bootstrap v4 based template could be an option (https://getbootstrap.com/docs/4.5/getting-started/introduction/).

My personal experience is that with the current theming and templating options of OpenACS, designs can be brought quite far. One of the most recent designs we did here in Vienna, was the design for the MOOC platform LearnPublic: https://learnpublic.wu.ac.at/ (Design by FONDA, implementation by WU).

I'd be interested to learn from your experience and to further exchange on ideas, e.g. where do you see the most need in advancing the design, what patterns could be used for that, etc.

Cheers, Monika

Collapse
Posted by Frank Bergmann on
Hi Monica,

Thanks for the links. LearnPublic looks great. Bootstrap also seems to be the preferred JS library by Brian and his friend who created the design referenced above.

Well, you have for sure seen this https://openacs.org/ Web site with all the different types of pages around and their embedded widgets.

The discussion with Maurizio mainly refered to a type of application that we call "business management applications" (biz apps):
- ]project-open[ (http://www.project-open.com)
- AIMS (http://www.quest.ie) and
- ALTER (https://openacs.org/forums/message-view?message_id=1451724).
- ... (please add if there are others...).

Biz apps have a different focus and different constraints than (what we call) "consumer Web sites" like LearnPublic:
- Biz apps are usually very large. ]po[ has > 1000 pages, and AIMS and ALTER surely have hundreds.
- They are very expensive to maintain. Somebody has to pay, if we need to tweak the GUI in 1000 pages...
- They are run in the corporate intranet, so page size and load times are not very important
- They have very long life cycles. One company is running ]po[ already since 20 years.
- Maintenance: Customer need to update their biz apps every few years, so poor souls like ourselves have to update the same code again and again. So we need to have _very_ clean code.

I am not aware of public demo web sites for AIMS and ALTER, but we have a public demo server for ]project-open[, so that you can see what I mean:

https://demo.project-open.net/
Please click on the "Ben Bigboss" photo for login.

If you look around a bit, you will see that most pages can be classified in one of the following types in terms of GUI structure (see below). The challenge that I see is:
1. Develop a GUI design for each page type and/or template used and
2. Technically adapt the pages (tag CLASS and ID etc.) in order to make the new GUI design work.

Types of pages in ]project-open[:

- "ListPages" show multiple objects according to a filter. Example: http://demo.project-open.net/intranet/projects/

- "ViewPages" with multiple portlets (grey boxes) show information about one specific business object. Example: http://demo.project-open.net/intranet/projects/view?project_id=45956 (shows everything about the "Car Configuration" project)

- "EditPages" with a form for creating or editing a business objects. Just press the "Edit" button in the "Project Base Data" portlet of the "Car Configurator" project above.

- "Reports" are similar to ListPages, but are technically created in a different way. Example: http://demo.project-open.net/intranet-reporting/timesheet-customer-project?start_date=2017-01-01&end_date=2021-01-01

- "XoWiki pages" are produced by the XoWiki. Example: http://demo.project-open.net/xowiki/

- "OpenACS pages" are just pages from OpenACS. These are mostly pages for administrators in the case of ]po[. Example: http://demo.project-open.net/notifications/

- "Admin pages" are pages only seen by system administrators. It really doesn't matter if these pages are ugly or pretty, as long as they are functional 😊

- "CustomPages" don't follow any of the conventions above. Example: http://demo.project-open.net/intranet-timesheet2/hours/

When selecting a new GUI, we would have to strike a balance between the "beauty" of the GUI and the effort required to adapt the various page types to the new GUI.

=> It would be great if you could have a look at demo.project-open.net and let us know what you think

Cheers
Frank

Collapse
Posted by Antonio Pisano on
I think it is useful to distinguish a few sub-tasks in this broader discussion about "redesign", which in my opinion would be:
1) upstream platform capability to implement a certain web paradigm in the first place
2) upstream platform support to make migration from one graphical design trend to the next one as easy as possible
3) downstream effort to adapt user's own design (markup, assets, fonts, layouts...)

An example of 1) is for me the set of tools needed to support Web 2.0 (database interaction, scriptable pages, validation and so on...) which the previous static Web 1.0 didn't need. There is to me no need for new OpenACS features to support the definition of Web 3.0 in [1], as this has only to do with graphics. This is not so clear with the definition in [2]... One feature for connectivity that we are discussing at WU that might fit in the 1) category is push notification support. Some work for this has been done recently by Gustaf on Naviserver. Semantic web boils down in the end to additional markup. 3D graphics has mostly to do with browser support and can also be just markup in the end (see e.g. [3]). It is unclear to me what a web platform can do to support "artifcial intelligence use cases", other than provide backend scripting capabilities for processing information...

2) is what we try to achieve via the templating system. As Monika pointed out, there is already quite much one can do to customize the look and feel of their applications. One shortcoming I see at the moment is the tcl duplication that right now is necessary when one overrides e.g. the vanilla login page with a new one in their own theme. In the future it might be possible to just let the adp file be customized. Apart from this, a sane application of best practices, such as the definition of the typical graphical elements of one's design in reusable components (includes, widgets, portlets, templates...) and the correct use of CSS styling, should help to migrate at least most of the design without touching every single program... even if I know from experience there will always be a few special pages which will need custom intervention.

to me 3) is the kind of effor that needs to be done every few years by everyone in order to maintain website sexyness... at some point a new theme package implementing a fresher graphic design might popup from upstream to be used as boilerplate for others as the current bootstrap theme has.

I would love to hear your take on the matter. What is Web 3.0 for you? What is that OpenACS is missing in your opinion that would enable this "new kind of web"? Is the templating system sufficient for your web design purposes?...

Cheers, Antonio

[1] https://nicepage.com/doc/article/20348/web-design-3-0-when-your-web-design-really-matters
[2] https://www.geeksforgeeks.org/web-1-0-web-2-0-and-web-3-0-with-their-difference/
[3] https://aframe.io/

Collapse
Posted by Maurizio Martignano on
Dear all, I would like to share my thoughts on this subject and I believe it might be very useful organizing these thoughts in three main topics.

I What needs to be taken into account/done?
Well I believe that here there are two different areas:
1. the user interfaces (e.g. here we have things like the Web framework: Bootstrap? React? Next.js? and the like...)
2. the "machinery" behind the user interface, actually connecting the user interface with the application and the data in the back-end.
On this second point it is not just a matter of implementing a REST API on the back-end, or to use things like GraphQL, etc...
Here the matter is for instance having data tables that paginate well, work properly on large datasets with many, many rows. Or as an other example the matter is having what Frank calls a view page, where the various blocks scale properly when the browser page is resized, do not disappear, but behave properly....
Now all of this, interface + machinery working well together is not at all a piece of cake....

II Who can do all this stuff (and Spazio IT approach)?
I'm a freelancer, my company is a single man operation and I'm getting old... (my revels now are ended) so for sure I do not have all the energy, capacity and know how to tackle such an immense endeavour. And I believe many SMEs are in my exact conditions.
So... So Spazio IT relies on interfaces+machineries developed by others, and they are: Microsoft .NET Framework; Oracle JET (https://www.oracle.com/webfolder/technetwork/jet/jetCookbook.html), Oracle ADF; SAPUI5 (https://open.sap.com/courses/ui51). Why did I only mention "corporate solutions"? Because it is only these types of solutions that do offer interface+machinery actually working together.

III How about OpenACS and ]project-open[?
Being an open community you'll have to take an open Web Framework as basis, say Bootstrap. But you'll have to implement for it the proper "machinery". I suggest you take a very close look at the corporate solutions, especially Oracle Jet (given that PostgreSQL is not so different from Oracle and OpenACS needs to support also Oracle). You take a very close look at how Oracle JET has implemented the "machinery" part and you implement some sort of equivalent "machinery" on top of OpenACS.
This is not an easy task.

I hope it helps,
Maurizio

Collapse
Posted by Antonio Pisano on
Dear Maurizio,

About your "machinery" subject I agree with you that no much support for "modern UIs" is provided by OpenACS out of the box.

Take the case of template::list. The way it was designed is nowadays less suitable for a modern UI, as it assumes "data retrieval" to happen together with rendering/sorting/pagination... at each and every request, while UIs today would fetch data from e.g. a JSON endpoint and update the UI dynamically and/or progressively without the user noticing a new request.

However, I am not 100% sure such fancier kinds of UIs cannot be achieved with just existing tools and plain HTML/javascript or if we should go for a "new and improved" template::list. For instance, one could render an existing template::list as JSON by defining a new style and hack it into a paginated JSON datasource that one could query from e.g. from ajax or websockets... For many use cases, a plain multirow can be very powerful as well to produce not only HTML, but many kinds of output (csv, JSON, XML...)

One issue with platform support for fancy UIs is the adoption of some javascript framework vs rolling up our own code. This has proven challenging to maintain in the past (e.g. ExtJS, YUI...). Even frameworks which used to be very established in the past such as jQuery UI or jQuery Mobile have declined over time. It will always be some kind of moving target.

Worth thinking about anyway.

Antonio

Collapse
Posted by Frank Bergmann on
Ciao Antonio,

not much support for "modern UIs" is
provided by OpenACS out of the box

Please have a look at the ]po[ REST API in this post here that I've just created:
https://openacs.org/forums/message-view?message_id=5467288

Please have a look at our demo-server and use the Chrome "Developer Tools" in order to see how it works:
https://demo.project-open.net/

So with this API you can create "forms" for HTML5 CRU(D) operations in minutes, with no need for any back-end code.

So this is just completely beyond ad_form and template::list.

Cheers
Frank

Collapse
Posted by Malte Sussdorff on
Thanks for sharing Maurizio.

At cognovis we have gone through multiple iterations of technological change in the past few years which might point to a direction OpenACS & ]po[ might go towards.

We started off with OpenACS and intranet functionality. This was plain OpenACS doing enhancements in the templating system for clients. We enjoyed using Listbuilder and played around with XoTCL.

Then we started to build upon .LRN, still using the OpenACS framework but learned more about the need for custom packages and how to do proper abstraction of custom(er) code from the main packages. At that point our thought process was „have custom packages and use callbacks / procedure overload wherever possible“.

After a few years we got invested in ]project-open[, more specific with Translation clients. At that point we faced the decision, stay with OpenACS and include ]po[ modules on top or Fork off ]po[ and allow for later merging of our code changes back to the main repository. We took the latter route, even though that resulted in us having to merge a lot of ]project-open[ code changes back into our fork, as the intended back-merging never happened.

With the advent of more need for interactive designs we embarked on utilizing Sencha ExtJS as our JavaScript framework, still very much Web 1.0 Style. We generated portlets with JavaScript code running in them (could have used jquery for that and guess what... we did as well. Why not make our live more complex.....). This made the funcationality work nicely, but it felt out of place within the rest of |project-open[.

So once we were asked to build a customer portal we decided to jump on the opportunity to build a native APP which just works using the REST API, splitting our small team of two into frontend and backend developer. This has proven successful in 2019 for other apps as well and we were happy with that approach, did it allow us to build Web 2.0(ish) applications. Sadly what we did not do is follow good coding practices (test coverage.... 0(ish) % - API Documentation ... Kindoff existing - Security - Handled individually for each endpoint etc.). Still, we were productive and the customers were happy with the „snappiness“ of the application.

Then the 2020 renewal for the Sencha License hit us. And lets just say a considerable price hike with reduced capabilities did not go well with my understanding of customer support. So we went looking again for a (JavaScript) Framework which allows us to build standalone application that are easily customizable. But we went further and added a few more things to our list:
* Ability for customers to engage in the UI design. We wanted to make sure that customers can easily design the UI and us having a fast turnaround time to make it work for them, even in a rudemantary fashion, so we can still have 2 week śprints and learn from them quickly after each iteration.
* Change over to Typescript. Not sure why, but the other two developers said that this is the much better choice for the frontend, so ... agreed 😊.
* Support for a test driven development process. At least not make it painfully difficult.

In addition to this, we changed our development ways to:
* API Documentation first. During refinement (of product backlog items) we nowadays write the API documentation, which (using a custom enhancement on top of intranet-rest) provides a mock server as well as OpenAPI 3 documentation
* UI Design sessions instead of User Story writing. We actually schedule meetings with the customer and design the UI in rough sketches together before we do the actual coding work.
* Allow coding by firing up a docker container (actually multiple ones), to make it VERY easy for a frontend developer to have something to connect against in the backend (without running a windows installer or firing up a virtual machine). docker-compose up -d is awesome 😊.
* Use NPM to potentially make our live miserable by including Typescript stacks for additional functionality instead of writing that within ]project-open[ & OpenACS.
* Test out that people with NULL experience in OpenACS & ]project-open[ can generate nice applications for customer consumption. Aka... we started working with a Student who’s current goal in live is Frontend Development using Typescript - definitely not learn TCL.

So... where are we now:

* We still have our apps written in ExtJS which work nicely and provide a good starting point for running standalone apps with ]project-open[
* We have rebuild out APP for customers and freelancers using Webix UI, building it based on UI Designs done between the frontend developer and the customers using the Webix UI designer ( UI element designer with automated code generation ).
* As a (paying) customer of Webix, we are really happy to utilize out of the box functionality for things we usually use like Chat & Kanban Board & Filemanager. Why do I mention this? Because we did not have to build it ourselves. We „just“ had to connect the API endpoints and translate the Widget „logic“ into something which we could store in the backend.
* API first works using the cognovis-rest package ( I will write up what it does and how to develop using it once I get around to it. Distribution is not something I would do without Frank‘s permission though, as I build upon / reused some of his code). But it is a lot of fun to sit in Sprint Planning Or Refinement and hash out the first version of the API between the whole (albeit smallish) team.
* We use Swagger Editor / Postman with the automatically generated OpenAPI documentation to access the endpoints (aka write endpoint tests) before actually fleshing out the TCL code doing the work. Helps a lot in the actual development (as we can do „code until successful entry“).
* Automated testing is working... ish..... We used Wallaby.js (which supports typescript despite the name) which allows us to integrate our testing into Visual Studio Code and immediately get notices if something does not compute.

If I had three things to be grateful for in our new approach:

* Improved customer happiness and interaction. Their (customer) willingness to play around with UI designs and testing options, if only half backed, is really encouraging. Though it is burdensome that one of them knows how to use CSS and styles the hell out of the application before we actually write functionality.. but story for a different day.

* Ability to build upon the shoulders of others (Apart from OpenACS & ]project-open[ obviously). As much as we see mocking of JavaScript Framework hell, using Webix has saved us from part of it and provides a consistent design language. Feels like writing iOS apps with Swift libraries, where all ties nicely together.

* Mixing CRUD operations on object types (similar to ::xo::db classes) with proper type support in OpenAPI (more than JSON obviously provides) as well as extensibility with „custom“ endpoints makes it so much easier for frontend developers to „just use„ ]project-open[, Yes, documentation is hard and it is boring, but it helps us so much to reevaluate what we are doing and how someone with zero experience in the OpenACS datamodel can still provide a simple community application with just a few days work (if the know their UI framework) Is gratifying.

We are still on a path of learning, adopting and tweaking how we are developing code. But the current split between frontend development and OpenACS endpoints feels go to us now and limits the need for me personally to do actual backend coding, so I can enjoy my second happiness (and income) stream: Teaching Product Owners how to do Scrum and Developers about XP practices.

Collapse
Posted by Benjamin Brink on
Hi,

This CSS style for building themes would be powerful with the current OpenACS template system:
http://johnpolacek.github.io/expressive-css/

Anyway, I'm planning to update the b-responsive-theme package to use it, once my current OpenACS app reaches beta..

cheers,
Ben

Collapse
Posted by Maurizio Martignano on
Dear Ben,
I believe that comparing something like
http://johnpolacek.github.io/expressive-css/
to something like
https://www.oracle.com/webfolder/technetwork/jet/index.html
is like comparing apples with oranges, from all points of view: e.g.
1. solution breadth: number and type of functions, or if you like, number of problems, needs covered by the solution
2. momentum: see who/what is behind the solution
3. effort needed in implementing not only the styling part but also all the "machinery" required by a web application (and not just a website).
The only thing they have in common is they are both on GitHub.
But I believe also that what you have in mind in terms of what needs to be done, to be achieved is very different from what for instance Frank has in mind for its ]project-open[.
If the goal is to have a "responsive" website than the solution you're suggesting might be more than adequate.
Maurizio
Collapse
Posted by Benjamin Brink on

You're right, Maurizio, like apples and oranges.

One has to be careful when adding complexity. A GUI for OpenACS should seek simplicity, because OpenACS historically has issues with overthinking solutions that results in technical debt and artificial limits to what one can do using the framework. By simplicity, I mean:

  • Define html forms by declaration of field names, values and data types.
  • Have form event-handling use a standardized flow.
  • Have code that defines a form take less keystrokes than building a form manually.
  • Have form generation code create accessible and responsive forms by default.
  • The GUI should make it easy to meet best practices by default, even for the most complex permutation of features.

These are things that ad_form may have intended to solve over time, and yet, complexity abounds. ad_form is avoided instead of used by default. There is a habit of bypassing ad_form by passing variables via urls, which is not good practice. For example, some browsers read ahead to cache pages, which puts more of a burden on the server. By changing these urls to form buttons, browsers (and web spiders) should not follow.

Also, consider re-factoring the table and list display API to default to show best practices in accessibility. For example, one row max for headers.

Also, javascript should be avoided for improving visual rendering, and yet is okay for client-side input validation when html5/browser cases are not adequate.

If one is going to refactor the GUI, why not take a slightly bolder approach with something like Next-Scripting Framework, and really rocket into the next decade?

https://next-scripting.org/xowiki/getstart#hello
https://next-scripting.org/xowiki/docs/nx/nx-migration/index1

Code could be written to migrate most code that is not oacs-core. And, since the majority of oacs-core functionality was coded in about a year that should be doable for an endeavor like this. You might actually get more volunteers. Also, according to the documentation, NSF is made for this type of endeavor --unlike OpenACS core.

While one is at it, make the system usable in a container / microservices environment ie cluster mode by default. You may get even more volunteers.

cheers,

Ben

Collapse
Posted by Benjamin Brink on
And if one were to take on refactoring of OpenACS into NSF, there are other shortcomings that would be strategic to address at the same time.

For instance, adding an 'ad_proc' feature that allows overwriting of release code procs with custom ones of the same name using some kind of standardized glob recognizable file pattern, such as *-custom.tcl in the same package, or maybe in sequential order from any package installed that follows core initialization on startup. This would make it easier to keep custom code separate, while allowing complete intercepts of the standard API.

cheers,

Ben

Collapse
Posted by Malte Sussdorff on
Three days ago I had a discussion „how to make ]project-open[ backend development ‚sexy‘“. And I did think of just using NSF for this. Or at least for all the endpoints which we code for the (typescript) frontend. but then I thought... well, not a very commonly used sexy language either.

Especially as we don‘t have to code that much backend code in TCL anymore (with the rest endpoints provided) we „sidestep“ the problem for the time being. Especially as this allows us to have people utilize OpenACS first (using the apps with a container running the Backend) and then maybe encourage more people to either write endpoints in NSF or any other language we write TCL code wrappers for. But as I said... first we focus on the Apps 😊.

Collapse
Posted by Benjamin Brink on
Hi Malte,

Yeah, I'm focusing on apps, too. As soon as I have a suite of apps working, I'm going to see about building OpenACS 6.0 on NSF --unless that project starts first, then I'll go with it! 😉

cheers,
Ben