Forum OpenACS Development: Re: GUI Redesign of OpenACS, ]po[ and Others?
Thanks for the links. LearnPublic looks great. Bootstrap also seems to be the preferred JS library by Brian and his friend who created the design referenced above.
Well, you have for sure seen this https://openacs.org/ Web site with all the different types of pages around and their embedded widgets.
The discussion with Maurizio mainly refered to a type of application that we call "business management applications" (biz apps):
- ]project-open[ (http://www.project-open.com)
- AIMS (http://www.quest.ie) and
- ALTER (https://openacs.org/forums/message-view?message_id=1451724).
- ... (please add if there are others...).
Biz apps have a different focus and different constraints than (what we call) "consumer Web sites" like LearnPublic:
- Biz apps are usually very large. ]po[ has > 1000 pages, and AIMS and ALTER surely have hundreds.
- They are very expensive to maintain. Somebody has to pay, if we need to tweak the GUI in 1000 pages...
- They are run in the corporate intranet, so page size and load times are not very important
- They have very long life cycles. One company is running ]po[ already since 20 years.
- Maintenance: Customer need to update their biz apps every few years, so poor souls like ourselves have to update the same code again and again. So we need to have _very_ clean code.
I am not aware of public demo web sites for AIMS and ALTER, but we have a public demo server for ]project-open[, so that you can see what I mean:
https://demo.project-open.net/
Please click on the "Ben Bigboss" photo for login.
If you look around a bit, you will see that most pages can be classified in one of the following types in terms of GUI structure (see below). The challenge that I see is:
1. Develop a GUI design for each page type and/or template used and
2. Technically adapt the pages (tag CLASS and ID etc.) in order to make the new GUI design work.
Types of pages in ]project-open[:
- "ListPages" show multiple objects according to a filter. Example: http://demo.project-open.net/intranet/projects/
- "ViewPages" with multiple portlets (grey boxes) show information about one specific business object. Example: http://demo.project-open.net/intranet/projects/view?project_id=45956 (shows everything about the "Car Configuration" project)
- "EditPages" with a form for creating or editing a business objects. Just press the "Edit" button in the "Project Base Data" portlet of the "Car Configurator" project above.
- "Reports" are similar to ListPages, but are technically created in a different way. Example: http://demo.project-open.net/intranet-reporting/timesheet-customer-project?start_date=2017-01-01&end_date=2021-01-01
- "XoWiki pages" are produced by the XoWiki. Example: http://demo.project-open.net/xowiki/
- "OpenACS pages" are just pages from OpenACS. These are mostly pages for administrators in the case of ]po[. Example: http://demo.project-open.net/notifications/
- "Admin pages" are pages only seen by system administrators. It really doesn't matter if these pages are ugly or pretty, as long as they are functional 😊
- "CustomPages" don't follow any of the conventions above. Example: http://demo.project-open.net/intranet-timesheet2/hours/
When selecting a new GUI, we would have to strike a balance between the "beauty" of the GUI and the effort required to adapt the various page types to the new GUI.
=> It would be great if you could have a look at demo.project-open.net and let us know what you think
Cheers
Frank
1) upstream platform capability to implement a certain web paradigm in the first place
2) upstream platform support to make migration from one graphical design trend to the next one as easy as possible
3) downstream effort to adapt user's own design (markup, assets, fonts, layouts...)
An example of 1) is for me the set of tools needed to support Web 2.0 (database interaction, scriptable pages, validation and so on...) which the previous static Web 1.0 didn't need. There is to me no need for new OpenACS features to support the definition of Web 3.0 in [1], as this has only to do with graphics. This is not so clear with the definition in [2]... One feature for connectivity that we are discussing at WU that might fit in the 1) category is push notification support. Some work for this has been done recently by Gustaf on Naviserver. Semantic web boils down in the end to additional markup. 3D graphics has mostly to do with browser support and can also be just markup in the end (see e.g. [3]). It is unclear to me what a web platform can do to support "artifcial intelligence use cases", other than provide backend scripting capabilities for processing information...
2) is what we try to achieve via the templating system. As Monika pointed out, there is already quite much one can do to customize the look and feel of their applications. One shortcoming I see at the moment is the tcl duplication that right now is necessary when one overrides e.g. the vanilla login page with a new one in their own theme. In the future it might be possible to just let the adp file be customized. Apart from this, a sane application of best practices, such as the definition of the typical graphical elements of one's design in reusable components (includes, widgets, portlets, templates...) and the correct use of CSS styling, should help to migrate at least most of the design without touching every single program... even if I know from experience there will always be a few special pages which will need custom intervention.
to me 3) is the kind of effor that needs to be done every few years by everyone in order to maintain website sexyness... at some point a new theme package implementing a fresher graphic design might popup from upstream to be used as boilerplate for others as the current bootstrap theme has.
I would love to hear your take on the matter. What is Web 3.0 for you? What is that OpenACS is missing in your opinion that would enable this "new kind of web"? Is the templating system sufficient for your web design purposes?...
Cheers, Antonio
[1] https://nicepage.com/doc/article/20348/web-design-3-0-when-your-web-design-really-matters
[2] https://www.geeksforgeeks.org/web-1-0-web-2-0-and-web-3-0-with-their-difference/
[3] https://aframe.io/
I What needs to be taken into account/done?
Well I believe that here there are two different areas:
1. the user interfaces (e.g. here we have things like the Web framework: Bootstrap? React? Next.js? and the like...)
2. the "machinery" behind the user interface, actually connecting the user interface with the application and the data in the back-end.
On this second point it is not just a matter of implementing a REST API on the back-end, or to use things like GraphQL, etc...
Here the matter is for instance having data tables that paginate well, work properly on large datasets with many, many rows. Or as an other example the matter is having what Frank calls a view page, where the various blocks scale properly when the browser page is resized, do not disappear, but behave properly....
Now all of this, interface + machinery working well together is not at all a piece of cake....
II Who can do all this stuff (and Spazio IT approach)?
I'm a freelancer, my company is a single man operation and I'm getting old... (my revels now are ended) so for sure I do not have all the energy, capacity and know how to tackle such an immense endeavour. And I believe many SMEs are in my exact conditions.
So... So Spazio IT relies on interfaces+machineries developed by others, and they are: Microsoft .NET Framework; Oracle JET (https://www.oracle.com/webfolder/technetwork/jet/jetCookbook.html), Oracle ADF; SAPUI5 (https://open.sap.com/courses/ui51). Why did I only mention "corporate solutions"? Because it is only these types of solutions that do offer interface+machinery actually working together.
III How about OpenACS and ]project-open[?
Being an open community you'll have to take an open Web Framework as basis, say Bootstrap. But you'll have to implement for it the proper "machinery". I suggest you take a very close look at the corporate solutions, especially Oracle Jet (given that PostgreSQL is not so different from Oracle and OpenACS needs to support also Oracle). You take a very close look at how Oracle JET has implemented the "machinery" part and you implement some sort of equivalent "machinery" on top of OpenACS.
This is not an easy task.
I hope it helps,
Maurizio
About your "machinery" subject I agree with you that no much support for "modern UIs" is provided by OpenACS out of the box.
Take the case of template::list. The way it was designed is nowadays less suitable for a modern UI, as it assumes "data retrieval" to happen together with rendering/sorting/pagination... at each and every request, while UIs today would fetch data from e.g. a JSON endpoint and update the UI dynamically and/or progressively without the user noticing a new request.
However, I am not 100% sure such fancier kinds of UIs cannot be achieved with just existing tools and plain HTML/javascript or if we should go for a "new and improved" template::list. For instance, one could render an existing template::list as JSON by defining a new style and hack it into a paginated JSON datasource that one could query from e.g. from ajax or websockets... For many use cases, a plain multirow can be very powerful as well to produce not only HTML, but many kinds of output (csv, JSON, XML...)
One issue with platform support for fancy UIs is the adoption of some javascript framework vs rolling up our own code. This has proven challenging to maintain in the past (e.g. ExtJS, YUI...). Even frameworks which used to be very established in the past such as jQuery UI or jQuery Mobile have declined over time. It will always be some kind of moving target.
Worth thinking about anyway.
Antonio
not much support for "modern UIs" is
provided by OpenACS out of the box
Please have a look at the ]po[ REST API in this post here that I've just created:
https://openacs.org/forums/message-view?message_id=5467288
Please have a look at our demo-server and use the Chrome "Developer Tools" in order to see how it works:
https://demo.project-open.net/
So with this API you can create "forms" for HTML5 CRU(D) operations in minutes, with no need for any back-end code.
So this is just completely beyond ad_form and template::list.
Cheers
Frank
At cognovis we have gone through multiple iterations of technological change in the past few years which might point to a direction OpenACS & ]po[ might go towards.
We started off with OpenACS and intranet functionality. This was plain OpenACS doing enhancements in the templating system for clients. We enjoyed using Listbuilder and played around with XoTCL.
Then we started to build upon .LRN, still using the OpenACS framework but learned more about the need for custom packages and how to do proper abstraction of custom(er) code from the main packages. At that point our thought process was „have custom packages and use callbacks / procedure overload wherever possible“.
After a few years we got invested in ]project-open[, more specific with Translation clients. At that point we faced the decision, stay with OpenACS and include ]po[ modules on top or Fork off ]po[ and allow for later merging of our code changes back to the main repository. We took the latter route, even though that resulted in us having to merge a lot of ]project-open[ code changes back into our fork, as the intended back-merging never happened.
With the advent of more need for interactive designs we embarked on utilizing Sencha ExtJS as our JavaScript framework, still very much Web 1.0 Style. We generated portlets with JavaScript code running in them (could have used jquery for that and guess what... we did as well. Why not make our live more complex.....). This made the funcationality work nicely, but it felt out of place within the rest of |project-open[.
So once we were asked to build a customer portal we decided to jump on the opportunity to build a native APP which just works using the REST API, splitting our small team of two into frontend and backend developer. This has proven successful in 2019 for other apps as well and we were happy with that approach, did it allow us to build Web 2.0(ish) applications. Sadly what we did not do is follow good coding practices (test coverage.... 0(ish) % - API Documentation ... Kindoff existing - Security - Handled individually for each endpoint etc.). Still, we were productive and the customers were happy with the „snappiness“ of the application.
Then the 2020 renewal for the Sencha License hit us. And lets just say a considerable price hike with reduced capabilities did not go well with my understanding of customer support. So we went looking again for a (JavaScript) Framework which allows us to build standalone application that are easily customizable. But we went further and added a few more things to our list:
* Ability for customers to engage in the UI design. We wanted to make sure that customers can easily design the UI and us having a fast turnaround time to make it work for them, even in a rudemantary fashion, so we can still have 2 week śprints and learn from them quickly after each iteration.
* Change over to Typescript. Not sure why, but the other two developers said that this is the much better choice for the frontend, so ... agreed 😊.
* Support for a test driven development process. At least not make it painfully difficult.
In addition to this, we changed our development ways to:
* API Documentation first. During refinement (of product backlog items) we nowadays write the API documentation, which (using a custom enhancement on top of intranet-rest) provides a mock server as well as OpenAPI 3 documentation
* UI Design sessions instead of User Story writing. We actually schedule meetings with the customer and design the UI in rough sketches together before we do the actual coding work.
* Allow coding by firing up a docker container (actually multiple ones), to make it VERY easy for a frontend developer to have something to connect against in the backend (without running a windows installer or firing up a virtual machine). docker-compose up -d is awesome 😊.
* Use NPM to potentially make our live miserable by including Typescript stacks for additional functionality instead of writing that within ]project-open[ & OpenACS.
* Test out that people with NULL experience in OpenACS & ]project-open[ can generate nice applications for customer consumption. Aka... we started working with a Student who’s current goal in live is Frontend Development using Typescript - definitely not learn TCL.
So... where are we now:
* We still have our apps written in ExtJS which work nicely and provide a good starting point for running standalone apps with ]project-open[
* We have rebuild out APP for customers and freelancers using Webix UI, building it based on UI Designs done between the frontend developer and the customers using the Webix UI designer ( UI element designer with automated code generation ).
* As a (paying) customer of Webix, we are really happy to utilize out of the box functionality for things we usually use like Chat & Kanban Board & Filemanager. Why do I mention this? Because we did not have to build it ourselves. We „just“ had to connect the API endpoints and translate the Widget „logic“ into something which we could store in the backend.
* API first works using the cognovis-rest package ( I will write up what it does and how to develop using it once I get around to it. Distribution is not something I would do without Frank‘s permission though, as I build upon / reused some of his code). But it is a lot of fun to sit in Sprint Planning Or Refinement and hash out the first version of the API between the whole (albeit smallish) team.
* We use Swagger Editor / Postman with the automatically generated OpenAPI documentation to access the endpoints (aka write endpoint tests) before actually fleshing out the TCL code doing the work. Helps a lot in the actual development (as we can do „code until successful entry“).
* Automated testing is working... ish..... We used Wallaby.js (which supports typescript despite the name) which allows us to integrate our testing into Visual Studio Code and immediately get notices if something does not compute.
If I had three things to be grateful for in our new approach:
* Improved customer happiness and interaction. Their (customer) willingness to play around with UI designs and testing options, if only half backed, is really encouraging. Though it is burdensome that one of them knows how to use CSS and styles the hell out of the application before we actually write functionality.. but story for a different day.
* Ability to build upon the shoulders of others (Apart from OpenACS & ]project-open[ obviously). As much as we see mocking of JavaScript Framework hell, using Webix has saved us from part of it and provides a consistent design language. Feels like writing iOS apps with Swift libraries, where all ties nicely together.
* Mixing CRUD operations on object types (similar to ::xo::db classes) with proper type support in OpenAPI (more than JSON obviously provides) as well as extensibility with „custom“ endpoints makes it so much easier for frontend developers to „just use„ ]project-open[, Yes, documentation is hard and it is boring, but it helps us so much to reevaluate what we are doing and how someone with zero experience in the OpenACS datamodel can still provide a simple community application with just a few days work (if the know their UI framework) Is gratifying.
We are still on a path of learning, adopting and tweaking how we are developing code. But the current split between frontend development and OpenACS endpoints feels go to us now and limits the need for me personally to do actual backend coding, so I can enjoy my second happiness (and income) stream: Teaching Product Owners how to do Scrum and Developers about XP practices.