Forum OpenACS Development: Re: GUI Redesign of OpenACS, ]po[ and Others?

About your "machinery" subject I agree with you that no much support for "modern UIs" is provided by OpenACS out of the box.

Take the case of template::list. The way it was designed is nowadays less suitable for a modern UI, as it assumes "data retrieval" to happen together with rendering/sorting/pagination... at each and every request, while UIs today would fetch data from e.g. a JSON endpoint and update the UI dynamically and/or progressively without the user noticing a new request.

However, I am not 100% sure such fancier kinds of UIs cannot be achieved with just existing tools and plain HTML/javascript or if we should go for a "new and improved" template::list. For instance, one could render an existing template::list as JSON by defining a new style and hack it into a paginated JSON datasource that one could query from e.g. from ajax or websockets... For many use cases, a plain multirow can be very powerful as well to produce not only HTML, but many kinds of output (csv, JSON, XML...)

One issue with platform support for fancy UIs is the adoption of some javascript framework vs rolling up our own code. This has proven challenging to maintain in the past (e.g. ExtJS, YUI...). Even frameworks which used to be very established in the past such as jQuery UI or jQuery Mobile have declined over time. It will always be some kind of moving target.

Worth thinking about anyway.

Antonio

At cognovis we have gone through multiple iterations of technological change in the past few years which might point to a direction OpenACS & ]po[ might go towards.

We started off with OpenACS and intranet functionality. This was plain OpenACS doing enhancements in the templating system for clients. We enjoyed using Listbuilder and played around with XoTCL.

Then we started to build upon .LRN, still using the OpenACS framework but learned more about the need for custom packages and how to do proper abstraction of custom(er) code from the main packages. At that point our thought process was „have custom packages and use callbacks / procedure overload wherever possible“.

After a few years we got invested in ]project-open[, more specific with Translation clients. At that point we faced the decision, stay with OpenACS and include ]po[ modules on top or Fork off ]po[ and allow for later merging of our code changes back to the main repository. We took the latter route, even though that resulted in us having to merge a lot of ]project-open[ code changes back into our fork, as the intended back-merging never happened.

With the advent of more need for interactive designs we embarked on utilizing Sencha ExtJS as our JavaScript framework, still very much Web 1.0 Style. We generated portlets with JavaScript code running in them (could have used jquery for that and guess what... we did as well. Why not make our live more complex.....). This made the funcationality work nicely, but it felt out of place within the rest of |project-open[.

So once we were asked to build a customer portal we decided to jump on the opportunity to build a native APP which just works using the REST API, splitting our small team of two into frontend and backend developer. This has proven successful in 2019 for other apps as well and we were happy with that approach, did it allow us to build Web 2.0(ish) applications. Sadly what we did not do is follow good coding practices (test coverage.... 0(ish) % - API Documentation ... Kindoff existing - Security - Handled individually for each endpoint etc.). Still, we were productive and the customers were happy with the „snappiness“ of the application.

Then the 2020 renewal for the Sencha License hit us. And lets just say a considerable price hike with reduced capabilities did not go well with my understanding of customer support. So we went looking again for a (JavaScript) Framework which allows us to build standalone application that are easily customizable. But we went further and added a few more things to our list:
* Ability for customers to engage in the UI design. We wanted to make sure that customers can easily design the UI and us having a fast turnaround time to make it work for them, even in a rudemantary fashion, so we can still have 2 week śprints and learn from them quickly after each iteration.
* Change over to Typescript. Not sure why, but the other two developers said that this is the much better choice for the frontend, so ... agreed 😊.
* Support for a test driven development process. At least not make it painfully difficult.

In addition to this, we changed our development ways to:
* API Documentation first. During refinement (of product backlog items) we nowadays write the API documentation, which (using a custom enhancement on top of intranet-rest) provides a mock server as well as OpenAPI 3 documentation
* UI Design sessions instead of User Story writing. We actually schedule meetings with the customer and design the UI in rough sketches together before we do the actual coding work.
* Allow coding by firing up a docker container (actually multiple ones), to make it VERY easy for a frontend developer to have something to connect against in the backend (without running a windows installer or firing up a virtual machine). docker-compose up -d is awesome 😊.
* Use NPM to potentially make our live miserable by including Typescript stacks for additional functionality instead of writing that within ]project-open[ & OpenACS.
* Test out that people with NULL experience in OpenACS & ]project-open[ can generate nice applications for customer consumption. Aka... we started working with a Student who’s current goal in live is Frontend Development using Typescript - definitely not learn TCL.

So... where are we now:

* We still have our apps written in ExtJS which work nicely and provide a good starting point for running standalone apps with ]project-open[
* We have rebuild out APP for customers and freelancers using Webix UI, building it based on UI Designs done between the frontend developer and the customers using the Webix UI designer ( UI element designer with automated code generation ).
* As a (paying) customer of Webix, we are really happy to utilize out of the box functionality for things we usually use like Chat & Kanban Board & Filemanager. Why do I mention this? Because we did not have to build it ourselves. We „just“ had to connect the API endpoints and translate the Widget „logic“ into something which we could store in the backend.
* API first works using the cognovis-rest package ( I will write up what it does and how to develop using it once I get around to it. Distribution is not something I would do without Frank‘s permission though, as I build upon / reused some of his code). But it is a lot of fun to sit in Sprint Planning Or Refinement and hash out the first version of the API between the whole (albeit smallish) team.
* We use Swagger Editor / Postman with the automatically generated OpenAPI documentation to access the endpoints (aka write endpoint tests) before actually fleshing out the TCL code doing the work. Helps a lot in the actual development (as we can do „code until successful entry“).
* Automated testing is working... ish..... We used Wallaby.js (which supports typescript despite the name) which allows us to integrate our testing into Visual Studio Code and immediately get notices if something does not compute.

If I had three things to be grateful for in our new approach:

* Improved customer happiness and interaction. Their (customer) willingness to play around with UI designs and testing options, if only half backed, is really encouraging. Though it is burdensome that one of them knows how to use CSS and styles the hell out of the application before we actually write functionality.. but story for a different day.

* Ability to build upon the shoulders of others (Apart from OpenACS & ]project-open[ obviously). As much as we see mocking of JavaScript Framework hell, using Webix has saved us from part of it and provides a consistent design language. Feels like writing iOS apps with Swift libraries, where all ties nicely together.

* Mixing CRUD operations on object types (similar to ::xo::db classes) with proper type support in OpenAPI (more than JSON obviously provides) as well as extensibility with „custom“ endpoints makes it so much easier for frontend developers to „just use„ ]project-open[, Yes, documentation is hard and it is boring, but it helps us so much to reevaluate what we are doing and how someone with zero experience in the OpenACS datamodel can still provide a simple community application with just a few days work (if the know their UI framework) Is gratifying.

We are still on a path of learning, adopting and tweaking how we are developing code. But the current split between frontend development and OpenACS endpoints feels go to us now and limits the need for me personally to do actual backend coding, so I can enjoy my second happiness (and income) stream: Teaching Product Owners how to do Scrum and Developers about XP practices.