Forum OpenACS Development: wiki code metrics for packages, add standards metrics?

Request notifications

Hi,

Since this is a coding community with coding standards becoming increasingly emphasized: http://openacs.org/xowiki/code-formatting

Might the wiki code metrics for packages (still [1]) be a practical way to promote standards and indicate how well each package's code adhere's to standards?

Also, wouldn't it be convenient/useful for a package developer to be able to view a local assessment from somewhere in "Developer tools" menu ( /acs-admin/developer ) or as a "metrics" page via the ACS API Browser?

I know this would be a useful feature for me. I'm probably one the worst violators some of the basic standards, such as keeping lines under 80 character width. ( I've tended to find code more difficult to read when an expression occupies lines and I have an option to see the full line unbroken on screen: A habit I'm breaking. ).

Wouldn't it be more fun to work with more sustainable code in the OpenACS community?

cheers,
Ben

[1] http://openacs.org/forums/message-view?message_id=352910

Collapse
Posted by Gustaf Neumann on
Hi Ben,

the more general coding guidelines page is [1]. Several of the mentioned code metrics are implemented since many years on the OpenACS web site for all packages (see e.g. [2]). Of course, more metrics can be added. Much of the work invested into OpenACS over the last three years went into improving code quality, such as removing obsolete idioms, deprecated functions, dead code, unused .xql statements, security concerns, obsolete coding practices, etc. This affects and Tcl/HTML/SQL (e.g. in much of the PostgreSQL code was developed having pg7 or earlier in mind). I was in many place reluctant with code reformatting according to the guidelines, since such a change makes it quite hard for installations with locally modified code to merge the code. Furthermore, several of the quality measures above are hard to quantify.

I am not sure, how many people look a t the code metrics, but it can be certainly improved. There are as well several "code smell detectors" in automated testing (e.g. checking naming conventions, comments, etc.). I am not sure, how many people are actively using automated testing. Improving this is as well worthwhile.

Anyhow, it is certainly possible to add various code metrics to e.g. the package manager (e.g. before making e.g. an .apm file) or the the OpenACS site. Contributions are always welcome.

-g

[1] http://openacs.org/xowiki/coding-standards-index
[2] http://openacs.org/xowiki/acs-subsite

Collapse
Posted by Benjamin Brink on
Thank you, Gustaf. I can appreciate the need to weigh changes vs. integration with existing code. Having re-factored thousands of lines of my own code, I can see how significant changes easily fit into a major number upgrade. At some point I imagine next script will contain the majority of new code.

I use automated testing, but wasn't aware of the code compliance tests. I'll investigate, and see if there is anything I can contribute to fostering practical coding standards.
cheers,
Ben

Dear Benjamin,
I'm afraid that for "pure" TCL/TK is rather difficult to find proper tools supporting code analysis and metrics.

For Naviserver (C/C++) please have a look at:
http://sonarsrv.spazioit.com/overview?id=42350

For OpenACS (pure Web/Html, no TCL) have a look at:
http://sonarsrv.spazioit.com/overview?id=myOpenACS

For general information about the topic of Quality Platforms, Code Analysis, Metrics and the like, please have a look at:
http://www.spazioit.com/pages_en/sol_inf_en/code_quality_en/

Hope this helps,
Maurizio

Still me...
I did forget the "experimental" CLANG Code Analysis:
http://clang.spazioit.com/
Collapse
Posted by Benjamin Brink on
Thank you, Maurizio. Code analysis software seems to be an interesting specialty. Tcl has these: http://wiki.tcl.tk/3162 I have not tried them on OpenACS tcl code, but imagine they are workable so long as one includes ACS core code when testing external packages.
Dear Benjamin,
as far as I understand none of the TCL tools you mentioned has a corresponding plugin in SonarQube.

Anyhow I believe the main question is another one:
Is there anyone willing to put energy and time in checking OpenACS TCL code and publishing the results?

I do that for Naviserver and I try to keep up with Gustaf fast pace. Effort wise we are talking about an activity which requires 15/20 minutes a day.
Is there anyone wanting to do something similar for OpenACS?

Thank you,
Maurizio

Collapse
Posted by Benjamin Brink on

Yeah, besides SonarQube, it looks like each of those TCL tools has limitations in the context of OpenACS. I think Gustaf smartly writes code to detect known problem patterns and edit code. How else can one maintain such a large code base?

My guess is, one can write snippets that focus on various parts of the code base, to identify patterns, make changes etc. or perhaps make large parts of existing code redundant.

I'm not qualified to analyze or suggest acs core changes. Yet, as a package developer, it's clear that this mature framework has some rigidity that I can avoid at will.

For example, right now I notice that db_dml seems clumsy in that every update/create has to be manually generated. Most of the tables I code have essentially the same 20+ lines that vary only by the table name and table elements. So a proc could replace all this replication: a [db_dml_update table_name] where the proc gets the element names, builds the sql and submits to db_dml; But submitting dynamically built sql to db_dml is not recommended. The way around this seems to suggest having to build a revised db_dml that accepts an extra parameter -table. If the revision is accepted into tcl core, then a proc could be written to detect these cases and revise them.. And yet..

There is NextScripting. Maybe this kind of pattern is already addressed in NextScripting. I would suspect so. And so learning and coding NextScripting becomes a way to regenerate a system from within. And then, maybe something like http://wiki.tcl.tk/2131 (XOtclide) but built inside NextScripting becomes part of an evolving solution.

And back to your question, Maurizio:

Is there anyone willing to put energy and time in checking OpenACS TCL code and publishing the results?

What does this entail exactly? Producing useful statistics? creating a proc spell checker and 80 column auto indenter to suggest changes?

If you have tool that potentially is able to identify hot spots, issues, code smells and the like... this tool is useless unless it is run regularly. On top of that the produced results have to me made available to the developers... who is willing to do that for OpenACS?

I do that for Naviserver, it is a matter of time and energy... of how much effort one wants to dedicate to this activity, if any...

When I see something like http://sonarsrv.spazioit.com/overview?id=42350 done also for the TCL code in OpenACS, something executed and maintained every single day, then we can talk...

About OpenACS the mere HTML analysis, as shown in http://sonarsrv.spazioit.com/overview?id=myOpenACS, the codebase uses plenty of deprecated HTML constructs. I believe it is time to get rid of these deprecated elements/attributes.

Maurizio

Collapse
Posted by Benjamin Brink on
Hi Maurizio,
Yes, html should be updated, but it looks like that OpenACS HTML analysis mainly points to issues with html created in the docbook documentation. Docbook writes its own html, so maybe docbook has become more of a maintenance burden than benefit.

Html in adp pages are not so simple to analyze. Wonderful work has gone into upgrading html/css partly by invoking per page html analysis at time of rendering. You can see this in the latest OpenACS release 5.9.

To me, at least 80% of that documentation should be removed, because it no longer fits the code, or is off topic etc. Some of it should be put directly in the code at the points of discussion to provide needed context, even if it is just historical. The remaining 20% should be integrated into the wiki.

As for Tcl testing, when I write a linter for my own code, then we'll see if it is applicable for acs-core --I think not. Because I write code as if I just learned the language. Much of acs-core uses syntax and terms I have little experience with.

I see that SonarQube has a ruleset for Python. Maybe that can be adjusted for Tcl with your experience and knowledge of naviserver?

cheers,
Ben

Collapse
Posted by Antonio Pisano on
Hello Benjamin,

having followed what direction OpenACS is going, I can tell you that very little changes will be made to the db_* API: all new development takes place in XoTcl + Nsf now, at least in the core team.

You're right about the big deal of replication one can incur when forcing a logically OO model onto a procedural language: you always end up writing the same procs (create, update, delete, read... boring) on tables that can be very similar.

Fortunately, the new OO API addresses a lot of this: one can define an object in a very declarative way and this will bring to you table creation, acs_object inherithance and all the CRUD methods for free. It is a big leap forward compared with how I am still used to develop (our codebase is currently all-procedural).

If you want to start digging as I did, you could consider this tutorial http://alice.wu-wien.ac.at:8000/s5-xotcl-core-tutorial/slides. Sooner or later I was thinking about writing a little demo package showing some of the OO potential of OpenACS and letting you fiddle with it. In the meantime, you could have a look at the code in xotcl-core package

All the best

Antonio

Collapse
Posted by Benjamin Brink on
Thank you, Antonio. You confirm that the OO/nsf paradigm is worth the time to learn, especially for anyone wanting to work on a team with new projects.

As for me, I need to finish what I have started already. I will cheerfully suffer through my commitments knowing that there is something even more fun to do.