The text file is your buffer, with something like multilog, you can have more than one, so you only process the ones that are closed. This provides a separation of writing/reading from the file.
Please note that I have no idea what data you are collecting, but only that eventually everything starts as a string, and Tcl is very good at producing strings. AOLserver is even better at handling strings (including binary data).
But I did look at the HDF, which I found out stands for Hierarchical Data Format, there appears to be a significant amount of up-front data-modeling. If you continue to work on this, it would be very interesting to hear from you on how you use HDF.
The details of applying this to AOLserver (how to use this in a filter) are simple by comparison, so continue to ask for help. A good starting point is simply writing data to a file, even from a Tcl shell, that does what you want, and can be used by whatever downstream tools you choose, after that, adding that to AOLserver is easy.
