Note: all examples are drawn from the random Tcl and Random Perl sample modules.
The moodss and moomps applications are composed of the core software and one or several modules. Modules are implemented as Tcl packages and thus usually comprise a Tcl, Perl or Python source file and a pkgIndex.tcl file as required by the Tcl package implementation.
All the module code and data are kept in a separate namespace. The module data is stored is a single array including some configuration data used when the module is loaded by the core, and variable data (displayed in the application table and possibly graphical viewers).
If a module is synchronous, it must start updating its data when requested by the core. If a module is asynchronous, its data may be updated at any time. The synchronous or asynchronous nature is specified in the configuration data for the module.
A module is a package, it must have a name and a version.
Note: version number management is important as the module major version number is taken into account when storing data cells history in a database (can be done from both the moodss GUI application and the moomps daemon). Modules developers must insure that no major changes occur between minor releases of the module. In other words, if there is a change in the module data structure, such as adding a new column, changing the order of columns, changing column data, such as label or type (message and anchor can be altered without problems), then a new major version number must be used, to preserve database integrity (a new major version number results in the new data to be stored with new, thus different identifiers).
package provide random 1.1 |
package Random;
...
BEGIN {
...
our $VERSION = 1.1;
}
|
__version__ = 1.1 |
Module names can be any combination of the following characters (minus the characters not allowed by the specific module language, of course):
All module procedures and data are kept in a specific namespace bearing the module name.
namespace eval random {
array set data {
...
}
proc update {} {
...
}
}
|
our %data;
our @data;
...
sub update() {
...
|
form = {}
data = []
...
def update():
...
|
The update function is not needed when the module is asynchronous.
Note: at this time, it is required for Perl and Python modules, as they cannot be asynchronous.
The module configuration defines the data table column headers, help information, ... This data never changes during the lifetime of the application.
namespace eval random {
array set data {
updates 0
0,label name 0,type ascii 0,message {user name}
1,label cpu 1,type real 1,message {cpu usage in percent}
2,label disk 2,type integer 2,message {disk usage in megabytes}
3,label command 3,type dictionary 3,message {most time consuming command} 3,anchor left
pollTimes {10 5 20 30 60 120 300}
sort {1 decreasing}
indexColumns {0 3}
helpText {This is a simple demonstration module ...}
}
...
}
|
our %data;
$data{updates} = 0;
$data{columns}[0] = {
label => 'name', type => 'ascii', message => 'user name'
};
$data{columns}[1] = {
label => 'cpu', type => 'real', message => 'cpu usage in percent'
};
$data{columns}[2] = {
label => 'disk', type => 'integer', message => 'disk usage in megabytes'
};
$data{columns}[3] = {
label => 'command', type => 'dictionary', message => 'command name',
anchor => 'left'
};
$data{pollTimes} = [10, 5, 20, 30, 60, 120, 300];
$data{sort} = {1 => 'decreasing'};
$data{indexColumns} = [0, 3];
$data{helpText} = 'This is a simple demonstration module ...';
...
|
form = {}
form['updates'] = 0
form['columns'] = [
{'label': 'name', 'type': 'ascii', 'message': 'user name'},
{'label': 'cpu', 'type': 'real', 'message': 'cpu usage in percent'},
{'label': 'disk', 'type': 'integer', 'message': 'disk usage in megabytes'},
{'label': 'memory', 'type': 'integer', 'message': 'memory usage in kilobytes'},
{'label': 'command', 'type': 'dictionary', 'message': 'command name', 'anchor': 'left'}
]
form['pollTimes'] = [10, 5, 20, 30, 60, 120, 300]
form['sort'] = {1: 'decreasing'}
form['indexColumns'] = [0, 4]
form['helpText'] = 'This is a simple demonstration module ...'
...
|
The updates member is a counter used to keep track of the number of times that the module data was updated, and is also used by the core to detect when module data display should be updated (see variable data for more information).
The label members ($data(n,label) in Tcl, $data{columns}[n]{label} in Perl, form['columns'][n]['label'] in Python, with n the column number), define the text to be displayed as column titles. There must be as many label members as they are columns. The titles must contain no ? character.
The type members ($data(n,type) in Tcl, $data{columns}[n]{type} in Perl, form['columns'][n]['type'] in Python) define the type of the corresponding column data. Valid types are simply those that the Tcl lsort command can handle: ascii, dictionary, integer and real, plus the clock type, which accepts any format that the Tcl clock format command can handle (see the trace module for an example). There must be as many type members as they are columns.
The message members ($data(n,message) in Tcl, $data{columns}[n]{message} in Perl, form['columns'][n]['message'] in Python) define the text of the help message to be displayed in a floating yellow window (widget tip, balloon, also see User Interface) as the user moves the mouse pointer over column titles. It can be composed of only a few words or multiple formatted lines. There must be as many message members as they are columns.
The anchor member ($data(n,anchor) in Tcl, $data{columns}[n]{anchor} in Perl, form['columns'][n]['anchor'] in Python) is optional. Column data is either centered by default, tucked to the left or right side of the column. Valid values are center, left or right.
Note that column numbers start at 0. There must be no hole in the column numbers sequence.
The pollTimes member is a list of valid poll times (in seconds) for the module. The list is not ordered, as its first element represents the default poll time value to be used when the moodss application starts. This value may be overridden by a command line argument. The smallest value in the list is used by the core as the lowest possible poll time and checked against when the user enters a new value through the poll time dialog box. The list must not be empty.
Note that the list is also used by moodss as a set of possible choices in the dialog box used to set the new poll time. The user may still directly input any value as long as it is greater than or equal to the minimum value.
If the module is asynchronous (data can be updated at any time and not in response to update procedure invocations (no polling required)), the pollTimes member must be a single negative integer value representing the preferred time interval for viewers that require one (only graphs at this point). For example, if you wish graph viewers to have a display interval of 10 seconds, use:
namespace eval random {
array set data {
...
pollTimes -10
...
}
...
}
|
In this case, the graph viewers time range (knowing that they feature 100 time points) would be 1000 seconds. I guess that the value that you should specify as the pollTimes member should be the expected average update interval for your asynchronous data. Note that the graphical viewers x axis always display properly labeled absolute time ticks in any case.
When several asynchronous modules are loaded with no synchronous modules, the interval used for all relevant viewers is the average (in absolute value) of all module intervals. For example, if you load 2 asynchronous modules, one with a pollTimes member of -10 and the other of -20, then a 15 seconds interval value is retained. Note that the interval can be forced through the --poll-time command line argument.
If at least one synchronous module is loaded concurrently with any number of asynchronous modules, the actual application poll time (the one that can be set with the then available poll time dialog box) is used.
The indices member is an optional list that specifies the table columns that should be displayed. If not specified, all the table columns are visible.
The sort entry is optional. It defines the index of the column which should be initially used as a reference for sorting the data table rows, and in which order (increasing or decreasing) the rows should be sorted. The column index for sorting works like the -index Tcl lsort command option, that is rows are sorted so that that specific column appears sorted in the specified order. The specified column must be visible (see indices member documentation above).
The indexColumns list specifies the columns required to uniquely identify a row in the table. In database talk, it represents the table key. To maintain backward compatibility, it is optional and defaults to 0, the leftmost column. The index columns are used when creating data viewer elements: their label is built by concatenating the key value for the cell row with the cell column title. The key value is the concatenation of the index column values for the cell. When specified, all the columns in the list must be visible (see indices member documentation above).
The helpText member specifies a text of any length, to be displayed when the user requests help information on the current module from within the help menu. The text can be HTML formatted or plain. HTML formatted help requires the <HTML> and <BODY> tags to be present, while tables and frames are not supported (and many other tags: stick to formatted text at the moment). The core will render HTML formatted or plain text in the module help window according to the help text contents.
The views member is optional. If specified, it defines one or more views to be used in place of the default view. One table will be displayed per view. For each view, 1 member must be defined: indices, the sort member being optional (syntax and usage are identical to the default table members). A swap optional member may be used if the table data is to be displayed with columns and rows swapped, which is generally the case when the data table has 1 or 2 rows, or a fixed number of rows (see memstats module for an example). The swap value is a boolean and must be either 0 or 1.
namespace eval random {
array set data {
...
views {
{indices {0 1 3 4} sort {1 decreasing} swap 1}
{indices {0 2 4} sort {2 decreasing}}
}
...
}
...
}
|
$data{views} = [
{indices => [0, 1, 3, 4], sort => {1 => 'decreasing'}, swap => 1},
{indices => [0, 2, 4], sort => {2 => 'decreasing'}}
];
|
form['views'] = [
{'indices': [0, 1, 3, 4], 'sort': {1: 'decreasing'}, 'swap': 1},
{'indices': [0, 2, 4], 'sort': {2: 'decreasing'}}
]
|
The switches member is optional. A switch is a single letter or a string with the - or + sign as header. The boolean value (0 or 1) specifies whether the switch takes an argument. If the switches member exists, an appropriate initialize procedure must be provided by the module (see Initialization). The core will take care of parsing the command line and reject any invalid switch / value combination for the module. The switches value may not be changed in the initialize procedure.
Note: if you wish to pass a sensitive password as an option to the module, use a special option name (-passwd or -password or --passwd or --password) so that the core knows when not to display or store readable characters, for example when entering a password in a dialog box, or storing password value in the moomps daemon data cells history database.
For example, a recent random module configuration is as follows:
array set data {
updates 0
0,label name 0,type ascii 0,message {user name}
1,label cpu 1,type real 1,message {cpu usage in percent}
2,label disk 2,type integer 2,message {disk usage in megabytes}
3,label memory 3,type integer 3,message {memory usage in kilobytes}
4,label command 4,type dictionary 4,message {command name}
pollTimes {10 5 20 30 60 120 300}
sort {1 decreasing}
indexColumns {0 4}
helpText {...}
views {
{indices {0 1 3 4} sort {1 decreasing}}
{indices {0 2 4} sort {2 decreasing}}
}
switches {-a 0 --asynchronous 0}
}
|
our %data;
$data{updates} = 0;
$data{columns}[0] = {label => 'name', type => 'ascii', message => 'user name'};
$data{columns}[1] = {label => 'cpu', type => 'real', message => 'cpu usage in percent'};
$data{columns}[2] = {label => 'disk', type => 'integer', message => 'disk usage in megabytes'};
$data{columns}[3] = {label => 'memory', type => 'integer', message => 'memory usage in kilobytes'};
$data{columns}[4] = {label => 'command', type => 'dictionary', message => 'command name'};
$data{pollTimes} = [10, 5, 20, 30, 60, 120, 300];
$data{sort} = {1 => 'decreasing'};
$data{indexColumns} = [0, 4];
$data{helpText} = '...';
$data{views} = [
{indices => [0, 1, 3, 4], sort => {1 => 'decreasing'}},
{indices => [0, 2, 4], sort => {2 => 'decreasing'}}
];
$data{switches} = {'-a' => 0, '--asynchronous' => 0};
|
form = {}
form['updates'] = 0
form['columns'] = [
{'label': 'name', 'type': 'ascii', 'message': 'user name'},
{'label': 'cpu', 'type': 'real', 'message': 'cpu usage in percent'},
{'label': 'disk', 'type': 'integer', 'message': 'disk usage in megabytes'},
{'label': 'memory', 'type': 'integer', 'message': 'memory usage in kilobytes'},
{'label': 'command', 'type': 'dictionary', 'message': 'command name'}
]
form['pollTimes'] = [10, 5, 20, 30, 60, 120, 300]
form['sort'] = {1: 'decreasing'}
form['indexColumns'] = [0, 4]
form['helpText'] = '...'
form['views'] = [
{'indices': [0, 1, 3, 4], 'sort': {1: 'decreasing'}},
{'indices': [0, 2, 4], 'sort': {2: 'decreasing'}}
]
form['switches'] = {'-a': 0, '--asynchronous': 0}
|
The identifier member is optional. It is string that uniquely identifies this module. If set, it is displayed by the core in the initial data tables title area and data cell labels in viewers. This feature can be used for example in modules that gather data from a remote host: in such a case, the identifier could be set to dataType(hostName) (the ps module uses ps(host) string). The allowed character set for the identifier is identical to the allowed set for a module name.
Note that the identifier member is usually set in the module initialize procedure, as it usually depends on the module options.
The resizableColumns member is optional. It is a boolean value (0 or 1) which specifies whether displayed data table(s) columns can be manually resized by the user with the mouse. Not available on a per view basis. If not present or set to 0, all the module data columns are automatically sized according to their content.
The persistent member is optional. It is a boolean value (0 or 1) that tells whether row numbers are identically mapped from data row keys across module instances. Most modules should have that capability as viewers, thresholds, database history, ... rely on the internal row/column pair to identify a cell: see row numbering section for detailed information. If absent, the module is considered non-persistent.
The 64Bits member is optional. It is a boolean value (0 or 1) which specifies whether row numbers are unsigned 64 bit integers (which requires a Tcl 8.4 or above core version). If absent or false, row numbers as unsigned 32 bit integers are assumed.
The initialize procedure, if it exists, is invoked by the core before any update occurs (when the update procedure is invoked if the module is synchronous).
In order for a module to accept command line arguments, the initialize procedure must exist (more information below). Otherwise, if the module can take no arguments, it stays optional. In that case, it is rather redundant with in-line module code (outside of any module function or procedure). When the module takes no arguments, so does the initialize procedure if it exists.
The initialize procedure is mandatory when the module supports command line arguments, and in such a case takes option values as arguments.
Let us use the following command line as example:
$ moodss random --asynchronous --other-option value -x 1234 |
options(--asynchronous) = options(--other-option) = value options(-x) = 1234 |
switches {-a 0 --asynchronous 0 --other-option 1 -x 1}
|
$options{--asynchronous} = 1
$options{--other-option} = value
$options{-x} = 1234
|
$data{switches} =
{'-a' => 0, '--asynchronous' => 0, '--other-option' => 1, '-x' => 1};
|
options['--asynchronous'] = 1 options['--other-option'] = value options['-x'] = 1234 |
form['switches'] =\
{'-a': 0, '--asynchronous': 0, '--other-option': 1, '-x': 1}
|
In all cases, data members other than updates and switches can be set or updated in the initialize procedure, and successfully taken into account by the core.
Example for a module that take arguments, where a module identifier is generated according to the -i or --identify command line switches:
proc initialize {optionsName} {
upvar $optionsName options
variable data
if {[info exists options(-i)] || [info exists options(--identify)]} {
# generate a random module identifier:
set data(identifier) "random [expr {int(rand() * 100)}]"
}
}
|
sub initialize(%) {
my %option = @_;
if ($option{'-i'} || $option{'--identify'}) {
# generate a random module identifier:
my $identifier = int(rand(100));
$data{identifier} = "random $identifier";
}
}
|
def initialize(options):
global form
identify = 0
try: identify = options['--identify']
except:
try: identify = options['-i']
except: pass
if identify:
form['identifier'] = 'randpy ' + str(randint(0, 100));
|
The core waits for the initialize procedure to be completed before initializing the next module. For modules likely to initialize slowly and/or susceptible to initialization failure, it is advised to allow the user interface to be updated in the meantime:
proc initialize {optionsName} {
...
set file [open "| /usr/bin/rsh -nl $remote(user) $remote(host) cat /proc/net/arp"]
fileevent $file readable {set ::arp::remote(busy) 0}
vwait ::arp::remote(busy)
...
}
|
Similar techniques must be used in such cases within the module update procedure, so that the user interface and any modules running concurrently are not prevented to update. The remote capable moodss modules contain various coding techniques achieving that functionality, which I am sure you can improve on.
If a procedure named terminate exists in the module namespace, it is invoked when the module is unloaded dynamically or when the application (moodss or moomps) exits. Make sure that it cannot potentially hang, for it would also hang the application.
That procedure must take no arguments. For example:
proc terminate {} {
# do some cleanup
}
|
sub terminate() {
# do some cleanup
}
|
def terminate():
# do some cleanup
pass
|
The tabular data (variable data) that the module code must update is stored in the data array (same as the module configuration data in Tcl, named after the module configuration data hash in Perl, and after the module configuration data list in Python).
In case of a synchronous module, the core invokes the module update procedure (which obviously must exist) when it is time to refresh the data display (tables and possibly graphical viewers). At this time, the update procedure may update the tabular data straight away (synchronous operation) or launch a request for later data update (asynchronous operation (only available in Tcl)).
In case of an asynchronous module, variable data may be updated at any time. The update procedure may not exist.
For all module types, it actually does not matter when the data is updated. The core will know that fresh data is available when the updates array member is set (actually incremented as it also serves as a counter for the number of updates so far).
It is the module programmer's responsibility to increment this counter right after all tabular data has been updated.
For example, retrieving information for the processes running on a machine is a local operation that can be achieved in a reasonably small amount of time. In such a case, data would be updated immediately and the updates variable incremented at the same time.
But if the data has to be retrieved from across a network, waiting for it to come back would cause a delay that the user would certainly notice, as the application would not respond to mouse or keyboard input during the whole time that it would take to fetch the whole data. In such cases, it is easier to let the update procedure return immediately without setting the updates variable, which would be incremented at a later time, only when the data would become available.
For example, in Tcl, when waiting for data to come across a network connection, the fileevent command could be used on a non blocking channel, where the script to be evaluated when the channel becomes readable would increment the updates array member.
In Perl, this is not yet possible, since the Perl interpreter is dormant outside of the update() function call (a solution to this problem is being studied).
In Python, it is possible to use threads, but at this time, there is no way to pass the information back to the Tcl interpreter outside of the Tcl interpreter driven call of the update function (a solution to this problem is being studied).
proc update {} {
variable data
array set data "
0,0 john 0,1 1234 0,2 4567 0,3 cc
1,0 william 1,1 8901 1,2 2345 1,3 xedit
2,0 anny 2,1 6789 2,2 0123 2,3 ps
4,0 peter 4,1 4567 4,2 8901 4,3 ls
6,0 laura 6,1 2345 6,2 6789 6,3 emacs
3,0 robert 3,1 1234 3,2 5678 3,3 top
"
incr data(updates)
}
|
our @data;
...
sub update() {
@data = (
['john', 1234, 4567, 'cc'],
['william', 8901, 2345, 'xedit'],
['anny', 6789, 0123, 'ps'],
['peter', 4567, 8901, 'ls'],
['laura', 2345, 6789, 'emacs'],
['robert', 1234, 5678, 'top']
);
}
|
data = []
...
def update():
global data
data = [
['john', 1234, 4567, 'cc'],
['william', 8901, 2345, 'xedit'],
['anny', 6789, 0123, 'ps'],
['peter', 4567, 8901, 'ls'],
['laura', 2345, 6789, 'emacs'],
['robert', 1234, 5678, 'top']
]
|
The column number must start from 0 up to the total number of columns minus 1 (no holes are allowed in the column sequence).
The row number can take any positive integer value (between 0 and 2147483647) and be defined in any order, as long as it is unique during the lifetime of the module data. If a new row is created, it must take a value that was never used: the index of a row that has disappeared cannot be reused. Row numbers need not be consecutive.
When all rows (or only those table cells that have changed) have been updated, the updates member array must be incremented so that the core knows that it can update the table data display.
The Tcl random module source code can be made to function asynchronously: please look into the random.tcl file.
The module data is internally organized in a 2 dimensional table with the traditional rows and columns. While column numbers must imperiously start from 0 and be consecutive, row numbers can take any value between 0 and 18446744073709551615 (any values allowed for a 64 bit unsigned integer) (even on 32 bit hardware, as long as Tcl/Tk 8.4 or above is used), and need not be consecutive.
However, row numbers need to be unique and the module code must maintain a one to one relationship between row numbers and table row keys. In other words, as in a database table, the module internal table has an associated key: the column or columns that uniquely identify a row of data.
This is very important in saved dashboards, as viewers, thresholds, database history, ... rely on the internal row/column pair to identify a cell. If the next time the module is started, the row number points to data belonging to another monitored item, any viewers containing data cells with that row number, for instance, would display unexpected results.
For example, in a module monitoring the traffic between computers in an enterprise Intranet, the table key will be formed by the 2 columns: source and destination hosts. It is vital to maintain a reliable mapping between the table key and the row numbers, function giving the same result every time the module is loaded. In the computer traffic module, the IP addresses of the source and destination computers could be used to form a unique row number, by concatenating the two 32 bit IP addresses to form a unique 64 bit row number.
As an example of what not to do, consider a module displaying the statistics per hard disk drive on a UNIX computer, with the row numbers assigned in sequence. For example, on a machine classically using 2 SCSI disks and an IDE CD-ROM device:
| row number | disk device | kilobytes/second written | kilobytes/second read |
|---|---|---|---|
| 1 | hdd | 0 | 0 |
| 2 | sda | 0 | 1214.2 |
| 3 | sdb | 511 | 7899 |
With this module loaded, a dashboard with a graphical viewer displaying the written data rate for the sda disk (data cell: (2,2) (row = 2, column = 2)) is created and saved.
One day, an IDE disk drive is added to the machine. After power-up, the internal data table then becomes:
| row number | disk device | kilobytes/second written | kilobytes/second read |
|---|---|---|---|
| 1 | hda | 7654 | 0 |
| 2 | hdd | 0 | 0 |
| 3 | sda | 0 | 67 |
| 4 | sdb | 1580 | 0 |
Now, next time the dashboard is reloaded in moodss, the graphical viewer will continue displaying the data cell (2,2) value, which is now the written data rate for the wrong disk, hdd, which since it is a CD-ROM drive, will remain at zero for ever.
Of course, the same error could occur on a threshold entry, data archived in a SQL database, ...
As a module developer, you must insure that row numbers are positive. Be careful with the incr and expr commands in Tcl, as they may generate negative row numbers, for example after incrementing 2147483647 with Tcl version 8.3. If you want to make sure to generate 32 or 64 bit numbers, the format command is a safe way:
set row [format %u $value] ;# 32 bits unsigned integer set row [format %lu $value] ;# 64 bits unsigned integer |
Various techniques can be used to generate unique row numbers. For example, let us have a look at some of the modules included in the moodss distribution:
The persistent data member should be set to 1 once the module implementation guarantees that row numbers are reliably generated (see configuration section).
Note: a hash library is included in the distribution.
The moodss GUI core is multi-lingual, thanks to Tcl internationalization support. Modules can also support different languages, for data column titles, help messages, module help, ... All that is required is the following code near the beginning of the module source code:
package require msgcat namespace import msgcat::* mcload . |
Then any string candidate for internationalization needs be coded as:
set message [mc "Good bye"] ;# = "Au revoir" in French |
msgcat::mcset fr "Good bye" "Au revoir" |
Note: the .msg files must be encoded in UTF-8.
A different technique must be used for the module HTML help file:
if {\
([info exists ::env(LC_ALL)] && [string match fr* $::env(LC_ALL)]) ||\
([info exists ::env(LANG)] && [string match fr* $::env(LANG)])\
} {
set file [open random-fr.htm] ;# (written in French)
} else {
set file [open random.htm] ;# (written in English)
}
|
Also note that the HTML file must be encoded in ISO8859-1 for French, EUC-JP for Japanese, ... (please refer to the Tcl documentation for more information).
A module is a package in the Tcl sense. When writing a module, you must then provide a pkgIndex.tcl file along with the module code file, placed in the module directory. The pkgIndex.tcl file is very simple, as the following example shows:
package ifneeded random 1.1 "source [file join $dir random.tcl]" |
package ifneeded Random 1.1 "source [file join $dir Random.pm]" |
package ifneeded randpy 1.1 "source [file join $dir randpy.py]" |
Modules can be installed at any valid place that the Tcl core allows (look at the pkg_mkIndex manual page for more information).
When you unpack moodss, you will find the sample modules in sub directories. The current directory (.) is appended to the auto_load global list variable so that sample modules can be found when moodss is run from the unpacking directory.
For example, if you unpacked moodss in /home/joe/moodss-X.x/, you will find the random module package in /home/joe/moodss-X.x/random/ so that the following will work:
$ cd /home/joe/moodss-X.x/ $ wish moodss random |
You can install your new modules in the default location: /usr/local/lib/ on UNIX. For example, if you move the files in /home/joe/moodss-X.x/random/ to /usr/local/lib/random/, moodss will still be able to find the random module (again, look at the pkg_mkIndex manual page for more information).
Note: when the --debug option is used in the command line, moodss will allow loading modules that are not in a moodss directory (a directory containing the "moodss" string in its full path).
Please take a look at the INSTALL file for the latest information on how to install the moodss application itself.
You may want to inform the user of the module activity. You may use the message area (called the messenger, across the bottom of the application main window) through the following API:
pushMessage "error: message..." popMessage flashMessage "warning: message..." numberOfSeconds |
Your message can be any kind of string (1 line only: it should fit in a reasonably wide main window), and numberOfSeconds being optional and defaulting to 1. Note that messages sent to the message area are also displayed in the trace module table(s) if it(they) exist(s) (see below). In such cases, popping messages has no effect on trace module tables.
One should use the message area facilities with discretion, as it is already used by the application core for informing the user of modules loading, initialization, updates, context sensitive help, ... My advice is to use it for important messages or errors pertinent to the module itself.
The module name should not appear at the beginning of the messages as it is automatically prepended internally (see pushMessage above).
I suggest using the importance level (as in thresholds) as the header. Possible values are, in increasing importance order: debug, info, notice, warning, error, critical, alert, emergency. Look for examples in included modules.
You also have the option of using trace tables (also see trace module) through the following API:
traceMessage "critical: message..." |
The difference is that messages from the module are displayed in the trace module tables if they exist (see trace, there can be more than 1 trace module loaded), remain visible to the user for a longer period of time, and can be multi-line.
Finally, displaying separate (toplevel) message windows using Tk is of course always possible (if you load Tk in the module). Keep in mind that the core, not being aware of the module event that has taken place, will continue to invoke the module update procedure (unless the module is asynchronous, of course). In such a case, you may want to use an internal (to the module) busy flag so that the update procedure immediately returns until the user acknowledges the informational message. Other strategies are of course possible (let me know if you have one that you think should appear here as an example).
In order to allow clean error handling with module loading either by command line arguments or dynamically while the application is running, the following rules need be followed:
Specifying the module name in the messages is not necessary as the core handles it.
Before moodss was packaged with a daemon (moomps), it was natural for such a GUI application to immediately react on module errors: when there is an error in a module initialization stage, the error is displayed in a dialog box and loading the module is aborted. Since the user is in front of the screen, he can try to fix the problem right away.
Now if the same module is used by the moomps daemon, in case of initialization error when the daemon is started unattended, the error message is logged but the module is never loaded. This is a problem if the error cause is a temporary communication breakage with a remote host, for example.
One solution would be for the moomps daemon to keep retrying loading a module until successful, but that may cause more problems than it solves, especially if the module loads a binary library or triggers some other mechanism which may not accept such forceful treatments (possibly when using secure channels, encryption, ...), or if the module contains bugs that prevents clean unloading, which may result in moomps hanging or crashing.
The module knows best if, when and how to retry its initialization, so if the module supports the --daemon option (no arguments), it is automatically set by moomps for the purpose of letting the module know whether it should keep trying to reinitialize on failures, or more generally do something different in daemon mode.
Moodss modules can also be written in the Perl language, and even internally use Perl threads for potentially processor intensive or blocking operations.
An embedded Perl interpreter is used for each loaded module, thus achieving complete independence between modules.
The tclperl library is required (available on my homepage), at least version 3.1 if Perl threads are to be used.
As described in the section about displaying messages in Tcl, such functionality has become available to Perl modules starting with moodss version 19.1. The API is very similar:
pushMessage("error: message...");
popMessage();
flashMessage("warning: message...", numberOfSeconds);
traceMessage("critical: message...");
|
Also, managing timers (timeouts, ...) is possible without blocking the core (sleep() would block the GUI, for example), by using the following function, which causes the core to invoke a piece of code (as in Perl eval()) back in the module Perl interpreter, after a specified number of milliseconds:
after(milliseconds, 'Perl code...'); |
Note: an example is given in the Random module, in order to implement the asynchronous mode.
Finally, when using Perl threads in a Perl module, which is the only way to ever prevent blocking the core (for example, the GUI could be otherwise blocked waiting for a connection to reestablish with a down computer), the following function must be used within a thread:
yield('updated');
|
It gives control back to the core once a thread is finished working. In turn, the core calls the updated() function in the Perl module. Only the 'updated' function is supported at this time. For example, Perl queues can be used to pass data from the threads to the Perl module interpreter. This functionality has become available starting with moodss 19.1.
Note: this is required as the parent Perl interpreter is generally no longer running at the time the thread has done its job, and thus needs to be awaken by the application core, which is always running.
Further documentation for programming Perl modules can be found in the fully commented Random.pm and Threaded.pm modules source files.
Moodss modules can also be written in the Python language.
An embedded Python interpreter is used for each loaded module, thus achieving complete independence between modules.
The tclpython library (version 3.0 or above, for Python 2.2 or above) is required (available on my homepage as source and Red Hat rpms).
As described in the section about displaying messages in Tcl, such functionality has become available to Python modules starting with moodss version 20.0 provided you use the tclpython extension version 4 or above. The API is very similar:
pushMessage("error: message...");
popMessage();
flashMessage("warning: message...", numberOfSeconds);
traceMessage("critical: message...");
|
Also, managing timers (timeouts, ...) is possible without blocking the core (sleep() would block the GUI, for example), by using the following function, which causes the core to invoke a piece of code (as in Python eval()) back in the module Python interpreter, after a specified number of milliseconds:
after(milliseconds, 'Python code...'); |
Note: an example is given in the randpy module, in order to implement the asynchronous mode.
Complete documentation for programming Python modules can be found in the fully commented randpy.py module source file.
The Tcl language allows non blocking implementations in 2 ways:
Non blocking code is very useful when coding modules, as it prevents the user interface (the moodss core) from being hung when a module, for example, cannot reach a remote host. Such coding techniques are therefore recommended for a graphical user interface, which should stay responsive at all times.
Unfortunately, handling all the different cases and errors is not easy, and coding using threads or events in Tcl involves 2 different techniques.
That is why the object oriented line task class (linetask.tcl file) was created and is included in the moodss source code. It allows, via a common interface, the transparent handling of communication with a data pipe, uses threads if available, or asynchronous event handling otherwise.
It is used in several modules, and especially in the pci module (documentation), where all the code is thoroughly commented and can be used as a reference for all remote capable modules that need to communicate with a remote machine via a pipe.
Note that threads are used in priority if available, as they are guaranteed to never block the main process, which is still not the case using events, as they can hang, for example, on a non responding DNS query.
You may find the hashes.tcl file (in the packlibs sub-directory), which includes a couple of unsigned 64 bit hash functions for hashing strings and lists of 32 bit integers.
The algorithm was chosen after a fair amount of testing, against other well known implementations. It produces very good results, especially considering its simplicity. It is a slightly modified version of the Tcl internal C implementation.
In order to use it inside a Tcl module, just include the following line early in the code:
package require hashes |
You may then invoke the following procedures, which return a 64 bit unsigned integer (between 0 and 18446744073709551615) either on a string of characters or a list of 32 bit integers:
hash64::string string ?repeatable? hash64::numbers32 list ?repeatable? |
The optional repeatable boolean argument (0 by default) should be used in most cases since it insures that the generated hash is always the same for a given input string or list, even across program instances, which is required if the module must be persistent. When a duplicate hash is found, which should be very rare, the problem should be caught as the following code sample (from kernmods.tcl) shows:
if {[catch {set row [hash64::string $name 1]} message]} {
flashMessage "warning: duplicate row ($name) removed: $message"
continue ;# skip row from data processing loop, for example
}
|
In such cases, the module documentation should suggest that the user reduces the number of displayed rows, possibly via a filtering option developed for the module.
Note: for Python and Perl modules, I suggest porting the Tcl implementation from the hashes.tcl file, which should be quite easy.
Modules that can communicate with a remote computer, and that use a TCP based protocol (as the rsh and ssh commands), may hang in case of communication problems. Unfortunately, such problems also cause the core (the moodss GUI or the moomps daemon) to hang until the timeout elapses or the communication error is caught or solved.
Fortunately, there are two ways to circumvent those problems in the Tcl language: event based programming or threads. Unfortunately, depending on the Tcl/Tk version used, threads facilities might not be available, and coding for either events or threads rapidly becomes cumbersome.
That is why the linetask library was made part of the moodss distribution, and is used in several included modules. This library, implemented as a stooop class, can be found in the linetask.tcl file in the packlibs sub-directory. In order to use it in a module, include the following line early in the code:
package require linetask 1 |
The lineTask class provides a unified implementation of a non-blocking channel library, by internally using either threads or events, while preserving the same API in both cases. The preferred method is the threads implementation, as even in the events case, timeouts caused by a failing remote host or network may still hang the module process.
In order to use threads, the Tcl threads library (binary compiled along the Tcl/Tk installation) and the included thread worker class implementation (see threads.tcl in the packlibs sub-directory) are needed:
package require Thread 2.5 package require threads 1 package require linetask 1 |
Then, if using ssh to connect to the remote host, pseudo-tty allocation is disabled as obviously the session is not interactive and the standard error channel redirected to the standard output so errors on the remote end can be detected and reported. A callback procedure for reading data lines from the remote shell is specified, while access is specified as bi-directional since scripts will be sent (written) to the remote host shell, translation is UNIX compatible, threaded code is enabled and command is not to be immediately executed:
set command "ssh -T -l $remote(user) $remote(host) 2>@ stdout"
set remote(task) [new lineTask\
-command $command -callback pci::read\
-begin 0 -access r+ -translation lf -threaded 1\
]
|
Later on, when the module is requested to update its data by the core, the task command is executed and a script initiating the data retrieval sent to the remote shell:
lineTask::begin $remote(task) lineTask::write $remote(task) "/sbin/lspci | tr '\\n' '\\v'" |
Note that the task is line oriented (hence the lineTask name), so all data lines are gathered into one using the tr command and the vertical tab character (any rare control character would probably do) as a separator. Having the data packed in a single line removes potential line buffering problems from the remote shell.
Until the data becomes available from the remote host, the core (moodss or moomps) can continue servicing user requests and update other modules. When the data arrives, the callback is invoked as follows:
proc read {line} {
process [split [string trimright $line \v] \v]
}
|
The packed data line is transformed back into a list of data lines before processing (the process procedure is an internal module procedure that formats the data, updates the module data array and finally lets the core know that new data has arrived).
The pci module contains a fully commented reference implementation of data retrieval and processing from a remote host, including events or threads programming automatic switch depending on Tcl/Tk core capabilities, user selectable ssh or rsh communication with remote host and reliable error handling and recovery.
Note: for Python and Perl modules, such facility is not available at this time.
When data to be displayed is an array of unrelated values, the only solution is to organize the data in a table with a single row and 1 column per value (which does not prevent the use of several views for displaying the data).
In such a case, no index column is required and as a matter of fact rather gets in the way. The trick is to use an empty column 0, use as index by the core by default, and to prevent it from being displayed by using 1 or more views.
Please look at the cpustats module code for a working example.
In the module data array, you cannot leave or set a numeric cell empty when no data is available. Use the ? character (also used in statistics tables) instead.