nsdbi(n) 0.3 nsdbi "NaviServer Database Interface"

Name

nsdbi - Commands for querying an SQL database.

Table Of Contents

Synopsis

Description

nsdbi provides an easy to use interface to databases.

COMMANDS

dbi_1row ?-db name? ?-timeout t? ?-bind array|set? ?-array name? ?--? query

Execute a query which is expected to return exactly one row. If no rows are returned then an error is raised.

The column values of the row are set as variables in the current stack frame using the column names as Tcl variable names. If the -array option is given then an array of that name is created instead with the column names corresponding to keys in the array.

dbi_1row {select first, last from peeps where id = 1}
set mysteryMan "$first $last"

Array-name syntax may be used to build an array result:

dbi_1row -array name {select first, last from peeps where id = 1}
set mysteryMan "$name(first) $name(last)"
dbi_1row {select first as 'name(first)', last as 'name(last)' from peeps where id = 1}
set mysteryMan "$name(first) $name(last)"
dbi_0or1row ?-db name? ?-timeout t? ?-bind array|set? ?-array name? ?--? query

Execute a query which is expected to return either zero rows or exactly one row. If no rows are returned the result is 0 (false), otherwise it is 1 (true). If more than 1 row is returned an error is raised.

if {[dbi_0or1row {select first, last from peeps where id = 1}]} {
    set mysteryMan "$first $last"
} else {
    set mysteryMan "Benny Hill"
}
dbi_rows ?-db name? ?-timeout t? ?-bind array|set? ?-columns varname? ?-max nrows? ?-append? ?-quote quoting-type? ?--? query ?template? ?default?

Execute a query which is expected to return zero or more rows. If no rows are returned then the result is the empty string. Otherwise a single, flat list of values is returned.

foreach {first last} [dbi_rows {select first, last from peeps}] {
    append result "<li>$first $last</li>\n"
}
set result
<li>Dirty Harry</li>
<li>Jolly Roger</li>
string ?template?

If template is given then for each row returned by the query, template variables are substituted for the values of the corresponding columns. The templated rows are concatenated and returned as a string.

dbi_rows {select first, last from peeps} {<li>$first $last</li>\n}
<li>Dirty Harry</li>
<li>Jolly Roger</li>

If a template variable name does not correspond to a column name in the result then a Tcl variable by the same name is substituted.

proc greet_the_people {{greeting "Hello"}} {
    dbi_rows {select first, last from peeps} {
        $greeting, $first $last!
    }
}
greet_the_people "Yo"
Yo, Dirty Harry!
Yo, Jolly Roger!

There are also three special variables:

dbi(rowidx)

The zero-based row index. The rows are numbered 0, 1, 2 etc.

dbi(rownum)

The 1 based row number. The rows are numbered 1, 2, 3 etc.

dbi(parity)

The string "even" or "odd", based on the rowidx.

dbi_rows {select first, last from peeps} {
    <tr class="$dbi(parity)"><th>$dbi(rownum)</th><td>$first $last</td></tr>
}
<tr class="even"><th>1</th><td>Dirty Harry</td></tr>
<tr class="odd"><th>2</th><td>Jolly Roger</td></tr>

If a template variable does not correspond to a column name, one of the three special variables, or an existing Tcl variable, an error is raised.

string ?default?

If no rows are returned then an error is raised, unless the default argument is given in which case it becomes the complete result.

dbi_rows {select leader from government} {${leader}! \n} {nobody here but us chickens}
nobody here but us chickens
-columns varname

If the -columns option is given then it is the name of a Tcl variable which will be created to hold a list of the column names of the result.

This is useful in queries of the form "select * from ..." (but don't do that), and for generic procedures which handle a variety of queries, such as formatting queries as an HTML table.

% dbi_rows -columns cols {select first, last from peeps}
{{Dirty Harry} {Jolly Roger}}
% set cols
{first last}
-max nrows

If the -max option is given then it is an assertion that the query will return no more than this number of rows. An error will be raised if it does. If -max is not given then the default maxrows for the db is used (see dbi_ctl below).

Useful values for -max are: >1000 when you're absolutely sure that a paricular query needs to return that much data (which would be unusual on the web) and 1, in which case dbi_rows acts like dbi_0or1row, except returning the result instead of setting variables.

-append

If the -append option is given then the result is appended efficiently to the ADP output buffer. If no ADP frame is active an error is raised.

<ul>
<% dbi_rows -append -- {select first, last from peeps} {<li>$first $last</li>} %>
</ul>
-quote quoting-type

Quote substituted variables (tcl variables or column values) in the template according to the quoting type. valid quoting types are "none", "html", or "js". This option can be only specified, when a template is given.

dbi_dml ?-db name? ?-timeout t? ?-bind array|set? ?--? query

Execute a DML query: insert, update, delete; or DDL query: create table etc. The result will be the number of rows affected by the query, for example the number of rows updated during an update query. If the query is not DML or DDL, an error will be thrown (the query is not run).

dbi_eval ?-db name? ?-timeout t? ?-transaction isolation-level? ?--? script

Get a database handle and then evaluate script, keeping the same handle open for the duration. Any dbi commands within the script will use the same handle if the handle the command needs would have come from the same db.

If the -transaction option is given then the commands are run in a transaction -- either all the commands succeed or they all fail. You may nest transactions.

The following transaction isolation-levels are supported. See your specific database for details.

readuncommited

May be abbreviated uncommitted.

readcommited

May be abbreviated committed.

repeatableread

May be abbreviated repeatable.

serializable
dbi_ctl command ?...?

Control various aspects of a configured database.

dblist

Returns a list of db names available to the current virtual server. This includes all globally configured databases plus, those loaded for this server.

default ?db?

The name of the default db for the current server, if any. The default db is used when the -db option is not specified.

If a db name is specified it becomes the new default db, and the return value is the old default db name.

driver ?db?

Returns the name of the driver for the specified db.

database ?db?

Returns the name of the database product for the specified db, e.g. postgresql or mysql.

bounce ?db?

Mark all handle in db stale. At the next checkinterval, or as active handles are returned to the db pool, their connection with the database will be closed.

maxhandles db ?maxhandles?

This setting controls how many handles, i.e. how many open connections, are made to the underlying database. This number determines how many threads may perform queries in parallel.

The default is 0 which means that each thread will open and cache a handle when first needed. The handle will be released when the the thread exits. This is ideal for an embedded database such as sqlite or the embedded mysql option, and is also appropriate if each thread which accesses the db, such as the conn threads, always query the database and your server can accommodate one database backend per thread.

If you have more threads accessing the database than you can afford backends then you will need to pick a number for maxhandles. Choose too high and your database will suffer excessive locking and make poor progress when your server is busy; memory consumption may be excessive. Choose too low and threads will serialize waiting for handles. Make a guess and then keep an eye on the handlegets and handlemisses stats. If both numbers are high and handlemisses is high relative to handlegets then you have a busy server which is starved of database handles, and you should increase maxhandles.

timeout db ?timeout?

This is the timeout in seconds that a thread will wait for a handle if none are available. You can override it for each command using the -timeout option. The default is 10 seconds.

Errors due to timeouts are tracked in the statistics.

maxrows db ?maxrows?

This defines the default maximum number of rows a query may return. The default is 1000 which is high enough that any query returning more rows is probably in error. You can override this setting on a per-query basis by passing the -max option to dbi_rows, or you can raise the default if you routinely run into this limit. -1 means no limit.

maxidle db ?maxidle?

This is the number of seconds that an open handle will linger unused, after which it is automatically closed (handles are checked every checkinterval seconds). The default is 0, which means handles are never closed due to idleness.

maxopen db ?maxopen?

This is the maximum number of seconds a handle can remain open, regardless of whether it is used or not. After this timeout, when the handle is returned it will be automatically closed. The default is 0, which means handle are not closed due to age.

maxqueries db ?maxqueries?

If maxqueries is set then a handle will be closed after this number of queries have been executed. The default is 0, which dissables this feature.

This option can be used to ensure that a cached plan for a prepared statement does not become stale after many inserts/updates. Set it to something high, like 10,000 or 100,000, as setting it too low it may negate the benefit of prepared statement caching.

stats ?db?

Return the accumulated statistics for db in array get format.

array set stats [dbi_ctl stats db1]
puts "handle miss rate: [expr {($stats(handlegets) * 100) / $stats(handlemisses)}]%"

The stats list contains the following keywords:

handlegets

The number of times a handle was successfully retrieved from the pool. If maxhandles is 0 this will increase by 1 for each thread that uses the db. If maxhandles is more than 1, this number will increase by 1 each time a dbi command is executed, except for commands within a dbi_eval block where a single handle is reused with the enclosed commands.

handlemisses

The number of times a handle was requested but none was available, perhaps after a timeout. If this number is high try increasing maxhandles (but make sure that handlefailures is not significant).

handleopens

The number of times a handle was connected/reconnected to the database. If this number is high relative to handlegets then maybe one of maxidle, maxopen or maxqueries is too aggressive. See also bounces.

handlefailures

The number of times a connection attempt to the database failed. This should be 0, or very close.

queries

Total number of queries by all handles for this db. This number always increases.

agedcloses

Number of times a handles was closed due to the maxopen time expiring.

idlecloses

Number of times a handle was closed due to the maxidle time expiring.

oppscloses

Number of times a handle was closed due to maxqueries being reached.

bounces

Number of times all handles for the db were bounced with the dbi_ctl bounce command.

OPTIONS

-db name

The name of the db to query. If not specified, the default db is used. Database names are defined each time a driver module is loaded -- the module name is the db name.

-timeout t

Time to wait for a database handle if none available immediately. t is in ns_time format and is either an absolute time in the future if large, or an offset from the current time if small.

If -timeout is not specified then the timeout configuration value is used (which defaults to 10 seconds).

If a handle does not become available during the timeout period a Tcl error is thrown with errorCode "NS_TIMEOUT". (If this is not caught the server will return a 503 Busy response).

-bind array|set

Specifies an array or ns_set which will be used to bind values to any variables in a query. The names of the query variables must match the keys in the array or set. The array or set must contain a key for each of the variables in the query.

If -bind is not specified and the query contains variables, the command will look for Tcl variables of the same name in the current stack level. There must be one Tcl variable for each query variable.

ARGUMENTS

string query

A query containing database specific SQL. The query may include bind variables of the form :variable - a name preceded by a single colon. Do not quote bind variables. Always use bind variables in preference to substituting Tcl variables into the query string.

set x 1
dbi_1row {select column from table where key = :x}

Bind variables automatically handle binary values. Simply pass a Tcl byte array type object and the raw bytes will be passed directly to the database. All other types must be in a string representation understood by the underlying database.

If a bind variable is the empty string "" then a NULL is passed to the database. Simillarly, NULLs in result sets become the empty string.

CONFIGURATION

To use the nsdbi commands you must first load a driver, e.g. the postgreSQL driver nsdbipg. Drivers can be loaded more than once to create multiple pools of database handles, each with it's own configuration options.

ns_section "ns/modules"
ns_param   global1       nsdbipg.so
ns_param   global2       nsdbipg.so
...
ns_section "ns/server/server1/modules"
ns_param   db3           nsdbipg.so
ns_param   db4           nsdbipg.so
...
ns_section "ns/server/server1/module/db4"
# The following parameters can be changed at runtime using dbi_ctl
# and are explained above.
ns_param   default       true
ns_param   maxhandles    0
ns_param   timeout       10
ns_param   maxidle       0
ns_param   maxopen       0
ns_param   maxqueries    0
ns_param   maxrows       1000
# The following parameters are configured at server-startup.
ns_param   user          dbusername
ns_param   password      dbpassword
ns_param   database      dbname
ns_param   cachesize     [expr 1024*1024]
ns_param   checkinterval 600

Standard configuration options which are available with every driver:

user

The user name with which to log in to the database.

password

The password with which to log in to the database.

database

Which database to connect to.

cachesize

The number of bytes used to cache the query text of prepared statements. The default is 1MB. There is one cache per-handle.

checkinterval

Check for idle handles every checkinterval seconds. The default is 600 seconds.

Each driver may also takes driver-specific parameters.

EXAMPLES

Select rows from a table using db x. Only rows in which column a matches the bind variable will be returned:

set a "someValue"
set rows [dbi_rows -db x {
    select a, b
    from table
    where a = :a
}]

The following example shows how to bind a Tcl variable in a query and also how to change the name of the result variable, which in this case is an array.

set a 1
dbi_1row {select foo as 'array(foo)' from table where a = :a}
ns_log notice "foo: $array(foo)"

See Also

nsdb, subst

Keywords

NaviServer, adp, database, query, sql