API Specification

- Preamble

RTA is a specialized memory resident database interface. It is not a stand-alone server but a library which attaches to your program and offers up the program's internal structures and arrays as database tables. It uses a subset of the Postgres protocol and is compatible with the Postgres bindings for "C", PHP, and the Postgres command line tool, psql.

This page contains the defines, structures, and function prototypes for the 'RTA' package. The information in this page is taken from the rta.h file in the RTA package. The rta.h file is the authoritative reference to the API.

- Limits

Here are the defines which describe the internal limits set in the RTA package. You are welcome to change these limits; just be sure to recompile the RTA package using your new settings.

- Data Structures

Each column and table in the database must be described in a data structure. Here are the data structures and associated defines to describe tables and columns.

The column definition (COLDEF) structure describes one column of a table. A table description has an array of COLDEFs to describe the columns in the table.

The table definition (TBLDEF) structure describes a table and is passed into the DB system by the rta_add_table() subroutine.

- Subroutines

Here is a summary of the few routines in the RTA API:

dbcommand() - Depacketize and execute Postgres commands
The main application accepts TCP connections from Postgres clients and passes the stream of bytes (encoded SQL requests) from the client into the RTA system via this routine. If the input buffer contains a complete command, it is executed, nin is decrement by the number of bytes consumed, and RTA_SUCCESS is returned. If there is not a complete command, RTA_NOCMD is returned and no bytes are removed from the input buffer. If a command is executed, the results are encoded into the Postgres protocol and placed in the output buffer. When the routine is called the input variable, nout, has the number of free bytes available in the output buffer, out. When the routine returns nout has been decremented by the size of the response placed in the output buffer. An error message is generated if the number of available bytes in the output buffer is too small to hold the response from the SQL command.

int
dbcommand(char *cmd - the buffer with the Postgres packet
          int  *nin - on entry, the number of bytes in 'cmd',
                      on exit, the number of bytes remaining in cmd
          char *out - the buffer to hold responses back to client
          int  *nout - on entry, the number of free bytes in 'out'
                       on exit, the number of remaining free bytes)
Return: RTA_SUCCESS   - executed one command
        RTA_NOCMD     - input did not have a full cmd
        RTA_CLOSE     - client requests a orderly close
        RTA_NOBUF     - insufficient space in output buffer

rta_add_table() - Add table to DB
Registers a table for inclusion in the DB interface. Adding a table allows external Postgres clients access to the table's content. Note that the TBLDEF structure must be statically allocated. The DB system keeps just the pointer to the table and does not copy the information. This means that you can change the contents of the table definition by changing the contents of the TBLDEF structure. This might be useful if you need to allocate more memory for the table and change its row count and address. An error is returned if another table by the same name already exists in the DB or if the table is defined without any columns. If a 'savefile' is specified, it is loaded. (See the rta_load() command below for more details.)

int
rta_add_table(TBLDEF *ptbl  -pointer to the TBLDEF to add);
Return: RTA_SUCCESS   - table added
        RTA_ERROR     - error

SQL_string()- Execute single SQL command
Executes the SQL command placed in the null-terminated string, cmd. The results are encoded into the Postgres protocol and placed in the output buffer. When the routine is called the input variable, nout, has the number of free bytes available in the output buffer, out. When the routine returns nout has been decremented by the size of the response placed in the output buffer. An error message is generated if the number of available bytes in the output buffer is too small to hold the response from the SQL command.

This routine may be most useful when updating a table value in order to invoke the write callbacks. (The output buffer has the results encoded in the Postgres protocol and might not be too useful directly.)

void
SQL_string(char *cmd - the buffer with the SQL command
           char *out - the buffer to hold responses back to client,
           int *nout - on entry, the number of free bytes in 'out'
                       on exit, the number of remaining free bytes);

rta_save() - Save table to file
Saves all "savetodisk" columns to the path/file specified. Only savetodisk columns are saved. The resultant file is a list of UPDATE commands containing the desired data. There is one UPDATE command for each row in the table.

This routine tries to minimize exposure to corrupted save files by opening a temp file in the same directory as the target file. The data is saved to the temp file and the system call rename() is called to atomically move the temp to the save file. Errors are generated if rta_save can not open the temp file or is unable to rename() it.

As a general warning, note that any disk I/O can cause a program to block briefly and so saving and loading tables might cause your program to block.

int
rta_save(TBLDEF *ptbl - pointer to the TBLDEF structure for the
                        table to save,
         char *fname  - null terminated string with the path and
                        file name for the stored data);
Return: RTA_SUCCESS   - table saved
        RTA_ERROR     - some kind of error

rta_load() - Load table from file
Load a table from a file of UPDATE commands. The file format is a series of UPDATE commands with one command per line. Any write callbacks are executed as the update occurs.

int
rta_load(TBLDEF *ptbl   - pointer to the table to be loaded,
         char *fname  - string with name of the load file);
Return: RTA_SUCCESS   - table loaded
        RTA_ERROR     - could not open the file specified

rta_config_dir() - Set directory prefix for file save/load
The string pointed to by configdir is saved and is prepended to the savefile names for tables with savefiles. This call should be used before loading your application tables. It is intended to make it simpler for applications which let the user specify an configuration directory on the command line.

If the savefile uses an absolute path (ie. one starting with '/') it is not prepended with the configuration directory.

This call is optional. If it is not used the current working directory is used instead.

int
rta_config_dir(char *configdir - path to the configuration files);
Return: RTA_SUCCESS   - config path set
        RTA_ERROR     - error, (not a valid directory?)

rtafs_init() - Initialize the virtual file system interface
The single input parameter is the mount point for the VFS. On success, the return value is a file descriptor to the VFS. On failure, a -1 is returned and errno is set. This file descriptor should be used in subsequent select() or poll() calls to notify your program of file system activity. An important SIDE EFFECT of rtafs_init() is that the signal handlers for SIGHUP, SIGINT, and SIGTERM are set. The signal handler tries to unmount the virtual file system. This routine is part of the librtafs library.

Note that FUSE requires that the owner and group of the mount point be the same as the owner and group of the program that does the mount. For example, if your mount point is owned by Apache, then your the UID of your program must be Apache as well.

int
rtafs_init(char *mountpoint  - desired mount point);
Return: int fd        - file descriptor or -1

do_rtafs() - Handle all virtual file system IO
This routine handles all file system IO for the virtual file system mounted by the rtafs_init() call. This routine should be called when there is activity on the file descriptor returned from rtafs_init(). It has no input or output parameters This routine is part of the librtafs library.

void
do_rtafs();

- RTA UPDATE and SELECT syntax

RTA IS AN API, *NOT* A DATABASE!

Neither the RTA UPDATE nor the RTA SELECT adhere to the Postgres equivalents. Joins are not allowed, and the WHERE clause supports only the AND relation. There are no locks or transactions.

SELECT:
    SELECT column_list FROM table [where_clause] [limit_clause]

SELECT supports multiple columns, '*', LIMIT, and OFFSET. At most MXCMDCOLS columns can be specified in the select list or in the WHERE clause. LIMIT restricts the number of rows returned to the number specified. OFFSET skips the number of rows specified and begins output with the next row. 'column_list' is a '*' or 'column_name [, column_name ...]'. 'where_clause' is 'col_name = value [AND col_name = value ..]' in which all col=val pairs must match for a row to match.

LIMIT and OFFSET are very useful to prevent a buffer overflow on the output buffer of dbcommand(). They are also very useful for web based user interfaces in which viewing the data a page-at-a-time is desirable.

Column and table names are case sensitive. If a column or table name is one of the reserved words it must be placed in quotes when used. The reserved words are: AND, FROM, LIMIT, OFFSET, SELECT, SET, UPDATE, and WHERE. Reserved words are *not* case sensitive. You may use lower case reserved words in your names.

Comparison operator in the WHERE clause include =, |=, >=, <=, >, and <.

You can use a reserved word, like OFFSET, as a column name but you will need to quote it whenever you reference it in an SQL command (SELECT "offset" FROM tunings ...). Strings may contain any of the !@#$%^&*()_+-={}[]\|:;<>?,./~` characters. If a string contains a double quote, use a single quote to wrap it (eg 'The sign says "Hi mom!"'), and use double quotes to wrap a string with embedded single quotes.

Examples:

 SELECT * FROM rta_tables

 SELECT destIP FROM conns WHERE fd != 0

 SELECT destIP FROM conns WHERE fd != 0 AND lport = 80

 SELECT destIP, destPort FROM conns
       WHERE fd != 0 
       LIMIT 100 OFFSET 0
 
 SELECT destIP, destPort FROM conns
       WHERE fd != 0
       LIMIT 100 OFFSET 0

UPDATE:
    UPDATE table SET update_list [where_clause] [limit_clause]

UPDATE writes values into a table. The update_list is of the form 'col_name = val [, col_name = val ...]. The WHERE and LIMIT clauses are as described above.

An update invokes write callbacks on the affected columns. All data in the row is written before the callbacks are called.

The LIMIT clause for updates is not standard Postgres SQL, but can be really useful for stepping through a table one row at a time. To change only the n'th row of a table, use a limit clause like 'LIMIT 1 OFFSET n' (n is zero-indexed).

Examples:

 UPDATE conn SET lport = 0;

 UPDATE ethers SET mask = "255.255.255.0",  
                   addr = "192.168.1.10"    
               WHERE name = "eth0"

 UPDATE conn SET usecount = 0 WHERE fd != 0 AND lport = 21

- Internal DB tables

The RTA API has five tables visible to the application:

The rta_tables table gives SQL access to all internal and registered tables. The data in the table is exactly that of the TBLDEF structures registered with rta_add_table(). This table is used for the generic table editor application. The columns of rta_tables are:

The rta_columns table has the column definitions of all columns in the DB. The data in the table is exactly that of the COLDEF structures registered with rta_add_table(). This table is used for the generic table editor. The columns of rta_columns are:

The rta_dbgconfig table controls which errors generate debug log messages. See the logging section below for the exact mapping. The RTA package generates no user level log messages, only debug messages. All of the fields in this table are volatile. You will need to set the values in your main program to make them seem persistent. (Try something like "SQL_string("UPDATE rta_dbgconfig SET dbg ....").) The columns of rta_dbgconfig are:

The rta_stat table contains usage and error statistics which might be of interest to developers. All fields are of type long, are read-only, and are set to zero the first time a table is added with rta_add_table(). The columns of rta_stats are:

- List of all error messages

There are two types of error messages available in the RTA package. The first type is the error messages returned in response to a failed SQL request. The messages of this type are:

The other type of error messages are internal debug messages. Debug messages are logged using the standard syslog() facility available on all Linux systems. The default syslog "facility" used is LOG_USER but this can be changed by setting 'facility' in the rta_dbg table.

You are welcome to modify syslogd in order to do post processing such as generating SNMP traps off these debug messages. All error messages of this type are sent to syslog() as: "rta[PID]: FILE LINE#: error_message", where PID, FILE, and LINE# are replaced by the process ID, the source file name, and the line number where the error was detected.

Following are the defines used to generate these debug and error messages. These are the messages that will appear in the syslog and on the stderr output. The "%s %d" at the start of each error string is replaced by the file name and line number where the error is detected.

         "System" errors 
"%s %d: Can not allocate memory\n"
"%s %d: Table save failure.  Can not open %s\n"
"%s %d: Table load failure.  Can not open %s\n"

         "RTA" errors 
"%s %d: Too many tables in DB\n"
"%s %d: Too many columns in DB\n"
"%s %d: Too many characters in table name: %s\n"
"%s %d: Too many characters in column name: %s\n"
"%s %d: Too many characters in help text: %s\n"
"%s %d: DB already has table named: %s\n"
"%s %d: Table already has column named: %s\n"
"%s %d: Column contains an unknown data type: %s\n"
"%s %d: Column contains unknown flag data: %s\n"
"%s %d: Too many columns in table: %s\n"
"%s %d: Not enough buffer space\n"

         "SQL" errors 
"%s %d: SQL parse error: %s\n"
"%s %d: Attempt to update readonly column: %s\n"

         "Trace" messages 
"%s %d: SQL command: %s  (%s)\n"

- How to write callback routines

As mentioned above, read callbacks are executed before a column value is used and write callbacks are called after all columns have been updated. Both read and write callbacks return an integer which is zero on success and non-zero on failure. Read callbacks have the following calling parameters:

Read callbacks are particularly useful to compute things like sums and averages; things that aren't worth the effort to compute continuously if it's possible to compute it just when it is used. A returned value of zero indicates that the read callback was successful.

Write callbacks can form the real engine driving the application. These are most applicable when tied to configuration changes. Write callbacks are also a useful place to log configuration changes.

Write callbacks have the same parameters as read callbacks with the addition of a pointer to a copy of the row before it was modified. Access to a copy of the unmodified row is useful to detect actual changes and not just updates with the same value.

Write callbacks are passed the following parameters: