Run Time Access |
This page gives a brief tutorial on the use of the RTA package by building a trivial application which uses RTA. We define the task, define the tables, give the code, describe how to build and link the application, and describe how to the test the application using the database interface.
The last section describes how to solve the problem using the FUSE package and librtafs to give a virtual file system interface to the application.
We want to build an application in which we expose an internal array of data structures as both a database and as a file system. The array has twenty structures and each structure has a writeable integer, a writeable float, a writeable string, and a read-only string. Both strings are thirty characters in length. A callback on the writeable string does two things: it replaces any '<' and '>' characters with a '.', and it copies the reversed string into the the read-only string.
The structure would be defined as:
#define NOTE_LEN 30
struct MyData {
int myint;
float myfloat;
char notes[NOTE_LEN];
char seton[NOTE_LEN];
};
We allocate storage for the data as:
#define ROW_COUNT 20 struct MyData mydata[ROW_COUNT];
Externally, we want this array of structures to be seen as a Postgres table called "mytable" or as a directory called "mytable" mounted under "/tmp/mydir".
We need to tell the RTA package about our table. To do this we need to build a COLDEF structure for each of the four columns, and to build a TBLDEF structure for "mydata", the array of structures.
We use an array of four COLDEFs to describe our columns. This is pretty simple:
COLDEF mycolumns[] = {
{
"mytable", /* the table name */
"myint", /* the column name */
RTA_INT, /* it is an integer */
sizeof(int), /* number of bytes */
offsetof(struct MyData, myint), /* location in struct */
0, /* no flags */
(int (*)()) 0, /* called before read */
(int (*)()) 0, /* called after write */
"A sample integer in a table"
},
{
"mytable", /* the table name */
"myfloat", /* the column name */
RTA_FLOAT, /* it is a float */
sizeof(float), /* number of bytes */
offsetof(struct MyData, myfloat), /* location in struct */
0, /* no flags */
(int (*)()) 0, /* called before read */
(int (*)()) 0, /* called after write */
"A sample float in a table"
},
{
"mytable", /* the table name */
"notes", /* the column name */
RTA_STR, /* it is a string */
NOTE_LEN, /* number of bytes */
offsetof(struct MyData, notes), /* location in struct */
0, /* no flags */
(int (*)()) 0, /* called before read */
reverse_str, /* called after write */
"A sample note string in a table"
},
{
"mytable", /* the table name */
"seton", /* the column name */
RTA_STR, /* it is a string */
NOTE_LEN, /* number of bytes */
offsetof(struct MyData, seton), /* location in struct */
RTA_READONLY, /* a read-only column */
(int (*)()) 0, /* called before read */
(int (*)()) 0, /* called after write */
"Another sample note string in a table"
},
};
For each of the four structure elements we gave the associated table name, column name, type, size, position in the struct, flags, read and write callbacks, and a short string to describe it.
We define the table in a similar way:
TBLDEF mytbldef {
"mytable", /* table name */
mydata, /* address of table */
sizeof(struct MyData), /* length of each row */
ROW_COUNT, /* number of rows */
mycolumns, /* array of column defs */
sizeof(mycolumns) / sizeof(COLDEF),
/* the number of columns */
"", /* no save file */
"A sample table"
};
Note the double quotes to specify no save file. This field is a pointer-to-string and the pointer can not be null, although the string can be.
The source code for our simple application is available here as myappdb.c. The first section of code of note is the list of include files:
#include <stdio.h> #include <stdlib.h> #include <stddef.h> /* for 'offsetof' */ #include <string.h> /* for 'strlen' */ #include <unistd.h> /* for 'read/write/close' */ #include <sys/socket.h> #include <netinet/in.h> #include "../src/rta.h"
You will need to set the path to the rta.h file to reflect where you install it.
We have already presented the code for the structures and arrays of structures. Let's look at the code in the main() program to perform all initialization:
int main()
{
int i; /* a loop counter */
int srvfd; /* FD for our server socket */
int connfd; /* FD for conn to client */
struct sockaddr_in srvskt; /* server listen socket */
struct sockaddr_in cliskt; /* socket to the UI/DB client */
int adrlen;
char inbuf[INSZ]; /* Buffer for incoming SQL commands */
char outbuf[OUTSZ]; /* response back to the client */
int incnt; /* SQL command input count */
int outcnt; /* SQL command output count */
int dbret; /* return value from SQL command */
/* init mydata */
for (i=0; i<ROW_COUNT; i++) {
mydata[i].myint = 0;
mydata[i].myfloat = 0.0;
mydata[i].notes[0] = (char) 0;
mydata[i].seton[0] = (char) 0;
}
/* init the RTA package and tell it about mydata */
rta_init();
if (rta_add_table(&mytbldef) != RTA_SUCCESS) {
fprintf(stderr, "Table definition error!\n");
exit(1);
}
This is pretty standard stuff. We allocate our socket structures and other local variables. We initialize the RTA package and add our one table.
The last piece of initialization is to set up the socket to listen for incoming client connections. Remember that each UI program will treat our program as if it were a Postgres database. We have to accept TCP connections from Postgres clients.
By-the-way: the following code is pretty horrendous. It uses blocking IO, ignores error conditions, and makes wildly optimistic assumptions about socket IO. My goal is to make the code understandable by getting it into as few lines as possible.
/* We now need to open a socket to listen for incoming
* client connections. */
adrlen = sizeof (struct sockaddr_in);
(void) memset ((void *) &srvskt, 0, (size_t) adrlen);
srvskt.sin_family = AF_INET;
srvskt.sin_addr.s_addr = INADDR_ANY;
srvskt.sin_port = htons (8888);
srvfd = socket(AF_INET, SOCK_STREAM, 0); /* no error checks! */
bind(srvfd, (struct sockaddr *) &srvskt, adrlen);
listen (srvfd, 4);
The main loop in our program waits for a TCP connection from a client, then loops reading the Postgres encoded stream of SQL commands, processing them with dbcommand(), and then writing any results back to the client. While a connection can close for errors, we normally expect the client to request an orderly close to the connection.:
/* Loop forever accepting client connections */
while (1) {
connfd = accept(srvfd, (struct sockaddr *) &cliskt, &adrlen);
if (connfd < 0) {
fprintf(stderr, "Error on socket/bind/listen/accept\n");
exit(1);
}
incnt = 0;
while (connfd >= 0) {
incnt = read(connfd, &inbuf[incnt], INSZ-incnt);
if (incnt <= 0) {
close(connfd);
connfd = -1;
}
outcnt = OUTSZ;
dbret = dbcommand(inbuf, &incnt, outbuf, &outcnt);
switch (dbret) {
case RTA_SUCCESS:
write(connfd, outbuf, (OUTSZ - outcnt));
incnt = 0;
break;
case RTA_NOCMD:
break;
case RTA_CLOSE:
close(connfd);
connfd = -1;
break;
case RTA_NOBUF:
close(connfd);
connfd = -1;
break;
}
}
}
The callback function is fairly straightforward. Remember that the write callback is called on a column after the write of all update data to the columns. Our write callback on the 'notes' field is called after the data has been written. It loops through the string replacing greater-than and less-than symbols with a period, and copying a reverse of the string into the 'seton' field.
int reverse_str(char *tbl, char *col, char *sql, void *pr,
int rowid, void *poldrow)
{
int i,j; /* loop counters */
i = strlen(mydata[rowid].notes) -1; /* -1 to ignore NULL */
for(j=0 ; i>=0; i--,j++) {
if (mydata[rowid].notes[i] == '<' ||
mydata[rowid].notes[i] == '>')
mydata[rowid].notes[i] = '.';
mydata[rowid].seton[j] = mydata[rowid].notes[i];
}
mydata[rowid].seton[j] = (char) 0;
return(0); /* no errors */
}
The information in this section should be sufficient to download, build, and run the sample application.
Download the RTA package from the download page given above. Untar it. Go to the src directory and do a make.
# tar -xzf rta-0.7.4.tgz
# cd rta-0.7.4
# cd src
# make librtadb.so.2
Download the myappdb.c file (available here). Place the file in the test directory. Build it with the command:
# gcc myappdb.c -o myappdb -L../src -lrtadb
Note that we are telling the application that the library it needs is in the sibling src directory.
To run the application we will need to tell it where to find the shared object library. From the test directory enter:
# export LD_LIBRARY_PATH=/home/rta-0.7.4/src
# ./myappdb &
That is all there is to it. Please use the contact page to report any problems you find.
If all has gone well, we now have an application running which pretends to be a Postgres database server. Only instead of a database, it is our sample application offering up its internal tables for use by various Postgres clients.
The first client to try is the Postgres browser tool, psql. This tool is part of the base Postgres install. Remember that we have our application table, mytable, as well as the RTA internal tables, rta_tables, rta_columns, rta_dbg, and rta_stat. See the API section for a description of these tables.
To start psql to our application enter:
# psql -h localhost -p 8888
Postgres should respond with:
Welcome to psql, the PostgreSQL interactive terminal.
Type: \copyright for distribution terms
\h for help with SQL commands
\? for help on internal slash commands
\g or terminate with semicolon to execute query
\q to quit
Let's try some SQL commands to play with the tables.
Give me a list of the tables:
SELECT name FROM rta_tables;
name
-------------
rta_tables
rta_columns
rta_dbg
rta_stat
mytable
(5 rows)
You do not need to use upper case for the SQL keywords. The psql program accepts both upper and lower case.
Display the contents of mytable:
SELECT * FROM mytable;
myint | myfloat | notes | seton
-------+----------------------+-------+-------
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
0 | 0.0000000000 | |
(20 rows)
Set all of the myint values to 44 and display the new table:
UPDATE mytable SET myint=44;
UPDATE 20
SELECT * FROM mytable;
Note that psql tells us how many rows were modified.
Set the notes field to "hi mom!"
UPDATE mytable SET notes="hi mom!";
UPDATE 20
Set the notes field of only row 0 to "<b>hi mom!</b>":
UPDATE mytable SET notes="<b>hi mom!</b>" LIMIT 1 OFFSET 0;
UPDATE 1
The LIMIT says how many rows to change, and the OFFSET tells where to start the changes. The default LIMIT is all rows, and the default OFFSET is zero.
Turn on the trace of all SQL commands and send the output to both stderr and syslog:
UPDATE rta_dbg SET trace=1, target=3;
You should now see all the commands echoed on the standard error output of the program. The commands are also being sent to your syslog facility.
The C program presented here, rta_client.c, illustrates how to get data out of our application from a C program. You need the postgresql-devel package to build this program. Build and run the program with:
# gcc rta_client.c -o rta_client -lpq
# ./rta_client
The program sets the field myint to a value of 44 and then gets and prints the fields myint, myfloat, and notes for all twenty rows in the table.
Note that we test for success in different ways depending on whether or not we expect data back from the command. Note also that the returned data is always a string. You need to scan it into a variable for other processing.
The PHP program presented here, rta_client.php, illustrates how to get data out of our application from a PHP program. You need the php-pgsql package to run this program. If you are running Apache, check for the PHP interpreter in modules directory. On RH8.0 look for libphp4.so in /etc/httpd/modules. You can verify that the PHP-Postgres library is loaded by checking for pgsql.so in the /usr/lib/php4 directory. Clearly your configuration for PHP and Postgres may be different than what is described here.
This program gets and displays the fields myint, myfloat, and notes for all twenty rows in the table.
Perhaps a better way to evaluate PHP is to install the table editor which comes with the RTA package. Just put the four .php files onto your web server (being sure that PHP and php-pgsql are installed). The table editor is the application running on the Live Demo link above.
The steps to build myapp with the file system interface are similar to those of the database interface. First let's consider the C code, myappfs.c.
The table definitions are exactly those shown above and are not included in this listing. The file system package is built on top of the database package and we still use the calls to rta_init() and rta_add_table(). The two new calls are to rtafs_init() and do_rtafs().
int main()
{
fd_set rfds; /* read bit masks for select statement */
int i; /* a loop counter */
int fsfd; /* FD to the file system */
/* init mydata */
for (i=0; i>ROW_COUNT; i++) {
mydata[i].myint = 0;
mydata[i].myfloat = 0.0;
mydata[i].notes[0] = (char) 0;
mydata[i].seton[0] = (char) 0;
}
/* init the rta package and tell it about mydata */
rta_init();
if (rta_add_table(&mytbldef) != RTA_SUCCESS) {
fprintf(stderr, "Table definition error!\n");
exit(1);
}
/* Mount the tables as a file system. */
fsfd = rtafs_init("/tmp/mydir");
/* Loop forever processing file system requests */
while (1) {
FD_ZERO(&rfds);
FD_SET(fsfd, &rfds);
(void) select(fsfd + 1, &rfds, (fd_set *) 0, (fd_set *) 0,
(struct timeval *) 0);
if ((fsfd > 0) && (FD_ISSET(fsfd, &rfds)))
{
do_rtafs();
}
}
}
We use a select() to detect arrival of a file system command and use do_rtafs() to do the read and execution of the command. You can use routines in the FUSE package directly if you want to have the read() statement in your main loop.
Note that an important side effect of the rtafs_init() call is that the signal handlers for SIGHUP, SIGINT, and SIGTERM are set to a routine that tries to unmount the file system prior to exit.
Be sure that the mount point, /tmp/mydir/, exists and has the same UID and GID as the user running the program.
Before compiling myappfs.c we need to build librtafs.so and
to do that we need the FUSE package installed. FUSE is available
from:
FUSE uses the usual download, ./configure, make, make install commands, and it requires that the kernel sources be installed prior to its build. Once FUSE is built, install the kernel module, fuse.o, and test your installation by trying several of the programs in the example directory.
Once FUSE is installed and verified, you can build the librtafs library.
# tar -xzf rta-0.7.0.tgz
# cd rta-0.7.0
# cd src
# make librtafs.so.2
Copy the myappfs.c program to the test directory and compile it with the command:
# gcc myappfs.c -o myappfs -L../src -lrtadb -lrtafs
As before, make sure sure the RTA libraries are installed in one of the system library directories or explicitly tell the system to look for the libraries in the src directory:
# export LD_LIBRARY_PATH=/home/rta-0.7.0/src
Make sure that /tmp/mydir exists and run the program.
# mkdir /tmp/mydir
# ./myappfs &
If all has gone well you should be able to see the application mounted on /tmp/mydir:
# mount
/proc/fs/fuse/dev on /tmp/mydir type fuse (rw,nosuid,nodev,user=bsmith)
An ls /tmp/mydir/* should show all of the tables as directories with all of the columns as subdirectories within each table directory.
# ls /tmp/mydir/*
/tmp/mydir/mytable:
myfloat/ myint/ notes/ ROWS/ seton/
/tmp/mydir/rta_columns:
flags/ length/ noff/ ROWS/ type/
help/ name/ readcb/ table/ writecb/
/tmp/mydir/rta_dbg:
facility/ priority/ rtaerr/ syserr/ trace/
ident/ ROWS/ sqlerr/ target/
/tmp/mydir/rta_stat:
nauth/ nrtaerr/ nselect/ nsqlerr/ nsyserr/ nupdate/ ROWS/
/tmp/mydir/rta_tables:
address/ help/ ncol/ rowlen/ savefile/
cols/ name/ nrows/ ROWS/
Remember that in each column directory is a file for each row in the table. The rows are just numbered starting with zero.
# ls /tmp/mydir/mytable/notes
0000 0002 0004 0006 0008 0010 0012 0014 0016 0018
0001 0003 0005 0007 0009 0011 0013 0015 0017 0019
The ROWS directory also has a file for each row in the table. A cat of one of the ROWS files displays the whole row with commas separating the columns and with strings enclosed in quotes.
# cat /tmp/mydir/mytable/ROWS/0000
0, 0.000000, "", ""
Either cat or echo can change a column or a row.
# echo "Babylon 5" > /tmp/mydir/mytable/notes/0000
There is no way to update the whole table at once, but it is pretty easy to update each row, one at a time.
# for i in /tmp/mydir/mytable/notes/*
do
date > $i
done
When updating a whole row, you may need to use single quotes to surround strings.
# echo "44, 12.345, 'Hi Mom!', ''" > /tmp/mydir/mytable/ROWS/0000