AolServer4.0 Module Porting Guide

$Header: /cvsroot/aolserver/aolserver.com/docs/devel/c/as4_moduleportingguide.html,v 1.6 2003/07/11 20:19:19 mpagenva Exp $

Critical API Changes

1. The following C APIs have changed. Calls to these functions should be examined to verify that the call is still valid.

Ns_TclGetConn
Ns_TclInterpServer
interp argument is no longer ignored, and must be specified with a good interp value. (documentation must be changed).

For the following API functions, server argument must specify a defined server, and unless otherwise specified, must be non-null:

Ns_AuthorizeRequest if NULL, will always return NS_OK
Ns_PageRoot
Ns_RegisterConnCleanup
Ns_RegisterFilter
Ns_RegisterProxyRequest
Ns_RegisterServerTrace
Ns_SetLocationProc
Ns_SetRequestAuthorizeProc
Ns_SetUrlToFileProc
Ns_TclAllocateInterp if server is NULL, a 'non-server' interpreter is returned.
Ns_TclInitInterps
Ns_TclInitModule (documentation must be changed)
Ns_TclLibrary (now takes server as parameter) if server is NULL, returns the shared library name. Otherwise, returns server's library name.
Ns_UnRegisterProxyRequest
Ns_UrlToFile
Ns_UrlIsFile
Ns_UrlIsDir

2. C Api Behavioral Changes

Ns_TclEnterSet This function will append the new setid into the given Tcl interpreter's result. Prior to AOLserver 4.0, the setid was set into the interpreter's result, replacing the previous value of result with the single new setid.

3. Tcl Api Behavioral Changes

ns_conn Prior to AOLserver 4.0, the ns_conn urlv form of this api had continued to allow the optional specification of a 'connid' argument, as in:
ns_conn urlv <connid> <idx>
In AOLserver 4.0, this form is no longer supported. Use of this form of the command will result in a Tcl error. The argument can be simply removed to provide a correct and equivalent statement.
ns_urldecode, ns_urlencode ns_urldecode will now convert '+' characters to spaces, and ns_urlencode will convert spaces to '+'.
ns_server The urlstats option is no longer supported. Calling 'ns_server urlstats' in AOLserver 4.0 will result in a Tcl error.
ns_thread The return value for a ns_thread join on a non-detached thread is now the return value from the script run in the corresponding ns_thread begin call.

4. New Header file: nsdb.h - contains Database Driver support definitions, required for modules which implement a Database API layer upon the nsdb services. In the 3.x AOLserver, these definitions were found in ns.h.

5. Define removed from ns.h - The 3.x ns.h had provided a definition of closesocket for non-WIN32 builds. This define is redundent and inconsistent in practice with the definition of ns_sockclose also supplied in this header file. This define has been dropped from the 4.x ns.h. Module code which had relied upon this definition of closesocket in ns.h to build on non-WIN32 systems need to be modified to use ns_sockclose.

Usage of ClientData in tcl commands

In AOLserver 3.x, the ClientData was normally ignored or used to trigger slightly different behavior for similar commands. In this case, AOLserver commands needing per-interp state would use thread specific data assuming a single interp for each thread. Virtual server specific data was simply global data because virtual servers were removed in 3.x.

In AOLserver 4.x, the ClientData is used extensively in a consistent manner. All commands are created with a pointer to the new NsInterp structure which packages up all the per-interp state (e.g., ADP buffers) and a pointer to the corresponding NsServer structure. This restored virtual server support without a performance penalty, enabled multiple interps per thread, and generally made the code more rationale.

It turns out the ClientData can be NULL for interps created without a virtual server such as the startup/config interp and interps created after libnsd.so is loaded into tclsh (which is a weird new possibility). A few commands deal with that directly (e.g., ns_info), others ignore ClientData anyway, and others simply don't exist in such interps. The command create code in nsd/tclcmds.c takes care of this.

One more trick to point out if you explore the code: The NsInterp structure is actually managed (although rarely accessed through) the Tcl_Get/SetAssocDatas APIs. This allows proper cleanup of the structure if you delete it directly via Tcl_DeleteInterp. Check nsd/tclinit.c to see how this is done.

The Right-Way to support Virtual Servers (as pertains to AolServer Module Implementation)

In general, a module can be defined to be associated with a Virtual Server. This allows configuration settings to be customized for each Virtual Server. AolServer will call the Module Initialization function for each virtual server for which the module is defined. To support this, a module must be able to organize any data which it retains to be keyed by the server parameter. This can be accomplished in a variety of ways. The code fragment following is snipped from nsperm.c as an example, and provides an illustration of a simple method to do this (note, numerous details have been suppressed). This module keeps a pair of hash tables , and a mutex. It keeps this data in a malloc'ed structure stored into a hash table keyed by server. Note that the 'context' parameter of Ns_TclInitInterps is used to pass the Server specific module data to the InterpInitProc function, which can then pass it to the Tcl_Create*Command as ClientData.

typedef struct PermServData {
    char          *server;
    Tcl_HashTable  users;
    Tcl_HashTable  groups;
    Ns_RWLock      lock;
} PermServData;
static int             uskey = -1;
static Tcl_HashTable   serversTable;
int
Ns_ModuleInit(char *server, char *module)
{
    PermServData *servPtr;
    char *path;
    Tcl_HashEntry *hPtr;
    int new;

    if (uskey < 0) {
        uskey = Ns_UrlSpecificAlloc();
        Tcl_InitHashTable(&serversTable, TCL_STRING_KEYS);
    }
    servPtr = ns_malloc(sizeof(PermServData));
    servPtr->server = server;
    path = Ns_ConfigGetPath(server, module, NULL);
    Tcl_InitHashTable(&servPtr->users, TCL_STRING_KEYS);
    Tcl_InitHashTable(&servPtr->groups, TCL_STRING_KEYS);
    Ns_RWLockInit(&servPtr->lock);
    Ns_SetRequestAuthorizeProc(server, AuthProc);
    Ns_TclInitInterps(server, AddCmds, servPtr);
    hPtr = Tcl_CreateHashEntry(&serversTable, server, &new);
    Tcl_SetHashValue(hPtr, servPtr);
    return NS_OK;
}
static int
AddCmds(Tcl_Interp *interpermPtr, void *arg)
{
    Tcl_CreateCommand(interpermPtr, "ns_perm", PermCmd, arg, NULL);
    return NS_OK;
}
static int
PermCmd(ClientData arg, Tcl_Interp *interp, int argc, char **argv)
{
    PermServData *servPtr = arg;
    /* and so it goes */
}

New AOLserver Versioning Information

A new set of APIs are provided with AOLserver 4.0, which provides compile time and runtime version information for use by Modules linking against AOLserver.

Compile-time Checking

For the compile-time version checking, there are a set of macros defined in the ns.h header file. Those closely resemble ones found in the Tcl distribution. Following macros are defined:

NS_MAJOR_VERSION    /* The "4" in 4.0 */
NS_MINOR_VERSION    /* The "0" in 4.0 */
NS_RELEASE_SERIAL   /* Patch release; would be "x" in 4.0.x */
NS_RELEASE_LEVEL    /* One of the release levels below */

NS_ALPHA_RELEASE
NS_BETA_RELEASE
NS_FINAL_RELEASE

One typical usage pattern for the above macros is to include them in your C-source/header files like this:

#if defined (NS_MAJOR_VERSION)
  /* Compile against 4.x or later */
  #if (NS_MAJOR_VERSION == 4) && (NS_MINOR_VERSION < 1)
    /* We're on the 4.0 server */
  #endif
#else
  /* Compile against pre-4.x server */
#endif /* NS_MAJOR_VERSION */

Run-time Checking

During runtime, it might sometimes be important to turn-on and/or off certain functionality. You can use the function:

void Ns_GetVersion(int *major, int *minor, int *patch, int *type);

to collect information stored in set of NS_ macros above.

This function is modeled after its Tcl counterpart, the Tcl_GetVersion, so everything what applies to this function, applies to the Ns_GetVersion as well.

Interpreter Creation, and availability of Tcl commands

Changes have been made in the method in which Interps are setup, significantly in the manner in which module commands are established. In 3.x AolServer just immediately invoked your callback on the "master interp". In 4.0 it registers your callback to be run when new interps are created for the given server - no matter how (detached thread, connection thread, etc.). This is a key difference in how 3.x and 4.0 Tcl is managed. The rationale was to enable more standard Tcl initialization eventually. In addition, a consistent mechanism for managing a partitioning by Virtual Servers is provided. As in the previous example, any data managed by the Tcl commands are organized and kept separate on a Server basis.

Shared vs Private Tcl Initialization - AolServer will source .tcl files from two distinct library sets. First are the 'shared' tcl files. These are found from ${home}/modules/tcl, where ${home} is defined under '/ns/parameters' (e.g., [ns_config ns/parameters home]). The second set are virtual server private, and are found from ${library}, where ${library} is defined under 'ns/server/${servername}/tcl', for each virtual server name ${servername} (e.g., [ns_config "ns/server/$server/tcl" library]), with a default of ${home}/${servername}/tcl. This is controlled from the virtual server's initfile. The initfile filename can be specified by ${initfile} under '/ns/server/${servername}/tcl' (e.g., [ns_config "ns/server/$server/tcl" initfile]), and defaults to ${home}/bin/init.tcl.

Note that if your Tcl thread is long running, you'll need to do the right thing to manage state manually as it won't benefit from the regular cleanup/update effects of Ns_TclAllocateInterp/Ns_TclDeAllocateInterp as with ordinary transactions. A call to "ns_cleanup" or perhaps "ns_ictl update" in the body of your while loop may work. Unfortunately this stuff is not well documented or understood as most folks don't need to bother with it - check the code in nsd/tclinit.c and nsd/init.tcl for details.

Configuration File Changes

1. DbPool support is now a separate module; must be configured into ns/server/${servername}/modules section to be used

2. The fastpath tcl code (tcl/fastpath.tcl) and fastpath core C code was updated to reflect a a change made earlier to server.c, in which all fast path related configuration was moved from ns/server/ to ns/server//fastpath. This change will require you to update your configuration file to reflect the new configuration path for any fast path options. Following is the list of configuration parameters to which this applies:

cache
cachemaxsize
cachemaxentry
mmap
pageroot
directoryfile
directorylisting 
directoryproc
directoryadp
hidedotfiles
returnmwtoppage
enableaolpress

In addition, the following configuration parameters will default to values specified in "ns/server/${servername}" if no values are defined within "ns/server/${servername}/fastpath":

directoryfile
pageroot

Server Behavioral Changes

ADP Start-page processing

The AOLserver 4.0 core no longer checks existance of startpage and errorpage. This functionality must be added to the config file, if desired. For example:
ns_section "ns/server/${servername}/adp"
if [file readable $startPageFName] {
    ns_param   startpage       $startPageFName
}