Connection and Socket Functions

$Header: /cvsroot/aolserver/aolserver.com/docs/devel/tcl/api/conn.html,v 1.2 2004/07/20 23:44:26 dossy Exp $


Note! Updated Tcl API documentation in progress is now available on the wiki.


ns_checkurl

Authorize a request using basic HTTP authentication.

Syntax

ns_checkurl method url authuser authpasswd ?ipaddr?

Description

This function does the same permission check that the AOLserver does before serving a URL. If the nsperm module is loaded, the algorithm is as follows.
1. If the authuser is "nsadmin", the password is correct, and the IP
   address of the client is allowed nsadmin access, then access is
   authorized.
2. Find the relevant permission record. If an exact match for the
   method and URL combination is not found, the end of the URL is
   pared down until a match is found. For example, if there is no
   match for `/products/cereals/raisin_bran.html,' then the server
   looks for a permission record for the URL `/products/cereals.' If
   that permission record is specified as "Exact URL match is NOT
   required", then that permission record is used.
By default, the server comes with a row that says GET on `/' is open to the world.

If no relevant permission record is found, access is denied (forbidden).

1. If the authuser is in the "Allow Users" list, access is permitted.
   If the authuser is a member of a group in the "Allow Groups" list
   and not in the "Deny Users' list, access is permitted.
2. If the host is in the "Hosts to allow" list, access is permitted.
   If the host is in the "Hosts to deny" list, access is denied.
3. If the request does not come in with authorization data, access is
   denied.
4. The user and password are verified. If there is no password
   specified in the database, any password is accepted.
5. Otherwise, access is denied.

Return Values

The following values can be returned by ns_checkurl.

OK
The user has permission to execute this URL and method.
DENIED
The user does not have permission to execute this URL and method.
FORBIDDEN
There is no possible user/password/IP Address combination that would give authorization.
ERROR
There was an error.

If the permissions module is not loaded, an error is returned.



ns_conn

Find information about the current HTTP connection.

Syntax

ns_conn authpassword
ns_conn authuser
ns_conn close
ns_conn contentlength
ns_conn driver
ns_conn form
ns_conn headers
ns_conn host
ns_conn isconnected
ns_conn location
ns_conn method
ns_conn outputheaders
ns_conn peeraddr
ns_conn port
ns_conn protocol
ns_conn query
ns_conn request
ns_conn url
ns_conn urlc
ns_conn urlv
ns_conn version

Description

ns_conn authpassword returns the decoded user password from the authorization data.

ns_conn authuser returns the decoded user name from the authorization data.

ns_conn close closes the connection so the script (or ADP) can do any time-consuming processing without making the client wait. If you use ns_conn close in an ADP, streaming should be turned on before closing the connection (i.e. <SCRIPT RUNAT=SERVER STREAM=ON>) or nothing will get sent out at all.

ns_conn contentlength returns the number of bytes in the content passed in.

ns_conn driver returns the name of the module (nssock or nsssl) that is acting as the communications driver for this connection.

ns_conn form returns any submitted form data as an ns_set. This form data may have been submitted with a POST or appended to the URL in a GET request. Note: ns_conn form is not suitable for multipart formdata file upload widgets.

ns_conn headers returns all the header data as an ns_set. The keys of the ns_set represent the field names. The case of the returned field names depends on the HeaderCase configuration parameter. By default, HeaderCase is "Preserve", which means case is preserved.

ns_conn host returns the host part of the URL in the HTTP request.

ns_conn isconnected returns 1 if you're in a connection thread, and you are therefore allowed to make calls to ns_conn. It returns 0 if you're not in a connection thread (such as when you're in a schedule procedure) and you are not allowed to make calls to ns_conn.

ns_conn location returns the location string for this virtual server in the form: protocol://hostname[:port].

ns_conn method returns the HTTP method, e.g. GET.

ns_conn outputheaders returns an ns_set containing the headers that will be sent out when a result is returned to the client. This ns_set can be manipulated like any other ns_set. You can also use this command to write to the set of output headers. For example: ns_set put [ns_conn outputheaders] key value.

ns_conn peeraddr returns the IP address of the client, i.e. the "other side" of the HTTP connection. The IP address is returned in the form of a string separated with periods (e.g., 155.164.59.75).

ns_conn port returns the port specified explicitly in the URL of the HTTP request. If the browser does not explicity send the ":port" part of the URL, the port number returned will be 0.

ns_conn protocol returns the protocol of the URL in the HTTP request (usually unspecified).

ns_conn query returns any query data that was part of the HTTP request.

ns_conn request returns the HTTP request line as presented by the client, e.g. GET / HTTP/1.1.

ns_conn url returns the URL of the HTTP request. This is the portion of the request after the hostname, for example [ns_conn url] on http://aolserver.com/ returns /index.adp.

ns_conn urlc returns the number of elements (delimited by `/') in the URL of the HTTP request.

ns_conn urlv returns a list containing the pieces of the URL delimited by `/'.

ns_conn version returns the version of the HTTP request. This is usually 1.0 or 1.1.



ns_conncptofp

Write content to a file

Syntax

ns_conncptofp fileId

Description

This function writes all the content (including any embedded null characters) to the specified file.



ns_connsendfp

Write contents of file to conn

Syntax

ns_connsendfp fp len

Description

This function writes len bytes of the specified channel or file to the conn.



ns_get_multipart_formdata

Handle the POST action of a form containing one Netscape file widget

Syntax

ns_get_multipart_formdata key fieldId ?formdataSet?

Description

If you have a Tcl script that is handling a POST of a form containing exactly one Netscape INPUT TYPE=FILE widget and 0 or more other widgets, you can call ns_get_multipart_formdata instead of ns_conn form. If you call ns_get_multipart_formdata at any other point, such as inside an ADP script, it may not work properly due to AOLserver's processing of the form data prior to execution of the ADP.

The key argument is the key for the file widget.

The fileId argument must be the file ID of a file that is open for write operations. ns_get_multipart_formdata will write the submitted file (from the file widget) to the file specified by fileId.

If you pass a formdataSet into the function, the rest of the form data is dropped into that formdata set.

Example

In the HTML page, the form is defined as:
<form enctype=multipart/form-data method=post action=/foo>
<input name=file type=file>
</form>
The POST action is handled by the /foo script defined below:
ns_register_proc POST /foo foo
proc foo {conn ignore} {
   set fp [open "/tmp/uploaded_file" w+]
   ns_get_multipart_formdata "file" $fp
   close $fp
   ## Process file
   ## return something
}



ns_getform

Return an ns_set that contains all of the query data that was part of the HTTP request.

Syntax

ns_getform

Description

If there is query data (from either a form or multipart form data), ns_getform returns an ns_set that can be queried for that data. If there is no query data, "" is returned.



ns_geturl

Fetch a URL.

Syntax

ns_geturl URL ?headersSetIdVar?

Description

This function retrieves the contents of the passed-in URL. If headersSetIdVar is passed in and it is a valid ns_set, then the header information received along with the request is inserted into it.

Example

ns_register_proc GET /wais getwais
proc getwais { conn context } {
    ns_return 200 text/html [ns_geturl http://www.wais.com/]
}

Notes

This function should be used with caution. If the server is running with 1 thread, and you perform ns_geturl back to the originating server, the server may deadlock.

Also, ns_geturl does not follow redirects or handle relative URLs. URLs that are server-relative (begin with "/") are translated into filenames, and the content of the file is returned.



ns_hrefs

Return list of HTML <A> links.

Syntax

ns_hrefs HTML

Description

This function returns a Tcl list of all the URLs that the HTML contains <A> links to.



ns_httptime

Convert time in seconds to HTTP header format.

Syntax

ns_httptime time_in_seconds

Description

This function converts the time (specified as the number of seconds from 00:00:00 UTC, January 1, 1970) to the appropriate format for an HTTP header or log file. The time and date is returned with a four digit year, for example: "Sun, 06 Nov 1994 08:49:37 GMT". You can use the ns_time function to get the current time in seconds like this:
set time [ns_httptime [ns_time]]



ns_httpget

Open an HTTP connection and fetch a page.

Syntax

ns_httpget url ?timeout? ?depth?

Description

ns_httpget opens an HTTP connection and fetches the page at the specified url. You can specify a timeout for opening the connection (the default is 30 seconds), and a maximum level of redirection (the default is 0). ns_httpget sends the HTTP/1.1 "Host:" header for proper support of virtual hosting but only supports HTTP/1.0.

Example

set page [ns_httpget http://www.aolserver.com]



ns_httpopen

Open an HTTP connection.

Syntax

ns_httpopen method url ?rqset? ?timeout?

Description

ns_httpopen opens an HTTP connection and performs the specified method at the specified URL. The timeout is the number of seconds to wait for the connection to open. ns_httpopen sends the HTTP/1.1 "Host:" header for proper support of virtual hosting but only supports HTTP/1.0.

ns_httpopen returns a list with these three elements: a file descriptor for reading, a file descriptor for writing, and a set ID for a set describing the connection. The three elements of the set are date, server, and content type.



ns_hostbyaddr

Resolve an IP address to a hostname.

Syntax

ns_hostbyaddr ipaddress

Description

ns_hostbyaddr resolves the specified IP address to its corresponding host name using AOLserver's built-in DNS cache. See the Administration Guide for adjusting the DNS cache.



ns_parseheader

Parse HTTP header.

Syntax

ns_parseheader set header

Description

This function parses the HTTP header specified by header into an ns_set specified by set.



ns_parsehttptime

Return number of seconds from HTTP time.

Syntax

ns_parsehttptime httptime

Description

ns_parsehttptime takes a properly formatted HTTP time and returns the number of seconds since 00:00:00 UTC Jan 1, 1970.



ns_parsequery

Parse a query string.

Syntax

ns_parsequery querystring

Description

This function parses the specified querystring into an ns_set, which is returned. The querystring takes the form: a=bcdefgh&b=123&c=rew.



ns_queryexists

Check for a key in the query data that was part of the HTTP request.

Syntax

ns_queryexists key

Description

ns_queryexists looks in the query data for the specified key. If the key exists, 1 is returned; otherwise 0 is returned. The key is interpreted in a case insensitive manner.

Example

ns_register_proc POST /queryexiststest queryexiststest
proc queryexiststest { } {
    if [ns_queryexists name] {
        # ...process the form...
    } else {
        ns_returnerror 400 "you need to supply your name in the form"
    }
} ;# queryexiststest



ns_queryget

Get a value from the query data that was part of the HTTP request.

Syntax

ns_queryget key ?value?

Description

ns_queryget looks in the query data for the specified key, and returns the value that was included in the HTTP request. If the key does not exist in the query data, "" is returned. The key is interpreted in a case insensitive manner.

If the optional value argument is specified, and the key does not exist in the query data, the specified value is simply returned. This capability allows for providing a default value if the key doesn't exist.

This function works for simple forms as well as for multipart formdata.

For files uploaded with the Netscape file upload widget, the file that was uploaded is an entry in the query data. See Example 3, below.

Examples

Example 1:
set x [ns_queryget name]
If "name" is a key in the query data, the variable x will be set to the value associated with the "name" key. If "name" is not a key in the query data, "" will be returned.

Example 2:

set x [ns_queryget name Hoover]
If "name" is a key in the query data, the variable x will be set to the value associated with the "name" key. If "name" is not a key in the query data, "Hoover" will be returned.

Example 3: Given this HTML form:

<form enctype=multipart/form-data method=POST
   action=/formtest>
Local file: <input name=clientfile type=file>
To remote file: <INPUT TYPE=text NAME=path VALUE="" SIZE=80>
<input name=submit type=submit value=Upload>
</form>
and this POST handler:
proc formtest { } {
    set remotefile [ns_queryget path]
    set localfile [ns_queryget clientfile]
    set where_the_data_is [ns_queryget clientfile.tmpfile]
} ;# formtest

Suppose the user specified "spoon.txt" as the Local File and "/oop/ack/tick.txt" as the Remote File, and then submitted the form. The variable values in the formtest procedure will be: remotefile = "/oop/ack/tick.txt"
localfile = "spoon.txt"
_the_data = something like: "/var/tmp/baaa29444"

If you want to use the contents of the uploaded file, you can open it by executing:

open [ns_queryget clientfile.tmpfile]
You can then read it and manipulate it as you want. Note, however, that this tmp file will be deleted once the connection closes.



ns_querygetall

Get multiple query values.

Syntax

ns_querygetall key ?def_result?

Description

This function gets multiple query values whose key is key. If there are none, the default result (def_result) is returned, or null is returned if def_result is not specified. This function is useful for checkboxes.



ns_requestauthorize

Ask the server to check permissions using nsperm.

Syntax

ns_requestauthorize method URL authuser authpassword ?ipaddr?

Description

This function does the same permission check that the AOLserver does before serving a URL. If the nsperm module is loaded, the algorithm is as follows.
1. If the authuser is "nsadmin", the password is correct, and the IP
   address of the client is allowed nsadmin access, then access is
   authorized.
2. Find the relevant permission record. If an exact match for the
   method and URL combination is not found, the end of the URL is
   pared down until a match is found. For example, if there is no
   match for `/products/cereals/raisin_bran.html,' then the server
   looks for a permission record for the URL `/products/cereals.' If
   that permission record is specified as "Exact URL match is NOT
   required", then that permission record is used.
By default, the server comes with a row that says GET on `/' is open to the world.

If no relevant permission record is found, access is denied (forbidden).

1. If the authuser is in the "Allow Users" list, access is permitted.
   If the authuser is a member of a group in the "Allow Groups" list
   and not in the "Deny Users' list, access is permitted.
2. If the host is in the "Hosts to allow" list, access is permitted.
   If the host is in the "Hosts to deny" list, access is denied.
3. If the request does not come in with authorization data, access is
   denied.
4. The user and password are verified. If there is no password
   specified in the database, any password is accepted.
5. Otherwise, access is denied.

Return Values

The following values can be returned by ns_requestauthorize.

OK
The user has permission to execute this URL and method.
DENIED
The user does not have permission to execute this URL and method.
FORBIDDEN
There is no possible user/password/IP Address combination that would give authorization.
ERROR
There was an error.



ns_respond

Build a complete HTTP response.

Syntax

ns_respond ?-status status? ?-type type? {?-string string? | ?-file file? | ?-fileid fileid? } ?-length length? ?-headers setId?

Description

ns_respond builds a complete response for the client with all of the specified information in the header.

Example

Using ns_respond, it's easy to do an HTTP redirect:
set headers [ns_set new myheaders]
ns_set put $headers location http://www.aolserver.com
ns_respond -status 302 -type text/plain \
    -string "redirection" -headers $headers



ns_return

Return the response to the client.

Syntax

ns_return status type string
ns_returnadminnotice status msg ?longmsg?
ns_returnbadrequest reason
ns_returnerror status msg
ns_returnfile status type filename
ns_returnforbidden
ns_returnfp status type fileId len
ns_returnnotfound
ns_returnnotice status msg ?longmsg?
ns_returnredirect location
ns_returnunauthorized

Description

These procedures provide a simple interface for returning information to the client. They build HTTP/1.0 headers and send the appropriate data out the socket to the client. The script does not end at the time ns_return* is invoked so you can continue processing data after the client has gotten the data and closed the socket.

type should be a MIME type (see ns_guesstype manual page for a list).
status is a three-digit number fitting the pattern below:

1xx Informational - Not used, but reserved for future use.
2xx Success - The action was successfully received, understood, and accepted.
3xx Redirection - Further action must be taken in order to complete the request.
4xx Client Error - The request contains bad syntax or cannot be fulfilled.
5xx Server Error - The server failed to fulfill an apparently valid request.
Some common status values and their meanings are:
201 Created
202 Accepted
203 Provisional Information
204 No Content
300 Multiple Choices
301 Moved Permanently
302 Moved Temporarily
303 Method
304 Not Modified
401 Unauthorized
402 Payment Required
403 Forbidden
404 Not Found
405 Method Not Allowed
406 None Acceptable
407 Proxy Authentication Required
408 Request Time-out
409 Conflict
410 Gone
501 Not Implemented
502 Bad Gateway
503 Service Unavailable
504 Gateway Time-out

ns_return sends back both the headers and the string.

ns_returnadminnotice performs the same function as ns_returnnotice, except that it appends a line with a link to "mailto:serveradministrator" based on the virtual server parameter "WebMaster".

ns_returnbadrequest returns a 400 status code and a formatted HTML message containing the reason text.

ns_returnerror wraps the text msg in some html and returns that to the client.

ns_returnfile sends back the headers and the contents of the file.

ns_returnforbidden returns a 403 status code.

ns_returnfp first sends the appropriate headers. Next, it writes out the contents of file from the current file position until the end of the file.

ns_returnnotfound returns a 404 status code.

ns_returnnotice wraps the text msg and longmsg in some html and returns that to the client.

ns_returnredirect returns a redirect to the passed in location.

ns_returnunauthorized returns a 401 status code.

Example

It's often handy to output a page to the user and close the connection but still process data afterwards. Your script will continue to run after you invoke an "ns_return*" function.
ns_returnnotice 200 "Thank you" "Thank you for your input!"
# Do more stuff...
#  Maybe get a cup of coffee...
#   It doesn't matter since the client has already left.
return TCL_OK  ;# not really needed but shown here for clarity.



ns_setexpires

Set the HTTP "Expires" header.

Syntax

ns_setexpires seconds

Description

This function sets the Expires header so the page will expire in the specified number of seconds. Setting a negative value typically causes broken images on browsers so use this option with caution.



ns_sockaccept

Accept a new socket connection.

Syntax

ns_sockaccept sockid

Description

ns_sockaccept accepts a new connection pending on sockid.

Example

#listen for connections on port 9000
set sock [ns_socklisten * 9000]
#wait for new connection
set fds [ns_sockaccept $sock]
set rfd [lindex $fds 0]
set wfd [lindex $fds 1]

puts $wfd "Hello!"

close $rfd
close $wfd
close $sock



ns_sockblocking

Set the blocking option on the socket.

Syntax

ns_sockblocking sockId

Description

This function sets the socket for the specified sockId to "blocking."



ns_sockcallback

Register a socket callback script.

Syntax

ns_sockcallback sockid script when

Description

ns_sockcallback registers a socket callback script. The script should accept the arguments sockid and when. The script will be called according to the value of the "when" argument as follows:

r = the socket is readable
w = the socket is writeable
e = the socket has an exceptional condition
x = the server is shutting down

Example

set sock [ns_socklisten * 9000]
ns_sockcallback $sock newconn r
# Keep $sock from closing after connection ends
detach $sock

# When a connection arrives, newconn will be called as:
# newconn $sock r
proc newconn {sock when} {
    set fds [ns_sockaccept $sock]
    set rfd [lindex $fds 0]
    set wfd [lindex $fds 1]
    puts $wfd "Hello!"
    close $rfd
    close $wfd
}



ns_sockcheck

Check if a socket is connected.

Syntax

ns_sockcheck fileid

Description

ns_sockcheck uses recv() or send() calls to check if a socket is still connected. The fileid is the read or write file id returned by ns_sockopen. This function is useful if you used the -nonblock option with ns_sockopen after calling ns_sockselect.

Example

An example containing ns_sockcheck is provided under ns_sockselect.



ns_socketpair

Create a pair of connected sockets.

Syntax

ns_socketpair

Description

ns_socketpair creates a pair of connected sockets and returns a list of file ids: the first one for reads and the second one for writes.



ns_socklisten

Create a new socket listening for connections.

Syntax

ns_socklisten address port

Description

ns_socklisten creates a new socket listening for connections at the specified interface address and port. "*" can be used as the address argument to specify all addresses.



ns_socklistencallback

Run script on connection.

Syntax

ns_socklistencallback address port script

Description

This function listens on the specified address and port and runs the specified script when new connections are received.



ns_socknonblocking

Set nonblocking option on the socket.

Syntax

ns_socknonblocking sockId

Description

This function sets the socket option for the specified sockId to "nonblocking."



ns_socknread

Return bytes to be read.

Syntax

ns_socknread sockId

Description

This function returns the number of bytes waiting to be read from the specified sockId.



ns_sockopen

Connect to a remote host on the specified port.

Syntax

ns_sockopen ?-nonblock | -timeout seconds? host port

Description

ns_sockopen uses socket(), bind(), and connect() calls to connect to the remote host (host) on the specified port (port). ns_sockopen returns a list of two file ids: the first one for reads and the second one for writes.

The -nonblock option specifies that ns_sockopen will return immediately, while the connect is still in progress by the operating system. If you use -nonblock, you then can call ns_sockcheck with the write file id to see if the connect was actually completed.

The -timeout option specifies how long to wait for the connect to be made in seconds before timing out.

Example

This is a simple example that fetches a page from an http server (www.aolserver.com) on port 80. The headers will be in the $headers list, and the body will be in the $page list.
set fds [ns_sockopen www.aolserver.com 80]
set rid [lindex $fds 0]
set wid [lindex $fds 1]
puts $wid "GET /index.htm HTTP/1.0\r\n\r"
flush $wid
while {[set line [string trim [gets $rid]]] != ""} {
    lappend headers $line
}
set page [read $rid]
close $rid
close $wid
A more advanced example containing ns_sockopen is provided under ns_sockselect.



ns_sockselect

Determine readable file id's.

Syntax

ns_sockselect ?-timeout seconds? rfds wfds efds

Description

ns_sockselect uses a select() call to determine which file id's re readable, writeable, or have exceptional conditions. ns_sockselect returns a list of three lists of: readable file id's, writeable file id's, and file id's with exceptions.

The -timeout option specifies the length of time to wait in seconds for ns_sockselect to return before timing out.

The rfds, wfds, and efds arguments are lists of readable file id's, writeable file id's, and file id's with exceptions, respectively.

Example

This example attempts to connect to nine servers at once and service the first available connections:
# Start nonblocking connect()'s to www01 through
# www09.foo.com and remember the read fileid which
# corresponds to each write fileid.

for {set n 1} {$n < 10} {incr n} {
    set host [format "www%2d.foo.com" $n]
    set fds [ns_sockopen -nonblock $host 80]
    set r [lindex $fds 0]
    set w [lindex $fds 1]
    set w2r($w) $r
    lappend wfds $w
}

# All connect()'s are in progress, use select to wait for one or
# more to become writable in the next two seconds which means #
# they may have connected.  We're not interested in readable or
# exception sockets so the corresponding lists are empty
# (i.e., {}).

set sel [ns_sockselect -timeout 2 {} $wfds {}]

# Select returned - get the sockets ready to write to.

set wfds [lindex $sel 1]

# Use ns_sockcheck to see if the sockets actually connected and
# didn't become writable because the connect() failed (e.g., no
# Web server was running on port 80 on the corresponding machine).
# Note that the wfds list may be empty, meaning all sockets timed
# out on connect.

set ok ""
foreach w $wfds {
    if [ns_sockcheck $w] {
        # Socket is connected - send a GET HTTP request.
        lappend ok $w
        puts $w "GET /index.htm HTTP/1.0\r\n\r"
        # The flush is important, otherwise the remote
        # server may never see the data.
        flush $w
    }
}

# Get the read ids for each socket which we sent the GET request to.

foreach w $ok {
    lappend rfds $w2r($w)
}

# Use select again to wait for the read sockets to have data
# available in response to the GET request.

set sel [ns_sockselect -timeout 2 $rfds {} {}]
set rfds [lindex $sel 0]

# Read the pages which came back.
foreach r $rfds {
    if [ns_sockcheck $r] {
        set page($r) [read $r]
    }
}

# Close all the sockets
foreach w [array names w2r] {
    close $w
    close $w2r($w)
}



ns_url2file

Return file pathname that corresponds to the URL.

Syntax

ns_url2file URL

Description

This function returns the file or directory corresponding to the specified URL. The file returned is located in the "pages" directory on the current virtual server.

Example

This is especially useful when using the ns_adp_include command.
<% ns_adp_include [ns_url2file /include/myfile.inc] %>



ns_write

Write raw content back to the client.

Syntax

ns_write string

Description

ns_write writes the string out the connection. You can use it instead of the ns_return or ns_respond functions to build HTTP responses. AOLserver will not include the usual headers on the output data. The script does not end at the time ns_write* is invoked so you can continue processing data after the client has gotten the data and closed the socket.

Example

See ns_return.



ns_writecontent

Write content to a file.

Syntax

ns_writecontent fileId

Description

This function writes all the content (including any embedded null characters) to the specified file.