13.1.1 Log File Format Symbol Definitions

 

%W (Log records in Webstar format)

%h (The hostname of the client, (or IP number if hostname is not available or if

DNSLookup is off.))

%u (remote user from authorization, if any.)

%t (date and time in CLF format: (day/month/year:hour:minute:second zone.))

%r (first line of request exactly as it came from the client (i.e., the file

name, and protocol requested.))

%s (original http request status code returned to client before internal

redirection. Indicates where or not the file was successfully retrieved,

and if not, what error message was returned.)

%>s (final http request status code.)

%b (number of bytes sent, not including headers.)

%U (url path requested)

%T (transfer time or time taken to serve a request in seconds)

%p (TCP port of the server servicing the request)

%P (process ID of the server servicing the request)

%l (the clients remote logname, if supplied)

%v (name of (virtual) server servicing request)

%w (WebSTAR result, i.e. "OK", "ERR", or "PRIV")

%d (date and time in WebSTAR format)

%{}n (contents of note from another module in brackets)

%{}i (Input header item in brackets)

%{}o (output header item)

%{Referer}i (The URL the client was on before requesting your URL.

%{User-agent}i (The identity of the client software (browser.))

 

 

Client Ident - [Timestamp1] "Method URI" Type Sizes

This logging configuration can not be changed, but you can add HTTP Header Fields. Near the end of the squid.conf file, you will notice the lines:

 

# TAG: log_http_hdrs

# Append individual HTTP request headers to CLF log entry

 

#log_http_hdrs Referer User-Agent

 

The last line is an example implementing the most popular HTTP Header Fields, Referer and User-Agent. Using the same format, you can add any Header Field you want. Or you can remove the # on the last line and log these example Header Fields.

13.2.1 Squid Log Rolling and Splitting

14.1.1 Required Shell Script Content

In addition to creating the text file, there are a few important considerations with respect to the content of the file. First, the top line of the file must contain the following text:

 

#!/bin/sh

 

This tells the system that this is a Bourne Shell script and that the Bourne Shell should be used to interpret the rest of the script.

 

Second, you can use the echo command to generate text which will be returned to the browser that initiated the URL. The first echo command must contain the following Bourne Shell commands to generate HTTP. This puts Web Ten and the browser in the proper mode to accept everything else:

 

echo Content-type: text/plain

echo

 

The first echo indicates that text/plain will follow. The second echo is necessary in order to get the HTTP interpreter to accept the Content-type request. After that, any text sent with an echo command is printed on the originating browser's screen as a response to the URL request.

 

Shell scripts are text files containing Bourne Shell commands that can generate a stream of characters in response to being executed. There are Bourne Shell commands for assigning integer and string values to shell variables, commands for prescribing conditional flow through the shell script, and commands for running other programs. Relatively sophisticated CGIs can be created by combining different Bourne Shell commands. There are a number of widely available books describing Bourne Shell programming.

 

Bourne Shell CGIs are used for low-performance, easy-to-develop CGIs. Each Bourne Shell script is text, and is interpreted by a Bourne Shell interpreter controlled by Web Ten . Since the interpreter interprets each command, shell scripts operate fairly slowly and use a large number of processing cycles. Therefore, Bourne Shell scripts should be used primarily for rapid CGI development or CGI prototyping. If a CGI will be used in high volume, you may want to consider constructing a more efficient C Language CGI or a Perl CGI.

14.1.2 Printenv.sh Example

 

A sample shell CGI is included in the printenv.sh file located in the Web Ten cgi-bin directory. The first few lines of the file establish the mandatory #!/bin/sh and echo Content-type: text/plain requirements for any shell script. The remaining shell script commands are used to output a few lines of constant text, followed by a dozen or more lines that output the values of a family of shell variables. The following is the content of the printenv.sh CGI:

 

#!/bin/sh

# disable filename globbing

set -f

echo Content-type: text/plain

echo

echo CGI/1.0 test script report:

echo

echo argc is $#. argv is "$*".

echo

echo SERVER_SOFTWARE = $SERVER_SOFTWARE

echo SERVER_NAME = $SERVER_NAME

echo GATEWAY_INTERFACE = $GATEWAY_INTERFACE

echo SERVER_PROTOCOL = $SERVER_PROTOCOL

echo SERVER_PORT = $SERVER_PORT

echo REQUEST_METHOD = $REQUEST_METHOD

echo HTTP_ACCEPT = "$HTTP_ACCEPT"

echo PATH_INFO = "$PATH_INFO"

echo PATH_TRANSLATED = "$PATH_TRANSLATED"

 

echo QUERY_STRING = $QUERY_STRING

 

echo SCRIPT_NAME = $SCRIPT_NAME

echo REMOTE_HOST = $REMOTE_HOST

echo REMOTE_ADDR = $REMOTE_ADDR

echo REMOTE_USER = $REMOTE_USER

echo AUTH_TYPE = $AUTH_TYPE

echo CONTENT_TYPE = $CONTENT_TYPE

echo CONTENT_LENGTH = $CONTENT_LENGTH

 

When the printenv.sh CGI is referenced by a URL, it produces the following output:

 

CGI/1.0 test script report:

 

argc is 0. argv is .

 

SERVER_SOFTWARE = Apache/1.2.6.36 WebTen/3.0

SERVER_NAME = www.tenon.com

GATEWAY_INTERFACE = CGI/1.1

SERVER_PROTOCOL = HTTP/1.0

SERVER_PORT = 80

REQUEST_METHOD = GET

HTTP_ACCEPT = image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*

PATH_INFO =

PATH_TRANSLATED =

SCRIPT_NAME = /cgi-bin/printenv.sh

QUERY_STRING =

REMOTE_HOST = 192.83.246.60

REMOTE_ADDR = 192.83.246.60

REMOTE_USER =

AUTH_TYPE =

CONTENT_TYPE =

CONTENT_LENGTH =

 

14.1.3 Shell Variables

 

Shell variables are pre-defined values set by Web Ten before the shell CGI is started. Shell variables are referenced by placing a "$" character in front of the

name of the shell variable. If the shell interpreter finds a name that matches the string of characters following any "$" character, it substitutes the value of that

variable in its processing. In the case of the echo command, the value of the $VAR shell variable is substituted as a parameter to the echo command and is

output to the browser as a partial response to the URL request.

 

14.2.1 Required Script Content

 

In addition to creating the text file, there are a few important considerations with respect to the content of the file. First, the top line of the file must contain the text:

 

#!/usr/bin/perl

 

This tells the Web Ten system that this is a Perl script and that Perl should be used to process the remainder of the file.

 

Second, you can use Perl print statements to generate text which will be returned to the browser that initiated the URL. The first print command must contain an HTTP header. This header indicates what format or kind of data will be output by the remainder of the print commands. The choices are usually plain text or text that is marked up using the HyperText Markup Language (HTML). This first print command puts Web Ten and the browser in the proper mode to accept everything else.

 

For Perl scripts that output plain text, use:

 

print "Content-type: text/plain \n\n";

 

For Perl scripts that output HTML statements, use:

 

print "Content-type: text/html \n\n";

 

The print indicates that text/plain or text/html will follow. After that, any text generated with a print command is sent to the originating browser as a response to the URL request.

 

Perl scripts are text files containing Perl language statements that generate a stream of text characters in response to being executed. There are Perl statements for assigning integer and string values to variables, statements for prescribing conditional flow through the script, and statements for running other programs. Very sophisticated CGIs can be created by combining different Perl statements. A number of widely available books describing Perl programming are available.

 

Programming Perl, Second Edition by Larry Wall, Tom Christiansen and Randal L. Schwartz, with Stephen Potter. 1996, O'Reilly & Associates

 

Perl is used for medium-performance, easy-to-develop CGIs. Each Perl program is text. The scripts are interpreted by a Perl interpreter controlled by Web Ten . Since the interpreter interprets each Perl statement, Perl scripts can consume a lot of memory and use a large number of processing cycles.

 

14.2.2 Printenv.pl Example

 

A sample Perl CGI is included in the .printenv.pl file located in the Web Ten cgi-bin directory. The first few lines of the file establish the mandatory #!/usr/bin/perl and print Content-type: text/plain requirements for any Perl script. The remaining two Perl statements output a dozen or more lines that contain the values of a family of environment variables. The following is the content of the printenv.pl CGI:

 

#!/usr/bin/perl

 

print "Content-type: text/html\n\n";

while( ($key,$val) = each %ENV ) { print "$key = $val<BR>\n"; }

 

When the printenv.pl CGI is referenced by a URL, it produces the following output:

 

SERVER_SOFTWARE = Apache/1.2.6.36 WebTen/3.0

GATEWAY_INTERFACE = CGI/1.1

DOCUMENT_ROOT = /usr/local/etc/httpd/WebSites/www.tenon.com

REMOTE_ADDR = 192.83.246.60

APACHE_PORT = 81

SERVER_PROTOCOL = HTTP/1.0

REQUEST_METHOD = GET

REMOTE_HOST = 192.83.246.60

QUERY_STRING =

HTTP_USER_AGENT = Mozilla/4.61 (Macintosh; I; PPC)

ADMIN_PORT = 84

PATH = /bin:/usr/bin:/usr/ucb:/usr/bsd:/usr/local/bin

HTTP_ACCEPT = image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, image/png, */*

REMOTE_PORT = 1138

HTTP_ACCEPT_LANGUAGE = en,pdf

HTTP_CACHE_CONTROL = Max-age=259200

SCRIPT_NAME = /cgi-bin/printenv.pl

SCRIPT_FILENAME = /usr/local/etc/httpd/cgi-bin/printenv.pl

HTTP_ACCEPT_ENCODING = gzip

SERVER_NAME = www.tenon.com

REQUEST_URI = /cgi-bin/printenv.pl

HTTP_ACCEPT_CHARSET = iso-8859-1,*,utf-8

HTTP_X_FORWARDED_FOR = 192.83.246.60

SERVER_PORT = 80

HTTP_HOST = www.tenon.com

SERVER_ADMIN = webmaster@tenon.com

HTTP_VIA = 1.0 www.tenon.com:80 (Squid/1.1.20.6)

14.2.3 Environment Variables

 

Environment variables are pre-defined values set by Web Ten before the Perl CGI is started. Environment variables are referenced by the Perl statement $ENV{<env var>}. The Perl statement:

 

$ENV{PATH} = "/bin:/usr/bin";

 

sets the PATH environment variable. The Perl statement:

 

print $ENV{PATH};

 

prints the current value of the PATH environment variable.

 

14.3.1 Printenv.c Example

The C Language CGI example included with Web Ten is in a file named printenv.c, which is located in the Web Ten cgi-bin directory.The printenv source code is in tenon/examples/printenv.c.text. Note that this code will not compile and run. It is only listed as an example of how to write C language CGIs. Below is the content of the printenv.c CGI:

 

 

#include <stdio.h>

#include <stdlib.h>

typedef struct {

char name[128];

char val[128];

} entry;

void getword(char *word, char *line, char stop);

char x2c(char *what);

void unescape_url(char *url);

void plustospace(char *str);

 

entry entries[10000];

 

main(int argc, char *argv[]) {

register int x,m=0;

char *cl;

 

printf("Content-type: text/html%c%c",10,10);

 

if(strcmp(getenv("REQUEST_METHOD"),"GET")) {

printf("This script should be referenced with a METHOD of GET.\n");

printf("If you don't understand this, see this ");

printf("<A HREF=\"http://www.ncsa.uiuc.edu/SDG/Software/Mosaic/Docs/fill-out-forms/overview.html\">forms

overview</A>.%c",10);

exit(1);

}

 

cl = getenv("QUERY_STRING");

if(cl == NULL) {

printf("No query information to decode.\n");

exit(1);

}

for(x=0;cl[0] != '\0';x++) {

m=x;

getword(entries[x].val,cl,'&');

plustospace(entries[x].val);

unescape_url(entries[x].val);

getword(entries[x].name,entries[x].val,'=');

}

 

printf("<H1>Query Results</H1>");

printf("You submitted the following name/value pairs:<p>%c",10);

printf("<ul>%c",10);

 

for(x=0; x <= m; x++)

printf("<li> <code>%s = %s</code>%c", entries[x].name, entries[x].val,10);

printf("</ul>%c",10);

}

 

This CGI prints the name/value parameter pairs that are available to any CGI when the CGI is invoked. The general flow of the printenv CGI is that it uses the printf statement to output Content-type: text/html\n\n. This is needed in order for the CGI to inform Web Ten and the remote browser of the type of content to follow.

 

The program then verifies whether or not a GET type of HTTP request was used to initiate the CGI. If a GET request was not used, an error message is returned with several printf statements and the program exits. If a GET HTTP request is found, the environment variable QUERY_STRING is requested. If that string is unavailable, an error message is printed and the program exits. If QUERY_STRING is found, a for loop is entered. The for loop calls the getword subroutine to parse the string into name and value pairs. Once all of the parameters have been parsed, the printf subroutine is called several times to output a constant string "QUERY RESULTS", followed by the string "You submitted the following name/value pairs:", followed by a name and value pair on each line until all of the name/value parameters have been displayed. When the printenv CGI is referenced by the URL:

 

/cgi-bin/printenv?company=Tenon Intersystems&addr=1123 Chapala St.&city=Santa Barbara

 

it produces the following output:

 

Query Results

You submitted the following name/value pairs:

company = Tenon Intersystems

addr = 1123 Chapala St.

city = Santa Barbara

 

use CGI::Fast;

while ($query = new CGI::Fast)

{

print "Content-type: text/html<BR>\n";

while (($key, $val) = each %ENV) {

print "$key = $val<BR>\n";

}

}

 

When a FastCGI such as this is run the first time, mod_fastcgi (an Apache module) spawns a process that keeps the script running while Apache is running. To have the FastCGI run automatically when Apache is first started, put the following lines in Web Ten 's httpd.conf file:

 

<IfModule mod_fastcgi.c>

FastCGIServer /usr/local/apache/cgi-bin/printenv.fcgi -processes 1

</IfModule>

 

These lines will create one instance of the printenv.fcgi script whenever Apache is run. The number of processes can be increased if more instances are needed to accommodate the volume of requests. All FastCGI scripts are named with the ".fcgi" extension by convention. Be sure to set the correct path to the FastCGI script in the Apache directives (/usr/local/apache/ is the path to the Web Ten folder.)

 

http://host.yourdomain.com/webmail

 

and entering your full e-mail address and password in the fields provided (see picture below).

 

The e-mail address entered must be fully qualified (e.g. joe@mail.tenon.com) as opposed to joe@tenon.com.)

 

Figure 72: WEBmail Login

 

When logging in, you must be sure to use your full e-mail address. For example, the address "user@domain.com" is not a full e-mail address (though you may receive mail at that address.) WEBmail requires that you include the hostname in your e-mail login so it can properly communicate with your mail server. A "fully qualified" e-mail address, four example, would be "user@mail.domain.com". You e-mail password is the same password you use to log into your e-mail account in any other mail client. Choose "GO" to log into WEBmail once you have entered your e-mail address and password.

 

Once successfully logged in, you will see a list of your e-mail messages. WEbmail is designed to offer all the features of an e-mail client. For help using the e-mail client portion of WEBmail, refer to the WEBmail documentation at "http://host.domain.com/web_mail/help/".

Figure 73: Choose WEBmail account password

 

The final step is entering a password for the new mailbox. Enter it twice and choose "Submit." WEBmail will reply with a message that confirms the account has been successfully created.

 

Figure 74: Default Indexing Options

 

The indexing form contains fields for specifying which URLs should be indexed. The Start URLs are the starting point for the indexing engine. The Exclude URLs are URLs that should not be indexed. The Limit URLs contains sets of patterns that the URLs must match.

 

The default Start URLs is a single URL matching the virtual host name used in the request. This default instructs the indexing process to visit all of the documents on this virtual host that are reachable (following any numbers of links) from the home page. The default Limit URLs specifies a set that exactly matches the set of Start URLs. In most cases, this is all that is needed to build a complete index of an entire virtual host. Additional URLs can be added to these lists.

 

The form also provides a field for an email address. It an email address is provided, the results of the indexing process will be emailed to that address.

 

Additional options may be displayed by clicking on the Options button. In this case, the form is displayed again with the default options shown (below). These defaults can then be modified. (The default options are used if the form is submitted without displaying the options.) The default settings are sufficient to create a search engine index (or database) file for the specified URLs.

 

Figure 75: All Indexing Options

 

To begin the indexing process, click on the Run! button. The CGI will start a batch indexing process (if the batch options is specified) that continues to run after the CGI has completed. A link to a file which will contain the detailed results of the indexing process is provided. Note that it may take some time for the batch indexing process to complete. (For example, a default Web Ten installation takes about 10 minutes.) If

the results are referenced before the indexing process is complete, only the completed parts of the indexing process will be shown. Providing an email address is the best way to be notified when the entire indexing operation is complete.

 

To continually monitor the progress of the indexing process, uncheck the batch option before clicking on the Run! button. In this case, the output from the indexing process is continually displayed in the CGIs output and the CGI does not complete until the indexing process completes.

 

17.1.1 Installing Plug-ins

• Install according to the instructions included with the plug-in package.
• Re-start Web Ten , log into the Web Ten admin server and make sure the plug-in has registered (see section See Plug-In Administration for information) and note the "Action" and "Suffix" entries in the plug-in Administration table.
• Configure an Action Handler for the plug-in with the Action that the plug-in reported in the plug-in Administration table (see See Configuring Plug-In Actions)
• Configure a MIME Extension for the plug-in with the extension reported in the plug-in Administration table (see See Configuring Plug-In Actions)
• Test the plug-in with the content provided with the plug-in package.

Figure 77: Included Apache Modules

 

In many cases, an Apache module provides the full functionality of a common WebSTAR-style plug-in.

17.2.1 Installing Apache Modules

To install an Apache Module, put the module file in the WebTen/Modules folder and re-start Web Ten . Then, if the module requires a MIME Extersion and an Action Handler, configure these as they would be configured for a plug-in (see See Installing Plug-ins). Every Apache Module should include documentation that defines the module's requirements.