Child pages
  • CGI Box Setup Guide
Skip to end of metadata
Go to start of metadata
  1. #Introduction
  2. #Prerequisites
  3. #Acquiring a CCIS Linux System
  4. #Initial Tasks
  5. #Apache 2.2
  6. #cgiwrap
  7. #extract_UserDir_logs
  8. #trimlog
  9. #MySQL
  10. #Host-Specific Files & Configurations

CGI Box

The CGI Box serves cgi scripts using an Apache HTTP Server on a modified CCIS Linux machine. The original CGI Box on a modified CCIS UNIX machine can be found at http://www.ccs.neu.edu/home/bass/cgibox/. This version of the CGI Box closely follows Ian's original instructions.

Introduction

This document describes the process of converting a CCIS Linux box (Ubuntu) into a cgibox.

Prerequisites

  1. Read the original instructions.
  2. a CCIS Linux (Ubuntu) machine (with root access)
  3. the packages
    % sudo apt-get build-dep apache2.2-common
    % sudo apt-get install apache2 ruby1.8 eruby \
                           libapache2-mod-auth-mysql mysql-server php5-mysql
    
    Note: all the project files live in /proj/crew/lee/cgibox

Acquiring a CCIS Linux System

There are a few solutions to "CCIS-ifying" (turning a machine into a CCIS machine). One would be to install a plain Ubuntu system and then go about installing all the necessary packages by hand. This method has a lot of room for error, and it is time-consuming. The other method is to preseed it (currently under development).

See /proj/crew/lee/lininstall or /home/www/ccs/groups/systems/lininstall

Initial Tasks

  1. Remove access from all users except the installer, systems.
  2. Setup #automounts.

SSL key and certificate

This is a duplicate of the original instructions.

% cd /etc/ssl
% openssl req -new -x509 -out server.crt -keyout server.key

Answer the questions.
Organization Name: Northeastern University
Organizational Unit Name: CCIS Systems
Common Name: cgi.ccs.neu.edu
Email Address: systems@ccs.neu.edu

% mv server.key server.key.encrypted
% openssl rsa -in server.key.encrypted -out server.key

Testing Environment

Because the /cgihome and /cgilogs directories contain user data, it is prudent to first set up a testing environment. I have created /testhome, /testlogs and /testbkp to mimic /cgihome /cgilogs and the backup directory for MySQL (all owned by root).

Apache 2.2

The biggest change from Apache 1.3 to 2.2 was in the modular structure in 2.x. As you can see in the original documentation of the CGI Box, there is only one configuration file. However, in 2008 with Apache 2.2, the configuration files have a new look.
Because the modules can be dynamically loaded, the configuration that loads these modules are located at /etc/apache2/mods-available.

To turn Apache 2.x on and off, see the message

% /etc/init.d/apache2

Turn off Apache, then back up the original configurations.

For the cgibox, we use the user cgiwww.

apache2.conf

The diff on apache2.conf

125,126c125,126
< User www-data
< Group www-data
---
> User cgiwww
> Group cgiwww
188a189,190
> Include /etc/apache2/cgi.conf
>

cgi.conf

The Include directive links the configurations specific to the cgibox.

cgi.conf

See attached cgi.conf

Specifies the UserDir, sets up the cgiadmin Directory, Options et cetera. {{Include}}s three configuration files

special-cases.conf

Handles special Apache configuration for those users who request it (e.g. authentication).

rewrite.conf

rewrite.conf is extracted from the original configuration. This part is used to redirect (rewrite) the cgi scripts / php / rhtml to cgiwrap.

rewrite.conf

########################################################################
##  Ian's Rewrite voodoo^[b^[drules for cgiwrap                       ##
##                                                                    ##
## Note: This is duplicated for both port 80 and 443  -Alex
##
RewriteEngine On
#RewriteLog logs/rewrite_log
RewriteLogLevel 0
#
# redirect root to cgiadmin user
#
RewriteRule ^(/home)?/?$ https://%{SERVER_NAME}/home/cgiadmin/ [R=303,L]
#
# redirect for email confirmation ("\d" doens't work?!?)
#
RewriteRule ^/confirm/([0123456789]+)/?$ \
        https://%{SERVER_NAME}/home/cgiadmin/cgi-bin/register.cgi?emailcookie=$1&go=ep \
        [R=303,L]
#
# cgiwrap magic
#
RewriteRule ^/(~|home/)([^/]+/.+\.)(php[34]?|php|phtml|cgi|py|rhtml)\.err(.*)$  \
            /cgi-bin/cgiwrapd/$2$3$4 [PT]
RewriteRule ^/(~|home/)([^/]+/.+\.)(php[34]?|php|phtml|cgi|py|rhtml)(.*)$       \
            /cgi-bin/cgiwrap/$2$3$4 [PT]
RewriteRule ^/(~|home/)([^/]+/.+\.ncgi)\.err(.*)$                          \
            /cgi-bin/nph-cgiwrapd/$2$3 [PT]
RewriteRule ^/(~|home/)([^/]+/.+\.ncgi)(.*)$                               \
            /cgi-bin/nph-cgiwrap/$2$3 [PT]
#
# /home/user/ and /~user/ transparency
#
RewriteRule ^/home/(.*) /~$1 [PT]
##                                                                    ##
##                                                                    ##
########################################################################

This snippet of configuration is at the very heart of the whole mechanism.

Explanation of the seven RewriteRule directives
  1. redirects to the main page (i.e. the registration pages).
  2. processes the confirmation code
  3. pipes the requested cgi script through cgiwrapd (debug)
  4. pipes the requested cgi script through cgiwrap
  5. pipes the requested ncgi scripts through nph-cgiwrapd (debug)
  6. pipes the requested ncgi scripts through nph-cgiwrap
  7. /home/user/... goes to /~user/... (for static data)

error.conf

404 and 403 pages are handled by /home/cgiadmin/cgi-bin/error.cgi.

See error.conf.

The case against suEXEC

suEXEC was introduced in Apache 1.2, and it provides the same functionality as cgiwrap. It is better supported, as it is currently integrated into Apache. How suEXEC works is similar to cgiwrap, however, this integration with Apache is what makes it so hard to customize.
The suEXEC module works with the userdir module and the cgi module. The user who runs suEXEC (the Apache user) has to be defined at compile time, along with the subdirectory under each user's $HOME directory. To view the source, customize, and build, look at the following.

% sudo su -
% cd /usr/src
% apt-get build-dep apache2.2-common
% apt-get source apache2.2-common
% cd apache2-2.\[the version\]/
% vi debian/rules
% dch -i
% dpkg-buildpackage

Notes:

  • "dch -i" increments the package number (to hinder upgrades)

In the file debian/rules, you can see the default configuration options for Ubuntu (Debian). If suEXEC were to be implemented for the cgibox, debian/rules should include

...
--with-suexec-caller=cgiwww \
--with-suexec-userdir=apache \
...

Note that suEXEC does not run php pages as the individual user. You need a separate module suphp to do that.

If you have incremented the package number (e.g. ubuntu1, ubuntu2 ...), installing the suphp package will give you a dependency error. If you haven't incremented, any upgrade will try to replaced the apache2 package.

cgiwrap

cgiwrap is what makes the CGI Box work. Through Ian's original rewrite configuration, all the user cgi scripts are piped into cgiwrap. Then, through setuid, cgiwrap executes the cgi scripts as the owner, and not as the Apache's user.

sudo apt-get install autoconf make php5-cgi eruby

Source

Hacking

I applied Ian's hack to cgiwrap-4.0

Here is the diff cgiwrap.c.orig cgiwrap.c
When testing, manually change the cgihome to the test directory.

41a42,43
>       char newHackedHomeDir[100];
>       /* used to reroute the homedirectory to /cgihome/ */
129a132,139
>        /* Ian's hack: force the user's home directory to be /cgihome/username */
>        /* (thanks Jay!) */
>        strcpy(newHackedHomeDir, "/cgihome/");
>        strncat(newHackedHomeDir, user->pw_name, (99-strlen("/cgihome/")) );
>        strcpy(user->pw_dir, newHackedHomeDir);
>        /* end hack */
>
>

Build / Install

% sudo su -
% mkdir /etc/apache2/cgi-bin
% ./configure \
      --with-perl=/usr/bin/perl \
      --with-local-contact-name="CCIS Systems" \
      --with-local-contact-email="systems@ccs.neu.edu" \
      --with-install-dir=etc/apache2/cgi-bin \
      --with-httpd-user=cgiwww \
      --with-cgi-dir=apache \
      --with-rlimit-cpu=10 \
      --with-rlimit-fsize=200000000 \
      --with-rlimit-nproc=10 \
      --with-rlimit-nofile=25 \
      --with-rlimit-rss=2500002 \
      --with-rlimit-as=100000000 \
      --with-rlimit-core=2500002 \
      --without-redirect-stderr \
      --without-check-group \
      --without-check-group-writable \
      --with-php=/usr/bin/php5-cgi \
      --with-php-interpreter \
      --with-eruby=/usr/bin/eruby \
      --with-eruby-interpreter
% make
% make install

See: ./configure --help
Warning: Some editing might be required on the Makefile

extract_UserDir_logs

This is an optional feature that parses the error_log and access_log for each individual user.

Source

Hacking

This is a copy of the original documentation.

379a380,389
>
>       /* begin ian's additions to allow for /home/username/ processing */
>       if (        *ptr == 'h' &&
>               *(++ptr) == 'o' &&
>               *(++ptr) == 'm' &&
>               *(++ptr) == 'e' &&
>               *(++ptr) == '/' )
>           extract_log_line(++ptr, line_g_size);
>       /* end ian's additions */
>

Building

See /net/ccs/src/daemons/extract_UserDir_logs

copy, make, make install

trimlog

This is somewhat of an optional feature. It is a matter of scale. This tool

Source

The original link http://www.securityfocus.com/tools/118 does not have trimlog on the page.
Try either

Building

See /net/ccs/adm/bin/trim_userdir_logs

MySQL

Source

Use the Ubuntu repository.

Install

sudo apt-get install mysql-server

Backing up

A few things changed not in this file, but in rotlogs-new. rotlogs-new was written pre-Ubuntu and the emergence of CCIS Linux machines. While the script is proven, all the paths to the binaries are hardcoded to UNIX directories (e.g. /arch/unix/... instead of using `which [binary]`).

Changing root's password

See http://dev.mysql.com/doc/refman/5.0/en/resetting-permissions.html#resetting-permissions-unix.

#!/bin/sh
#
# mysql-cgibox-backup - backs up mysql on the cgibox
#
# Ian Langworth <bass@ccs>
# Thu May 29 13:26:11 EDT 2003
#
# Instructions derived from:
# http://www.mysql.com/doc/en/Backup.html

ROTLOGS=/cgihome/cgiadmin/apache/tools/rotlogs-new
MYSQLDUMP=`which mysqldump`
DESTFILE=/testbkp/cgibox.sql
DBPASS=xxxxxxxxxxxxx

################################################

umask 077

$ROTLOGS -c gzip $DESTFILE

$MYSQLDUMP --user=root --password=$DBPASS \
     --opt --all-databases > $DESTFILE

Startup Scripts

Find the startup script for apache2 in /etc/rc3.d/

ls -al | grep apache2

Insert the startup script for extract_Userdir_logs, BEFORE the startup script for apache2.

See /proj/systems/bass/cgibox/rc

Host-Specific Files & Configurations

cron

/etc/cron.allow

root

In the test configuration, cgiwww and cgiadmin do not need cron jobs. (This may change depending on the trust relationship between the cgibox and the net drive (earthsea|narnia).

/etc/cron.daily/daily.priv

See #automount.

root's crontab
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.

SHELL=/bin/sh
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin

# m h dom mon dow user  command
17 *    * * *   root    cd / && run-parts --report /etc/cron.hourly
25 6    * * *   root    test -x /usr/sbin/anacron || \
                        ( cd / && run-parts --report /etc/cron.daily )
47 6    * * 7   root    test -x /usr/sbin/anacron || \
                        ( cd / && run-parts --report /etc/cron.weekly )
52 6    1 * *   root    test -x /usr/sbin/anacron || \
                        ( cd / && run-parts --report /etc/cron.monthly )

#cgi-box
0,5,10,15,20,25,30,35,40,45,50,55 * * * * root /testhome/cgiadmin/apache/queue/processqueue \
                                               >> /testlogs/priv/processqueue.log 2>&1
0,30 * * * * root /testhome/cgiadmin/apache/tools/iansession_purge \
                  >> /testlogs/priv/iansession_purge.log 2>&1
0 * * * * root /usr/bin/trimlog \
               >> /testlogs/priv/trim_userdir_logs.log 2>&1
30 4 * * * root /testhome/cgiadmin/apache/tools/mysql-cgibox-backup \
                >> /testlogs/priv/mysql-cgibox-backup.log 2>&1
trim_userdir_logs

See the attachment trim_userdir_logs.

Ran every hour, this script updates trimlog.conf by looking at the log directory /net/cgilogs/apache for the user (error|access) logs. Then, extract_UserDir_logs daemon is stopped, the trimlog runs, extract_UserDir_logs daemon is started back up.

automount

In the cgibox, the automount maps are not read from nis. Using a cron script, the maps are updated nightly. The script reads through auto.proj and auto.net and switches everything except the sub-directories that are cgi(home|logs) (e.g. /net/cgihome, /net/cgilogs) from read write to read-only. Here is a snippet.

ypcat -k $map | sed \
    -e '/^cgi[lh]o[gm][se]/!s/\([-,]\)rw\([, ]\)/,ro,/' \
    -e '/\/vol\/homedirs/d' \
    -e '/\/vol\/scratch/d' \
    | sort > /etc/$map.new
  • No labels