Processing Form Data in Shell CGI Scripts

Note: Writing this script was interesting, but the program is also kind of a joke. I wanted to prove that it is indeed possible to pick apart CGI data with a shell script, and I succeeded. Yet the script has some deficiencies.

Don't get me wrong. Programming CGI shell scripts isn't a bad idea in general, it indeed works quite well for small jobs. But evaluating the form datashouldn't be done in a script.

One problem is that this script is slow. For biting its way through the data string, it must continuously invoke the afore-mentioned utilities. Hundreds of times. Second, there are a few tiny differences in the tools' implementations on various platforms, so in rare cases, obscure problems may occur.

I never expected the overwhelming feedback on my script. Unfortunately, it didn't always work out perfectly. It is much better having an integrated piece of C code that does the job than a shell script. I have decided to write such a tool, proccgi, to help all folks that were having problems getting the script to work. Unless you know what you're doing, or are looking for fascinating shell scripts, please use that program to evaluate form data in your shell CGI scripts instead, it's much easier to use.

If you are currently using the script on your site, you can simply switch over to proccgi. It works the same, and you only need to change a single line of code in your CGI scripts.

A big thank you to all who have used this shell script in the past, present, and future, and to those who commented on it.

Yet another option is to switch to the Tcl language for CGI scripting. It's powerful and easy to use; I use it for most of my scripting these days. Of course I also have a Tcl script for CGI data evaluation: proccgi.tcl.

Features

• Handles both GET and POST methods transparently.
• Also accepts assignments on the command line (great for testing).
• If the extra pathname features assignments (like /A=trash), it is processed as well.
• Handles '+' (replaced by space) and '%XX' sequences.
• Exports all found variables to the shell environment.
• Can also handle multiline "Textarea" input fields.

Bugs

• All exported variables are case sensitive.
• Cannot properly handle newlines (%0A); they are replaced by spaces.
• Hex escapes must be uppercase. This should be guaranteed by the remote browser.
• Slow.

Usage

eval `proccgi.sh $*`

If something goes wrong, you can also enable some debugging output by setting the variable DEBUG to 1, causing a log to be printed to standard error. To get a debugging logfile, you would call

DEBUG=1
export DEBUG
eval `proccgi.sh $* 2> logfile`

Example

The Form

<form action="http://our-server/cgi-stuff/mailer" method="post">
<dl>
  <dt> Your Email <dd> <input name="email" size="50">
  <dt> Filename   <dd> <input name="file"  size="50">
</dl>
<input type="submit" value="Submit">
</form>

The Script

#!/bin/sh
eval `proccgi.sh $*`
mail $FORM_email < $FORM_file
cat - << \END
echo Content-type: text/plain
echo
echo done.
END

Note

The script may be freely used and distributed at no charge provided that the copyright notice remains at its top.

Script & Code

Don't forget to download the code, proccgi.sh.

http://www.fpx.de/fp/Software/ProcCGIsh.html

#!/bin/sh
#
# Process input to a CGI script. Written and Copyright 1995 Frank Pilhofer
# You may freely use and distribute this code free of charge provided that
# this copyright notice remains.            fp@informatik.uni-frankfurt.de
#
# All variables in here are prefixed by _F_, so you shouldn't have
# any conflicts with your own var names
#
# get query string. if $REQUEST_METHOD is "POST", then it must be read
# from stdin, else it's in $QUERY_STRING
#
if [ ${DEBUG:-0} -eq 1 ] ; then
	echo --Program Starts-- 1>&2
fi
#
if [ "$REQUEST_METHOD" = "POST" ] ; then
	_F_QUERY_STRING=`dd count=$CONTENT_LENGTH bs=1 2> /dev/null`"&"
	if [ "$QUERY_STRING" != "" ] ; then
		_F_QUERY_STRING="$_F_QUERY_STRING""$QUERY_STRING""&"
	fi
	if [ ${DEBUG:-0} -eq 1 ] ; then
		echo --Posted String-- 1>&2
	fi
else
	_F_QUERY_STRING="$QUERY_STRING""&"
	if [ ${DEBUG:-0} -eq 1 ] ; then
		echo --Query String-- 1>&2
	fi
fi
if [ ${DEBUG:-0} -eq 1 ] ; then
	( echo "  " $_F_QUERY_STRING
	  echo --Adding Arguments-- ) 1>&2
fi
#
# if there are arguments, use them as well.
#
for _F_PAR in $* ; do
	_F_QUERY_STRING="$_F_QUERY_STRING""$_F_PAR""&"
	if [ ${DEBUG:-0} -eq 1 ] ; then
		echo "  " arg $_F_PAR 1>&2
	fi
done
if [ ${DEBUG:-0} -eq 1 ] ; then
	( echo --With Added Arguments--
	  echo "  " $_F_QUERY_STRING ) 1>&2
fi
#
# if $PATH_INFO is not empty and contains definitions '=', append it as well.
# but replace slashes by ampersands
#
if echo $PATH_INFO | grep = > /dev/null ; then
	_F_PATH_INFO="$PATH_INFO""//"
	if [ ${DEBUG:-0} -eq 1 ] ; then
		( echo --Adding Path Info--
		  echo "  " $_F_PATH_INFO ) 1>&2
	fi

	while [ "$_F_PATH_INFO" != "" -a "$_F_PATH_INFO" != "/" ] ; do
		_F_QUERY_STRING="$_F_QUERY_STRING""`echo $_F_PATH_INFO | cut -d / -f 1`""&"
		_F_PATH_INFO=`echo $_F_PATH_INFO | cut -s -d / -f 2-`
	done
fi
#
# append another '&' to fool some braindead cut implementations. Test yours:
# echo 'i am braindead!' | cut -d '!' -f 2
#
_F_QUERY_STRING="$_F_QUERY_STRING""&"
#
if [ ${DEBUG:-0} -eq 1 ] ; then
	( echo --Final Query String--
	  echo "  " $_F_QUERY_STRING ) 1>&2
fi
#
while [ "$_F_QUERY_STRING" != "" -a "$_F_QUERY_STRING" != "&" ] ; do
	_F_VARDEF=`echo $_F_QUERY_STRING | cut -d \& -f 1`
#	_F_QUERY_STRING=`echo $_F_QUERY_STRING | cut -d \& -f 2-`
	_F_VAR=`echo $_F_VARDEF | cut -d = -f 1`
	_F_VAL=`echo "$_F_VARDEF""=" | cut -d = -f 2`

#
# Workaround for more braindead cut implementations that strip delimiters
# at the end of the line (i.e. HP-UX 10)
#

	if echo $_F_QUERY_STRING | grep -c \& > /dev/null ; then
		_F_QUERY_STRING=`echo $_F_QUERY_STRING | cut -d \& -f 2-`
	else
		_F_QUERY_STRING=""
	fi

	if [ ${DEBUG:-0} -eq 1 ] ; then
		( echo --Got Variable--
		  echo "  " var=$_F_VAR
		  echo "  " val=$_F_VAL
		  echo "  " rem=$_F_QUERY_STRING ) 1>&2
	fi
	if [ "$_F_VAR" = "" ] ; then
		continue
	fi

#
# replace '+' by spaces
#

	_F_VAL="$_F_VAL""++"
	_F_TMP=

	while [ "$_F_VAL" != "" -a "$_F_VAL" != "+" -a "$_F_VAL" != "++" ] ; do
		_F_TMP="$_F_TMP""`echo $_F_VAL | cut -d + -f 1`"
		_F_VAL=`echo $_F_VAL | cut -s -d + -f 2-`

		if [ "$_F_VAL" != "" -a "$_F_VAL" != "+" ] ; then
			_F_TMP="$_F_TMP"" "
		fi
	done

	if [ ${DEBUG:-0} -eq 1 ] ; then
		echo "  " vrs=$_F_TMP 1>&2
	fi

#
# replace '%XX' by ascii character. the hex sequence MUST BE uppercase
#

	_F_TMP="$_F_TMP""%%"
	_F_VAL=

	while [ "$_F_TMP" != "" -a "$_F_TMP" != "%" ] ; do
		_F_VAL="$_F_VAL""`echo $_F_TMP | cut -d % -f 1`"
		_F_TMP=`echo $_F_TMP | cut -s -d % -f 2-`

		if [ "$_F_TMP" != "" -a "$_F_TMP" != "%" ] ; then
			if [ ${DEBUG:-0} -eq 1 ] ; then
				echo "  " got hex "%" $_F_TMP 1>&2
			fi
			_F_HEX=`echo $_F_TMP | cut -c 1-2 | tr "abcdef" "ABCDEF"`
			_F_TMP=`echo $_F_TMP | cut -c 3-`
#
# can't handle newlines anyway. replace by space
#
#			if [ "$_F_HEX" = "0A" ] ; then
#				_F_HEX="20"
#			fi

			_F_VAL="$_F_VAL""`/bin/echo '\0'\`echo "16i8o"$_F_HEX"p" | dc\``"
		fi
	done

#
# replace forward quotes to backward quotes, since we have trouble handling
# the former ones.
#

	_F_VAL=`echo $_F_VAL | tr "'" '\`'`

#
# if debug, send variables to stderr
#

	if [ ${DEBUG:-0} -eq 1 ] ; then
		( echo --Final Assignment--
		  echo "FORM_$_F_VAR"=\'$_F_VAL\' ) 1>&2
	fi

#	/bin/echo "FORM_$_F_VAR"=\'$_F_VAL\'
	/bin/echo "FORM_$_F_VAR"="'"$_F_VAL"'"
done
#
if [ ${DEBUG:-0} -eq 1 ] ; then
	echo done. 1>&2
fi
#
# done.
#
exit 0

http://cyent.blog.51cto.com/905592/829034