Help for MF3

PURPOSE

MF3   allows   the  user  to  create C -like 
expressions to perform general mathematical operations on 
one  or more IBIS/graphics file columns.   The  expressions 
are  written as a parameter string.   The parameter  is 
interpreted  to determine the input and output  columns 
and   operations  to  be  performed.    Applies a user 
specified arithmetic expression to columns of a cagis table.
All results are computed in double precision (15 decimal
places) even if the input columns are single precision or
integer.

The functions available are: @sqrt, @alog, @alog10,
@aint, @sin, @cos, @tan, @asin, @acos, @atan,
@atan2, @abs, @min, @max, @mod, along with
standard binary operations +,- *, / and ^ (pow), and
logic operations <, >, <=, >=, ==, !=, && (and), || (or),
and ! (not).  The main difference with C is the use
of ^ for power of two integers or reals is implemented;
and ^^ (xor) is not implemented (use (a||b)&&(!(a&&b))).

The old FORTRAN constructs .EQ. etc. and also **
are no longer allowed.

String functions are also available.  The arguments
can be column names (must contain strings) or string
constants enclosed in single quotes, except for some arguments
which are numeric (e.g., see fstr below).  Examples:
	      @cat(a,b)  or @cat(a,'xxx')

The string functions are:

@cat(a,b)                concatenates a to b

@break(a,b)              outputs a up to first occurrence of a
			 character in b   (e.g., @break(a,'.,;:?'))

@fstr(a,m)               outputs the first m characters of a

@bstr(a,m)               outputs from the m'th character to the end of a

@adelete(a,b)            deletes any of b's characters from a
			 (e.g., @adelete(a,'.,;:?'))

@sdelete(a,b)            deletes occurrences of the whole string b from a
			 (e.g., @sdelete(a,'dog')

@trim(a,b)               trims from the low order end of a, all characters
			 in b, but stops trimming at the first non-b char

@ucase(a)                outputs a in upper case

@lcase(a)                outputs a in lower case

@ljust(a,n)              left justifies a in an n-character field.  if too
			 long, keeps high order part of a

@rjust(a,n)              right justifies a in an n-character field.  if too
			 long, keeps high order part of a

@replace(a,'dog=cat')    replaces all occurrences of the string before the
			 = with the string after the =

@strlen(a)               outputs the length of the string a

@pos(a,b)                finds the pattern b in a and returns its starting
			 position.  ^ is left anchor % is right anchor
			 ? matches any single character * matches a run
			 (e.g., @pos(a,'^a??.*z*%'))

@streq(a,b)              returns TRUE or 1 if a equals b else FALSE or 0

@strsub(a,b)             returns TRUE or 1 if a contains b else FALSE or 0

@strpat(a,b)             returns TRUE or 1 if a contains the pattern b
			 else FALSE or 0.  see the syntax for @pos(a,b)

@num(a)                  returns the numeric value of string a, which must
			 contain an integer or floating number, can have exponent
			 such as 2.73e-06 (use e, E, d, or D).

@i2str(n)                converts the integer n to a string; zero goes to 0

@f2str(f,n)              converts the float or integer f to a floating
			 string with n digits of precision to the right of
			 the decimal; n=0 omits the decimal; rounding is
			 performed

@dmsstr(a)               converts the string degree-min-second into a
                         degree number.   Acceptable formats include
                         1332727.666, 1332727.666W, 1.332727E+06W,
                         133d27m27.666 where the - can be any non-numeric
                         separator other than .+-Ee.  The EWNS can be
                         lower case and can be at the front e1332727.666.
                         A minor point, exponent e or E can be followed
                         by a number or a sign, but d or D must be
                         followed by a sign.

@dmsnum(f)               converts the number degree-min-second into a
                         degree number.   Acceptable formats include
                         1332727.666, -1332727 (real or integer).

All operations work as in the c language with
except for the column operations described below.
The special variable @index may be used to insert
the record number into an expression.  The special
variable @rand may be used to put a random number
between 0 and 1 in the column.  If @rand is used, the
parameter seed can be used to vary the random sequence
Multiple formulas may be given by separating them with the
$ character.

Column operations are added features that perform
specialized functions to the table.  Two restrictions
must be observed:

1. Column operations cannot be used in a formula.
2. The arguments must be column names, not constants
   or expressions.

They perform an operation on columns placing
results in a column.  The operations @fill and
@interp require a column of values separated by
zeros.

The column operations are:

@average(col)           calculates the average in
			the column

@sigma(col)             calculates the standard deviation in
			the column

@sum(col)               sum the values in the column

@rsum(col)              running sum of values in the
			column

@csum(col1,col2)        controlled sum; sum the values
			in col1 using col2 as a control
			column, restarts the sum for a
			change in the value in col2

@crsum(col1,col2)       controlled running sum; running
			sum of values in col1 but restarts
			the sum for a change in the value
			in col2

@vmax(col)              calculates the maximum in the column

@vmin(col)              calculates the minimum in the column

@cvmax(col1,col2)       controlled maximum; calculates the
			maximum in col1 using col2 as a control
			column, restarts the max for a change
			in the value in col2

@cvmin(col1,col2)       controlled minimum; calculates the
			minimum in col1 using col2 as a control
			column, restarts the min for a change
			in the value in col2

@diff(col)              subtracts the value in the previous
			record from the value in the current
			record

@cdiff(col1,col2)       subtracts the value in the previous
			record from the value in the current
			record; restarts the operation for a
			change in the value in col2

@shift(col,n)           shifts downward n records,
			negative n for upward shift;
			downward shift replicates first
			value in column while upward
			shift replicates last value

@rotate(col,n)          same as shift except values that
			are rotated off the end of the
			column are wrapped around to the
			other end

@interp(col1,col2)      replace zero values between non-zero
			values in col1 by interpolating
			between the non-zero values in col1
			to corresponding values in col2;
			col2 may contain @index in which case
			interpolation is linear or it may
			contain some other function
			(i.e. logarithmic or exponential)

@fill(col)              fill the zeros in the column with the
			previous non-zero value in the column

@dist(lon1,lat1,lon2,lat2,dist)     calculate the distance in meters
				    between the two geographic points
				    on the Earth.  A spherical formula
				    is used above 1.05 degrees and a
				    plane formula is used below .95
				    degrees of central arc.  Between
				    these values, both formulas are used
				    and the result is a linear
				    interpolation of both formulas.
				    This is done to give a continuous
				    result.  Results near the poles
				    are not guaranteed accurate.

@head(lon1,lat1,lon2,lat2,head)     calculate the heading of the line
				    from the first to the second point
				    in degrees clockwise from north.
				    The interpolation technique used
				    in @dist is applied here.

@bear(lon1,lat1,lon2,lat2,lon3,lat3,bear) calculate the bearing of the
					  line from the first to the
				    second point clockwise in degrees
				    from the line from the first to the
				    third point.  The interpolation
				    technique used in @dist is
				    applied here.


A full example of an fstring to calculate a
time increment dt from a column t is
fstring="dt=t$shift(dt,-1)$dt=t-dt"

TAE COMMAND LINE FORMAT

     MF3 INP=int PARAMS

     where

     int                 is a random access file.  Since it
                         is used for both input and  output, 
                         no output file is specified.

     PARAMS              is   a  standard  VICAR   parameter 
                         field.
METHOD

     MF3 performs arithmetic operations on an interface file.  
     The  program  uses  two  library  routines SP_KNUTH  and 
     SP_XKNUTH,   to   compile  and  interpret  C-like 
     expressions  entered by the parameters in an expression 
     such as:

                     C135 = (100*C34)/C4

     In this expression,  C34 and C4 are the input  columns.  
     SP_KNUTH    compiles   the   expression   into  pseudo-machine 
     instructions.   The  expression is applied to the input 
     column in SP_XKNUTH to produce the output column, C135.


RESTRICTIONS

1.     Maximum number of columns in one execution is 100. (oops, 9 until bug is fixed)
2.     The number of columns in the IBIS file is not limited here.
3.     Maximum input string length is 10,000 (40 x 250).
4.     Maximum number of operations is 3000.
5.     Maximum number of temp locations is 938.
6.     Maximum number of constants from the expression is 960.

notes:

1.  Column numbers greater than 100 are mapped sequentially 1,2,3...
    so there is no limit on the number of columns in the IBIS file.
3.  The input parameter is a string array (40) each with 250 chars.
    The array is concatenated by the program into a single array.
4.  These can be counted by setting debug to one and counting the
    lines that begin with "xknuth:op,opnd".  The count is not
    easily determined by looking at a long input.
5.  These can be counted by setting debug to one and counting the
    lines that begin with "xknuth:op,opnd" and having an opnd
    value above 1061.  The count is not easily determined by
    looking at a long input.
6.  These can be counted in the input, or by setting debug to one
    and counting the lines that begin with "xknuth:op,opnd" and
    having an opnd value between 103 and 1061, inclusive.
EXAMPLE

     MF3 INP=FILE.INT FUNCTION=("C5 = C2/C3+100+@SQRT(C2)")

     In this example,  C2 is divided by C3 and added to  100 
     plus the square root of C2.   The results are placed in 
     C5.  Further examples of allowable functions follow:

                FUNCTION=("C5 = !(C3  || C2)")

     logical   operations  are  performed  bitwise  on   the 
     operands. The  logical values T and F are converted to 1.  and 0. 
     for storage in column C5

                FUNCTION=("X5 = X3<=INDEX")

     Column 5 is 1.0 if column 3 has a value < its row value (INDEX).
     
                FUNCTION=("@average(C3)")

     In this example, the mean of column 3 is calculated and 
     that  value is placed in every row entry in  column  3.  
     This  operation  is different than the  arithmetic  and 
     logic operations given earlier because it operates on a 
     vertical  column instead of horizontally across a  row.  
     These  operations  cannot  be  used  in  an  arithmetic 
     expression  such as C5 = @average(C3)*10.   See the FUNCTION
     help for more examples.

Original Programmer:  A. L. Zobrist, 15 December 1976

Cognizant Programmer:  N. D. Ritter

Revision:
  12 December 1999 Double precision and strings, etc. A. L. Zobrist
  06 February 2000 Enlarge all Function restrictions  A. L. Zobrist
  Fri Dec 28 2007 wlb switched to USES_ANSI_C AND LIB_CARTO; misc cleanup

PARAMETERS: