README for NETLE 3.01


*************************************************************************
**    PROGRAM:   NETLE                                                 **
**    PURPOSE:   Feedforward Neural Networks and Lyapunov Exponents    **
**                  Estimation                                         **
**    COPYRIGHT: Chung-Ming Kuan, Tung Liu, and Ramazan Gencay         **
**    VERSION:   3.01, January 1997                                    **
*************************************************************************

UPDATE: (1/2/1997)

NETLE 3.01 is a Windows 95/NT program to estimate feedforward neural 
network models and compute Lyapunov exponents (LE). The maximum 
number of data values is 7000. The maximum inputs and hiddens are 20. 
Current version runs under the DOS prompt. The future version will 
be run under Windows.

Changes: 1/2/1997 Compute LE for inputs up to 20. 

CONTENTS:

Section 1. INTRODUCTION
	What is NETLE? Installation and run the program.
Section 2. SYSTEM REQUIREMENTS
Section 3. PROGRAM RESTRICTIONS:
	The data and neural network restrictions for this program.
Section 4. DATA FILE:
	The structure of the data file.
Section 5. CONTROL FILE:
	A file with the information and options for running this program.
Section 6. PROGRAM OUTPUT:
	How to read the program output.
Section 7. CORRESPONDENCE:
	The address, phone number, and email address for the authors.

========================================================================

1. INTRODUCTION

This program, NETLE.EXE, estimates feedforward neural network models
and computes Lyapunov exponents (LE).  Neural networks are estimated
by the method of nonlinear least squares (NLS) (Kuan and Liu (1995));
Lyapunov exponents are calculated from the derivative matrices of
estimated network models (Gencay and Dechert (1992)).  Note that a
positive Lyapunov exponent indicates that the underlying series is
chaotic.

This disk (or zip file) contains the main program, a sample data, 
and a control file:

README.TXT	This file.
NETLE.EXE	The main program.
HENON.IN	A sample control file for NETLE.EXE.
HENON.DAT	An example data set with 600 observations and 4 inputs.

To run NETLE, unzip the file NETLE3.ZIP to a directory. Put the main 
program, your data file, and the control file in one directory. And run 
the program from the DOS prompt by typing:

C:> NETLE control_file_name

The structure of your data file must follow the format described in 
the later section. The detail of the control file is also explained
in this document. You may test NETLE with the supplied sample control 
file and data file by typing:

C:> NETLE HENON.IN

This disk is distributed for academic purposes only; proper citation of
this program is gratefully appreciated.

REFERENCES:

Kuan, Chung-Ming and Tung Liu (1995).
  "Forecasting exchange rates using feedforward and recurrent networks,"
  Journal of Applied Econometrics, Vol. 10, 347-364.

Gencay, Ramazan and W. D. Dechert (1992).
  "An algorithm for the n Lyapunov exponents of an n-dimensional unknown
  dynamical system," Physica D, 59, 142-157.

========================================================================

2. SYSTEM REQUIREMENTS

NETLE.EXE requires an IBM or IBM compatible PC. Since the NLS estimation 
requires a lot of CPU time, a high speed 486 or Pentium PC is recommended.
The program is for 32 bits OS. This requires a Windows 95 or NT system.


========================================================================

3. PROGRAM RESTRICTIONS

Current version of NETLE.EXE has the following restrictions:

    (a) Data:	 The maximum number of observations is 7000.
    (b) Network: The maximum number of inputs is 20;
		 The maximum number of hidden units is 20;

You may run NETLE with the above maximum feature only if your
PC has enough RAM.

========================================================================

4. DATA FILE

The first column of the data file must be a series of ID numbers (e.g.
1,2,3,....,n); the second column should contain the target (dependent)
variable; the other columns should be input (explanatory) variables.

========================================================================

5. CONTROL FILE

NETLE is not an interactive program.  The execution of this program
is controlled by a control file which contains information of the
data file name, network structure, and other options for computation.
A sample of this control file SAMPLE.IN is included in the disk.  The
simplest way to create your own control file is to revise the content of
SAMPLE.IN and rename it with a new file name.  After you create your
own control file, you can run NETLE from the DOS prompt by typing

C:> NETLE  control_file_name

For example, to test the program using SAMPLE.IN, type

C:> NETLE  SAMPLE.IN

5.1 Contents of a Control File:

This file contains 16 instruction lines.  Each line is followed by
a single information line.  The user must supply all the information
in these information lines.  If no information is needed, a blank line
must exist between two instruction lines.  The program will not function
properly if wrong information or more than one information line is supplied.
If two numbers are required in an information line, these numbers must be
separated by space(s).	DO NOT erase the instruction lines in your control
file as the program will read them as NULL lines.

The following are the instruction lines and explanation.

(1)  [* NETLE Version 2.1 (neural NETwork for LE) by Kuan, Liu, & Gencay. *]

     The TITLE you would like to have in your output file.

(2)  [* Data File Name: *]

     The name of the data file to be read by the program.

(3)  [* Output File Name: *]

     The name of the output file.

(4)  [* Number of Observations: *]

     Total number of observations in the data file specified in (2).

(5)  [* Number of Inputs: *]

     The number of inputs (explanatory variables).

(6)  [* Minimum and Maximum Number of Hidden Units: *]

     Two integer values.  The first value, q, is a number specifying the
     minimum number of hidden units; the second value, Q, is a number
     specifying the maximum number of hidden units.  The program will
     estimate total (Q-q+1) networks.

(7)  [* Calculating the Lyapunov Exponent: 0/No, 1/Yes *]

     Enter 0 or 1.  The Lyapunov exponents will be computed if 1 is entered.

(8)  [* Method of Scaling Inputs: *]

     Enter one of the following 4 choices:
     0: no scaling is made;
     1: each input is standardized by subtracting its mean and divided
	by its standard deviation;
     2: each input is standardized between 0 and 1 using a logistic
	transformation: x*=1/(1+exp(-x));
     3: each input is standardized between -1 and 1 using a logistic
	transformation: x*=2/(1+exp(-x)) - 1;
     Notice that this value will be set to 0 by the program if the
     calculation of Lyapunov exponents is requested.

(9)  [* Seeds (Positive Number); NTRY: *]

     The first number specifies the seed of the (standard normal)
     random number generator;  the second gives the number of
     sets of initial parameters to be generated.  The program will
     select the set of initial parameters which results in the lowest
     MSE as the starting values for NLS estimation.

(10) [* Number of Iterations; Tolerance in NLS: *]

     The first number specifies the maximum number of iterations in
     NLS estimation; the second specifies the tolerance level
     for NLS convergence.

(11) [* Creating the Parameters File: 0/No, 1/Yes *]

     Enter 0 or 1.  A file for parameter estimates will be created under
     the name specified in (12) if 1 is entered.

(12) [* Parameters File Name: (The Option Above Must Be "1") *]

     The file name for parameter estimates.  If 0 is specified in
     (11), you may leave this line blank, or the program will ignore the
     content of this line.

(13) [* Creating the Fitted Values File: 0/No, 1/Yes *]

     Enter 0 or 1.  A file for fitted values will be created under the
     name specified in (14) if 1 is entered.

(14) [* Fitted Values File Name: *]

     The file name for fitted values.  If 0 is specified in (13),
     you may leave this line blank, or the program will ignore the content
     of this line.

(15) [* Creating the Derivatives: 0/No, 1/Yes *]

     Enter 0 or 1.  A file for derivatives will be created under the
     name specified in (16) if 1 is entered.

(16) [* Derivatives File Name: *]

     The file name for derivatives which is used to compute Lyapunov
     exponents.  If 0 is specified in (15), you may leave this line blank,
     or the program will ignore the content of this line.


5.2. A Sample Control File

Below is a sample control file HENON.IN.  The user may create a new
control file by suitably modifying this sample file.

   [* NETLE Version 3.0 (neural NETwork for LE) by Kuan, Liu, & Gencay. *]
Henon Map
   [* Data File Name:	*]
henon.dat
   [* Output File Name: *]
henon.out
   [* Number of Observations: *]
200
   [* Number of Inputs: *]
2
   [* Minimum and Maximum Number of Hidden Units: *]
1   2
   [* Calculating the Lyapunov Exponent: 0/No, 1/Yes *]
1
   [* Method of Scaling Inputs: *]
0
   [* Seeds (Positive Number); NTRY: *]
56666459   500
   [* Number of Iterations; Tolerance in NLS: *]
900  1.0d-9
   [* Creating the Parameters File: 0/No, 1/Yes *]
1
   [* Parameters File Name: (The Option Above Must Be "1") *]
henon.par
   [* Creating the Fitted Values File: 0/No, 1/Yes *]
1
   [* Fitted Values File Name: *]
henon.fit
   [* Creating the Derivatives: 0/No, 1/Yes *]
1
   [* Derivatives File Name: *]
henon.der

========================================================================

6. PROGRAM OUTPUT

The results of the program will be displayed on the screen and
written to an output file as specified in the control file.

Part of the result from the control file SAMPLE.IN is as follows.

  Hiddens  1:	    MSE =    .226113	      SIC =   -1.35426
     Lyapunov Exponents:
	lambda( 1) =	 -1.451576
	lambda( 2) =	 -4.839406

  Hiddens  2:	    MSE =    .744204E-08      SIC =   -18.4777
     Lyapunov Exponents:
	lambda( 1) =	  .4331328
	lambda( 2) =	 -1.636965

The values of lambda's are the estimated Lyapunov exponents. If the
calculation of Lyapunov exponents is set to "0" in SAMPLE.IN, the
values of lambda will not be given, and the result reads:

  Hiddens  1:	    MSE =    .226113	      SIC =   -1.35426
  Hiddens  2:	    MSE =    .744204E-08      SIC =   -18.4777


6.1 The file for Parameter Estimates

If the parameter file is requested, both the initial values from the
random number generator and the final estimates from NLS will be written
to the parameter file.	There are two lines above each set of parameters:

  Initial Est.:    ntobs   nv	  nq	nw
		           xxx     x	   x	xx

  NLS Est.:	   ntobs   nv	  nq	nw
          	    xxx     x	   x	xx

Here, "ntobs" is the total number of observations in the data file,
"nv" is the number of inputs, "nq" is the number of hidden units, and
"nw" is the total number of parameters of the estimated network.  The
parameters of each network in the parameter file are arranged according
to the following order:

Output bias (constant term),
Parameters connecting hidden and output units,
The first hidden bias,
The parameter connecting the first input and first hidden units,
The parameter connecting the second input and first hidden units,
...
The parameter connecting the v-th input and first hidden units,
...
The q-th hidden bias,
The parameter connecting the first input and q-th hidden units,
The parameter connecting the second input and q-th hidden units,
...
The parameter connecting the v-th input and q-th hidden units,

6.2. The file for Fitted Values

The file for fitted values has 3 columns: the first contains
original target values, the second contains network fitted values,
and the third contains network errors (target - fitted values).

========================================================================

7. CORRESPONDENCES

Questions of this program should be addressed to:

Prof. Tung Liu
Department of Economics
Ball State University
Muncie, IN 47306
U.S.A.
(765) 285-3724
E-Mail: tliu@bsu-cs.bsu.edu

Other authors can be reached at:

Prof. Chung-Ming Kuan
Department of Economics
National Taiwan University
Taipei 10020
TAIWAN
(886-2) 351-9641 ext. 522
E-mail: ckuan@ccms.ntu.edu.tw

Prof. Ramazan Gencay
Department of Economics
University of Windsor
Windsor Ontario
Canada, N9B3P4
(519) 253-4232 ext. 2382
E-mail: gencay@uwindsor.ca


Home
Author: Tung Liu | Last Modified: 4-September-96 | Comments to Webmaster