1. Introduction

Given a chaotic time series, this project examines how much is known about the underlying system. In practice, this question occurs when one measurable source of data is available, but modeling the underlying system requires an unknown amount of variables. Using time-delay coordinates and various methods described in [1]-[3], one can reconstruct the phase space and estimate the underlying dimension, also called degree of freedom. Some of these methods are described in section 2.

In this project, the shapes of the reconstructed attractors are examined. The technique of time-delay coordinates is applied to 4 sample systems, Logistic, Henon, Lorenz and Rossler. A java applet is written to test the results, and is described in section 3. Section 4 discusses on how accurate these reconstructed attractors resemble the shapes of real attractors. Based on observations, linear components in the underlying equations seem to directly affect the accuracy of the shapes.
 

2. Background Material

2.1 Time-delay Coordinates

Given a time series {s₀, s₁,...}, a d-dimensional attractor is reconstructed using coordinates (s_i, s_i+T, s_i+2T, ..., s_i+(d-1)T), where T is the time-lag, and i = 0,1,2... It is believed that information of each variable of the underlying system is intricately embedded into the time series so that phase space reconstruction is possible.

For example, the underlying equations of the Rossler system are

               x' = -y -z
               y' = x + ay
               z' = b + z(x-c),

where x', y' and z' represent the derivatives of x, y and z respectively in terms of time t. A time series may be obtained from an observation of the x-, y- or z-coordinate alone.

2.2 Choosing Time Lag

In order to reconstruct the phase space, an appropriate time-lag, T,  must be chosen. If the time-lag is too small, the coordinates s_i and s_i+T  will be almost identical; thus the resulting attractor will look like a diagonal line. On the other hand, if the time-lag is too large, the correlation between successive data points will be lost; the coordinates s_i and s_i+T will come from different regions of the underlying attractor, creating erroneous lines that hop from one side of the attractor to the other.

To find an appropriate time-lag, one can measure the average mutual information between successive data points using different values of T.

A practical solution is to use T that gives the first minimum in function I(T) as the time-lag. The rationale behind this choice is that the first minimum in average mutual information signifies the time when successive data points are independent enough but not completely uncorrelated.

2.3 Finding Embedded Dimension

The embedding theory says that if the underlying system contains an attractor of dimension d_A, the attractor reconstructed using time-delay coordinates can be unfolded in dimension  d_E < 2d_A+1; unfolding means that the trajectory does not cross itself.

To find d_E, one can use the technique of false nearest neighbors (FNN) described in [1]. False neighbors are data points that are close together only because they are projected onto a lower dimension. As the dimension is increased, false neighbors will be seperated apart. Thus the FNN method checks for decreases in number of false neighbors as the dimension goes up. The true embedded dimension d_E is found when the count of false neighbors drops to the lowest value. After d_E is found, one can estimate the true dimension d_A.
 

3. Java Applet

As part of this project, a java applet is written to test some of the issues concerning attractor reconstruction. Emphasis is placed on how 2- and 3-dimensional reconstructed attractor can accurately capture the shapes of the real attractors. Some of the applet features are described in this section. (Detailed usage of the applet is found in the document Using the Applet.) Results are discussed in section 4.

(1) The user can generate a time series from a choice of 4 sample systems, Logistic, Henon, Lorenz and Rossler, and adjust the coefficients in the system equations. Fourth-order Runge-Kutta method is used to solve the ordinary differential equations in the Lorenz and Rossler systems. The fourth-order Runge-Kutta formula is:

                    k₁     =     h * f( x_n , y_n)
                    k₂     =     h * f( x_n + h/2 ,  y_n + k₁/2)
                    k₃     =     h * f(x_n + h/2 , y_n + k₂/2)
                    k₄     =     h * f(x_n + h , y_n + k₃)
                y_n+1       =     y_n + k₁/6 + k₂/3 + k₃/3 + k₄/6

where h is a small increment of approximately 0.01. The formula is applied to each of the x-, y-, and z-coordinate.

(2) Using the time series generated in (1), the user can reconstruct the attractor in 2- or 3-dimension by adjusting the time lag. The best tested results are pre-set into the default parameters of the java applet.

(3) To facilitate viewing, features of zooming, rotation, re-centering, and resetting are provided.

(4) For the Lorenz and Rossler systems, the user can display the real attractor for comparison; the real attractor is plotted using all 3 coordinates of the system instead of time-delay coordinates coming from a time series.

(5) A crude estimation of the fractal dimension is calculated using box-counting method. The image is divided into 100 squares on each side. Then box-counting is done using box sizes of 2, 5, and 10 squares respectively. The fractal dimension is the slope of the graph log(n_i) vs. log(s_i), where s_i  is the box size and n_i  is the corresponding box count. The slope is found using a regression formula provided by the course instructor, Dr. Randall Pyke.
 

4. Shape of the Reconstructed Attractor

4.1 Overview

The Logistic and Henon time series are generated from discrete-time systems. Therefore, a time-lag of 1 is apropriate. The shapes of the reconstructed attractors are identical to the real attractors.

The Lorenz and Rossler systems provide more interesting results. The time series generated by the x- or y-coordinate creates an attractor similar to the real attractor, but not truely identical. Contrarily, the z-coordinate does not carry enough information to reconstruct the obvious features of the real attractor.

4.2 Lorenz Attractor

The Lorenz attractor looks like two circles interlocked perpendicularly, like a pair of butterfly wings twisted in the middle.

First we look at the two time series generated by the x- and y-coordinate respectively. In the real attractor, the x- and y-coordinate loop around in circles; therefore, each of the x- or y-coordinate alone contains enough information to capture the cycling effect. In the reconstructed attractor, the two wings are present. However, the height, i.e. the perpendicular interlocking effect, is not accurate.

In search for an explanation, the x and y equations are examined. The equations are x' = a(y-x) and y' = -xz+bx-y. One notices that both equations contain a linear term in x and a linear term in y; but the term z is not linear. Therefore, it is suspected that the accuracy in the x- and y-coordinate is due to these linear terms; and that the inaccuracy in the shape of z is due to non-linearity.

Then we look at the time series generated by the z-coordinate. The reconstructed attractor contains only one wing. The two-wing effect in the underlying system is not captured by the time series. Looking closely at the z equation, z' = xy - cz, one notices that the x- and y-coordinate are multiplied together. Hence, large x multiplied by small y, as well as small x multiplied by large y, both give the same value. Some information is lost in the process. The two circles are merged into one.
 

4.3 Rossler Attractor

The Rossler attractor looks like a flat disc that occasionally jumps up in the z-coordinate and then falls back down.

First we look at the two time series generated by the x- and y-coordinate respectively. In the real attractor, the x- and y-coordinate loop around like a disc; therefore, the circling effect is preserved by each of the x- or y-coordinate alone. Also, the jumping effect is preserved by the time series, and can be seen if a correct time-lag is chosen. A small time-lag creates a flatter attractor. A large, but not too large, time-lag creates a taller attractor.

To explain this result, one has to look closely at the x and y equations. The x equation, x' = -y-z, contains a simple term of -z; therefore, it is not surprising that the x-coordinate alone contains enough information of the z-coordinate for reconstruction. The y equation, y' = x + ay, contains a plain term of x; therefore, the shape of the z-coordinate is relayed through the x value to the y value. It is also not surprising that the y-coordinate requires a larger time-lag than the x-coordinate to capture the jumping effect. To see why this is so, one notices that a linear term in the equation affects the rate of change of a variable. The z term affects x with a factor of roughly h, where h is the time increment used in solving differential equations. The x term affects y with a factor of roughly h. Therefore, z affects y with a factor of roughly h².

Then we look at the time series generated by the z-coordinate. In the real attractor, the z-coordinate is mostly close to zero, with some abrupt increases in value followed by abrupt drops back to the zero range. Therefore, only the jumps are recorded in the z value, and are shown as loops of different sizes in the reconstructed attractor. The circling effect of the x- and y-coordinate is simply not present in the time series. Looking closely at the z equation, z' = b+z(x-c), one notices that the equation does not contain a linear term in x or y; therefore, it is believed that the absense of a linear term is causing the information lost.
 

5. Further Explanation

Not all information of a chaotic system is present in a single coordinate. As seen in the Lorenz and Rossler system, some important effects cannot be retrieved from the z-coordinate.

Based on observation, a linear term seems to be crucial to the preservation of information of a particular coordinate. An explanation may be possible based on how the time series is generated using Runge-Kutta method, which involves only linear addition.

In the forward direction, ie. the presence of a linear term leads to preservation of information, one notices that as the time series progresses, linear terms are being added linearly by Runge-Kutta. Suppose the x equation contains a linear term in y, and the y equation contains a linear term in y. Since we use (x_i, x_i+T, x_i+2T) as coordinates of the reconstructed attractor, the coordinate x_i+T will be ky + g(x,y,z), where ky is a linear term in y, and g(x,y,z) contains other terms. If g(x,y,z) is small compared to ky, then the shape of the y-coordinate can be accurately reconstructed using the x-coordinate alone.

In the backward direction, ie. information is not preserved if a linear term is not present, one notices that, using a similar argument from above, x_i+T will be g(x,y,z) which contains no linear term in y. Since x_i+T does not contain an obvious value of y, reconstruction will be inaccurate.
 

6. Conclusion

This project examines various issues concerning attractor reconstruction based on data obtained in the form of a time series. The time-delay coordinate method can be used to estimate the degree of freedom of the underlying system. However, the shape of the attractor may not be accurate, depending on the equations of the underlying system.
 

7. References

1. H. D. I. Abarbanel, Analysis of Observed Chaotic Data, Springer-Verlag, 1996 .

2. E. Ott, T. Sauer, J. A. Yorke, Coping with Chaos, John Wiley & Sons, 1994.

3. J.-O. Peitgen, H. Jurgens, D. Saupe, Chaos and Fractals: New Frontiers of Science, Springer-Verlag, 1992.