Echo state network ile sistemlerin modellenmesi

Abstract

Modelleme genel anlamda, bir sistemin değişen koşullar altındaki davranışlarını incelemek, benzetimini yapmak ve kontrol etmek amacıyla elemanları arasındaki bağlantıları matematiksel olarak ifade etmektir. Gerçek hayatta kullanılan çoğu sistem doğrusal değildir ve bu sistemlerin iç yapıları epey karmaşıktır. Bu durum, sistemlerin matematiksel modellerinin çıkarılmasını zorlaştırmakta hatta bazen imkansız kılmaktadır. Modellemede karşılaşılan bu zorluklar, bulanık modelleme ve yapay sinir ağları ile modelleme yöntemlerinin doğmasına sebep olmuştur. Gerçek sistemden elde edilen giriş – çıkış verileri ile sistemin bulanık modelinin oluşturulması bulanık modelleme; yine gerçek sistem verileri ile sistemin yapay sinir ağı modelinin oluşturulması ise yapay sinir ağları ile modelleme olarak tanımlanabilir.Yapay sinir ağ modellerinden tekrarlı yapay sinir ağlarınını eğitmek için kullanılan gradyal azaltma yöntemine alternatif olması amacıyla son yıllarda `rezervuar hesaplama` yöntemi ortaya atılmış ve bu konuda çalışmalar yapılmıştır. Rezervuar hesaplamanın anahtar yöntemlerinden olan echo state network, bu alandaki aktif çalışmalardan biridir. Pratik, kavramsal olarak basit ve kolay uygulanabilir olduğu için pek çok alanda başarılı bir performans göstermektedir.Bu çalışmanın amacı, herhangi bir sistemden toplanan giriş – çıkış verilerini kullanarak sistemin echo state network kullanarak modellenmesinin yapılmasıdır. Bu amaçla dört farklı sistemden alınan giriş – çıkış verileri ile sistemlerin modelleri echo state network ile oluşturulmuş ve ardından en popüler bulanık modelleme yazılımı olan ANFIS ile başarıları karşılaştırılmıştır.

In general, modeling means expressing the connections between its elements mathematically to analyze the system behaviour under changing conditions, to simulate and to control it. Most of the systems used in real life are not linear and internal structures of these systems are quite complex so it is hard or sometimes even impossible to design mathematical models of these systems. The difficulties encountered in modeling has led to the creation of fuzzy modeling and artificial neural network modeling methods. The creation of fuzzy model by using input – output data that are collected from real system is fuzzy modeling; again the creation of a model with ANNs with actual system data can be defined as artificial neural network model of the system.Moreover, fuzzy logic was first introduced by Lotfi Zadeh in 1965. Zadeh and his coworkers continued to develop fuzzy logic and implement at the time. In the beginning the idea of fuzzy sets and fuzzy logic were not accepted in academic areas, because some of the basis mathematics of it had not yet been explored. Applications of fuzzy logic develops slowly because of this, except in the east. The idea of fuzzy logic is more closer to the culture and the history of the Eastern people than Western people. Looking at the development of fuzzy logic, it can easliy be seen that the rate of development was more faster in the East in the compare of the West. For example in Japan fuzzy logic was fully accepted and applied in products simply because fuzzy logic worked, regardless of mathematicians agree or not. In the early 80s, the success of many of the products based on fuzzy logic in Japan gave a lead to an appearance of fuzzy logic in the U.S in the late 80s. Since that time, in the area of fuzzy logic, U.S. has been trying to catch up with this.Zadeh introduced his idea with a paper `Fuzzy Sets`. A fuzzy set is a set without a crisp boundary. The transition from `belonging to a set` to `not belonging to a set` is gradual and smooth and this is characterized by membership functions. Determination of membership functions is subjective. Fuzzy sets have nothing to do with randomness. Subjectivity and randomness are two important differences between fuzzy sets and probability theory.In using fuzzy logic there is a need to a fuzzy inference system which uses fuzzy set theory to map inputs to outputs. Mainly there are three types of fuzzy inference system; Mamdani type of fuzzy inference system, Sugeno type of fuzzy inference system and finally singleton type of fuzzy inference system. In Mamdani type of fuzzy inference system, it is necessary to determine a set of fuzzy rules. Fuzzifying the inputs using the input membership functions is the next step of this process. Then it is combined the fuzzified inputs according to the fuzzy rules to establish a rule strength. Continuely, it is suggested to find the consequence of the rule by combining the rule strength and the output membership function. Finally, combining the consequences to get an output distribution, and defuzzifying the output distribution (this step is only if a crisp output (class) is needed). In Takagi – Sugeno type of fuzzy inference system, the first two parts of the fuzzy inference process, fuzzifying the inputs and applying the fuzzy operator, are exactly the same. The main difference between Mamdani and Sugeno is that the Sugeno output membership functions are either linear or constant.Artificial neural network modeling is one of the most important modeling ways in academic areas. It has basically two steps; mathematical structure of neuron and activation part. In activation part, learning is quite important to have a descent system model. Learning is a process by which the free parameters of neural networks (generally the weights and biases) are adopted through a process of stimuliation by the environment in which the network is embedded. The sequence of events can be written as follows; neural network is stimulated by the environment, neural network undergoes changes in its free parameters as a result of this stimulation and neural network responds in a new way to the environment because of the changes that have occurred in its' internal structure.Learning has two main parts; learning with a teacher and learning without a teacher. Learning without a teacher also divided into two different subclasses asrainforcement learning and unsupervised learning. In learning with a teacher, teacher has the knowledge of the environment while the neural network does not have it. Teacher and the neural network are both exposed to training vector drawn from environment. Teacher provides the neural network with a desired response. This is the optimum action to be performed by neural network. In learning without a teacher in rainforcement learning the main idea is sourced by animal learning. The basic idea is awarding the learner from correct actions and punishing wrong actions. Rainforcement learning is a process of trial and error, combined with learning. In unsupervised learning neural network develops the ability to form internal representations for encoding features of the input and thereby to create new classes automatically. Moreover, in unsupervised learning neural network tries to capture statistical proper of the input. Provision is mode for a task independent measure of the quality of representation that the network is required to learn and the free parameters of the neural network are optimized with respect to that measure.One of the leading toolbox in fuzzy modeling that has been accepted by many scientists who work in this area is ANFIS (Artificial Neuro – Fuzzy Inference System) because of its user interface and flexible which is available in MATLAB software. ANFIS combines the fuzzy and artifical neural network ideas that make it popular in academic area that much.Similar to many popular softwares, ANFIS has its own weakness. As instance it only can have a single output, obtained using weighted average defuzzification. Also all the output membership functions must be the same type and either be linear or constant. In addition, ANFIS has no rule sharing which means different rules cannot share the same output membership function, namely the number of output membership functions must be equal to the number of rules.It also has the unity weight for each rule, which means each rule should have same weight. This situation can reduce the success of the modeling systems.Reservoir computing has emerged in the last years as an alternative to gradient descent methods for training recurrent neural networks. Echo State Network is one of the key methods for reservoir computing. While being practical, conceptually simple, and easy to implement, ESNs require achieve the hailed good performance in many tasks.Echo state network was introduced by Prof. Dr. Herbert Jaeger, a professor of Computational Science in Jacobs University, as an academic report in 2001. Corrected and full version of this report was published in 2010. This report introduces a constructive learning algorithm for recurrent neural networks, which modifies only the weights to output units in order to achieve the learning task and that is the main idea of echo state network. He also put the free toolbox of echo state network on internet. The perspective taken is mainly that of mathematics and engineering, where a recurrent network is seen as a computational device for realizing a dynamical system. The RNNs considered here are mostly discrete-time, sigmoid-unit networks. The basic idea of echo states has been independently investigated in a complementary fashion. In his work, the emphasis is on biological modeling and continuous-time (spiking) networks.Because it is easy to implement and use ESN has been very popular in academic area. There are many publications including echo state network so far. In general, recurrent neural networks are potentially powerful approximators of dynamics. There are many ways to make this statement more precise. For instance, RNNs can be casted as representations of the vector field of a differential system or they can be trained to embody some desired attractor dynamics.In ESN there is a term for network which is called `reservoir`. Reservoir should be rich and should have a random structure. One simple method to prepare such a rich `reservoir` echo state network is to supply a network which is sparsely and randomly connected. Sparse connectivity provides for a relative decoupling of subnetworks, which encourages the development of individual dynamics.In this work, three different systems, single input – single output system (SISO), Rossler time series and multiple input – single output system (MISO) have been modeled by ANFIS Toolbox in MATLAB software and ESN Toolbox. In final step of this study is a real nonlinear system `Active Suspension Plant` and all results have been compared with each other. In Active Suspension Plant, system consists of two masses, each supported by a spring and damper. The sprung mass represents the mass of the vehicle body while the unsprung mass represents the tire in the quarter – car model. This system is fourth order because there are four independent storage elements, the two masses and the two springs. The spring and the damper support the body weight over the tire. The spring and the damper model the stiffnessof the tire in contact with the road. To derive the dynamical model of this plant free body diagram method has been used and derived the general dynamical equations. The purpose of this study to desing an ESN model by using the input - output data collected from any system. For this purpose, system models are developed with echo state network from the input – output data of four different systems and then the performance of the models are compared with ANFIS, the most popular fuzzy modeling tool.