Evolution Strategies + Neural Networks = EVASION

Hallo!!

I found some weeks ago an article about Misfire Detection. It is from Ford Motor Company and can be read here: Aston Martin DB9 is High-Tech Under the Hood.

The Aston Martin DB09:



It is an interesting application of a Neural Network (NN) to help to solve a real world problem. The 2005 Aston Martin DB09 is the first Ford car with a Neural Network embedded to identify misfires. As some of you know, Neural Networks are tools that help to identify/detect patterns inside a set of data, these data can be random, noisy, etc, and some important patterns can occur so many times inside the data. But when a misfire occurs, also happens some disruptions of those patterns and the further pattern recognition can be difficult (or even incorrect), because "the misfires themselves may be isolated events, or they may form a pattern. That pattern is the signal we're looking for in all the noise and on a V-12, the frequency of firing events is so high that the legislative requirements for misfire detection could not be met with conventional computing resources. Neural networks offered us a whole new paradigm for computing and the potential for a misfire detection system that would be fully capable of meeting every detail of the regulations, something that the whole industry struggles with on any engine with eight or more cylinders", as explains Craig Stephens, manager of Research & Advanced Powertrain Controls at Ford Motor Co.

They could improve the Neural Network performance to identify patterns using Evolutionary Computation. An interesting method to to that is the EVASION Method. The EVASION Method uses an Evolution Strategy to optimize the Neural Network's structure (number of inputs, number of weights, number of neurons in the hidden layer, number of outputs, number of feedbacks and etc). "EVASION means EVAcuation out of the dimenSION. Valleys have to be formed at the edges of the optimization space (the zero weight axes), so that the gradient path is leading from the hyper-space to the adjacent hyper-subspace. Following the gradient-path evolution descends into the smallest possible subspace. Evolution-strategic learning will eliminate the superfluous weights.".

Below there is an Evolutionary Neural Network.

Generation 0 (Zero): At this generation the Neural Network is oversized and its performance is not so good.



Generation 2000: This one is better than the above, but we can improve more its performance.



Final Compressed Neural Network: The best model found through the EVASION Method, its performance is much better than the other two above. I would be so much arrogant if I said that this final version is The Optimum structure for the Neural Network. Surely it is not, because that Neural Network could be very good to solve problems from a certain set, but it would be very bad to solve problems from other categories/types.



In the figures the thickness of the connections represents the weight strength of the Neural Network.

So, James Bond's Aston Martin would be much more intellingent and he could spend more time with girls and at ultra-exclusive parties. I am joking. :)

Just to remember the persons who visit my Blog: There are other methods and/or Evolutionary Algorithms (such as Genetic Algorithms, Evolutionary Programming and Genetic Programming) to evolve a Neural Network. Yes, Evolutionary Computation is much more than (only) Genetic Algorithm.

Até Mais!!

Marcelo

References:

http://www.bionik.tu-berlin.de/institut/xs2neuro
http://media.ford.com/newsroom/feature_display.cfm?release=18677
Ingo Rechenberg: Evolutionsstrategie '94. Stuttgart: Frommann-Holzboog 1994.

Auf Wiedersehen, Evolutionsstratege!!!

Hallo!!



Tomorrow (February 11th 2006) will be a sad day to the Evolutionary Computation field, because our revered colleague and friend Professor Hans-Paul Schwefel (Der Evolutionsstratege) is retiring from the academic area (Professor Schwefel is in the center of the picture above - [see note 3]). There are not enough words to express all his contributions to the Evolutionary Computation, since his first steps doing calculations on a mechanic device to demonstrate the working principle of the proto-Evolution-Strategy, still in the 1960's, until now with a very rigorous and robust analysis of Evolutionary Algorithms (meanly Evolution Strategies), offering us well fashioned, very analitic and elegant investigations of that kind of algorithms (such as time complexity, convergence properties and other complexity-based studies [see note 1]) without the necessity to appeal to very weird/strange/creep/non-algorithmic concepts, approaches and models. If there was an approach he applied, it was the Engineering and the Science spirits, because Engineering is Art and Ars, sine Scientia, nihil est!! (see note 2)

The University of Dortmund will hold a Festkolloquium to celebrate this occasion. The program is below and will have the presence of important persons inside Evolutionary Computation. (Click here to see the original program at University of Dortmund)

Program

Greeting

Prof. Dr. Bernhard Steffen
Dekan des Fachbereichs Informatik

Prof. Dr. Eberhard Becker
Rektor der Universität Dortmund


Lectures

HPS: Humble, Pioneer, Scholar
Prof. Dr. Kenneth A. de Jong
George Mason University, Fairfax VA, USA

Hans-Paul Schwefel, Predator-Prey, and Multi-Objective Optimization
Prof. Dr. Kalyanmoy Deb
Indian Institute of Technology, Kanpur, India

Chairs, Evolution, and Hans-Paul Schwefel
Prof. Dr. Marc Schoenauer
Université Paris Sud, France

- Snack -

Der SFB 531 und die Rolle der Theorie
Prof. Dr. Ingo Wegener

Erste Eindrücke am Lehrstuhl 11
Prof. Dr. Petra Mutzel

Zur frühgeschichtlichen Theorie der Evolutionsstrategie: Eine Zeitreise
Prof. Dr. Günter Rudolph

Evolution eines Lehrstuhls: Intelligenter Entwurf oder selbstorganisiertes Chaos?
Mike Preuß und Boris Naujoks

Schlussworte
Prof. Dr.-Ing. Hans-Paul Schwefel

And, now, the final words (in slide presentation) of Professor Schwefel at the Festkolloquium:

"Could my career have been planned? I don?t
think so! It is the result of many improbable events
and bifurcations. Richard Dawkins used the nice
metaphor climbing mount improbable for the course of
life on Earth (I do not agree with all of his theses).

My advice for your career: To achieve all that is
possible, you must attempt the impossible !

I am content with my life and hope that you have not
suffered from me too much.

Many people have contributed, in different ways, to my
zigzag career.

More names should be listed here than there are seats
in this room!

Let me go with a shortlist only:

My wife Antje, also honorary member of the group,
responsible, e.g., for the biennial report Blaues Heft
(take one on leaving together with a decision aid from
this jar)

My current secretary Gundel Jankord, former secretary
Heike Bracklo, and the longterm technical
administrator Uli Hermes

All other members of the Chair, current and emeriti
and you, who have come from all over the world (alas,
nobody from other planets :) ) to meet me again at
this somehow final event.

Grand merci mes amis ? hartelijk bedankt beste
vrienden ? thank you very much dear friends
? vielen herzlichen Dank!"

Thank you, Professor Hans-Paul Schwefel, for all your contributions and pioneer work inside Evolutionary Computation and, also, Engineering!!

Your contributions to the Evolutionary Computation field are undeletable!!


Auf Wiedersehen, Evolutionsstratege!!!

:)


Marcelo

P.S: Soon I will post again the "Evolutionary Computation Classics - Vol. I" here. I am doing some corrections and writing new texts to improve it.


Notes:

1 - You might ask me: "Why would someone need all those Complexity-based investigations to use an algorithm ??" Well, my friend, all those Complexity-based properties/features are things ihnerent to algorithms and it is really complicated to disconect algorithms from their characteristics, because, without these, algorithms are not an Engineering/Science object, but, (maybe) only, flies around a lamp in the dark.

2 - Art, without Science, is nothing!! - Jean Vignot

3 - I would like to thank Mrs Antje Schwefel for have sent me that picture above and, also, to have answered some questions about chinese and japanese language. :D