New Book On The Block

Jason Brownlee has published a new book on intelligent algorithms and machine learning: Clever Algorithms Nature-Inspired Programming Recipes.

I read the topic on evolution strategies and genetic algorithms it is a well crafted text for quick reading, rapid implementation, code examples and good references at the end of each topic.

You can download the PDF, read it online or buy a real copy!

Worse Is Better!

David Tow has written an instigating article on the role artificial intelligence (AI) methods are playing and will play concerning its use in e-business.

He outlines a set of major methods he believes will become a hotbed for e-commerce and web intelligence:

1. Evolutionary algorithms
2. Bayesian networks
3. Fuzzy logic
4. Swarm intelligence
5. Neural networks
6. Intelligent agents

Except for #4 and #6, I have seen so many enterprise applications of the remaining methods -- being pretty honest, not only enterprise ones, but a highly diversified spectrum of uses of those methods (#1, #2, #3, and #5).

But what David Tow argues is the increasing use of those methods in e-commerce and web intelligence. I think his arguments are well reasoned, since those kinds of endeavours are recent phenomena -- they arose after the Internet and the World Wide Web --, it is normal AI methods have not been widely applied in them so far.

His future trends prognostics are even more instigating:

"The enterprise of the future will increasingly depend on the wide range of rigorous artificial intelligence algorithms and methods outlined above, to drive its operations at all levels of management. 

Decision Engineering techniques are at the forefront of this revolution- while IBM has recently set up a new services unit focussed on applying predictive modelling to automate business decisions.  

These techniques will continue to be enhanced and packaged in different combinations, to provide immensely powerful problem solving capability as well as integrating with the global intelligence of the Web 4.0.  

Major decisions incorporating sophisticated levels of intelligent problem-solving will increasingly be applied autonomously and within real time constraints, in order to achieve the level of adaptability required to survive in an ever-changing and uncertain global environment.

I do not think traditional methods are sentenced to die out, just take a look at the operations research guys that have bred a shining research city on the hill since the early stages of that field around the 1930s and I do not consider they will suddenly vanish from the surface of this planet.

The author did not point an important issue: Better not always means better. The better definition here is not intented to slur other methods, but it embeds what an enthusiast of some research area thinks about his own field/research: Not so rarely, we can see researchers colourfully speaking when it comes to their own research. I consider that a sensible attitude, but hyper-hyped words do not help to solve problems in the real world and the real world, as an old saying states, is a cruel place that may tear into shreds any hype. The chills of the AI Winter can still be felt.

Other important point has to do with the user base a given method owns. Someone may have designed the best time complexity algorithm, the best neural network, the best fuzzy set tunning method, and so on, but if they do not have a significant user base, all those methods are useless -- they may serve as an interesting academic investigation endeavour, but not for application in the real world, since no one would be using them. The ultimate example that illustrates this is the so called worse is better approach:

[...] [S]omething can be "inferior" but still "better". For example, to a particular market or user, software that is limited but exceptionally simple to use may be "better" than software that is more comprehensive but harder to use.

I hope I am not slurring the work of anyone else out there, but the Elitist Simple Genetic Algorithm is, in my humble and insignificant opinion, a perfect example of worse is better in evolutionary computation.

No, I do not consider the elitist SGA a bad method. On the contrary: It is an amazing method devised by Professor John Holland and it has succeeded in so many fields -- thanks to the embedded knowledge its users have included in it and its simplicity. BUT, the elitist SGA, despite its drawbacks, is the most applied genetic algorithm (and evolutionary algorithm) until now, even though there are some new genetic-based techniques that are direct descendants of the SGA, such as EDAs (Estimation of Distribution Algorithms), that were made to overcome some drawbacks inherent to the SGA. Fifteen years after the publication of Baluja's seminal EDA paper, the SGA still is strong, well alive, and bigger than all of its offsprings. Worse is better! :)

Even the SGA siblings (evolution strategies, evolutionary programming and genetic programming) have not got the wide range SGA has. The SGA dominance and its offsprings struggle to obtain a bigger amount in the application market is, in my opinion, an example of worse is better. Would the SGA be an evolutionarily stable strategy? :)

So, I do not believe future problem solving will be strongly tied to artificial intelligence methods. At least not in a fatalistic manner as some believe. Those methods will play a role -- major or minor -- in different problem solving situations, but not to the point of relying 100% on them.

Evolving Two-Wheeled Small Vehicles Through Simulated Evolution

Very interesting pointer this one, see it here.

It is a flash animation that shows you the evolution of two-wheeled small vehicles and their ability to move around a track. Some designs are very strange and deliver no motion at all. Others are extremely similar to real world vechicles, such as motocyles and automobiles, being able, untill a certain extent, to move.

So many thanks to GreyThumb.com guys for the link!

Automatic Tuning Of Car Controllers - UPDATED

Our blog friend and official Viking, Julian Togelius, is asking all the machine learning/artificial intelligence researchers, practioners, enthusiasts, etc. to submit a car controller tuned using your favorite learning algorithm. Further informations may be found here and here. The controllers will be subjected to a competition against each other during a set of races.

If were you, I would submit it immediately! If you don't, then, flee from your hometown or... face the Viking raid!! :)

The first place will be awarded a weekend staying at Valhala, being guided by Valkyries. :)

-------------------------------------------
I was told, in the comments, by Julian Togelius that there is a video from the last racing car competition they held this year at the last WCCI. See it here.

It is one of the bloodiest races I have ever seen in my whole life!

The blue car is very skilled.

Machine learning researchers of the world, unite! And send your controller to the competition!
-------------------------------------------

Lawrence J. Fogel Highlighted By The American Society For Cybernetics

Lawrence J. Fogel has been highlighted by the American Society For Cybernetics. See it here. For a complete list of cyberneticians, see here.

Lawrence Fogel is the father of the so called evolutionary programming, which his son, David B. Fogel, later improved and extended to work on real valued domains.

Via Natural Selection, Inc.

Swarms

Very good point from the masterminds behind the Grey Thumb blog. It's a link to Dr. Deborah Gordon's lecture on how ants know to do what they do, see here.

Understanding the inner working of ants' behaviour - and others swarms too - may be important to further the current state of those bio-inspired methods relying on that.

I read her book Ants at Work: How an Insect Society is Organized. She even applied neural networks to model the way that ants interact.

Maybe, the main lesson we may learn from those little insects is directly linked to the fact that ants (and other kinds of swarms) do what they do without a central control and the nest works fine. Sure, when it comes to the evolutionary computation realm, it is always sensible to remember that ants' nests own a very big population and in optimization (and machine learning in general) the population size may be a drawback if set up so big in some problems. Let alone that keeping a small population is very useful when dealing with expensive fitness/objective function evaluations.

Evolving Electrical Circuits Through Simulated Evolution

Another application of genetic algorithm to optimize circuit components' values, see here.

The resistor's and NTC thermistor's values must be set up such that the resulting curve (from the circuit equation) matches the curve on the chart below:



At a first glance, it seems a very simple problem, but, as the article observes, it may get hard to solve as the circuit complexity increases.

It remembers me of a tale of a senior student in a class of electrical circuits. The professor put some hard circuit problems to be solved through variable substitutions and so on. There were so many derivatives to be solved and etc. The senior student used a tricky way to solve them:

01. He organized all the equations.

02. He used the Laplace transform to all of them.

03. He put all the parameters on a matrix.

04. He solved all the equations through the Gauss-Jordan method.

I was not there, but some "persons" state that the professor refused his solution!!!

I wonder if that professor is the same guy which once said that 1/2 Ohm is not the same as 0.5 Ohm!!!!

Evolving Grid Computing Optimization Through Simulated Evolution (Updated)

Interesting stuff I got via my webnews service: Evolving towards the future of science: genetic algorithms and grid computing.

The main objective has to do with the optimization of job scheduling on a computing grid. Through this, lots of valuable resources are saved, such as energy consumption, CPU time, human resources, and so on.

It seems that the genetic algorithm (GA) applied is the good old CGA or Canon Genetic Algorithm (likely holding some slight modifications, for instance elitism, stochastisc tournament selection, etc.), what stands for bit flip mutation, one point crossover and a fitness based selection scheme. Of course, I may be wrong about that, since I have not read nothing (except that news) upon the genetic algorithm they use for that task.

From the article:

"Standing for Distributed Optimal GENEtic algorithm for Grid applications Scheduling, DIOGENES quickly determines the most efficient way to schedule of a set of jobs on a computing grid, optimizing both time and resources in the process."

I have noticed an interesting (strange?) phenomenon which has taken place in the, let's say, New Wave Of Genetic Algorithms (NWOGA): It seems as though no one is applying them! From my surveys inside the Internet and looking for nice real world applications of evolutionary algorithms, what I have noticed is that when it comes to genetic algorithms, the most applied one still is the... Canon Genetic Algorithm! Surely, as stated above, that CGA which is being applied to solve a particular problem, very frequently, holds modifications, be these on the selection scheme, the evolutionary operators (mutation, crossover, and, from time to time, inversion), population size, or even on the probability tunning of mutation and crossover. The fiddling CGA game has generated, sometimes, strange algorithms.

By the way, I have nothing against NWOGA.

Our blog friend, Julian Togelius, wrote an interesting observation upon that situation. I consider that he aimed at the right target, see:

"I think the reason so few people are using other types of evolutionary algorithms than standard GAs is that few people know of, or understand, anything else. It's really amazing how little people know of what's going on next doors to their own little research field.

[...]

So, the people who are busy coming up with new algorithms don't have much time for learning about applications, and vice versa..."

Absolutely! :)

The big academic/technical organizations/publishers (ACM, IEEE, Springer, etc.) could sponsor more crossover events in which different research areas could come a little closer to share their respective experiences, new ideas and so on trying to find common points so that one area could help the other. Perhaps, that attitude could bring more open air to both fields.

P.S: To better understand why I wrote NWOGA, please, see here. :)

Update: Also via MEDAL Blogging.

Feng Gui

Nice visualization tool for a simulated gradient attention map on a website. See here. Its name is Feng Gui and creates through artificial intelligence techniques a gradient map for the human attention on a picture and/or webpage.

Thanks to François from Intelligent Machines and to MEDAL Blogging for the pointers! :)

The picture above is the Genetic Argonaut's gradient map!

P.S: Well, if the current picture was that one from Lenna, maybe the gradient map could be very different! :)

2007 IEEE Congress On Evolutionary Computation (CEC)

Our blog friend, Julian Togelius, has written his own account upon the 2007 IEEE Congress On Evolutionary Computation (CEC), see here.

You can also check some pictures of the congress here.

No complaints on the whole event. It's nice, since the attendee must pay a fee for some basic services - such as, I suppose, internet facilities, lunch place, etc -, what means that the fee was worthwhile. There was even somekind of "safari" through the local zoo!

It seems that the event was O.K.!

ECAL 2007

Juan Julián Merelo Guervós wrote an interesting post on the ECAL 2007, see here.

There is a blog upon the event, see here.

Don't forget to read the fairytale!

1984

No, this it not an allusion to the famous Orwellian work, but, let's say, the first time that someone posted on Usenet (now Google Groups) something that deals with genetic algorithms, see it here.

It is an announcement about the 1984 AAAI Conference, held in Texas, USA. Guess what? Who did speech on genetic algorithms? Let me give you a tip: JH... That's it! John Holland!

There were, I suppose, just two genetic algorithm presentations:

Michael L. Mauldin, Carnegie-Mellon University: Maintaining Diversity in Genetic Search

ABSTRACT
Genetic adaptive algorithms provide an efficient way to search large function spaces, and are increasingly being used in learning systems. One problem plaguing genetic learning algorithms is premature convergence, or convergence of the pool of active structures to a sub-optimal point in the space being searched. An improvement to the standard genetic adaptive algorithm is presented which guarantees diversity of the gene pool throughout the search. Maintaining genetic diversity is shown to improve off-line (or best) performance of these algorithms at the expense of poorer on-line (or average) performance, and to retard or prevent premature convergence..

And a machine learning tutorial session in which John Holland was involved in, see below:

9:00 a.m. to 10:30 a.m.

TUTORIAL AND PANEL: PARADIGMS FOR MACHINE LEARNING
Concert Hall in the Performing Arts

The session will be divided into two parts:

Center

Part 1. Tutorial.
A single presentation defining and outlining each
major approachto Machine Learning, and contrasting them with each other on the basis of objectives, techniques, limitations, and applications.

The role of the tutorial is to:

- Introduce each paradigm and the contrastive dimensions listed above.
- Present some meaningful comparative analysis.
- Raise potentially controversial issues to be addressed in the ensuing panel discussion.

Tutorial presenter: Jaime Carbonell

Part 2. Panel discussion.

Each Machine Learning paradigm will be represented by a panelist advocating that particular approach. The panelists are active researchers with considerable experience in ML in general and their approach in particular.

Discussion Leader: Patrick Winston

Panelists:

Tom Mitchell, Rutgers University (Analytical Generalization)
Ryzsard Michalski, University of Illinois (Empirical Induction)
John Holland (Genetic Algorithms)
Doug Lenat, Stanford University (Discovery Systems)
Jaime Carbonell, Carnegie-Mellon University (Learning by Analogy)

The panel discussion will center on addressing specific issues raised in the preceding tutorial (the panelists will be informed ahead of time of these issues). We are explicitly disallowing prepared statements by the panel -- we hope to have a real discussion focused around a few burning issues.

The picture above is Usenet circa 1985/1986.