Wednesday, October 27, 2010

Evolving Astronomy Data Mining Through Simulated Evolution

Interesting stuff this one: The Future of Astronomy is Automated.

It seems genetic algorithms may give another kind of role for traditional astronomers -- and not only for them, but also for other kinds of data miners.

Keep your eyes wide open data miners!


Labels: , , , , , , , ,

Saturday, July 17, 2010

Genetic Algorithms, Proteins, Nuclear Power And More!!!

A nice summary of a new book on natural computing and, of course, genetic algorithms are a part of it: The Lessons Of Living Things.

Well, it is not a novelty, since computers have been helping us to build things and achieve results since the early beginnings of computing. Surely, the natural approach to computation was a nice move and helped to open a new and wide research field.

Labels: , , , , , , ,

Wednesday, May 05, 2010

Rosenbrock Function And The Steepest Descent

It's interesting how the steepest descent method sweats a lot if not correctly set up inside the "right" parameters' interval of the Rosenbrock Function:



It's a well known testbed optimization function and evolutionary algorithms can find its optimum with no problems at all -- surely, as long as you try a traditional parameters' setting that any introductory Evolutionary Computation book gives you.

Using the steepest descent and setting it up as:

x = 2

y = 2

Step Size = 0.001

Stop Criterion = 10-6

I got the following results (see the image below):





Pay attention to the path the steepest descent takes until finding the optimum at x = 1 and y = 1. It is said that an evolution strategy would follow a similar path if its population could be infinite. Since, so far, there are no real computers with infinite memory, such assumption cannot be verified in the real world.

During a second run I set the parameters x = 5 and y = 5. The optimization path run very far away from the previous one! And from the optimum too!

Labels: , , , , , , , , ,

Saturday, September 19, 2009

Evolving Selling Rates Through Simulated Evolution

There's an interesting article in E-Commerce Times on the use of innovations to improve sales. The author states that artificial intelligence techniques -- such as genetic algorithms, expert systems, and fuzzy systems -- could help in such an endevour. An excerpt:

"Every customer touch point will capture more information that the system can then analyze to increase the chances of making decisions on how to sell more to customers. The software cloud will be using concepts like expert systems, genetic algorithms, and fuzzy logic to make recommendations on how the business will sell more, what to sell where, who to sell to -- and it will modify itself to be able to accomplish these optimizations."


I think big vendors, such as Amazon.com, are already using those AI stuffs.

Labels: , , , , , , ,

Thursday, August 27, 2009

Evolving Brachystochrone Through Simulated Evolution

In 1696, the famous mathematician of the Bernoulli family, Jean Bernoulli, challenged his peers posing an interesting physical problem:

"Given two points P and Q in a vertical plane and taking into account the straight line holding them is not vertical neither horizontal, what is the curve connecting them such that a particle starting from the higher point P with no initial velocity (V0 = 0) and sliding down through that line without friction, under the influence of gravity, takes the smallest amount of time to reach the lower point Q?"


A simple and crude schematic of that problem may be seen below:



That problem was presented in the June issue of the famous mathematical journal Acta Eruditorum.

Five other mathematicians replied with a solution and only four of them were published in the next year's May issue of that journal. The whole group was:



The solution curve was named brachistochrone (from the Greek words βραχίστος, brachistos - the shortest; and χρόνος, chronos - time) by Leibniz.



Jean's solution was valid only under certain conditions what made him later propose another and harder version of the same problem, resulting in what became known as calculus of varitations.

This problem itself is an optimization one, in this case optimization of time. So... what about using evolutionary computation to solve the same problem some 320 years later? You may not be of such genius stock as those guys above, but evolutionary computation makes easier to you solve the very same problem! Enjoy it!

Labels: , , , , ,

Monday, December 15, 2008

Even More Mona Lisa Optimization

Nicolas has made interesting simulations on the same grounds of Mona Lisa's -- he even used the same painting. :)

You may find his post here and an YouTube video here.

Labels: , , , , ,

Friday, April 11, 2008

On The Analysis Of Biological Mutations




Browsing MEDAL Blogging, I got a nice link to a post at Olivia Judson's New York Times blog, see it here: A Mutual Affair.

That link was the pointer to another Olivia Judson's post: A Random Analysis. It deals with the biological nature of DNA mutations. Are they really random? Do they follow some kind of probabilistic distribution (or density) function? Are they weighted? Do small mutations happen much more often than the larger ones?

Of course, those questions are mine, not hers. :)

She states that mutations are small modification on the biological blueprint and, depending on the way they happen - small or large -, may affect seriously or not the living being's evolutionary path.

The manner through which mutations occur - be these deleting, inserting, and so on - can also affect the genome's configuration of a specimen. She says that when deletions are more frequent than insertions, the genome's configuration is prone to be compact and small, what may influence the specimen phisiology - those creatures holding small genomes own a heavy metabolic and growing rates.

Mutations are not only random, but also work different from species to species and the mutation flavours (deletion, insertion, etc.) also occur at different rates when taking into account other species. For example, in humans, deletions are more common and prominently working on DNA bases' repetitions, such as ATATAT or AGCAGCAGC. Why do mutations often occur on those repetitions? She explains:


"The reason mutations to repeated sequences are so common is that, in such repeats, it’s easy for the DNA copying machinery of the cell to slip and lose its place, and then put in too many repeats, or too few. (Even for a person, copying something like AAAAAAAA is harder than copying ACTGTCAG. Ahhh!) And although, obviously, these mutations can only happen in part of the genome where there is a repeated sequence, they happen at such a high rate that each of us probably carries as many new slippage mutations as “point mutations” — mutations that swap one base for another, say A for C."


Genomes, as Judson said, hold mutational hotspots and coldspots. These hotspots seem to be long repetitions of DNA bases, since "copying a long repeated segment without slipping is more difficult than copying a short one." Some creatures seem to have evolved their own kinds of mutational hotspots to aim for evolutionary profits - for instance, a pathogen developing the "stealth" ability before the immune system "eyes".

She advises that the manner mutations happen may trap a species in a specific evolutionary path, denying it the exploration of other evolutionary routes:


"But the mutational peppering has a consequence. As I mentioned at the start of this article, an important source of evolutionary novelty is when one member of a pair of duplicated genes evolves to take on a new function. In Neurospora
[bread mold] this can’t happen: duplicated gene pairs get destroyed. Its use of mutations to defend its genome from invasion may have inadvertently blocked off some evolutionary paths."



Very nice article! :)


My Very Own Biased Comments

I think that there are so many lessons the evolutionary computation field may learn from Olivia Judson words. Of course, her small and layman article is just an initial step in that direction.

01. On Genetic Coldspots And Hotspots

For example, there is nothing in genetic algorithms that models the so-called mutational hotspots. On the contrary, take a common genetic algorithm book and the author - likely - advises you configuring the mutation rate between the range [0.0001, 0.001]. Crossover rates are set up at some point between [0.6, 0.9]. It seems that macromutations are avoided when it comes to genetic algorithms. When dealing with problems holding strong dependencies among theirs parameters, such a set up may even be harmful for the whole optimization process, since high crossover rates - along small mutation rates - tend to allow gene pool diversity losses, what can stuck the whole population in a specific local optima and/or optimization track. Sure, the GA community has made works on modelling genetic phenomena, such as gene linkage and viral infection.

Of course, detecting and handling the mutational coldspots is very valuable too, since we could avoid flipping bits (or a set of them) that must undergo small modifications or none at all.

I think that the optimization process of a given real world problem could benefit in some sense from that flavour of genetic phenomenon, since mutational hotspots and coldspots seem, at a first glance, to be useful for escaping local optima - remember: some problems require the modification of ALL parameters at same time to escape a local optimum. BUT... they could be harmful too. To avoid that drawback, mechanisms of self-adaptation could help to overtake that type of situation and could even implicitly identify mutational hotspots and coldspots, handling each one according to their respective nature.

Mutational hotspots and coldspots in non-decomposable problems should be addressed considering the relationship between those bits located at those spots and the others - sure, and between those spots' bits themselves. A well designed mutation operator would be very important here. (Again, a self-adaptation mechanism would be helpful.)

Upon the evolution without mutation, see her another nice article: Stop The Mutants!

Another important point has to do with the detection of those mutational hotspots and coldspots. Although there are some works from the Estimation of Distribution Algorithms (EDA) community handling problems that loosely resemble those genetic phenomena (see the Extended Compact Genetic Algorithm or the Hierarchical Bayesian Optimization Algorithm), they were designed under the statistical philosophy of the EDAs - an approach that throws away the genetic stuffs in genetic algorithms replacing them by statistical sampling and probabilistic distribution functions. Let alone that their bioinspired aspect is completely flawed, since in nature there is not an entity that captures data, analyses them, and etc. I guess those methods were designed much more as an optimization tool rather than intended to be a medium to increase our understanding of biological evolution. As optimization tools, those methods have a nice performance, mainly when it comes to discrete nearly decomposable problems. Summing up this paragraph: Despite a loosely resemblance between some genetic phenomena and the inner working of some EDAs, the later were not bioinspired.

For the EDA enthusiasts, I left a simple question: Is there a probability distribution that behaves in the same way mutational hotspots and coldspots do? :)

My hunch: I guess there is!

02. On Gene Deletion And Duplication

One of the few works I am aware of using abstractions of those two genetic phenomena is Professor Hans-Paul Schwefel's nozzle experiment, see it here: Optimization of a Two-Phase Nozzle with an Evolution Strategy. John Koza has used a kind of deletion and duplication too.

He got impressive results applying those genetic phenomena as an experimental optimization abstraction.






Those nozzle designs were obtained through a (1+1)-ES - without computer!

03. On Long DNA Base Repetitions

I have little to say here, since in "01" I have said a lot about a similar phenomenon. But, the way the genes interact in DNA seem to be much more important than their arrangement itself.

In the evolutionary computation realm, that problem has been addressed by linkage learning techniques in genetic algorithms and through correlated mutations in evolution strategies.

Labels: , , , , , , ,

Tuesday, March 25, 2008

Evolving Crystal Structure Through Simulated Evolution



From my news webservice I got this interesting sample of genetic algorithm application, see here.

An excerpt:


"[Julio] Facelli, director of the University of Utah's Center for High Performance Computing and a biomedical informatics professor, uses NCSA's Mercury cluster to predict crystal structures for organic molecules that are frequently used in pharmaceuticals, fertilizers, and explosives."



"Modeling the crystal structure of a given substance, Facelli and his team begin with nothing more than the atoms in the molecule and the nature of their bonds. They're looking for structures with the lowest energies, which typically mark the molecules' standard crystal structures or something very close. The problem is that this straightforward data and straightforward goal create billions of possible solutions. Just imagine finding the needle of the lowest energy in that haystack of possible structures."



"'An exhaustive search is not feasible, so we have to have a way to direct the search,' says Facelli. '[Genetic algorithms] are based on the principle of survival of the fittest. Trial solutions compete with one another in the population for survival and produce offspring for the next generation of tests. These algorithms offer excellent scaling properties, which make them good for large-scale parallel computing systems like those at NCSA and emerging computational grids like TeraGrid.'"


Here the article outlines through a simple manner the inner working of the genetic algorithm used:


"For example, an initial calculation on NCSA's Mercury may run 20 simulations on 20 different processors simultaneously, calculating possible crystal structures for a given set of atoms and their bonds. The energies for these structures are compared. The 10 with the lowest energies are kept, and the features of those 10 are mixed and matched to generate the structures for another 10 possible structures. Energies are calculated again, comparisons are made, best candidates are kept, and the cycle continues.



The 'mating operation,' as the mixing and matching is called, stagnates quickly, producing very similar structures over the course of thousands of generations. To combat this lack of variety, the genetic algorithm also introduces arbitrary mutations into the process, occasionally taking one variable from one of the best candidates and including a random number for that variable in the next generation."


Stagnation has been, since immemorial times, a hard drawback to overcome, mainly when using an elitist selection method. Taking into account that those individuals holding the lowest levels of energy are the best ones, then it is, in some sense, normal that the genetic algorithm got stuck quickly. Surely, there already are techniques to treat that, such as truncation selection.

If I were the author and considering he is using mutation and crossover, I would set up a high mutation rate (0.85 to 0.95) and a mild crossover rate (0.45-0.6), of course I would put the elitist rate (the number of fittest individuals saved per generation) as a rate of 1/7 of population size. Inversion could help too!

Labels: , , , , , , , ,

Saturday, December 15, 2007

Leonard A. Rastrigin




I found the academic website of Leonard A. Rastrigin, see here.

He is the author of the famous Rastrigin Function, see it here. That function is hard to optimize, since it has so many local optima and the global optimum exhibits an inclination/slope. It also is a traditional benchmark in Evolutionary Computation to measure the optimization features of an Evolutionary Algorithm.

The general form of that function is the following:



The 3D shape of it:



As you can see, bigger the value of the steepness, more complex the fitness landscape becomes.

It's a beautiful function!

You can check Rastrigin's publication here.

Labels: , , , , , , , ,

Charles Darwin Has A Posse Check Google Page Rank