Editorial to tenth anniversary issue on progress in genetic programming and evolvable machines
The Genetic Programming and Evolvable Machines journal is 10 years old. This sizable special issue is intended to mark this special anniversary and take the opportunity to look back at what has been achieved in the last 10 years of Genetic Programming (GP) and to start charting the terrain for potential new developments in the next 10 years.
When putting together this special issue, we were particularly interested in high quality papers that reviewed or analysed progress in the field, presented the state-of-the-art in the evolution of software and hardware, described promising new approaches and paradigms, suggested important new application areas, or provided an in-depth coverage of foundational topics and theoretical analyses in genetic programming and evolvable machines.
Three articles were invited to act as reference points for the special issue: one on theoretical results in GP, a review of human-competitive applications of GP and one aimed at providing a snapshot of the published literature in GP. We then encouraged submissions that would complement these seeds thereby providing a full coverage of the topics we aimed for in this special issue. Twelve further papers were obtained through the open submission process. We would like to thank their authors for their willingness to contribute to this special issue.
Some of the papers submitted, while very good contributions to the field, did not have the necessary breadth for inclusion in this special issue. These were thus excluded, but resubmission as ordinary papers was encouraged and it is likely that some of these papers will appear in this journal at some point in the future. All remaining papers underwent one or more rounds of peer review (including the invited papers, thereby ensuring that all authors were pushed to produce the best possible product).
Understanding the importance of this special issue, we had an extremely high responsiveness in the reviewers, with only a couple of reviewers declining due to conflicts on interest that had gone inadvertently undetected. We would like to take this opportunity to thank all reviewers for their willingness, care and dedication in helping us produce this high-quality special issue.
The final result of this complex selection process led to the acceptance of 3 out of the original 12 open submissions. You will therefore find 6 articles in this special issue. The articles are briefly described in the next section.
2 The papers in this special issue
It is doubly fitting that this tenth anniversary issue begins with an invited article by John Koza. Firstly, John is a pioneer of GP and one of its most influential figures. Secondly, John and his collaborators, Martin Keane, Jessen Yu, Forrest Bennett III and William Mydlowec, published a paper in the first volume of this journal on the “Automatic Creation of Human-Competitive Programs and Controllers by Means of Genetic Programming”. In this issue he describes the history and current status of human competitive results produced by GP. In 1994, 10 years before the establishment of the annual human-competitive results competition at the 2004 Genetic and Evolutionary Computation Conference (GECCO 2004), he obtained the first human competitive result on a problem in bioinformatics. He goes on to describe tens of human competitive results that have been found using GP since then. The diversity of these results is very great and it is clear that GP is genuinely producing advances in many areas of science, mathematics and engineering. Researchers are not only obtaining novel solutions to formerly issued patents but are frequently surpassing them. Over the years, John has consistently argued that obtaining human competitive results is correlated with Moore’s law and he concludes with a prediction that the flow of human competitive results produced by GP will increase markedly in the future.
The second invited article is on advances in the theoretical foundations of genetic programming. The authors of the article include some of the major contributors to the theory of genetic programming. The decade since the first issue of Genetic and Evolvable Machines has witnessed considerable strides forward in the theoretical understanding of GP. Indeed, much of this progress has been reported in this journal. However, the authors describe how, despite this, undoubted progress, precise mathematical results in GP are still relatively scarce so there is still much to be done. One of the key difficulties in making progress in this area is that GP now contains a diversity of different representations of programs (e.g., linear, tree, graph, stack and grammatical) each potentially requiring different mathematical techniques. The article here begins with a comprehensive review of the state-of-the-art of GP theory; this will be invaluable for researchers who wish to understand progress so far. It moves on to discuss fundamental open questions and promising directions for future work. This paper and its substantial bibliography will form an excellent starting point for existing practitioners and new theoreticians to refine and construct GP theory for many years to come.
In almost every issue of this journal there has been a review. In the third invited article, Bill Langdon and Steve Gustafson give a “master review” and analysis of published literature. It covers books, conference proceedings, software packages, web pages, on-line resources, and a variety of other products. The article includes an analysis of publication outputs and authors in GP and evolvable hardware (EH) using the extremely useful GP bibliography as a resource, which itself now lists over 5,200 articles. The article charts how the field of GP+EH has grown and continues to grow. The field owes a substantial debt of gratitude to the maintainers of the GP bibliography, Bill Langdon, Steve Gustafson and John Koza, but particularly to Bill for his untiring efforts sustained over a long period.
The first article of the open submissions on “Open Issues in Genetic Programming” is highly apposite for this anniversary issue. The authors are Michael O’Neill, Leonardo Vanneschi, Steven Gustafson and Wolfgang Banzhaf. The article had its genesis in a number of panel discussions at the European Conference on Genetic Programming. It begins with a brief history of the field and then moves on to discuss ten main categories of open issues in GP that require more study. This paper will be extremely useful to the field in focusing attention on many of the important questions that require much more investigation. It is an inviting list! It should be particularly useful to students and researchers who are entering the field and are looking for topics for research where they could make significant impact. The authors convey their optimism about the future of GP, arguing that it is receiving increasing visibility and adoption in industry. Also the availability of parallel processing and generally increasing computer power will be likely to lead to even more impact in the future. It will be most interesting to review progress in another 10 years!
The second article of the open submissions is a survey on a hot topic in GP, namely, grammar-based approaches. The authors are Bob McKay, Nguyen Hoai, Peter Whigham, Yin Shan, and Mike O’Neill. Grammars have played an important role in GP almost from the beginning. However, their role has expanded to the point where one form, Grammatical Evolution, is now one of the most widely applied GP methods. The article begins with a history of “Grammar Guided” GP (GGGP) and then analyzes GGGP and contrasts it with other GP approaches, finally identifying important issues for the future. This sub-field of GP is still rapidly developing and the authors have necessarily limited their survey until the end of 2008. Despite this the paper has a substantial list of references which will be an excellent starting point for future research in grammatical GP methods.
The final article is a paper on “Developments in Cartesian Genetic Programming” by Simon Harding, Wolfgang Banzhaf and Julian Miller. Cartesian Genetic Programming (CGP) is a graph-based form of GP that, it could be argued, has its “official” birthday in 2000, the year the GPEM journal was first published. In that year a paper entitled “Cartesian Genetic Programming” was presented by Julian Miller at European Conference on Genetic Programming in Edinburgh. In addition, an article applying CGP to digital circuit design was published in the first volume of GPEM. CGP has undergone a number of enhancements since then. The article presented here discusses a form of CGP that includes graph re-writing through the use of self-modification instructions. It is called “Self-modifying Cartesian Genetic Programming” (SMCGP). The new technique is shown to find completely general solutions to some problems (i.e., even-parity, n-bit binary addition and computing π) and also is shown to improve on the efficiency of CGP on a wide variety of problems. Self-modification, though included in Lee Spector’s “PushGP”, has received little attention in the field, so the inclusion of this paper in this anniversary volume may help to encourage other researchers to consider including self-modification in their GP systems.
3 The overall picture drawn by these articles
In the first issue of Genetic Programming and Evolvable Machines there were five research articles covering the evolutionary design of digital circuits, solving high-order parity problems, molecular evolution, crossover techniques in tree-based GP, and human competitive controllers evolved with GP. There was the first review on GPEM books and other resources and a review of an evolvable hardware conference. It is interesting that many of the authors of these articles are not only still in the field of GP but indeed are also authors for this anniversary issue. How the field has moved on since then!
Overall, what have we learned from this issue? We know that we can expect many more human competitive results in the future. This is an enticing prospect. Great strides have been made to move forward the theory of GP but we know that there still remain many fundamental theoretical challenges. We hope very much that the article in this issue will encourage others so that much more progress can be reported in 10 years time. How much larger will the GP bibliography be in another 10 years? Bill Langdon mentions that at first it grew exponentially with time, and now it still grows steeply but in a linear way. Will this be maintained or will some new development in GP attract even greater attention leading once again to exponential growth? It is clear from the article of Langdon and Gustafson that GP will be taught at more universities so this almost certainly will lead to greater interest in industry and academia. A major factor in the future could be the uptake of GP in developing countries. Many fundamental issues facing GP have been addressed in this issue and substantial progress on these almost certainly will be made. Which issues will prove the most difficult? Will entirely new issues arise? The use of grammars is widespread in many areas of computer science. Perhaps GP based grammatical approaches will be taken up in other areas, hitherto unknown? Self-modification may be routinely used in GP systems in 10 years. What problems will this have allowed us to solve? The future of GP looks bright with possibilities and we hope that this anniversary issue will have played a significant part in this.
We would like to thank the editor-in-chief, Lee Spector, for suggesting and supporting this special issue.