Controlling maximum evaluation duration in on-line and on-board evolutionary robotics

On-line evolution of robot controllers allows robots to adapt while they perform their proper tasks. In our investigations, robots contain their own self-sufficient evolutionary algorithm (known as the encapsulated approach) where individual solutions are evaluated by means of a time sharing scheme: an individual controller is given the run of the robot for some amount of time and fitness corresponds to the robot’s task performance during that period. In this paper, we propose and provide a detailed analysis of two on-the-fly control schemes to set the evaluation time in highly dynamic scenarios with completely different tasks. One scheme, called the roulette-wheel selection scheme, stochastically selects evaluation time from promising intervals similar to multi-armed bandit schemes. The other scheme, named Heuristic-Rule (H-Rule), tweaks the evaluation time using specific heuristics. Our experiments show that H-Rule gives stable performance in different scenarios and can serve as a viable alternative to pre-selected optimal evaluation time.