After that, as you so eloquently put in your post, it is a head-scratcher. Personally, I would do a formula mixture of “scroll-to-the-right,” score, and time. I think these all have to be taken into account, with their weights depending on your personal preference on “optimal” gameplay. I, myself, would be keenly interested in seeing the fastest possible Mario completion possible w/o a concern over warps and the such, and would thus give scroll-to-the-right and score high emphasis.

Also, I remember at least one game (one of the Donkey Kong games, I believe) that allowed you to hit select then start to exit a level…doing so near the bottom of a chasm you carelessly jumped into proved a very beneficial strategy.

Similarly, there shouldn’t be a need to weight certain buttons more heavily when randomly generating candidate solutions. After all, solutions with a lot of “bad” moves should get culled from the population pretty quickly. Though I suppose there would be a lot to be said to throw Select and Start out of consideration entirely, since pausing would never be helpful and Select does nothing in-game anyway.

One interesting metric I didn’t think of originally is penalizing every time the candidate solution presses or releases a button. Factoring that in should favor “smoother” or more “elegant” solutions — it’s better to hold Right for a while than to repeatedly press and release it.

On the other hand, in my (admittedly limited) understanding of using genetic algorithms, it’s best for the fitness function to just consider how close a candidate solution is to some “ideal” solution, rather than trying to coax it into strategies you think are beneficial. After all, part of what makes this approach interesting is its ability to “invent” novel approaches, and perhaps some of the things we might want to rule out could turn out to be helpful in some cases.

In any event, actually being able to play around with an implementation of this stuff would hopefully settle a lot of these questions. I just have to get my hands on the source code for a good NES emulator….

Perhaps you could build a simple virtual mock up of level 1-1 and incorporate your own metrics that can be measured definitively- at least this would save you the effort of having to try to measure values obtained from the actual game.

Of course, you could always get a *real* job and not waste your time with such problems.

I would say priorities should go:
a) not dying (since the game can be beaten without dying, death is sub-optimal performance)
b) level advancement (measured by what world you’re on and screen scroll rate)
c) time (possibly tie this in with level advancement, as in screen scroll RATE)
c) score
d) extra lives (possibly work into a function of score, since the idea is you never need to use them)

Also, the criteria themselves could be graded by how fast they produce the optomized code for completing a stage. Criteria would be based on how many iterations are required to produce a winning combination. Furthermore, the order of random trials could be tweaked. Say for instance that 01000000 is moving right, and 11000000 is jumping right. When running iterations, if it starts from 00000000 and moves up, most of the initial trials won’t involve much jumping. It’s quite likely (and from observing the speed trials of people who perform nearly perfectly) that the majority of an optimal iteration of winning moves is spent jumping. Therefore, in the controller input, instead of initially trying 00000000 and counting up, it would be faster to start at 11111111 and count down. This may be more applicable to some leves that are more conducive to almost continuous jumping, but it still has the potential of seriously impacting calculation time. Another excellent (and simpler example, I wish I had thought of this first), would be running right, as opposed to running left (something that is rarely, if ever, needed in Mario Brothers), and thus it would be preffered if the program always attempted running right before trying a solution that involves running left.