## SUMMARY

For early vertebrates, a long-standing hypothesis is that vertebrae evolved
as a locomotor adaptation, stiffening the body axis and enhancing swimming
performance. While supported by biomechanical data, this hypothesis has not
been tested using an evolutionary approach. We did so by extending biomimetic
evolutionary analysis (BEA), which builds physical simulations of extinct
systems, to include use of autonomous robots as proxies of early vertebrates
competing in a forage navigation task. Modeled after free-swimming larvae of
sea squirts (Chordata, Urochordata), three robotic tadpoles (`Tadros'), each
with a propulsive tail bearing a biomimetic notochord of variable spring
stiffness, *k* (N m^{-1}), searched for, oriented to, and
orbited in two dimensions around a light source. Within each of ten
generations, we selected for increased swimming speed, *U* (m
s^{-1}) and decreased time to the light source, *t* (s),
average distance from the source, *R* (m) and wobble maneuvering,
*W* (rad s^{-2}). In software simulation, we coded two
quantitative trait loci (QTL) that determine *k*: bending modulus,
*E* (Nm^{-2}) and length, *L* (m). Both QTL were mutated
during replication, independently assorted during meiosis and, as haploid
gametes, entered into the gene pool in proportion to parental fitness. After
random mating created three new diploid genotypes, we fabricated three new
offspring tails. In the presence of both selection and chance events
(mutation, genetic drift), the phenotypic means of this small population
evolved. The classic hypothesis was supported in that *k* was
positively correlated (*r*^{2}=0.40) with navigational prowess,
*NP*, the dimensionless ratio of *U* to the product of
*R*, *t* and *W*. However, the plausible adaptive
scenario, even in this simplified system, is more complex, since the remaining
variance in *NP* was correlated with the residuals of *R* and
*U* taken with respect to *k*, suggesting that changes in
*k* alone are insufficient to explain the evolution of *NP*.

## Introduction

To test hypotheses about biomechanical adaptations, selection experiments on living populations offer the opportunity to analyze connections between genotype and phenotype (Garland, Jr, 2003), construct models that relate morphology, performance, and fitness (Arnold, 2003), and understand the separate impact of selection, chance and history (Travisano et al., 1995). Without living taxa, however, it is at best difficult to determine plausible mechanical functions from morphology alone (Alexander, 2003; Hutchinson and Gatesy, 2006; Lauder, 1995), let alone the adaptive value of a trait (Amundson and Lauder, 1994). Seeking a bridge between the morphology and fitness of extinct taxa, biologists have combined biomimetics and phylogenetics (Bels et al., 2003; Kingsolver and Koehl, 1985), building biologically inspired physical or computer simulations to determine plausible performance changes associated with changes in morphology (Long, Jr et al., 2006a) or behavior (Long, Jr et al., 2006b). Inference from performance to fitness requires that the investigator also reconstruct a selection environment, an ecological situation in which to judge adaptive value (Brandon, 1990). Thus the goal is to understand how individuals and their environments might have plausibly interacted to produce the derived state from the ancestral. We call this approach biomimetic evolutionary analysis (BEA).

Inspired by evolutionary robotics
(Nolfi and Floreano, 2000), we
extend traditional BEA by adding fundamental evolutionary features: (1)
genetics: quantitative trait loci (QTL) coding biomimetic morphologies, (2)
mutation and genetic drift: random processes creating variable offspring, (3)
autonomous individuals: mobile robots competing against each other, (4)
selection environment: a behavior-based environment and criteria judging the
relative fitness of individuals, and (5) evolution *per se*: the
population changing over generational time by selection, drift and mutation
(Gillespie, 2004). Central to
the explanatory power of this evolutionary approach are the random processes
of drift and mutation. Without randomness, decisions about how to vary the
phenotypes under selection are made by the investigator, leaving open the
opportunity for unintentional bias to canalize evolutionary trajectories.

The morphology we analyze is the vertebrate axial skeleton, which has
undergone at least four episodes of convergent evolution from the ancestral
state of a continuous notochord to the derived state of a jointed vertebral
column (Gardiner, 1983;
Koob and Long, Jr, 2000). The
classical interpretation (Goodrich,
1930) is that bony or calcified vertebral elements evolved to
strengthen and stiffen the axis as an adaptation for more powerful swimming
(Laerm, 1976;
Laerm, 1979;
Rockwell et al., 1938). The
presence of vertebral centra, the elements that surround and constrict the
notochord, is correlated with increased fast-start performance in
osteichthyans (Lauder, 1980)
and increased undulatory swimming speed in actinopterygians
(Webb, 1982;
Weihs and Webb, 1983).
Conversely, species that possess an adult notochord, e.g. Atlantic hagfish
*Myxine glutinosa*, white sturgeon *Acipenser transmontanus*,
and shovelnose sturgeon *Scaphirhynhcus platorynchus,* swim voluntarily
at very slow speeds, less than one body-length per second in the laboratory
(Long, Jr et al., 2002;
Long, Jr, 1995;
Adams et al., 1997).
Correlations, however, hide the complex relationship between morphology and
performance: sturgeon, even though they possess a notochord, can swim fast
enough to breach (Sulak et al.,
2002), and no single morphology alone determines burst swimming
performance (Brainerd and Patek,
1998; Ghalambor et al.,
2003).

To identify influential vertebral morphologies and relate them to a
specific mechanical function, we focus on mechanical properties. For example,
the addition of rigid artificial vertebrae to a hagfish's notochord increases
its bending modulus, *E* (in Pa)
(Long, Jr et al., 2004a).
*E* combines with the second moment of area, *I* (in
m^{4}), a cross-sectional shape factor, and length, *L* (in m)
of a cantilevered beam (Summers and Long,
Jr, 2006) to produce spring stiffness, *k* (in N
m^{-1}):
(1)

Spring stiffness has been linked to swimming performance
(Pabst, 1996). Experimental
alterations to the body's *k* in swimming gar
(Long, Jr et al., 1996),
swimming whole-body preparations (Long, Jr
et al., 1994) and swimming physical models
(McHenry et al., 1995) has
provided indirect evidence that *k* increases swimming speed by
increasing tailbeat frequency, tailbeat amplitude or propulsive wavelength,
respectively. Moreover, the axial skeleton provides 25% of the body's spring
stiffness in pumpkinseed sunfish *Lepomis gibbosus*, and 70% in
Atlantic hagfish *Myxine glutinosa*
(Summers and Long, Jr, 2006).
These results support the classical hypothesis: vertebrae have an adaptive
function in swimming, stiffening the axial skeleton, increasing swimming speed
and, hence, enhancing swimming performance.

To test this adaptation hypothesis using BEA, we first created biomimetic
notochords constructed of cross-linked gelatin, based on a process used to
make artificial tendons (Koob and
Hernandez, 2003). Varying among notochords are two QTLs,
*E* and tail length *L*, which, given constant *I*,
determine *k* (Eqn 1). We placed these phenotypically variable
biomimetic notochords into propulsive tails and attached those constructs to
three swimming robots. The robots, called `Tadros', were modeled after the
one-eyed, free-swimming tadpole larvae of ascidians (Urochordata, Phylum
Chordata), which to our knowledge is the chordate swimmer with the simplest
sensorimotor system (McHenry and Strother,
2003). Ascidian tadpole larvae navigate in three dimensions using
helical klinotaxis, an algorithm that also works in two dimensions for
surface-swimming robots (Long, Jr et al.,
2003; Long, Jr et al.,
2004b). Information from ascidians informs the reconstruction of
the common ancestor of the urochordate-vertebrate clade, recently posited as a
monophyletic taxon based on phylogenomic evidence
(Delsuc et al., 2006). To
assess the relative performance of the Tadros, we created a selection
environment in which their performance was judged on their ability to forage:
to swim quickly, smoothly and accurately to a food source and then to hold
station in close proximity. In proportion to their fitness, the tails were
allowed to reproduce in simulation, with mutation randomly varying the QTL for
*E* and *L*, gametogenesis assorting the chromosomes, and
mating, with its accompanying genetic drift, producing new offspring tails.
Lack of a significant relation between *k* and *NP* would
refute, in this system, the classical hypothesis.

## Materials and methods

Since this work involves many steps, processes and experiments, we provide an overview of the BEA methodology (Fig. 1). Each element of the method is detailed in the sections that follow.

### Biomimetic notochords and propulsive tails

Notochords were manufactured in a two-step process: (1) cylindrical
hydrogels were fabricated from gelatin and (2) they were cross-linked with
glutaraldehyde to achieve targeted values of *E*. All hydrogels were
made of a 0.1 g l^{-1} concentration of powdered gelatin (porcine
skin, Type A; Sigma, St Louis, MO, USA) dissolved in distilled water heated at
temperatures below 100°C. Trapped air bubbles were allowed to escape
before the liquid was poured into cylindrical molds of 1.0 cm inner diameter
and cooled at 6°C. Once solidified, the hydrogels were carefully pushed
out of the molds and were visually inspected for damage; those with tears,
pits or other imperfections were discarded. Undamaged hydrogels were cut to
the desired length.

To adjust the *E* values, hydrogels were cross-linked using various
% concentrations of glutaraldehyde (25% EM grade; Polysciences, Warrington,
PA, USA). The glutaraldehyde was diluted in phosphate buffer solution (0.1 mol
l^{-1} NaH_{2}PO_{4}, 0.15 mol l^{-1} NaCl, pH
7.0). Hydrogels bathed in the solution were placed on a shaker bed, left to
cross-link for about 17 h, and then rinsed five times in distilled water to
remove any remaining unreacted glutaraldehyde. Hydrogels were stored
aseptically in an aqueous solution of 20% ethanol. To determine the
relationship between concentration and *E*, we created a preliminary
range of hydrogels crossed-linked at various glutaraldehyde concentrations.
The resulting hydrogels were bent at 1 Hz in three-point bending (Mini Bionix
858 with 10 N load cell; MTS Systems, Eden Prairie, MN, USA), resulting in the
following equation:
(2)

where *E* is in units of MPa.

After producing ten generations of biomimetic notochords using the formula
(Eqn 2) for evolutionary trials, we validated the hydrogels' targeted
*E* values in a custom-built cantilevered bending machine
(Long, Jr et al., 2006a) that
mimicked the cantilevered bending of the hydrogel mounted in the robots during
swimming. We bent biomimetic notochords at a frequency of 1.76 Hz, near the
operating tailbeat frequency of 1.70 Hz during swimming
(Table 1), at flexions of 0.287
rad, near the operating tail flexion of 0.267 (see
Table 3). All 30 evolved tails
were tested in random order; two tails were randomly chosen and were retested
three times throughout the procedure to measure repeatability, one minus the
ratio of the measurements' standard deviation (s.d.) and mean value. We had an
average repeatability of 93% and the 30 *E* values did not differ
significantly (two-tailed paired *t*-test, *P=*0.137) from the
targeted *E* values from Eqn 2. These were the *E* values we
used.

The biomimetic notochords were used as the axial skeleton in the propulsive tail (Fig. 2). The ends of the notochords were gripped with heat-shrink tubing that was, in turn, glued on one end to an insert screwed into the driveshaft and on the other end to a rigid and flat hemi-circular tail of acrylic (Fig. 2). To prevent the tail from sagging or lifting in the vertical plane, a vertical septum was created from two conjoined sheets of Press and Seal™ wrap (Glad, Oakland, CA, USA); the septum cradled the notochord and secured the caudal fin to the driveshaft. While the depth of the driveshaft side of the vertical septum was held constant, the axial length of the vertical septum co-varied with the length of the tail. Finally, in lateral bending tests, we were unable to detect the presence of the vertical septum, since its contribution to stiffness was within the margin of repeatability (see above).

### Biomimetic robotic tadpoles: Tadros

The Tadro is an autonomous, self-propelled agent, capable, using a single photoresistor `eye', of detecting a light gradient, swimming up the gradient, and orbiting around the spot of greatest light intensity (Long, Jr et al., 2003; Long, Jr et al., 2004b). For surface swimming, we call this behavior two-dimensional, cycloptic helical klinotaxis (2D cHK). We modified the previous 2D cHK Tadro design by replacing the analog circuit implementation of the neural algorithm with a digital microcontroller (MIT HandyBoard, Newton Labs, Renton, WA, USA). The microcontroller allows us to use a single servomotor (MPI MX-400, Maxx Products, Lake Zurich, IL, USA) to both flap and turn the tail (Fig. 2). Controlled by software (Interactive C, v. 4; Newton Labs; working code available from J.H.L. upon request), the servomotor flapped the tail ±30° at a nearly constant cycle frequency of 1.7 Hz producing lateral tail-tip amplitudes that operated the tail at Froude numbers of 0.226 (Table 1), within the range seen in fish and close to but not within the lower limit considered optimal for thrust production (Terada and Yamamoto, 2004; Triantafyllou et al., 1993). The Tadros operate at Reynolds numbers near 11 730 (Table 1). Voltage across the single cadmium sulphide (CdS) photoresistor `eye' (Fig. 2) varied inversely with light intensity and was converted to a tail offset that varied continuously and independently from the sinusoidal flapping.

### Competition experiments with forage navigation

In each generation, we tested three tails, each bearing a biomimetic
notochord with a different value of spring stiffness, *k*. We limited
the population size to three so that we could compete three Tadros
simultaneously, allowing their physical interactions to be part of the
selection environment (Fig.
3A). While we built three Tadros of identical specifications and
parts, we did not assume that they would perform identically. To control for
differences among the Tadros, independent of those caused by the different
tails, we ran 12 competition experiments in each generation, with all six
possible combinations of tails and robots replicated twice.

Each trial was 3 min long, and began with Tadros directed towards the center of a 2.5 m diameter circular tank (Fig. 3B). Starting position was randomized for all twelve trials. A light source was suspended above the tank, creating on the water surface an area of high intensity (Fig. 4A) and a rapidly attenuating gradient (Fig. 4B). We measured the Tadros' ability to perceive the light gradient, and confirmed that their perception varied with both distance from and orientation to the light source (Fig. 4C).

The three Tadros were videotaped from above (JVC digital video; 30 Hz temporal resolution; 1.2 cm spatial resolution), under light provided only by the target light source. To mark the bow, stern and identity of each Tadro, they carried colored LEDs. Digital videos were converted to QuickTime movies and analyzed using VideoPoint (ver. 2.5, Lenox Softworks, Lenox, MA, USA). Bow and stern points on each of the three Tadros were manually digitized every 1 s, for a total of 120 frames and 720 points per trial.

For our behavioral metric, we recognized the coordination required between
functional systems to enact behavior (Rice
and Westneat, 2005). The navigational task was to move towards the
light source, find it quickly, and hold station in a tight orbit around that
source. This involves possible trade-offs: swim quickly to get to the target
but then maneuver to stay as close to the target as possible. An idealized
agent would swim quickly to the source and then stop, moving only as needed to
stay on the target. For our simple swimmers, however, they are behaviorally
constrained because they are constantly swimming; the only motor output
controlled by the brain is turning direction. Since forage navigation involves
detection, propulsion, maneuvering, and station-holding, we created
`navigational prowess,' *NP*, a dimensionless number:
(3)

where *U* (m s^{-1}) was the average forward speed of
propulsion, taken as the finite difference in midpoint position of the Tadro's
hull along its trajectory over each 1 s time increment (0.01 m s^{-1}
resolution). *R* (m) was the orbital radius, the average distance of
the Tadro from the light target (Fig.
3) and a measure of station-holding ability. *t* (s) was
the time to the food source, which was defined as within 0.5 m of the spot of
highest intensity (Fig. 3);
Tadros that failed to get this close received the maximum time of 120 s.
*W* (rad s^{-2}) was the yaw wobble of the Tadro, defined as
the s.d. of the Tadro's angular acceleration in yaw (rad s^{-2}), as
determined by finite differences of the angular velocity (rad s^{-1}),
which was also determined from finite differences of the heading (rad) between
frames. *W* measures both recoil caused by periodic tail beats and
accelerations caused by turning maneuvers
(Fig. 5), and was developed as
a proxy of path tortuosity, a key parameter in animal orientation and
searching that is inversely related to the efficiency of the orientation
mechanism (Benhamou, 2004).
Taken together in *NP*, these kinematic variables define more effective
forage navigation, which we defined as higher values of *NP*. Higher
values of *NP* can be generated in a variety of ways given the number
of variables. Thus the complexity of *NP* as a behavioral metric
permits the evolution of different phenotypes that produce equivalent
performance.

Individual fitness, ω_{i}, is the chance of survival of the
genotype (Ridley, 1993), a
number between 0 and 1. The `individual' here, denoted by the subscript
*i*, was one of three tails each generation. The individual relative
fitness, ω_{ij}, relative to that of other individuals in the
particular generation, where generation was indicated by the subscript
*j*, was computed, following 12 experimental runs per generation, as
follows:
(4)

where the *j*_{max} and *j*_{min} terms were
the maximum and minimum values, respectively, for all individuals performing
in that generation, *j*. Without *a priori* biological knowledge
about the relative importance of each, we weighted the kinematic phenotypes
equally.

### Biomechanical analysis

In biomechanical analyses, we sought more detail about the motion of the tails during swimming. Each Tadro was filmed separately from below as they swam straight in a clear Plexiglas™ tank (JVC digital video; 30 Hz temporal resolution; 0.25 cm spatial resolution). Two points on the hull were tracked (bow and stern) in addition to three points on the notochord, one on either end and one in the middle. All five points were manually digitized as above every 1/30 s for three tailbeat cycles. Each of the 30 tails was tested three times, once on each of the three Tadros, for a total sample size of 90.

In addition to calculating swimming speed *U* and wobble *W*,
as above, we used the positional data on the notochord to calculate tailbeat
frequency *f* (Hz), tail flexion φ(rad), lateral amplitude of the
tail tip *a* (m), phase of lateral tail displacement δ (% cycle),
and Froude number η (non-dimensional). Tailbeat frequency *f* was
measured as the inverse of the average period (s) of the beating tail over
three tailbeats (resolution of 30 Hz). Tail flexion φ was measured as the
angular deviation of the three points on the tail from a straight line
(resolution of 0.009 rad). Lateral amplitude of the tail tip *a* was
measured as half of the average maximal difference between positions of the
tail (resolution of 0.001 m). Phase of lateral tail displacement was measured
as the difference in time between the attainment of maximal lateral position
of the middle and distal points on the tail (resolution of 5 % of cycle).
Froude number η was calculated as the ratio of the product of *f*
and *a* to *U.*

### Genetic algorithm

Genetic algorithms were used to simulate the interaction of mutation, genetic drift, mating and selection in artificial evolution (Nolfi and Floreano, 2000). Genotype frequencies after selection, , were computed as follows: (5)

where *p*_{ij} was the genotype frequency prior to
selection, which, before we applied selection, was equal for all three
genotypes with *p*_{ij}=0.333. After selection, the frequencies
were as follows:
(6)

Since the sum of the all the genotype frequencies no longer equaled one, we
calculated the normalized genotype frequency, *p*_{ij}, which
was used to determine the number of gametes that each parent contributes to
the gene pool:
(6)

where
was the mean relative fitness (Ridley,
1993) of the population at generation *j*:
(8)

### Gametogenesis and mating

Prior to mating, each parent genotype underwent simulated gametogenesis
*via* meiosis with mutation but no recombination. For each QTL, we
assumed simple additive genetic variance without dominance or epistasis. The
diploid adult genotype (E_{1}/E_{2},
L_{3}/L_{4})_{ij} consisted of two QTLs, *E*
and *L* (where the trait denotation is a capital letter italicized and
the chromosome letter is not) with a large but unspecified finite number of
loci, with loci for each trait located on separate chromosomes as indexed by
the subscripts. Given an isomorphic relation between genotype and phenotype
(narrow-sense heritability of one), we represented genotype quantitatively
using the continuous values for the adult phenotype. For simplicity, we
partitioned the quantitative phenotype equally between the two sister
chromosomes.

During meiosis, we mutated each of the four chromosomes, E_{1},
E_{2}, L_{3}, L_{4}, of each individual. The magnitude
and sign of mutational change was determined by randomly selecting a value
from a Poisson distribution, where the center of the distribution was zero
change and the range was scaled to encompass ±25% of the trait's median
value. (For *E*, range and median were 0.1 to 1.4 MPa and 0.65 MPa,
respectively; for *L*, range and median were 20 to 120 mm and 70 mm,
respectively.) Mutation created four new chromosomes for each individual,
E_{1*}, E_{2*},
L_{3*}, L_{4*}, and 12 new chromosomes
for the population. By segregation and independent assortment, each of the
three individuals creates, four gamete types with the following haploid
genotypes:

For mating, six gametes were chosen for the gene pool. Each individual
contributed to the pool on the basis of their post-selection normalized
genotype frequency, (Eqn 7),
which determined their number of gametes, *N*_{ij}:
(9)

Thus, for an individual, *N*_{ij} gametes were chosen
randomly from the four possible types. The combined gene pool of six gametes
was then randomly combined in pairs to create the new generation of three
diploid individuals.

Because of the small population size of three, selection results in
differential reproduction, and hence evolution by selection, only if the
differences among individual fitnesses results in a value of *p*″
greater than 0.5 for one individual. When all values of *p*″ are
below 0.5, each individual contributes two gametes and evolution by selection
is therefore not present. This is the distinction we use with the phrases
`with selection' and `without selection', respectively. Please note that
always present is evolution by chance, which includes mutation and drift.

### Statistical design

To normalize the response variables from the competition trials, a variety
of transforms were run, and the best, as judged by the linearity of
probability plots, were used (Zar,
1996). *E* was normally distributed and untransformed;
*L* and *k* were log_{10}-transformed. *R, U, W*
and *NP* were arcsine-square-root transformed, with *R* first
divided by 10; *t* was inverse transformed. For the biomechanical
analysis, data were log_{10}-transformed. All statistics were
performed using **JMP** (ver. 5.0, SAS Institute).

The competition trials were analyzed in four ways. First, to determine if
evolution had occurred, we conducted a nested ANCOVA with generation as the
main effect, Tadro as the covariate, and tail nested within generation
(*N*=360; for each of 10 generations, three tails tested in 12 trials).
This model was used on the response variables, *R, t, U, W* and
*NP*, with planned *a priori* contrasts (α=0.05) to
examine generation-to-generation changes in the population mean. Second, to
address the same question for the morphological phenotypes, *E, L* and
the mechanical behavior, *k*, we simplified the model to a one-way
ANOVA, with generation as the effect (*N*=30), which included planned
*a priori* contrasts (α=0.05) to examine generation-to-generation
changes in the population mean. Third, to determine if evolutionary changes in
the population mean differed when the population was under selection and when
it was not, we created new response variables, for example Δ*E*,
that took the difference in the population mean from generation *j* to
*j*+1. Since there are nine intergenerational intervals, this yielded a
sample size for each new response variable of nine. We ran a one-way ANOVA
with selection (absent, present) as the effect. Fourth, to examine the
correlation structure between *k* and *NP*, as mediated by the
kinematic variables, *R, t, U* and *W*, we ran a one-way MANOVA
(*N*=30), which yielded partial correlation coefficients and univariate
regressions of the responses onto *k*. Using those regressions, we
generated predicted values of *R, t, U* and *W* and compared
those to the observed; residuals were calculated as the difference between
observed and predicted values. The residuals of *R, t, U* and
*W* were used as a set of possible predictor variables for the
residuals of *NP*, and stepwise linear regressions (forward, backward,
mixed models) were run.

The biomechanical analysis of *k* and swimming kinematics (*a, f,
U, W*, δ, φ and, η recorded in straight-swimming trials)
was conducted using linear regression (*N*=30, from the mean of three
trials for each of 30 tails).

## Results

The thirty tails evolved over ten generations had variable morphological
and mechanical properties (Table
2). In nonnavigational rectilinear swimming during the
biomechanical analysis, the Tadros and tails also had variable kinematics
(Table 3). Unexpectedly, we
encountered two uncontrolled sources of variation in the fabrication of the
notochords and the Tadros. First, the notochords' second moment of area,
*I* (m^{-4}), which was meant to be held constant (see
Materials and methods), instead varied in inverse proportion to *E*
(MPa):
(10)

with a coefficient of determination of 0.375 (*N*=30;
*P*<0.001). Because of this covariation, calculations of *k*
for each tail used an actual *I* and not an *I* assumed from the
fabrication process. Second, even though the three Tadros were designed to be
identical, they performed differently, as revealed by ANCOVA from the
competition experiments. Treated as the covariate, the Tadro effect was highly
significant (*P*<0.001) for *R, U* and *W*. This
result justified our experimental design, where each tail was tested 12 times
per generation, four times on each of the three Tadros.

The Tadros showed variation in navigational behavior that was sometimes,
but not always, correlated with phenotypic variation in *k*
(Fig. 3). Over ten generations,
the population mean of *NP* and *k* evolved, changing
significantly in five and six generational intervals, respectively
(Fig. 6). Evolution occurred in
all other phenotypes as well. With every phenotype, the direction of evolution
varied. Moreover, evolution of at least three of the eight phenotypes occurred
in every inter-generational time step (Fig.
6).

In the presence of selection in generations 1, 5, 6 and 9, unidirectional
evolution of the population mean, relative to that when selection is absent,
occurred in five of the eight phenotypes
(Fig. 7). The morphological
phenotypes, *E* and *L*, showed no significant unidirectional
trend; likewise for *k*, although the non-significant trend was for
*k* to increase in the presence of selection. The behavioral phenotype
*NP* increased in the presence of selection; this was not surprising,
since *NP* is a composite of the kinematic phenotypes *R, t, U*
and *W*, all of which changed unidirectionally in the presence of
selection (Fig. 7). Note that
the directions of change differ: *R* and *t* increased while
*U* and *W* decreased in the presence of selection. Thus the
increases in *W* correlated with the increases in *U* counteract
(Eqn 3).

From the biomechanical analysis and competition trials, *k*
predicted *U* of the biomimetic robots
(Fig. 8A). The only other
kinematic phenotype that was measured in both the biomechanical analysis and
the competition trials, *W*, was predicted by *k* only in the
competition trials (Fig. 8B).
Since *W* includes angular acceleration from both yaw recoil of the
flapping tail during rectilinear swimming and unsteady turning maneuvers (see
Fig. 5), the difference between
competition and biomechanics trials was a measure of the portion of the
*W* caused by unsteady maneuvering. Thus *k* was unrelated to
the rectilinear swimming *W* (biomechanical analysis) and was
positively correlated with greater unsteadiness during maneuvers, *W*
(competition trials). For the competition experiments, the kinematic
phenotypes were correlated, with negative correlations between *R* and
*U, R* and 1/*t, R* and *W*, and positive correlations
between *U* and 1/*t, U* and *W*
(Table 4). Note that these
correlations were performed on transformed data and that the sign of the
correlations with 1/*t* will be opposite those with *t*.

From the biomechanical trials alone, *k* predicted only *a*
(Fig. 9A), which, in turn,
significantly predicted *U* (Fig.
9B). All other biomechanical response variables
(*f*,δ,φ and η) are not predicted by *k* in
linear regression.

The trajectory of the population through [*E, L*] morphosphace was
erratic (Fig. 10), as expected
if both chance (drift and mutation) and selection were operating. Unexpected
by the classical prediction of the benefit of increased *k* on
navigational performance, however, was that when selection was present
sometimes *k* increased (steps 1-2 and 5-6) and sometimes it decreased
(6-7 and 9-10). Changes in the log_{10} of *k*, mediated
through the kinematic variables (Eqn 3;
Table 4), predicted 40% of the
variation in *NP* (Fig.
11A). Of the variance in *NP* unexplained by *k*,
measured as the *NP* residuals, 81% was inversely and positively
correlated with the residuals in *R* and *U*, with slopes of
-0.115 and 0.610, respectively (Fig.
11B). The trajectory of the population through [*NP, k*]
behavior space appeared erratic, but a pattern emerged
(Fig. 12). Whenever selection
was present, *NP* increased, independent of changes in *k*. At
this landscape level, the evolutionary histories of *k* and *NP*
appeared decoupled, in spite of the linkage between the two seen by the
significant relation between *k* and *U*
(Fig. 6A), *k* and
*W* (Fig. 8B), and
*k* and *NP* (Fig.
11A). When taken in sum, the functional relation between
*k* and *NP* was complex enough
(Fig. 13) to permit multiple
evolutionary outcomes.

## Discussion

This is the first study, to our knowledge, that evolves autonomous
biomimetic robots to test an adaptation hypothesis for vertebrates. This
extension of biomimetic evolutionary analysis (BEA) permits the functional
relation between morphology, mechanics, and behavior to be varied randomly, by
mutation and drift, and judged explicitly by artificial selection for adaptive
value. Even in this case, using simple robots, genetics and ecology, adaptive
value can be predicted only when the functional linkage between biomechanics
and behavior is assessed in a specific selection environment
(Brandon, 1990). When we
selected for improved navigational prowess, *NP*, stiffness of the
axial skeleton, *k*, determined by the biomimetic notochord's
genetically based traits bending modulus, *E* and length, *L*
(Eqn 1), increased as an adaptive response. However, *k* was neither
the only trait that determined *NP* nor did it always evolve in
synchrony with *NP*. Thus, even though *k* has both a mechanical
and adaptive relation to *NP*, selection for *NP* did not in all
cases evolve *k*. Below we discuss the complexities of this asymmetric
relation by examining first mechanical and then adaptive function of the axial
skeleton of biomimetic robots during forage navigation.

### A biomechanical approach: mechanical function

In terms of mechanical function, our results support Blight's
hybrid-oscillator theory (Blight,
1977): stiffness of body and tail was the key parameter in
controlling thrust production and, hence, swimming speed, *U*. When we
swam our tadpole robots, `Tadros', with a constant tailbeat frequency in a
straight line, *k* predicted 74% of the variation in *U*. The
causal connection between *k* and *U* was explained by only one
other of the five kinematic variables measured
(Table 3), lateral tailbeat
amplitude, *a* (Fig. 9).
Hence *k* in this system modulated thrust by increasing *a*.

This is not, however, the first study to demonstrate a causal relation
between *k*, kinematics and *U*. Inspired by the `halfmyotome'
preparation (Johnsrude and Webb, 1981), Long, Jr et al.
(Long, Jr et al., 1994) swam
whole-body preparations of pumpkinseed sunfish at constant tailbeat frequency,
*f*, finding that reduced *k* also reduced *a* and
*U*. McHenry et al. (McHenry et
al., 1995) swam flexible casts of pumpkinseed sunfish,
demonstrating that increased *k* increased the speed of the body's
flexural wave of bending. In live longnose gar, reductions of the body's
*k* reduced the *f* at which animals chose to swim
(Long, Jr et al., 1996).

These results speak to the evolution of vertebrates insofar as they
demonstrate that increases in the *k* of the axial skeleton (this
study) and the body increase thrust production and hence *U*. Given
that the classical adaptation hypothesis links the evolution of vertebrae with
increased *k* (Webb,
1982; Weihs and Webb,
1983), we also need to know if and how vertebrae control
*k* of the body. In preliminary experiments, in which we placed
artificial vertebral ring centra onto the excised notochord of Atlantic
hagfish, the presence of vertebral centra dramatically increased *k* of
the axial skeleton by reducing the amount of connective tissue available to
undergo strain during bending (Long, Jr et
al., 2004a). This result is consistent with correlations between
number of vertebrae and minimal propulsive wavelength during steady swimming
(Long, Jr and Nipper, 1996)
and body curvature during fast starts
(Brainerd and Patek,
1998).

One remaining question for the biomechanist is how much of the body's
*k* is provided by the axial skeleton? The axial skeleton provides 75%
of the body's passive (without muscle contraction) flexural stiffness in
hagfish and 25% in sunfish (Summers and
Long, Jr, 2006). Those percentages may fall if we consider
situations in which the muscle might be used to actively increase the body's
flexural stiffness (Long, Jr and Nipper,
1996; Long, Jr,
1998).

From biomechanics alone, we see evidence to support the classical
hypothesis that vertebrae stiffen the body and that increased body *k*
increases thrust production and steady swimming speed. This formulation,
however, avoids a central evolutionary question: under what ecological and
selective conditions might vertebrae evolve? Unlikely is a simple relation
between mechanical function (*k* controlling *U*) and adaptive
function (increased *U* improves every behavior), given the
complexities and interactions of fundamental behaviors such as navigation
(Dittman and Quinn, 1996),
predator evasion (Walker et al.,
2005), feeding (Rice and
Westneat, 2005) and foraging
(Martini et al., 1997).

### An evolutionary approach: adaptive function

We examined the adaptive value of *k* in a selection environment in
which robots were rewarded for improvement in forage navigation relative to
other individuals in the same population. Our fitness function (Eqn 4) gave
equal weight to the four variables: orbital radius *R*, time to target
*t, U* and robot wobble *W*, which determine the performance
metric, *NP* (Eqn 3). Because of the biomechanical connection between
*k* and *U* (see above), we designed *NP* to trade-off
*U* against other important aspects of foraging, *R, t* and
*W*, for which we had no *a priori* knowledge of how they
related to *k*. We sought to create a selective environment that
allowed for the emergence of different and equally successful behavioral
strategies. On one end of the spectrum, a `maneuver specialist' may evolve if
the product of *R, t* and *W* decreased while *U* held
constant; selection would act on the kinematic phenotypes related to
maneuvering alone. On the other end of the spectrum, a `speed specialist' may
evolve if *U* increased while the product of *R, t* and
*W* remained constant; selection would act on the speed phenotype
alone.

While specialist or generalist behavioral strategies might be rewarded by
selection, depending on available phenotypic variance, evolution requires that
selected phenotypes are heritable. In this system, therefore, the only
behavioral strategies that may evolve are those linked to *k, E* and/or
*L*. We found evidence for heritability of the four kinematic
phenotypes that determine *NP*: *R, t, U* and *W* (Eqn
3). Both *U* and *W* are directly correlated with *k*
(Fig. 8), while *R* and
*t* are indirectly correlated with *k* through their correlation
with *U* (Table 4; for
summary, see Fig. 13).

From a `top-down' view starting with behavior, selection for increased
*NP* evolved a single generalist behavior in which *U* and
*W* increased while *R* and *t* decreased (Figs
6,
7). It is interesting to note
that *NP* increases in spite of the fact that increasing *W*
removes a portion of the gains afforded by the productive changes in *U,
R* and *t*. This maladaptive effect of *W*, seen in the
positive correlation between *U* and *W* seen during the
competition experiments (Table
4), arises from maneuvering, since *W* was unrelated to
*k* during the propulsion-only biomechanical tests
(Fig. 8). While we did not test
directly for the mechanical relation between *U* and *W* during
competition, we suspect that turning maneuvers were exaggerated when the robot
was swimming at higher *U* because the propulsive tail was also
functioning as the turning rudder.

From a `bottom-up' view starting with *k*, the single generalist
strategy just described emerges from the summation over generational time of
two different behaviors, a `*k*-direct' and a `*k*-indirect'
mode. Recall that over all four bouts of selection, *k* is
statistically invariant while *NP* increases
(Fig. 7). A step-by-step view
reveals that *k* does change under selection, two times increasing and
two times decreasing while *NP* increases each time
(Fig. 12). One explanation is
that the relation between *k* and *NP* alternated between a
*k*-direct (*k* increasing) and a *k*-indirect
(*k* decreasing) mode. Evidence for the *k*-direct mode comes
from *NP* values generated using *k* to predict *R, t, U*
and *W via* regressions (Fig.
11A). In this *k*-direct mode, *k* predicts 40% of
the variance in *NP* and operates directly through unequal,
countervailing changes in *U* and *W*
(Table 4;
Fig. 8). Evidence for the
*k*-indirect mode comes from the remaining 60% of the variance in
*NP* unexplained by *k*
(Fig. 11). When this variance
was measured as the *NP* residuals, it was significantly correlated
with the residuals of *R* and *U* regressed onto *k*
(Fig. 11B). Hence *R*
and *U* are the causal variables in the *k*-indirect mode,
working together, by virtue of their inverse and direct proportionality to
*NP*, respectively (Fig.
11B), to increase *NP*.

### Vertebrates and vertebrae

Regarding the origin of vertebrae, as represented here by *k*, we
examined only one of many plausible adaptation scenarios. We found evidence
that vertebrae may function as an adaptation for improved navigational
prowess, *NP*, with the stiffness they give to the axial skeleton
increasing *U* and *W* directly or *R* and *U*
indirectly (see previous section). While *U* figures in both adaptation
scenarios, it is inaccurate to say that vertebrae are an adaptation for
increased *U*. By focusing on a single aspect of behavior, like
*U*, we miss the point that adaptation always occurs in a particular
environment where selection acts upon behavior directly, with the result that
many aspects of that behavior (such as *R, t, U* and *W* in this
case) evolve simultaneously in integrated, independent and/or maladaptive
ways.

The search for plausible adaptation scenarios for vertebrae is limited only by the imagination and resources of the investigator. We recognize that many plausible scenarios exist, and that they await testing. For example, adaptation for predation is suggested by phylogenetic reconstruction of extant taxa correlating the origin of jaws and vertebral centra (Koob and Long, Jr, 2000). Adaptation for benthic, low-speed or fin-based life histories are suggested by the phylogenetic distribution of the adult notochord, with this ancestral state retained or secondarily reacquired in taxa such as jawless hagfish, six-gilled sharks, sturgeon, lungfish and coelocanths (Janvier, 1996). Even in closely related fish taxa, possible adaptation scenarios are numerous, with fish size, life history, acceleration and environment all correlated with changes in number of vertebrae within a family (Brainerd and Patek, 1998; McDowall, 2003). The same can be said of divergent taxa (Long, Jr and Nipper, 1996), and even the terrestrial undulatory locomotors, snakes (for a review, see Shine, 2000). Finally, selection for and pleiotropy among immune, reproductive and nervous systems may create regulatory gene networks that influence the number and morphology of vertebrae (Anand et al., 2003; Galis, 1999; Narita and Kuratani, 2005).

### Caveats

When attempting to reconstruct the locomotor dynamics of fossil species, the multitude of unknown parameter values guarantees a wide range of plausible results (Hutchinson and Gatesy, 2006). Add to this the uncertainty introduced by simplifications required to model the complex interactions of genetics, phenotypes, and selection related to swimming in living vertebrates (Aubret et al., 2005; Blake, 2004; Ghalambor et al., 2003; Walker et al., 2005). At best, then, attempts to determine the adaptive value of any trait in extinct taxa can yield only what have been called `how-possibly' adaptation explanations (Brandon, 1990). While poor resolution blurs its hindsight, BEA's simulated evolution, driven by selection and chance, offers the opportunity for unexpected results that force consideration and ranking of viable, alternative pathways.

For some, a more primary concern is the use of robots as model simulations of biology (see Webb, 2001 and the multiple critiques appended thereto). In our case, it is easy to show how the Tadros, their biomimetic tails, and the selection environment are unrealistic: in broad strokes, the individuals, their genetics, and their environment are too simple and too invariant. We have grossly oversimplified anything close to actual fish body design (Blake, 2004; Gemballa and Roder, 2004), we ignore the possibility that burst speed and tail length are related (Aubret et al., 2005), we fail to consider alternative adaptation explanations related to acceleration performance or unrelated to swimming mechanics, and we isolate locomotor systems at the cost of losing important interactions between swimming and prey capture (Rice and Westneat, 2005). The list goes on. In our defense, any modeler, whether their canvas is software or hardware, must decide what to include and what to omit. Our premise was akin to that used to justify parsimony: in the absence of a reason to include one feature over another, start with the simplest possible model that produces the desired behavior, in this case, autonomous navigation during swimming. This start-simple approach can also be supported phenomenologically, since even a single addition to a simple autonomous agent creates novel navigation behavior that is difficult to understand analytically (Braitenberg, 1984).

- List of symbols and abbreviations
- 2D cHK
- two-dimensional, cycloptic helical klinotaxis
- a
- lateral tail amplitude (m)
- BEA
- biomimetic evolutionary analysis
- E
- bending modulus (Pa)
- f
- tailbeat frequency (Hz)
- I
- second moment of area (m
^{4}) - k
- spring stiffness (N m
^{-1}) - L
- length of tail or beam (m)
- N
- number of gametes
- NP
- navigational prowess
- p
- genotype frequency
- QTL
- quantitative trait loci
- R
- orbital radius (m)
- t
- time to target (s)
- U
- swimming speed (m s
^{-1}) - W
- wobble of Tadro (rad s
^{-2}) - δ
- phase lateral tail displacement (% tailbeat cycle)
- φ
- tail flexion (rad)
- η
- Froude efficiency
- ω
- relative individual fitness
- mean fitness

## ACKNOWLEDGEMENTS

Special thanks go to Janese Trimaldi, who spent many days optimizing the hydrogel fabrication process. Matthew McHenry, Hugh Crenshaw and Charles Pell graciously shared their ideas about helical klinotaxis and its robotic implementation. Robot design and fabrication was made possible by Ken Livingston, Carl Bertsche, John Vanderlee, Betsy Ketcham and Jerry Calvin. Assistance in conducting and analyzing experiments was provided by Kurt Bantilan, Nicole Doorly, Yusuke Kumai, Chun Wai Liew, Gianna McArthur, Meghan Overgaard, Robert Root and Mike Zubrow. This work was funded by National Science Foundation grants DBI-0442269 and BCS-0320764.

- © The Company of Biologists Limited 2006