Collision-avoidance behaviors of minimally restrained flying locusts to looming stimuli

The American grasshopper (Schistocerca americana) has as its natural habitat the southern shrublands of North America (Kuitert and Connin, 1952; Capinera et al., 2001; Capinera et al., 1997). The relatively dense tree canopy constitutes a highly cluttered environment that requires effective collision-avoidance strategies in flying animals. During flight, grasshoppers and locusts also have to avoid predation on the wing by a wide variety of bird species (Kuitert and Connin, 1952; Capinera et al., 1997; Branson, 2005). The neural circuits mediating collision-avoidance behaviors are well defined in S. americana, as well as in the closely related species Schistocerca gregaria (desert locust) (Sword, 2003) and Locusta migratoria (migratory locust) for which many extensive investigations have been carried out (for review, see Fotowat and Gabbiani, 2011). The mechanisms underlying jump escape behaviors to looming stimuli, the two-dimensional (2D) expanding shadows associated with objects approaching on a collision course with the animal, have been most studied (Fotowat and Gabbiani, 2007; Fotowat et al., 2011; Santer et al., 2005b; Santer et al., 2008). The lobula giant movement detector (LGMD)/descending contralateral movement detector (DCMD) pathway is thought to underlie these behaviors, based on extensive work carried out since its initial anatomical and electrophysiological characterization (O’Shea and Rowell, 1974; O’Shea and Williams, 1974; Schlotterer, 1977; Rind and Simmons, 1992; Hatsopoulos et al., 1995).

Collision-avoidance behaviors have also been studied by a number of authors in simulated flying conditions where the animal’s position is rigidly fixed in a wind stream to elicit wing flapping close to that observed under natural conditions (e.g. Robertson and Reye, 1992; Robertson and Johnson, 1993a; Robertson and Johnson, 1993b; Gray et al., 2001; Santer et al., 2005a; Santer et al., 2006; Ribak et al., 2012). These experiments investigated either responses to head-on collision, as would occur with a stationary obstacle on the animal’s flight path, or responses to looming stimuli appearing from the side, mimicking a predator’s strike. In head-on collisions, the animals’ body and wing movements as well as the forces generated during such episodes are consistent with attempts to fly around the potential obstacle (Thüring, 1986; Robertson and Johnson, 1993a; Kutsch et al., 2003). Looming stimuli presented from the side have mainly been reported to evoke brief interruptions of the wing beating pattern in an effort to avoid the potential threat (Santer et al., 2005a), a behavior consistent with natural gliding and diving (Roffey, 1963; Baker and Cooter, 1979). Another behavior similar to a banked turn away from the perceived object was also observed when the animals retained rotational freedom about one axis (Ribak et al., 2012). Rigid restraint on the animal’s position presents undeniable advantages for the experimenter’s ability to monitor the animal’s performance, but may limit the range of behaviors expressed by the subjects. Such experiments may also lead to ambiguities in interpretation, as by design the subtle changes in body posture or wing motion elicited by the stimulus are not directly related to changes in flight trajectory. In this paper, we thus sought to investigate the collision-avoidance behaviors generated by minimally restrained animals to looming stimuli presented from the side, in an effort to better characterize the range of strategies used by flying animals to avoid the approach of a potential predator. In a second step, we used a more restrained set-up to obtain finer information on wing kinematics during such collision-avoidance behaviors. This allowed us to investigate the relationship between wing beat asymmetries and the resulting collision-avoidance behaviors, as well as the potential involvement of wing deformations that contribute to the generation of straight flight in the desert locust (Walker et al., 2009a; Young et al., 2009).

All experiments were performed with adult S. americana Drury 1773 of both sexes preselected by first testing their ability to fly adequately in a large room and then by verifying that they were able to generate positive lift in a wind tunnel when 500 mg of their own weight was counterbalanced, as explained in ‘Experimental setup’, below. The subjects’ ranged in mass from 1 to 2.5 g. In contrast to S. gregaria or L. migratoria, which can fly for hours at a time (Uvarov, 1977), S. americana is not a long distance flier. Under our experimental conditions, individual animals placed in a wind tunnel flew robustly for 5–10 min with sustained wing beat frequencies of 20 Hz and flapping amplitudes of 85 deg on average. In an endurance test carried out in the wind tunnel at 30°C and in the dark, only 1 subject out of 3 flew continuously for 45 min. Additionally, the flight stance of the animals usually differed from the long-distance flight posture of S. gregaria, with legs tucked along the femurs (Weis-Fogh, 1956). Specifically, the tibias were extended instead of being tucked up, although we observed intermittently the characteristic stance of S. gregaria. Under minimally restrained conditions, the subjects were also seen to oscillate in flight. This is expected from the aerodynamic and inertial forces induced by wing flapping and unsteady air flow during unrestrained flight (Dudley, 2000). The flight behavior reported above is consistent with observations of S. americana in the wild, where they fly skillfully, but briefly, and fairly unpredictably in their cluttered natural environment.

Subjects were flown in a custom-designed, open-circuit wind tunnel with a test section measuring 61×61×91 cm (width×height×length; Engineering Laboratory Design, Inc., Lake City, MN, USA). The wind flow was laminar and had a speed of 2 m s^–1 at an ambient temperature of 25°C. The temperature was selected in preliminary experiments to give the most stable flight performance under our loosely restrained experimental conditions.

Two high-speed movie cameras were used at varying speeds to capture the collision-avoidance behavior as detailed below (FASTCAM SA3, Photron, San Diego, CA, USA). They were placed symmetrically above the center of the test section on the left and right, respectively, and viewed it at an angle of 60 deg from the vertical. These two views allowed us to reconstruct the three-dimensional (3D) trajectory of the subject as described below. Lighting was provided by two 1000 W theatrical lights (Lowel DP Light, Hauppage, NY, USA) with red filters (Roscolux no.26, Rosco Laboratories Inc., Stamford, CT, USA) mounted in front of them. The filters blocked wavelengths between 420 and 580 nm, minimizing stimulation of the visual system as the peak sensitivity of the subject’s photoreceptors is around 450 nm (Lillywhite, 1978).

The ‘loosely tethered’ setup was used to investigate flight in a minimally restrained environment allowing full 3D movement. A rare earth magnet (100 mg) was fixed with wax onto the subject and attached to a thin cotton thread via a magnetic harness (400 mg). The other end of the thread was attached to a counter-weight balance through a hole at the top of the wind tunnel as shown in Fig. 1A. The balance allowed us to cancel the weight of the magnet and harness, as well as an additional 500 mg (equivalent to between one-half and one-fifth of the subject’s weight), which reduced the lift load and resulted in locusts flying longer in the setup. The subject is perfectly counter-balanced when it flies directly under the hole in the wind tunnel, i.e. when the counter-balancing force is exerted in the direction of vector a in Fig. 1A. In practice, this almost never happened, leading to a non-zero force component in direction b as well. Neglecting friction, the component b was only 3% of the total force or 0.3 mN on average across all trials. The thread never touched the wings of the subjects in any trial. A recording rate of 250 frames s^–1 at a resolution of 1000×1000 pixels was used to capture changes in trajectory. Each trial was subdivided in a ‘free-flight’ epoch lasting 4.6 s and an ‘encounter’ epoch of the same duration.

The ‘tightly tethered’ setup kept the subject in a small restricted 3D volume allowing close focusing on wing kinematics. Instead of swinging freely, the tether was threaded through a thin rigid hollow tube with only 2.5 cm of thread dangling from the bottom. In practice, the subjects preferred to fly as close to the bottom of the tube as possible. Hence, in this setup only body orientation information was obtained. A recording rate of 500 frames s^–1 was used to track both body orientation and wing kinematics. Because of the higher frame rate and memory constraints on the cameras, we reduced the duration of the free-flight epoch to 2.3 s. Thus, in this case the free flight to encounter epoch duration ratio was 1:2.

All experimental trials were carried out ∼10 min apart to allow data transfer from the cameras to a personal computer. This long interval minimized habituation of the response, which is typically small in flight (Rind et al., 2008).

Preliminary experiments showed that salient stimuli are required to generate escape behaviors with high probability. Visual stimuli were thus projected onto a large, 70×61 cm (width×height) rear-projection screen positioned on the right wall of the test section (relative to the flying subject) with a high-powered projector running at 60 Hz (XG-PH50X, Sharp Corporation, Osaka, Japan). Electrophysiological experiments have shown that this refresh rate is sufficient to appropriately stimulate the locust visual system as photoreceptors act like temporal low-pass filters (Howard, 1981; Faivre and Juusola, 2008; Jones and Gabbiani, 2010). During the free-flight epoch, the screen displayed a white background. During the encounter epoch, a simulated object approaching on a collision course was presented. At the end of the encounter epoch, the last frame of this stimulus was kept for an additional 1.15 s before the screen returned to its background luminance level. Instead of a real approaching object, the looming stimulus employed in these experiments reproduced the expanding shadow of a black square on a white background (Fotowat and Gabbiani, 2007; Fotowat et al., 2011). The scotopic luminance of the white background was 2760 cd m^–2 and that of the black square 3.32 cd m^–2.

A schematic diagram of the looming stimulus is shown in Fig. 1B. We assume that the eye is located at a fixed distance D from the screen. The associated virtual object has a profile of half-height l and is approaching the locust’s eye at velocity v<0. The time to collision is denoted by t (<0 before collision) while vt corresponds to the object’s distance from the eye. When projected onto the rear-projection screen, the object appears as an expanding square with half-height r(t)=(l/v)(D/t). All experiments were performed with l/|v| equal to 40 ms. This value was selected because it was in the range of values yielding high escape probabilities in earlier jump escape experiments (Fotowat and Gabbiani, 2007). Higher l/|v| values were impractical because of limitations imposed by the camera memory. Lower ones have been shown to elicit glide responses characterized in earlier work as escape maneuvers of last resort (Santer et al., 2005a). The looming stimulus expanded from its initial (1 deg) to its final size (∼86 deg) during the 4.6 s of the encounter epoch. An example trial is illustrated in Fig. 1C, with its free-flight and encounter epochs separated by a dashed line.

For the fixed-tether experiment, the screen distance was fixed at D=32 cm which translates into a maximum view angle of 86 deg. For the loose-tether experiments, the screen distance varied in time and across trials with a maximum of 47.3 cm and a minimum of 17.1 cm as the subjects moved in the test section. The maximum view angle was 111.9 deg but when trajectories were averaged across trials, the mean maximum view angle was 84.7 deg. However, this variability of screen distance over time did not change the projected time to collision substantially, with only a 40 ms difference between the slowest and fastest times estimated across trials. Hence, we used the averaged time to collision as our reference (zero) time point.

In the loosely tethered setup, we semi-automatically tracked the black colored harness on the subject’s back. The initial step was for the user to select a reference point for the harness in the first frame of the movie. The next frame was then loaded and a 50×50 pixel window surrounding the previously selected point was extracted. A threshold was applied to the window to extract the darkest pixels. The biggest blob was taken to be the harness and the center of this blob was the new reference point. The cycle was repeated for all movie frames to obtain a trajectory for that trial. The automatically tracked points were then validated by eye to ensure accuracy.

The tightly tethered setup allowed us to automatically track the orientation of the subject. This required a segmentation process to place an ellipse around the body on each image, followed by reconstruction of the orientation from all the images of the trial. The segmentation process, summarized below, relied mainly on morphological image analysis techniques (Soille, 2003).

(1) Separate the subject from the background: sum three consecutive images into one to increase subject–background contrast. Apply a bottom hat transformation with a large square structural element (128 pixels, ∼1/2 of the locust body length) to correct uneven lighting. Threshold the image to obtain the whole subject.

(2) Separate the subject’s wings from the background: sum three consecutive images into one to increase subject–background contrast. Apply a bottom hat transformation with a small circular structural element (4 pixels) to remove slow varying areas across time and space, like the body. Remove noise with a Weiner filter. Threshold the image to obtain the subject’s wings.

(3) Separate the body from the wings: take the difference between the images from step 1 and 2. Apply a closing transformation to the difference image to remove salt and pepper noise. Find the largest blob, which usually consists only of body pixels. Fit an ellipse to the blob.

Based on the ellipses across all images, we reconstructed a line segment running from head to abdomen tip that represented the body orientation, using the following algorithm. (1) Smooth the time-varying parameters of the ellipse (center location, angle of major axis with the horizontal, length of major and minor axes) across images by low-pass filtering the corresponding Fourier-transformed time series (low frequency thresholding at a cut-off frequency of 7.33 Hz). (2) Extract the line running through the major axis of each ellipse. (3) Based on the principle of epipolar geometry, rotate and scale each image so that for any point in one image, its matching epipolar line in the other image is horizontal (image rectification) (Hartley and Zisserman, 2008). The depth information about a point in 3D space can then be obtained from the disparity of its image points in each camera view. In one image of each stereo pair, pick the two points marking the head and abdomen tips at the intersection with the ellipse on the orientation line. (4) Find the matching head and abdomen tip points in the other image of the stereo pair as the intersection of the corresponding epipolar lines and the orientation line in that image.

Using the tracking methods described above, we obtained, for each tracked 3D point, two time series of points belonging to the two image spaces associated with the cameras. These pairs of 2D trajectories were further processed to reconstruct the underlying trajectory in 3D space. Generally, a point in 3D space can be reconstructed using images of that point from two (or more) separate views. Based on the affine camera assumption, we computed calibration camera matrices up to affine ambiguity using the factorization algorithm (Yekutieli et al., 2007; Hartley and Zisserman, 2008; Tomasi and Kanade, 1992). We then used a calibration object with ground truth points in 3D space to remove all ambiguity from these calibration camera matrices and obtain complete metric reconstructions (Hartley and Zisserman, 2008). The 3D coordinate system used to track the animal’s trajectory is illustrated in Fig. 1E, with the origin placed at the center of the visual stimulation screen, to the right of the flying subject in the test chamber.

To quantify the collision-avoidance behavior for loose-tether experiments, we first computed a confinement ellipsoid around the cloud of points representing the trajectory of the subject in the free-flight epoch using singular value decomposition (Trefethen and Bau, 1997). The exit point of the collision-avoidance behavior was defined as the point at which the trajectory exited an ellipsoid twice the scalar dimensions of the one computed from the free-flight epoch (doubled confinement ellipsoid, or DCE). Such an ellipsoid is illustrated in pink in Fig. 1E, where the red trace corresponds to the trajectory during the ‘free-flight’ epoch and the blue trace corresponds to the animal’s trajectory during the ‘encounter’ epoch. The direction of the collision-avoidance behavior was taken to be the vector from the center of the ellipsoid to the exit point of the collision-avoidance behavior. The animal’s response point, corresponding to the presumed onset of the collision-avoidance behavior, was defined as the point immediately preceeding the exit point where the trajectory’s curvature exhibited a local maximum. The corresponding time relative to projected collision in the looming stimulus sequence was called the response time. The curvature measures the deviation of the trajectory from a straight line. In two dimensions, the curvature is κ=1/R where R is the radius of a circle tangent to the curve at the considered point, as shown in Fig. 1F. This definition was adopted as it identifies the point where the trajectory last changed significantly in its course before leading to the exit point. During data analysis, we tested several alternative definitions of collision-avoidance behaviors; none of the conclusions presented below were affected by changes in the parameters around the values given above.

To factor out variability in the size and orientation of the confinement ellipsoids between and across subjects, as well as variability in the animals’ escape direction, we submitted the escape trajectories to the following normalization procedure: (1) the coordinates along the ellipsoid’s principal axes were rescaled so as to transform the free-flight DCE into the unit sphere; (2) all directional information was discarded by computing the animal’s distance from the ellipsoid center; (3) for each such distance trajectory, time was referenced relative to the exit point. After this normalization, all trajectory points within the DCE had a normalized distance (ND) smaller than 1 and time was negative along such a distance trajectory before it reached its exit point.

The tight tether experiments were analyzed in a similar fashion, except that the point cloud used was the free-flight trajectory in yaw and pitch space obtained from the subject’s orientation. As this space was two dimensional, ellipses (2D ellipsoids) were fitted.

We chose to focus on forewing kinematics as it had been shown previously that hindwing kinematics is unchanged during turning maneuvers (Robertson and Johnson, 1993a). To track wing deformations, on each forewing we physically painted four to five lines approximately equidistant from each other and perpendicular to a central line on the wing, running perpendicular to its base towards its tip. This yielded in each camera image plane four to five points on the leading and trailing edge of the wing, in addition to the two points corresponding to the extremities of the wing base and the point characterizing the wing tip. A coordinate system adapted to the wing was found by first fitting a reference plane to these 11–13 points, such that the plane was constrained to pass through the base of the wing. The distance from the wing base along the central line on the wing was normalized by wing length and thus varied from 0 (wing base) to 1 (wing tip). The elevation of the wing’s leading or trailing edge as a function of this normalized distance was defined by projection onto the plane perpendicular to the reference plane.

Next, we fitted the elevation of the leading and trailing wing edges relative to the reference plane to two separate polynomials of degree 3, denoted by p_l and p_t. As they were constrained to yield zero elevation at the wing base, each polynomial was characterized by three parameters, p_l/t(x)=a_l/tx+b_l/tx²+c_l/tx³, where x denotes the normalized distance from the wing’s base, and p_l/t(x) is the fitted elevation of the leading/trailing edge, respectively. The cordwise deformation of the wing was defined as: d_c(x)=[p_l(x)–p_t(x)]/l_c, where l_c is the average wing cord length. The spanwise deformation was defined as the derivative of mean elevation, d_l(x)=[p_l(x)+p_t(x)]/2. Geometrically, these definitions are equivalent to modeling the wing as a ruled surface spanned by p_l(x) and p_t(x), with the spanwise and cordwise deformations corresponding to the slope of tangent vectors along the generating curve of the surface, d_l(x), and perpendicular to it, respectively (Kühnel, 2006). A ruled surface can be visualized as a ribbon with the generating curve running through the center such that the sectional line perpendicular to the generating curve is straight. Moving to this framework allowed us to de-dimensionalize the wing and compare our results across different subjects.

To check the assumption that the wing surface approximates a ruled surface, we estimated the camber along one wing over an entire wing beat cycle in a single flight trial. For each of the four to five lines marked on the wing, this was done by computing the distance between its midpoint and the midpoint of the line defined by its two end points on the wing edges, normalized by cord length. The camber was always less than 9%, thus justifying our assumption. This result is in agreement with previously published camber data in straight flight (Walker et al., 2009a).

We further reduced the dimensionality of the cordwise and spanwise deformation spaces by principal component analysis (PCA) performed separately on these 3D vectors pooled across time and across 10 trials in five animals (two trials per animal). As illustrated in Fig. 8A,B, the first principal component in cordwise and spanwise deformation explained more than 85% of their respective variance.

This method of analyzing wing deformations is slightly different from earlier approaches, such as the strip method (Willmott and Ellington, 1997; Walker et al., 2009a; Walker et al., 2009b). Instead of focusing on building a detailed map of wing deformation across time using a large number of parameters, we reduced the number of parameters to only two per wing, one for ν_c and one for ν_s. In contrast, the strip method and its more advanced variants divide a wing into a number of cordwise strips, each of which is then individually characterized using an angle of incidence. Using our polynomial approximation, the cordwise wing deformation easily proxies for the angle of incidence. As far as we are aware, the spanwise wing deformation has no analog in the extant flapping wing literature. As in the original strip method, camber cannot be reconstructed using our method.

The wing parameters described above operate on a time scale of individual wing beats, while body trajectory and orientation parameters operate on substantially larger time scales. To compare these parameters, we smoothed out the effects of wing beats by first finding the sets of maxima and minima in each wing trajectory. We interpolated each set of points with a piecewise cubic Hermite polynomial and then took the average of both interpolated functions, as shown in Fig. 7C.

The stroke plane of a wing was approximated by using the first two principal components of the wing tip trajectory, with the stroke angle computed based on the first principal component.

Boxplot conventions follow those of McGill and colleagues (McGill et al., 1978). The central line of each box plot is the median; the top and bottom edges of the box are the upper and lower quartiles, q_u and q_l, respectively. The dashed lines and bounding whiskers extending from q_u and q_l represent the extent of the data up to 1.5(q_u–q_l) away from q_u and q_l. Outliers (e.g. Fig. 2B) are represented by crosses. Triangles represent 95% confidence intervals. We initially tested for multimodality of a data distribution by fitting Gaussian mixture models with a different number of modes using the expectation maximization algorithm (McLachlan and Peel, 2000). We then used the Akaike information criterion to estimate the most likely number of modes (Akaike, 1974). If the data distribution was deemed multimodal according to this criterion, we performed a more refined clustering using the k-means algorithm with the appropriate number of clusters determined using a silhouette graph (Seber, 1984; Kaufman and Rousseeuw, 2008). This last method was used because it is robust to eventual deviations from the Gaussian assumption, which were also tested graphically using quantile–quantile plots.

Locusts avoid collisions in flight even under highly cluttered conditions [see p. 216 of Uvarov (Uvarov, 1977)]. We studied the generation of such collision-avoidance behaviors by presenting looming stimuli to flying animals under minimally restrained conditions (Fig. 1). Each trial was separated in an initial, free-flight epoch followed by an encounter epoch, when the looming stimulus was presented.

During the free-flight epoch, the animals tended to occupy a restricted portion of space within the wind tunnel test section. We characterized this behavior by fitting a minimal confinement ellipsoid to each free-flight trajectory (see Materials and methods). On average, the volume of the confinement ellipsoid represented 0.0025% of the wind tunnel test section volume. Based on the clustering of ellipsoid centers in Fig. 2A, we can see that some individuals had a preference for a fixed location across trials when flying freely in the wind tunnel. The tendency to prefer a fixed location across trials was most apparent in subjects 5 and 6. In contrast, subject 2 had a wide spread of free-flight locations across trials, while subjects 1 and 3 were somewhere in between. Similarly in Fig. 2B, we can see a wide spread in the length of the longest ellipsoid axis (axis 1). The interquartile range when pooled across individuals varied from 30 to 70 mm, while the interquartile ranges for each individual were typically dissimilar from each other. In contrast, the axes directions showed a similar clustering across individuals, the pooled results of which are shown in Fig. 2C. As expected, the mean axes across all trials were roughly orthogonal to each other. A higher variability was observed for the two axes that tended to be laterally and vertically oriented (axes 1 and 3) and the smallest variability was found for the axis oriented in the wind direction (axis 2). On average, the aspect ratio of the ellipsoids, defined as the ratio of the longest to the shortest axis length was 2.1 (axes 2 and 3 had the same length on average).

We identified the occurrence of a collision-avoidance behavior when a subject crossed an ellipsoid twice the size of that used to confine the free-flight trajectory (DCE; see Materials and methods), as illustrated in Fig. 1E. According to this criterion, the fraction of trials leading to an escape behavior was high among our six subjects (68%). If we exclude one of the subjects (subject 5) that was a clear outlier in terms of its response rate, the overall escape rate increases to 76%. To obtain an upper bound on the false-positive rate, i.e. the fraction of trials where an escape behavior did not occur but may have been registered according to our criterion, we recorded trials where no stimulus was presented in the encounter period and analyzed them with the same detection algorithm. Based on 52 such trials, we obtained a positive response in 15% of the cases. Thus, we estimate that the true rate of escape behavior lies between 53 and 68% of the trials on average across subjects (between 61 and 76% of trials when excluding subject 5).

Fig. 3A–D illustrates the stimulus time course and subjects’ trajectories pooled across all animals and trials. The blue traces in Fig. 3B–D correspond to the example trajectory illustrated in Fig. 1E, with the blue circle indicating the onset of the behavior. In this example, the escape behavior had a fast onset time. When considered collectively, the trajectories illustrated in gray in Fig. 3B–D reveal no obvious pattern in the data, building a noisy band across time. One escape behavior with a slow onset time drawn from this homogeneous set is illustrated in dark gray. The mean trajectory (red) was equally uninformative as its standard deviation effectively bracketed zero for all three coordinates axes, corresponding to the center of the confinement ellipsoid. This is in contrast to the fast example trial (blue), which showed a clear deflection during the encounter epoch. Fig. 3E plots distances of the subjects from the center of the confinement ellipsoid after factoring out the variability of the free-flight trajectory and the direction of escape, and after aligning them in time at their exit point. This analysis reveals a much more coherent pattern in the data; the fast escape trajectory depicted in blue is now very close to the mean trajectory and to the slow escape trajectory depicted in dark gray. Both fast and slow trajectories are well within the standard deviation bounds of the data. Fig. 3E also reveals that the subjects’ excursions out of DCEs lasted on average over 500 ms with a lower standard deviation bound in excess of 100 ms. Thus, the animals’ escape behaviors were prolonged in time and represented substantial excursions from their free-flight behavior. Much of the variability seen in Fig. 3B–D must therefore derive from variability in the response times, response directions and free-flight confinement ellipsoids. We further analyze these causes of variability immediately below.

We turn first to the large variability in response times. On average, response times preceded exit times from the DCEs by 33 ms (s.d. 40 ms). From the data pooled across all trials in Fig. 3F, we can see that their interquartile range covers more than 2.5 s or almost half the looming period. Within subjects, we see equally large interquartile ranges, spanning 2–3.5 s. Because of this large variability, even within subjects, we could not obtain reliable estimates of their median response times as their confidence intervals lie outside the interquartile range (except for subject 6). The histogram in Fig. 3G also shows broad variability, with a very long tail stretching from –2 s to the start of the stimulus. A multimodality test based on Gaussian mixture models applied to this response time distribution indicated the presence of two modes. This was confirmed by k-means clustering and silhouette analysis (see Materials and methods; see also supplementary material Fig. S1), allowing us to split the 78 escape behaviors into 46 slow and 32 fast responses, as illustrated on top of Fig. 3F by the outlined and solid black circles. To investigate whether the trajectories of the fast and slow responses were significantly different, we compared their means and found no substantial differences (supplementary material Fig. S2). This analysis also revealed that the averaged trajectory may not be representative of true behavior in this data set. The median response time when pooled across all trials had a value of –0.5 s, which was before projected collision. This was 1 s earlier than the apparent response time computed from the averaged trajectories that occurred after collision. In 1/4 of the trials (29/115), the animals initiated their collision-avoidance responses after projected collision. This result is in agreement with earlier jump escape behavior experiments that yielded average escape times around projected collision time for a looming stimulus parameter l/|v| of 40 ms (Fotowat and Gabbiani, 2007). Larger l/|v| values, corresponding to slower approach speeds, yielded earlier response times and vice versa.

What strategy do the animals use to avoid collision with the simulated object? To address this question, we analyzed the statistics of the directions of response, pooled across all the trials. As shown in Fig. 4, the response directions were similarly varied and widely spread out, in both elevation and azimuth (Fig. 4A). The median responses were close to 0 deg, which was included in the confidence interval of the pooled data for both elevation (Fig. 4Ai,ii) and azimuth (Fig. 4Aiv,v). In the case of elevation, a distinct distribution peak at 0 deg indicates that staying level is the preferred strategy, while Fig. 4B shows no statistically significant preference for diving compared with climbing as the observed counts for these behaviors lie in the 95% confidence interval of a P=0.5 binomial model. This result also held true when the data were split into slow and fast component responses (P=0.11, 0.85 and 0.46 for fast, slow and all responses, respectively). In eight out of 115 trials, we observed a brief interruption in wing flapping, a behavior similar to gliding in natural flight. Furthermore, the eight trials were approximately equally distributed between fast (3/8) and slow responses (5/8; P=0.57, binomial test). These flap interruptions were treated as regular dives and included in the data set. In the case of azimuth, most of the responses were localized at the ends of the range, with a dip at 0 deg as shown in Fig. 4Av. The bimodal character of the azimuthal distribution of escape behaviors was validated statistically using the Gaussian mixture model multimodality test. Similar to the case of elevation, there was no statistically significant difference between banking away from and towards the stimulus as shown in Fig. 4C (binomial test; P=0.19, 0.09 and 0.37 for fast, slow and all responses, respectively).

These results suggest that subjects did not have a strongly stereotypical response timing and direction, even for a looming stimulus with fixed parameters. As we could not find any systematic relationship between escape behavior variability and free-flight variability, it is highly likely that escape behavior variability is generated on the spot as the looming threat becomes imminent. At first glance, such a high degree of variability may seem surprising, but it represents a reasonable survival strategy when facing an approaching predator, as predictability would certainly lead to capture.

How does wing motion relate to the changes in trajectories described above? To find out, we increased our spatial and temporal resolution of wing movements at the expense of restricting the subjects’ body movements in space by using a tighter tether setup. Although subjects retained the ability to move within 2.5 cm of a central reference position, in practice they preferred to stay close to it. Hence, we saw little translational motion under these behavioral conditions. Instead, the animals initiated turns with high probability (83%), as illustrated in the body orientation trajectories of Fig. 5. As earlier, the blue and dark gray traces in Fig. 5A–C highlight slow and fast example trials, respectively, with the onset of the response indicated by a circle of the same color, while the red traces and circle correspond to the average over all trials, which are depicted in gray. Once again, a great deal of variability both within and across subjects was observed, but the details were slightly different from the loose-tether setup. Thus, although the mean yaw and pitch were close to zero as observed previously for the animal’s position, we saw a large increase in their standard deviation after the projected time of collision. Such an increase was barely seen, if at all, in the loose-tether experiments and corresponds to escape behaviors consisting of large swings in yaw and/or pitch generated by the subjects around that time.

Fig. 5D reveals that the interquartile range of the pooled response times covers 4 s or 3/4 of the looming stimulus span, which was much longer than the range found in the loose-tether experiments. Except for subject 3, we saw a similarly large range in individual subjects. The median of the pooled response times was at –1 s which is 1/2 s earlier than in the loose-tether experiments. Yet, as illustrated in Fig. 5E, the observed response times were very similar to those of Fig. 3G. Specifically, the pooled distribution peaks around the time of collision with a long tail all the way to the start of the stimulus, as seen in the loose-tether experiments. The multimodality test based on Gaussian mixture models indicated the presence of two or three modes in the data distribution. Subsequent k-means clustering and silhouette analysis confirmed the existence of two distinct modes; their associated data clusters were highly non-Gaussian in shape (supplementary material Fig. S1). Fig. 5F depicts the normalized distance trajectories relative to their exit points. As in Fig. 3E, the blue trace of the example trial is close to the average escape behavior indicated in red. In addition, the normalized distance trajectories differ from those seen in the loose-tether conditions. Specifically, the animals’ excursions outside of the DCEs were smaller in amplitude and shorter on average, but hovered longer near its boundary. This may be due in part to the restoring force exerted on the animals by the stationary air stream in the wind tunnel under the tight-tether conditions. As in the loose-tether condition, there were only minor differences in the fast and slow trajectories, based on their means and standard deviations (supplementary material Fig. S3).

The distribution of the directions of the collision-avoidance responses, illustrated in Fig. 6A, was again widely spread out in yaw and pitch, but its general shape differed from that of the loose-tether experiments. The pooled medians in Fig. 6Aii and 6Aiv indicate a preference for slightly upward pitch as well as yawing away from the stimulus, although they were not significantly different from zero. Fig. 6B,C shows a similar trend, with significantly more yaw turns biased away and pitch turns biased upwards (binomial test; P=0.048 and 0.024 for all pitch and yaw responses, respectively; red asterisk in Fig. 6B,C, All). Fast escape responses showed a significant upward pitch bias (binomial test; P=0.002; red asterisk in Fig. 6B, Fast) while slow responses were unbiased (binomial; P=0.5). However, both fast and slow responses considered individually did not show a significant bias in yaw, in contrast to the pooled data (binomial test; P=0.08 and 0.11, respectively). The multimodality test based on Gaussian mixtures indicated that both pitch and yaw were bimodal, which is in line with the data being distributed within an annulus as seen in Fig. 6Aiii. At the individual level, we saw that subjects 2 and 5 had a bias towards a positive pitch change as their interquartile ranges were well above 0. For subject 5 only, we saw biased yaw changes away from the stimulus as well. Analogous biases across individuals were not seen in the loose-tether experiments. In five out the 53 trials, the animals briefly interrupted the flapping of their wings, which would have led to a ‘glide’ in the loose-tether experiments. All these trials belonged to the slow category, suggesting a predominance of such ‘glides’ immediately around collision time, although their number was slightly below significance (P=0.059, binomial test).

Thus, although the collision-avoidance behaviors in the two conditions were broadly similar in their range of response times, the spread in response direction distributions was different. We therefore conclude that the manner of tethering had some effect on the collision-avoidance response directions but very little effect on the timing of responses. Additionally, our results suggest that separate mechanisms may underlie fast and slow escape behaviors based on differences in the pitch response and the occurrence of ‘glides’ in tight-tethered experiments.

From the large scale changes in orientation observed in the tight-tether setup, we moved to the finer scale changes in wing motion during the collision-avoidance response. Out of 53 tight-tether trials, 10 (two per subject) were selected for wing reconstruction based on the visibility of wings in all camera views and the long time required for manual placement of points on the images. These trials formed a representative subset in terms of collision-avoidance onset times (Fig. 5D, red triangles). Fig. 7A–D shows a trial with a late collision-avoidance response, as was often observed in our data set. From the full yaw orientation trace in Fig. 7A, a smaller window was selected in Fig. 7B with accompanying forewing kinematics illustrated in Fig. 7C. This window was selected to contain a single yaw oscillation and varied from 284 to 530 ms across trials. In this case, our algorithm detected the animal’s collision-avoidance behavior ∼370 ms after the time of projected collision for the looming stimulus. This collision-avoidance behavior consisted of a change in yaw towards the stimulus (vertical dashed magenta line), preceded by smaller and followed by increasingly stronger oscillatory yaw turns towards and away from the stimulus. As illustrated in Fig. 7B, we manually reconstructed the body orientation in a small window around the collision-avoidance response time (dashed cyan lines). Fig. 7C shows the height of both forewings over the same time period and their smoothed version across wing beats (see Materials and methods). As can be seen from this panel, the onset of the collision-avoidance behavior is accompanied by a change in wing height. Specifically, the ipsilateral wing height increases while the contralateral wing height decreases over a time interval of ∼100 ms following the onset of the response. Calculating the correlation coefficient between yaw and the smoothed difference in forewing height (black trace in Fig. 7D), we saw that there was indeed a relationship between them around the collision-avoidance response time. Furthermore, the correlation is largest at a lag of ∼100 ms. As positive lag indicates that the smoothed wing height difference anticipates changes in yaw, this result was consistent with the change in wing height difference causing the change in yaw. The correlation between yaw and ipsilateral or contralateral forewing height similarly peaks around the same time with positive and negative extrema, respectively, consistent with the result for wing height difference. Fig. 7E illustrates the same finding in another trial during which the collision-avoidance maneuver was initiated 4437 ms prior to projected collision.

The results presented in Fig. 7 show that changes in forewing height are strongly correlated with changes in body orientation during collision-avoidance behavior. Similar results were observed in all 10 trials where detailed wing reconstructions were carried out (see also Fig. 9B below). By inspecting the movies of these collision-avoidance behaviors, we also observed that the subjects’ forewings typically underwent substantial deformations over the course of a wing beat cycle. We therefore analyzed how wing deformations could contribute to collision-avoidance behaviors in the same 10 trials, as illustrated in Fig. 8. For this purpose, we fitted the wings’ leading and trailing edges with third-order polynomials and used the ruled surface they spanned as a model of the wings (average root mean squared reconstruction error: 0.37 mm or 3% of the wing’s cord length). This allowed us to characterize wing deformation along its length by the two tangent vectors of the associated ruled surface in cordwise and spanwise directions. Both the cordwise and spanwise tangent vectors were 3D (see expressions for ν_s and ν_c in Materials and methods). As these vectors were time varying during wing beat flapping cycles, each of them followed a specific 3D trajectory. We investigated whether these trajectories could be dimensionally reduced by PCA. Fig. 8A,B shows that the first principal component of the cordwise and spanwise tangent vectors yielded a good approximation for the deformations that we encountered in the entire data set. The corresponding wing deformations observed over the effective range of first principal component values describing the wings are illustrated in Fig. 8C,D. These two panels depict the angles of the cordwise and spanwise tangent vectors along the wing, respectively. The associated spatial deformations are illustrated in one example in Fig. 8E,F. Based on this dimensional reduction, we could visualize the trajectory of wing deformations over the course of a given trial, as illustrated in Fig. 8G. The blue connected circles illustrate the trajectory before the onset of the collision-avoidance response and the red circles illustrate those after it. Clearly, these circles cluster in different regions of space, suggesting that changes in wing deformation are an integral part of the collision-avoidance behavior. We observed similar clustering in the nine other trials that were analyzed in the same manner, although the exact pattern and extent of clustering differed from trial to trial (Fig. 8H).

Next, we studied the inter-relationships between body orientation, wing motion and wing deformation in our data set. In Fig. 9, we applied the method of smoothing across wing beats introduced in Fig. 7 to calculate correlation coefficients between yaw and a larger set of smoothed wing kinematic variables, including ipsilateral and contralateral wing heights, the wing height difference and deformations of the ipsilateral and contralateral wings along the cordwise and spanwise directions. Fig. 9A,B shows that wing height difference correlated very strongly with yaw, more so than the ipsilateral or contralateral wing heights, as the median is the highest and the interquartile range lies mostly above the 0.75 mark. The stereotypical relationship in six out of 10 trials was for the difference in wing height to correlate positively at positive lag (Fig. 9C), while ipsilateral and contralateral wing heights correlated positively and negatively, respectively.

The relationship between yaw and the difference in wing height was the most dominant effect we observed based on correlation strength when compared with the other variables we tested, including pitch (not shown). The correlation between these variables amounted to 0.74 or 0.78 when absolute correlation values were considered. In contrast, the absolute correlation between yaw and all deformations was 0.55 (median taken first over all trials and then all deformations). As the difference in wing height and wing deformations was correlated as well at 0.45 (median over all trials and then all deformations), we computed partial correlations of yaw and wing deformations that took into account wing height differences, as illustrated in Fig. 9D–F. This allowed us to obtain the correlation between yaw and wing deformations that were not accounted for by an underlying change in wing height difference. Fig. 9E shows that the spanwise deformation of both wings correlated most strongly with yaw. Furthermore, the coefficients between deformations and yaw obtained after partial correlation were just as strong or stronger than those obtained before taking into account wing height difference (median over all trials and deformations, 0.54). This suggests that the effects of wing height difference and wing deformations were largely independent of each other. However, the correlation effect was not consistent across trials as both positively and negatively correlated trials were obtained (Fig. 9D), as well as trials where the lag between wing deformations and yaw was either positive or negative (Fig. 9F). The cordwise deformation variables showed a similar mixed variability across trials, but had weaker correlation strengths.

Flight is essential for grasshoppers and locusts to forage, reproduce and seek shelter from potential dangers. In their natural setting, where obstacles and predators abound, flight sometimes requires complex and rapid maneuvers, such as those mediating collision avoidance. In this work, we studied the collision-avoidance behaviors of loosely tethered locusts to a looming stimulus presented on the animal’s side, mimicking the approach of a predator on a collision course. Our results demonstrate that a wide range of possible escape strategies were applied by the animals. When their behavior was more tightly restrained, it was possible to relate some aspects of flight kinematics to the observed collision-avoidance behaviors.

To the best of our knowledge, this study is the first to investigate how looming stimuli presented from the side affect flight behavior in minimally restrained animals. When animals are fully restrained, the clearest response was a brief interruption in wing flapping, consistent with gliding that would allow the animal to dive in an attempt to avoid collision (Santer et al., 2005a). This gliding behavior was most reliably elicited by very fast looming stimuli.

In terms of the parameter l/|v| that fully characterizes the looming stimulus, the maximal frequency of gliding (70–80%) was reached for values of 10 ms or less, but rapidly decreased for larger values [fig. 2 of Santer et al. (Santer et al., 2005a); corresponding to a speed of 4 m s^–1]. At l/|v|=40 ms, the frequency of gliding behavior was reduced to ∼15%. In our experiments, which were also carried out at l/|v|=40 ms, brief wing flapping interruptions were observed between 7 and 9% of the time in minimally restrained and more tightly restrained animals, respectively. Thus, the glide frequencies observed in this study are broadly consistent with those of Santer and colleagues (Santer et al., 2005a). Additionally, our results suggest that glides become less frequent as the animals’ restraints are relaxed and that they tend to be more evenly distributed among fast and slow escape responses in conditions better approximating free flight.

Thus, glides may not be exclusively used as a last resort manoeuver in the animals’ escape repertoire. When the animal kept one rotational degree of freedom, the other reported response to side presentation resembled a banked turn away from the stimulus (93% of the time) (Ribak et al., 2012). In contrast, we observed a much wider variety of escape behaviors, both in our loosely tethered and in our tightly tethered conditions where 6 and 2 degrees of freedom were maintained, respectively. Specifically, we found that escape behaviors can consistently be subdivided into two subtypes, fast and slow, based on their onset time (Fig. 3G, Fig. 5E). Within each group, a substantial fraction of the variability in escape trajectory is explained by variability in the animal’s free-flight trajectory. A complementary variability component is escape direction, which is broadly distributed towards or away from the stimulus, either upwards or downwards, with a slight preference for upwards trajectories in fast escape behaviors of tightly tethered animals (Figs 4, 6). Yet, even after taking all these factors into account, residual variability remains (Fig. 3E, Fig. 5F).

This rich diversity of responses to looming stimuli presented from the side also contrasts with the behavior of tethered locusts presented with objects approaching from the front. In such situations, particularly when the objects have a lateral offset >2 cm from the animal, the avoidance behavior is fairly stereotyped, consisting of steering to the opposite side of the approaching object (Robertson and Johnson, 1993a). This differs from our results, as we saw escapes both towards and away from the looming stimulus. Thus, objects approaching from the front and from the side may trigger qualitatively different collision-avoidance responses. A likely reason is that a frontal approach will more probably be due to the subject moving towards a static obstacle (at least at the speeds used in earlier work), while a sideways approach will unambiguously be related to an object actively moving towards the animal. In this last case, the locust may increase its chances of survival by strategically eschewing ‘predictable patterns of response’(see Tzu’s The Art of War) (Camhi and Tom, 1978; Domenici et al., 2008).

The second set of behavioral experiments, carried out on more tightly restrained animals, allowed us to investigate the aerodynamic origin of collision-avoidance behaviors. After exploring a number of locust flight characteristics, we observed a high correlation between changes in forewing height asymmetry and changes in yaw. These correlations generally peaked when forewing height asymmetry led yaw, strongly suggesting a causal relationship. Changes in forewing height asymmetry have been related to changes in yaw during turning behavior in several past experiments (Thüring, 1986; Robertson and Johnson, 1993a; Kutsch et al., 2003). Thus, our results confirm that wing height asymmetry is often the main determinant of yaw turning behavior and specifically establishes its role in the avoidance of stimuli looming from the side. Yet, our results do not preclude other mechanisms of collision-avoidance behavior. Ribak and colleagues, for instance, consistently observed assymmetries in hindwing pronation during collision-avoidance behaviors resembling banked turns away from a looming stimulus (Ribak et al., 2012). In contrast, earlier work had reported little change in hindwing kinematics in fully fixed animals (Robertson and Johnson, 1993a). Hindwings are thought to provide about 2/3 of the animal’s lift (Jensen, 1956). Their use in collision-avoidance behaviors may thus depend on specific requirements for lift production at the time of the collision-avoidance behavior. In our setup, animals actively generated more than half of the required lift to remain aloft, which may have interfered with hindwing usage for collision avoidance.

Recently, the contributions of wing deformation to forward flight in tethered locusts have been analyzed in detail (Walker et al., 2009a; Young et al., 2009). A natural question was then whether and how such deformations may contribute to collision-avoidance behaviors to looming stimuli. In our experiments, we found substantial deformations of the front wings during their flapping cycle. When visualized as trajectories of their principal components, the time course of these deformations was qualitatively different before and after the onset of a collision-avoidance behavior. Furthermore, we found sizeable correlations between wing deformations and yaw. Interestingly, these correlations remained essentially unchanged when their relationship with wing height asymmetry was taken into account through partial correlations. Thus, our results strongly suggest that wing deformations play a role in the generation of collision-avoidance behaviors in addition to that played by wing height asymmetry. However, we also found substantial trial to trial variability that precluded a more definitive explanation of their contribution. The study of wing deformation is a relatively new direction in flapping wing aerodynamics, awaiting a more complete mathematical characterization. Our results indirectly point to their role in collision avoidance but a definitive connection will require a deeper understanding of their aerodynamic effects.

A comparison of collision-avoidance behaviors under loose- and tight-tether conditions revealed several differences. Under tight tether, we found a lack of response directions with small pitch or yaw values, substantial oscillations after escape behavior onset, asymetries between positive versus negative pitch (or yaw) responses, and a different temporal distribution of glide events. In loose tether, response directions with small pitch or yaw values are associated with the animal either accelerating or decelerating in the wind stream to avoid collision. This option is not available in tight tether, which explains the lack of such responses. Similarly, in tight tether, the restoring force exerted by the air stream likely contributes to amplify the oscillations that follow the onset of collision-avoidance behaviors. While the last two differences do not seem as simply related to tethering, neither of them pertains to the onset of yaw turns. As similar yaw turns are seen in loose and tight tether, we expect them to be generated similarly under these two conditions. Additionally, it is likely that other yaw turning maneuvers – not necessarily caused by a looming stimulus – will have similar characteristics, including a role for wing deformations.

One neuron that has been implicated in the generation of collision-avoidance behaviors in locusts is the DCMD, in conjunction with its partner in the ipsilateral nerve cord, the descending ipsilateral movement detector neuron or DIMD (Fotowat and Gabbiani, 2011). The peak time of the DCMD firing rate has been tied to jump escape behaviors as well as to the glides elicited by lateral looming stimuli discussed above (Fotowat et al., 2011; Santer et al., 2006). The DCMD is, however, less likely to be involved in frontal collision avoidance as its firing rate does not co-vary with the observed collision-avoidance behaviors (Gray et al., 2001). Given that the peak DCMD firing rate occurs around 100 ms before collision for the l/|v| value used here, it appears unlikely to contribute to those collision-avoidance behaviors, which occur substantially earlier in our experiments. Yet, this does not preclude a contribution based on other aspects of its firing rate (Fotowat et al., 2011). Other neurons, such as those involved in flight stabilization, would appear to be natural candidates to explain these behaviors as well (e.g. Hensler and Rowell, 1990). The description of collision-avoidance behaviors in flight we provide here should allow for detailed investigation of their neural basis, based on the extensive knowledge of the sensory–motor neural pathways of the locust (Burrows, 1996) and recent telemetry techniques for neural recordings (e.g. Fotowat et al., 2011).

In conclusion, we have shown that looming stimuli presented sideways to minimally restrained animals elicit both fast and slow escape behaviors whose diversity relies on several complementary factors, including free-flight variability, onset time and direction. Furthermore, the execution of these escape behaviors relies at least on changes in wing beat asymmetry and wing deformations. Thus, minimally restrained flight presents an attractive opportunity to study the neural basis and the aerodynamic mechanisms underlying the implementation of collision-avoidance behaviors in flight.

We thank Richard Dewell, Peter Jones and Rajeev Kumar for their insightful comments on our preliminary draft.