Flies exhibit extraordinary maneuverability, relying on feedback from multiple sensory organs to control flight. Both the compound eyes and the mechanosensory halteres encode angular motion as the fly rotates about the three body axes during flight. Since these two sensory modalities differ in their mechanisms of transduction, they are likely to differ in their temporal responses. We recorded changes in stroke kinematics in response to mechanical and visual rotations delivered within a flight simulator. Our results show that the visual system is tuned to relatively slow rotation whereas the haltere-mediated response to mechanical rotation increases with rising angular velocity. The integration of feedback from these two modalities may enhance aerodynamic performance by enabling the fly to sense a wide range of angular velocities during flight.
Flies rely on feedback from a variety of different sensory organs to perform agile and robust flight maneuvers. Two sensory systems, the compound eyes and the mechanosensory halteres, are sufficiently complex to encode angular velocity about the three body axes during flight (Blondeau and Heisenberg, 1982; Dickinson, 1999). The inherent differences between the mechanisms of photo-transduction and mechano-transduction impose different constraints on the bandwidth of these two sensors. Photo-transduction involves a chemical cascade that intrinsically limits the rate of response. In contrast, due to the direct physical linkage between membrane deformation and the activation of ion channels, mechanoreceptors can respond rapidly. The way in which neural systems fuse information from sensory systems with different temporal dynamics is not well understood. A key step in understanding this integration is to determine the dynamic characteristics of each system.
Fruit flies are equipped with compound eyes that provide low spatial resolution visual information (Buchner, 1976). Although there is no precise measure of the flicker fusion rate for Drosophila, photoreceptor dynamics measured in other flies indicate that dipteran visual systems display unusually high temporal resolution (Autrum, 1958; Laughlin and Weckstrom, 1993). For instance, the flicker fusion rate of Calliphora was measured as approximately 250 Hz (Autrum, 1958). Downstream of the photoreceptors, motion-sensitive neurons in the visual system allow flies to track both small objects and large field rotations (Borst and Egelhaaf, 1989; Egelhaaf and Borst, 1993). In the blowfly Calliphora erythrocephla, Krapp and coworkers (1998, 1996) have shown that certain visual interneurons encode the optical flow fields that would be generated by self-motion such as forward translation, roll or pitch. Blondeau and Heisenberg (1982) measured the torque exhibited by Drosophila melanogaster in response to visual rotation about the three body axes. In each case, the fly generated a torque that would rotate it in the direction of the imposed stimulus. The magnitude of a fly's response to large field image rotation is not a function of the true angular velocity of the rotation, but instead a function of the contrast frequency, defined as the angular velocity of the rotation divided by the spatial wavelength of the image (for a review, see Srinivasan et al., 1999). This feature is thought to result from the intrinsic properties of elementary movement detectors, which measure temporal correlations of local luminance and not absolute image velocity (Reichardt and Poggio, 1976).
In contrast to the visual system, the halteres can potentially provide an accurate measure of angular velocity as the fly rotates in space. Halteres are small evolutionarily modified hind wings (Fig. 1A) that beat anti-phase to the wings and serve a purely sensory function during flight. Derham (1714) was the first to note that flies cannot maintain stable flight once their halteres are removed. The halteres beat through an amplitude of roughly 180°, in a plane reclined approximately 30° to the transverse axis of the body in both Calliphora (Nalbach, 1993) and Drosophila (Dickinson, 1999). As the fly rotates around the roll, pitch and yaw axes, angular velocity dependent Coriolis forces act on the beating halteres (Nalbach, 1993; Pringle, 1948). Campaniform sensilla and chordotonal organs at the base of the halteres are thought to encode strains generated as the Coriolis forces cause the haltere to deviate from the beating plane (Pflugstaedt, 1912; Pringle, 1948). Although Pringle proposed that halteres were sensitive solely to yaw rotation, it was later shown by Faust (1952) that flies adjusted wing kinematics in response to rotation around all three axes.
Interconnections between the halteres and the flight control system have been examined in the blowfly. Motor neurons controlling the neck muscles receive input from both haltere afferents and motion sensitive neurons in the visual system (Strausfeld and Seyan, 1985). Additionally, haltere mechanoreceptors provide synaptic input to the motor neurons of the steering muscle B1 (Fayyazuddin and Dickinson, 1996), an observation that has been repeated in Drosophila (Trimarchi and Murphey, 1997). In Calliphora, motor neurons of the haltere muscles also receive excitatory input from visual interneurons (Chan et al., 1998). Haltere-mediated behavioral responses include both head movements and changes in wing kinematics in Drosophila (Dickinson, 1999), Calliphora (Nalbach, 1993, 1994; Nalbach and Hengstenberg, 1994) and Lucilia (Sandeman, 1980). Although flight torques have never been measured during stimulation of the halteres, the observed changes in wing kinematics are consistent with compensatory reactions that would act to rotate the fly against the direction of imposed motion.
Our focus in this study is the interaction between feedback from the halteres and the compound eyes. Using a flight simulator, we decouple these sensory inputs and characterize their temporal sensitivity to imposed rotational stimuli. Our results show that the two systems are complimentary. Whereas the gain of the visually mediated response decays with increasing frequency, the haltere-mediated response rises with stimulus velocity. The fusion of information from these two modalities would result in a broader bandwidth for detection of angular velocity during flight.
Materials and methods
We collected data from a total of 80 female fruit flies Drosophila melanogaster Meichen, 2-4 days old. During the tethering process, flies were anesthetized on a 4°C Peltier stage and glued to the end of a short piece of 0.4 mm diameter tungsten wire. The flies were given a minimum of 1 h to recover from the anesthesia before being placed in the simulator. The heads of the flies were fixed with glue to prevent any relative motion between the head and thorax.
The flight simulator consists of a visual display mounted within a rotational gimbal (Fig. 1B,C). The wrap-around display, composed of 11 340 light-emitting diodes (LEDs), spans 316° horizontally and 88° vertically. Visual patterns, refreshed at a minimum rate of 1 kHz to accommodate the high flicker fusion rate of flies, were generated and controlled using a digital signal processor (Texas Instruments TMS320C6701 EVM). Three brushless d.c. motors attached to pulleys controlled the orientation of the visual display mounted within the gimbal (Fig. 1C). The gimbal was designed to achieve velocities up to 2000° s-1 and accelerations up to 20 000° s-2 around each of the three rotational axes. These specifications were based on velocity profiles that were experimentally determined by tracking fruit flies during free flight (Tammero and Dickinson, 2002).
Visual closed-loop flight
During experiments, the tethered fly was positioned in the center of the visual display using a micromanipulator. An infrared LED, mounted directly above the fly, illuminated the two photocells of a wingbeat analyzer beneath the fly (Götz, 1987; Lehmann and Dickinson, 1997). A shadow, created as the wings pass beneath the infrared (IR) light, falls onto the photocells, generating output signals that are proportional to the amplitude of each wing. The wingbeat analyzer also provides a precise measure of the wingbeat frequency. Previous work has shown that flies adjust their left and right wingbeat amplitudes to keep a dark vertical stripe centered frontally in their field of view, a behavior known as fixations (Götz, 1987). A closed-loop environment is created in which the output of the wingbeat analyzer is used to control the angular position of the dark stripe, essentially simulating the visual motion that would be generated by the flies' wingbeat adjustments. Flies flying in the arena under closed-loop conditions tend to display more robust behavioral responses than those operating in open-loop mode, a condition where the stimulus is not dependent on the behavior of the fly (Dickinson, 1999). The basic experimental procedure was to superimpose visual or mechanosensory rotation in open-loop while the fly performed closed-loop fixation of the vertical stripe.
During mechanical rotation experiments, the gimbal was oscillated in open-loop about either the roll, pitch or yaw axis. We allowed the fly to fixate a stripe in visual closed-loop during presentation of mechanical rotations. The angular position of the gimbal was modulated in a modified sinusoidal sweep, stepping through frequencies from 0.8-3.0 Hz in discrete increments of 0.2 Hz with an amplitude of ±32°, representing a peak angular velocity range of 160-700° s-1. Each frequency was presented for 5 cycles within the sweep. Each sweep was separated by a pause of sufficient length to allow wingbeat amplitude and wingbeat frequency to return to pre-trial values.
The visual open-loop stimuli consisted of striped spherical patterns presented on the cylindrical LED display. The motion of the pattern created the illusion of a sphere being rotated around the fly (Fig. 1D). To create behavioral conditions comparable to mechanosensory experiments, a dark stripe, which the fly fixated in closed-loop, moved independently of the open-loop spherical pattern. The pattern position was oscillated along a sinusoidal sweep that stepped in frequency from 0.1 to 3.9 Hz in 0.2 Hz increments. The amplitude of the image rotation was fixed at 45°, and thus the peak angular velocity changed according to the oscillation frequency of the visual stimulus. The oscillation frequencies within the sweep produced peak angular velocities ranging from 30-1100°s-1. One stimulus sweep was composed of six consecutive oscillation cycles at each frequency. Each fly was tested with at least five sweeps, alternated with recovery periods during which wingbeat parameters were able to return to pre-stimulus levels.
The data, sampled at 200 Hz, included right and left wingbeat amplitude, wingbeat frequency, the position of the vertical stripe, the orientation of the gimbal, and the position of the spherical pattern. The data were filtered digitally (zero phase delay) with a low-pass cut-off of 40 Hz to remove any high-frequency noise resulting from vibration of the motors. The relationship of the wingbeat analyzer output to the stroke amplitude is known to be linear over the operating region (Lehmann and Dickinson, 1997). Although we could not calibrate the wingbeat signal for each fly, in this study we used the output of the wingbeat analyzer as a measure of the relative behavioral responses to sensory input, and not for accurate measurement of stroke amplitude. For the group of flies comprising each experimental treatment, the standard deviations (S.D.) of the left and right wingbeat amplitude signals were calculated over a pre-stimulus period of approximately 10 s. The wingbeat signals of each individual were then scaled such that this pre-stimulus S.D. was normalized to that of the group. To evaluate the flies' behavioral response representing the control of roll and yaw, we focus on the difference between the amplitudes of the left and right wings, since a disparity generates torque along these axes. The behavioral response representing the control of pitch is the sum of the left and right wingbeat amplitudes, since a bilateral increase or decrease in total amplitude modulates flight forces around the pitch axis. The responses to each stimulus frequency were averaged over multiple trials. We used an FFT algorithm to determine the amplitude and phase of the sine curve that best fit the averaged response at each stimulus frequency. In the subsequent sections and figures, the amplitude of these calculated sine fits is referred to as the change in wingbeat amplitude (ΔWBA), whereas raw wingbeat amplitude data is labeled WBA. All analyses were performed using custom software written in MATLAB (Mathworks).
In response to a mechanical rotation about the roll axis, flies modulated the difference between the left and right wingbeat amplitude. When mechanically rolled to the right side, flies increased the wingbeat amplitude of the right wing and decreased the wingbeat amplitude of the left wing. Thus, as reported previously (Dickinson, 1999), the reflexes driven by mechanical rotation would function to counteract the imposed rotation. The magnitude of the fly's response rose linearly as the angular velocity of the roll stimulus increased (Fig. 2A).
Similarly, a visual rotation presented in the absence of mechanical stimuli elicited a compensatory response in wingbeat amplitude. In response to sinusoidal oscillations, flies modulated the difference between left and right wingbeat amplitude in a roughly harmonic pattern. As with mechanical rotation, the amplitude of the fly's response was a function of the angular velocity of the visual stimulus. However, when tested over roughly the same range of frequencies as that used for mechanical rotations, the magnitude of responses to visual roll decreased with increasing velocity (Fig. 2B), a trend that was consistent across flies (Fig. 3).
We tested the responses in wingbeat amplitude (left—right for roll and yaw, left+right for pitch) to mechanical and visual rotations across a wide range of angular velocities. The mean amplitude of the sine wave fits to the raw wingbeat responses is plotted against peak stimulus velocity in Fig. 3 For all three rotational axes, the visually elicited motor response behaved as a band-pass filter of the sensory stimulus. The response is highly sensitive to intermediate rotational speeds, but decays for both large and small angular velocities. In contrast, the response to mechanical rotation rose with increasing angular velocity for all three stimulus axes. Furthermore, whereas the visual responses were similar for yaw, pitch and roll, the response to mechanical oscillation varied significantly, depending on the axis of rotation. Of particular note is the relatively weak effect of mechanical yaw. This finding is consistent with the trend observed in previous studies measuring haltere-mediated equilibrium reflexes at a single frequency (Dickinson, 1999), although the relative magnitude of the yaw response is even smaller in the present study. It is unclear whether the weak yaw response is a due to the response properties of yaw-sensitive mechanoreceptors in the haltere, or to downstream sensory-motor circuitry.
The visually and mechanically elicited responses to comparable rotational stimuli are approximately 180° out of phase, as seen in the averaged response of a single fly to multiple presentations of rotation about the roll axis (Fig. 4). This phase disparity arises because flies rotate in the direction visual motion, attempting to stabilize the image, whereas they move against mechanical rotation, attempting to regain equilibrium. This phase difference is consistent with the reflexes being compensatory, because mechanical rotation of the fly's body will generate optic flow across the retina in the opposite direction.
One important determinant of behavioral response to motion of a periodic visual pattern in flies is contrast frequency (Götz, 1972). In an independent set of experiments at a constant oscillation frequency, we measured the effect of spatial wavelength on the behavioral response to sinusoidal pattern rotation about the yaw axis. For patterns with three different stripe widths (8.8°, 17.5°, 35°) we found no statistically significant difference in the magnitude of the response (analysis of variance, ANOVA; P=0.5290) (Fig. 5). Because our stimuli consisted of bars, and not sinusoidal functions of intensity, each stimulus comprised a series of contrast frequencies. However, most of the stimulus energy resides in the fundamental. Our three different stripe widths corresponded to contrast frequency fundamentals of 4.8 s-1, 2.4 s-1 and 1.2 s-1. Thus, our visually elicited equilibrium responses did not vary significantly over a fourfold range in contrast frequency.
In addition to stroke amplitude adjustments, flies also modulated wingbeat frequency during compensatory reactions to visual and mechanical rotation. Fig. 6 shows the responses of both wingbeat amplitude and wingbeat frequency to pitch rotations. When subjected to mechanical pitch at two different frequencies, the fly's wingbeat amplitude response was larger for the faster oscillation (Fig. 6, top row). In contrast, the amplitude of wingbeat frequency modulation was smaller for the higher oscillation frequency. In response to visual pitch, modulation of both wingbeat amplitude and frequency decreased as the stimulus frequency increased. This result was consistent for all flies and indicates that the amplitude of wingbeat frequency modulations decreases with increasing stimulus velocity, even in cases when the wingbeat amplitude response rises (Fig. 7). This suggests that either wingbeat amplitude and frequency are controlled independently or that the wingbeat frequency response is dynamically limited.
The results of these experiments indicate that feedback from multiple sensory modalities drives equilibrium motor reflexes in fruit flies. Flies subjected to either mechanical or visual rotations about each of the three body axes respond by modulating the relative amplitude of their left and right wingstrokes (Fig. 2). Flies tend to track visual motion and counteract mechanical rotation, thus responses to these two stimuli are offset by 180°. This phase offset indicates that both responses act as compensating reflexes to oppose externally applied rotation (Fig. 4). However, the dynamic responses of the two systems differ in that the visually mediated response is strongest for slow rotations, whereas the haltere-mediated response is maximal during fast rotations (Fig. 3). In contrast, the dynamics of the wingbeat frequency response are similar when driven by either visual or mechanical stimuli (Fig. 6).
The main limitation of these experiments is the necessity of analyses restricted to tethered flight conditions. Sensory feedback to the halteres and other organs is attenuated under tethered conditions. Consequently, the response of a tethered fly may be quite different than that of the animal in free flight, making extrapolations between the two conditions difficult. Further, although we can measure wingbeat amplitude and frequency, we have no information about many other significant stroke parameters such as angle of attack and deviation of the wing from the stroke plane. Sensory-motor control of these parameters may play a significant role in equilibrium reflexes. Despite these limitations, we have observed robust behavioral responses that constitute a good, if not complete, measures of flies' sensitivity to imposed visual and mechanical rotations.
Filter characteristics of the halteres and visual system
Both the visual system and halteres act as band-pass filters of sensory stimuli. The visually elicited response decays both for fast and very slow rotations (Fig. 3). Similar results were found in Hengstenberg's study of visually elicited head roll responses in Calliphora (Hengstenberg, 1991). In a study of the optomotor responses in Drosophila, a decrease of the visual mediated torque was seen at slow speeds for both roll and pitch (Blondeau and Heisenberg, 1982). In the present study, we found that the wingbeat amplitude response to mechanical rotation decayed linearly with decreasing stimulus velocity. This decay was expected because the Coriolis force on the haltere is linearly proportional to the angular velocity of the stimulus. Hengstenberg found that the haltere-mediated head roll response was insensitive to roll rotation below 50° s-1 in the absence of visual cues. Although the haltere-mediated responses rise with stimulus velocity, this reflex also possesses band-pass characteristics because mechanosensory neurons must eventually fail as stimulus frequency increases. A measure of the upper limits of halteres responses would provide further insight into the significance of haltere feedback in flight maneuvers. Unfortunately, we could not operate our apparatus fast enough to map this upper limit.
Function of multi-sensory feedback in flight performance
In all closed-loop control systems, the performance of the overall system is dependent on the dynamics of the sensory feedback channels. Combining the fast mechanosensory halteres with a slower visual system may allow the fly to optimize bandwidth without sacrificing sensitivity. In addition to simply extending the bandwidth of equilibrium reflexes, a visually independent source of feedback would enable a fly to distinguish motion of its external environment from that generated by self-motion. Along with their role in equilibrium reflexes, the halteres probably provide critical feedback during active flight maneuvers. The flight trajectories of many flies consist of short straight flight segments punctuated by rapid turns called saccades (Collett and Land, 1975; Zeil, 1986). The extended high frequency response of the haltere system might enable flies to achieve these straight segments by detecting quick perturbations beyond the range of the visual system. Drosophila saccades vary little in amplitude, and are typically 90° to the left or right (Tammero and Dickinson, 2002). Although the stereotyped nature of saccades might result from a simple feed-forward motor command, evidence suggests that haltere feedback plays a role in terminating free flight saccades, which are significantly shorter than their tethered flight analogs (Mayer et al., 1988). Angular velocity exceeds 1000° s-1 during free flight saccades (Fry and Dickinson, 2001; Tammero and Dickinson, 2002). As shown in Fig. 3, the visual feedback is considerably weaker than haltere-mediated responses at speeds greater than 500° s-1. Although we could not measure the haltere response at speeds above 800° s-1, there is good evidence that the halteres encode angular velocities well above 1000° s-1. For example, Hengstenberg (1988) has shown in Calliphora that the maximal halteremediated head roll response occurs at 1500° s-1. Thus it is likely that haltere feedback plays an important role in regulating saccade amplitude.
In addition to the compound eye and haltere systems, flies receive visual feedback from ocelli, as well as mechanosensory feedback from specialized receptors on the neck, wings, legs and antennae. Although the role of these sensory systems in flight is not as well characterized as that of the visual system, they likely provide important feedback about the relative motion between the fly's body, its head and its environment. For example, as in other insects, the ocelli are thought to play a role in the orientation of the fly with respect to the sky (for reviews, see Mizunami, 1999; Stange, 1981; Taylor, 1981). Asymmetrical stimulation of the ocelli in Calliphora will elicit a transient head roll response as the fly tries to correctly position itself with respect to light (Hengstenberg, 1993). Additionally, campaniform sensilla at the base of the wing sense asymmetrical wing load and elicit compensatory head movements (Hengstenberg, 1988).
Convergence of visual and mechanosensory feedback on the flight motor
The neuroanatomy of the halteres and visual system, as well as of the flight motor, provides insight into the functional interaction between these feedback channels. In Calliphora, lobula plate tangential neurons synapse with a subset of motor neurons controlling neck muscles (Strausfeld and Seyan, 1985). Also in Calliphora, there is physiological evidence that visual interneurons project to the motor neurons of haltere control muscles (Chan et al., 1998). Motor neurons controlling two of the 17 wing steering muscles (B1 and B2) and descending neurons from the visual system are dye-coupled in male flesh flies, Neobellieria bullata (Gronenberg and Strausfeld, 1991). Thus, the combination of evidence from anatomical and physiological studies indicates that descending visual interneurons contact motor neurons in all three thoracic segments, controlling the motion of the head, wings and halteres.
Studies suggest that feedback from haltere afferents influences motor activity in flies. Neck motor neurons in Calliphora receive input from haltere afferents (Strausfeld and Seyan, 1985). This connection is quite fast, with latencies from haltere stimulation to activity in the neck motor neurons of approximately 2-3 ms (Sandeman and Markl, 1980). The steering muscle B1 receives electrical synaptic input from haltere afferents in both Calliphora (Fayyazuddin and Dickinson, 1996) and Drosophila (Trimarchi and Murphey, 1997). In both species, B1 is known to control changes in wingbeat amplitude (Heide and Götz, 1996; Tu and Dickinson, 1996).
Whereas the pathways by which visual and haltere feedback influence wingbeat amplitude have been at least partially identified, the circuits controlling wingbeat frequency are not as well understood. Specialized pleurosternal control muscles (ps1 and ps2) are thought to alter wingbeat frequency via changes in the mechanical resonance of the thorax (Kutsch and Hug, 1981; Nachtigall and Wilson, 1967). The firing patterns of these muscles correlate with changes in the wingbeat frequency of Calliphora and Muscina (Kutsch and Hug, 1981; Nachtigall and Wilson, 1967). The maximum firing rate of pleurosternal muscles is 20 Hz, approximately one tenth of the firing rate of B1 (Kutsch and Hug, 1981; Nachtigall and Wilson, 1967). The relatively slow firing rate of ps1 and ps2 might explain in part the slow adjustments of wingbeat frequency in the flight responses. Although the factors that limit the bandwidth of the wingbeat frequency response are not entirely understood, these results imply that this response is qualitatively different from the wingbeat amplitude response.
In summary, we have characterized the dynamics of the visual and mechanosensory systems as a step towards understanding how flies integrate these two sensory modalities for flight control. Our results show that the halteres are more responsive to fast rotations, while the visual system is more sensitive to slow rotational stimuli. This study on isolated visual and mechanosensory responses will serve as a basis for future work on more natural, concurrent, sensory stimuli.
The authors wish to thank J. Birch, L. Bentley, M. Frye and L. Tammero for their helpful comments on this manuscript. We would like to acknowledge the work of E. Vaaler (Vaaler Engineering) who designed the mechanical structure of the flight simulator, and D. Smith who built the visual display. Finally, we are grateful to A. Flynn for her technical contributions to the project. This work was supported by grants from the National Science Foundation (FD97-23424), the Packard Foundation and DARPA (N00014-98-1-0855).
- © The Company of Biologists Limited 2003