QUICK SEARCH:   [advanced] Author: Keyword(s):
Home     Help     Feedback     Subscriptions     Archive     Search     Table of Contents

First published online April 17, 2009
Journal of Experimental Biology 212, 1324-1335 (2009)
Published by The Company of Biologists 2009
doi: 10.1242/jeb.025502

This Article Summary Full Text Full Text (PDF) Supplementary Material Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Ristroph, L. Articles by Cohen, I. Search for Related Content PubMed Articles by Ristroph, L. Articles by Cohen, I. Social Bookmarking                         What's this?

Automated hull reconstruction motion tracking (HRMT) applied to sideways maneuvers of free-flying insects

Leif Ristroph^1,*, Gordon J. Berman¹, Attila J. Bergou¹, Z. Jane Wang² and Itai Cohen¹

¹ Department of Physics, Cornell University, Ithaca, NY 14853, USA
² Department of Theoretical and Applied Mechanics, Cornell University, Ithaca, NY 14853, USA

View larger version (11K): [in this window] [in a new window]   Fig. 1. An experimental assembly for filming free-flying insects. (A) Three orthogonal cameras C aim toward a focal volume in a flight chamber FC, magnifying the image with bellows B and zoom lens L. Opposite each camera is a film slide projector P that illuminates the chamber. (B) The cameras are triggered to begin filming when crossed laser beams are broken by the flying insect. A laser L emits a beam that diverges at a beam-splitter BS and is re-routed by mirrors M to intersect through the flight chamber. Beam expanders BE inflate the beam to the size of the focal volume, and photodiodes PD detect the beam breakage. Simultaneous breakage of the beams initiates filming via a modified Schmitt trigger switching circuit (not shown).

View larger version (38K): [in this window] [in a new window]   Fig. 2. Aligned silhouettes are rendered by image processing and registration. (A) The three orthogonal cameras provide images of a fruit fly in flight (top). To obtain silhouettes from these raw images, a background picture is subtracted and the resulting image is thresholded (bottom). (B) Because the cameras are not perfectly aligned, the pixel coordinates in different views may not correspond to the same spatial coordinate. In order to register the images, we form a minimal bounding rectangle around the shadow in each view and then shift and scale the images such that the rectangle corners are consistent between views.

View larger version (23K): [in this window] [in a new window]   Fig. 3. Visual hull reconstruction forms a 3D shape that is consistent with the three silhouettes. Our implementation seeks 3D volume pixels, voxels, that project onto the silhouette in each view. Hull reconstruction is equivalent to the exercise of first forming extended 3D shadows from the silhouettes (A), and then finding the intersection in space of these extended shadows (B). The resulting object is the visual hull of the insect, the largest volume shape that is consistent with the three silhouettes. The hull data consist of an array of voxel coordinates.

View larger version (18K): [in this window] [in a new window]   Fig. 4. The body, right wing and left wing of the insect are identified by applying a clustering algorithm. The top view (A) and two side views (B and C), show that the right (red) and left (dark blue) wings are clearly distinguished from the body (light blue).

View larger version (24K): [in this window] [in a new window]   Fig. 5. The positions and orientations are extracted for each of the body, right wing and left wing. The centroid is defined to be the mean of the voxel coordinates for each respective grouping. (A,B) To identify the three Euler angles of the body, we define two vectors on the body. The first is the axial unit vector, Â, which is found by applying principal components analysis (PCA) to the body voxel coordinates and gives the yaw angle, , and the pitch angle, β. The second is the lateral unit vector, , that runs from the insect's right to left and is identified as the normal to the plane formed by the centroids of the head, thorax and abdomen clusters. (C) The roll angle, , is the angle between and the unit yaw vector, . (D) For each wing, the span vector, , is identified by PCA and gives the stroke angle, , and the stroke deviation angle, . (E) The chord vector, , is parallel to the longest diagonal of the parallelogram cross-section of the wing hull. (F) The wing pitch, , is the angle between and unit stroke vector, . For other definitions, see Table of abbreviations.

View larger version (49K): [in this window] [in a new window]   Fig. 6. A test of the automated tracking algorithm on a computer model of a flapping fly. (A) A morphologically appropriate model fly consists of five ellipsoids. Three ellipsoids form the head, thorax and abdomen of the body, and two flat plates represent the wings. The wings act as three degree-of-freedom hinges that rotate about a point on the surface of the thorax. (B) Measured flapping motions are imposed on the wings of this model fly, the shadows in each of three views are generated, and finally the tracking algorithm is run on these shadows. For this case, the body is held fixed at a typical orientation of (, β, )=(0, 59, 0) deg. (C) A comparison of the imposed body position (open circles) and the measured position (filled circles) for the centroid (x, y, z). A histogram of the residuals, measured value minus the actual value, is shown to the right for each coordinate. The reconstruction method measures the body centroid to within the voxel size of 2 pixels. (D) A similar comparison for the body orientation angles reveals an accurate recovery, with errors of a few degrees. (E) The right wing centroid is recovered to within 2 pixels. (F) The right wing orientation angles can be resolved to better than 5 deg. The left wing shows similar statistics.

View larger version (8K): [in this window] [in a new window]   Fig. 7. Errors in the wing angles are nearly independent of phase in the wing stroke. To arrive at the displayed mean and standard deviation of residuals, we orient the model fly, impose wing motions and measure errors in each wing orientation angle. Left and right wing residuals are similar, so we lump these data together. Residuals in each angle are plotted as a function of the imposed stroke angle. The stroke angle b is measured in the body frame such that b is approximately –90 deg. at the dorsal flip and b is approximately 50 deg. at the ventral flip.

View larger version (31K): [in this window] [in a new window]   Fig. 8. Errors in the recovered coordinates depend on the body orientation relative to the cameras. To reveal this dependence, we set the model fly of Fig. 6A in various orientations and measure the residuals in all coordinates. (A) Dependence of body position error on typical values of , β and . In each plot, the residuals of the three position variables x, y and z are plotted next to one another and are shaded differently. The errors show little dependence on orientation and are generally smaller than the voxel size of 2 pixels. (B) Errors in body orientation as a function of orientation. The body roll is more difficult to resolve than and β and becomes particularly error prone when the body is rolled considerably. As might be expected, heading is highly inaccurate when the insect is pitched up vertically near β=90 deg. (C) The right wing position is generally resolved to within 2 pixels. (D) The right wing orientation is accurate to within 3 deg. for most typical orientations of the body. For high pitch β and high roll , the wing pitch, , is not as well resolved. The left wing has similar error statistics.

View larger version (13K): [in this window] [in a new window]   Fig. 9. Fruit flies undergo lateral acceleration during two maneuvers. Lateral acceleration is the horizontal component of acceleration that is perpendicular to the insect yaw direction. (A) Top view of a `dodge' maneuver. The fly yaw orientation is indicated by the black arrowheads, and the horizontal component of acceleration is shown as the red vectors. During the dodge, the insect moves from one forward trajectory to a nearly parallel forward trajectory. (B) To execute the dodge maneuver, the fly accelerates leftward and then rightward while moving forward. (C,D) In this `sashay' maneuver, the fly initially generates a large rightward acceleration that switches to become leftward near the end of the maneuver. Here, the lateral acceleration is as large as 0.4 g. Lateral acceleration is calculated from the body position and orientation data using a window-averaging method for differentiating noisy data (A.J.B., L.R., G.J.B., I.C. and Z.J.W., manuscript in preparation).

View larger version (54K): [in this window] [in a new window]   Fig. 10. Wing orientation angles for two maneuvers. In both cases, the lateral acceleration crosses through zero, and we display wing orientations near such a transition. For the dodge maneuver, the lateral acceleration is leftward before time t 0.075 s and rightward thereafter. For the sashay maneuver, the lateral acceleration is rightward before t 0.033 s and leftward thereafter. (A–C) The time course of b, and for the dodge. In order to facilitate comparison of the right and left wings, we have plotted the body frame stroke angle, b. (E–G) The time course of b, and for the sashay. In both maneuvers, the kinematic data reveal that the wing motion consists of a flipping motion of the wings superposed on the flapping back and forth. Asymmetries in the right (red) and left (blue) wing motions are associated with lateral acceleration. (D,G) These asymmetries lead to differences in the aerodynamic angle of attack, , the angle between the chord and the instantaneous wing velocity. This angle is calculated from the other wing orientation angles and has typical errors of 5–8 deg.

View larger version (29K): [in this window] [in a new window]   Fig. 11. A drag-based mechanism of lateral force generation. Fruit flies primarily flap their wings back and forth with the upstroke and downstroke separated by rapid wing flips at the stroke reversal. (A) Four snapshots of the wing orientations near stroke reversal for flight with no lateral acceleration. When no lateral force is produced, the wing motions are nearly symmetrical between left and right wings. (B) When the insect is accelerating to its left, the right and left wings have different angles of attack, as evidenced by the different projected areas of the wings in this top view. (C) An idealized representation of the wing motion that generates leftward force. By selecting different angles of attack for the two wings near stroke reversal, asymmetric drag forces lead to a lateral force imbalance. (D) This asymmetry can be simply actuated by having the left wing rotate prior to the right, consistent with the timing difference observed in the angle of attack data for laterally accelerating fruit flies.

View larger version (8K): [in this window] [in a new window]   Fig. 12. Lateral acceleration is correlated with the timing difference in the angle of attack of the right and left wings. Each point represents a single wing stroke during the dodge (open circles), sashay (black circles), and three additional sideways flight maneuvers (red, blue and green circles). The timing difference, t, is the shift in time between the right and left wing angles of attack, R and L, and has been normalized by the flapping period, T. The value of the lateral acceleration, a, is the average during each wing stroke and has been normalized by gravitational acceleration, g.

View larger version (19K): [in this window] [in a new window]   Fig. A1. Comparison of coordinates tracked by the HRMT method and a manual method. Over 200 frames from the dodge sequence are tracked by both methods, and the differences in the measured coordinates are plotted as a histogram. Comparisons are displayed for the body centroid position (A), the body orientation (B), the right wing centroid position (C), and the right wing orientation (D). The left wing shows similar statistics to the right wing. The mean differences in position coordinates are as high as 8 pixels. With the exception of the roll angle, orientation angles recovered by the two approaches are similar, with no mean difference greater than 4 deg.

CiteULike

Complore

Connotea

Del.icio.us

Digg

Reddit

Technorati

Twitter