3D reconstruction and analysis of wing deformation in free-flying dragonflies

Most of nature's fliers are equipped with flexible wings. Wing flexibility is of interest in both aerodynamics (Combes and Daniel, 2001; Sane, 2003; Wootton, 1993; Zhao et al., 2010; Zhao et al., 2011) and structural dynamics (Combes and Daniel, 2003a; Combes and Daniel, 2003b). It is widely believed that wing flexibility and deformation can potentially provide new mechanisms of aerodynamic force production, which presents itself as passive cambering patterns as a result of wing flexural stiffness, kinematics and fluid-structure interactions not present in completely rigid wings (Combes and Daniel, 2001; Sane, 2003).

Insect wings are of particular interest because of their small size and the wide variety of passive cambering patterns they undergo as a result of wing and fluid interaction. The role that wing venation plays in these deformations has been studied elsewhere (Combes and Daniel, 2003a; Combes and Daniel, 2003b). It was shown that wing flexural stiffness varies by four orders of magnitude across insect taxa, wing flexibility strongly correlates with absolute wing span and chord length accounts for more than 95% of the observed variation in flexural stiffness for insect wings.

As pointed out previously (Walker et al., 2009b), high-speed photogrammetry is more suitable for studying wing deformation than the 'strips' method in which wings are modeled as a series of chord-wise strips (Willmott and Ellington, 1997) because the latter cannot capture wing camber. Photogrammetry also has advantages over the projected laser line method in which static lasers are used to project a fringe pattern onto flapping wings (Song et al., 2001; Sunada et al., 2002; Wang et al., 2003; Zeng et al., 1996; Zeng et al., 2000) as specific points on the wings cannot be tracked because of the fixed laser alignment.

Photogrammetry was used to study hummingbird wing kinematics in free flight (Hedrick and Biewener, 2007; Tobalske et al., 2007) and 3D fish fin kinematics in forward swimming have been successfully recreated using photogrammetry-based reconstructions (Lauder et al., 2006). In addition, bats have been filmed flying in a wind tunnel, revealing that bat wing kinematics change predictably with time (Lindhe Norberg and Winter, 2006). Photogrammetric reconstructions of free-flying bats were also used to quantify the effects of flight speed and reconstruction accuracy on the dimensional complexity observed in reconstructed bat wing kinematics (Riskin et al., 2008). The inertial effects of wing movement in the flapping flight of a fruit bat have also been studied previously (Iriarte-Díaz et al., 2011).

More recently, photogrammetry has been used to study insect flight, which is more challenging because of their small size, the faster flapping rate and the fact that it is not feasible to place large reflective markers (like those used in the studies of bats) on their tiny wings. For instance, Ristroph and colleagues used automated volumetric hull reconstruction on the images output from a photogrammetry system to study sideways free flight of Drosophila (Ristroph et al., 2009). In addition, Fontaine and colleagues studied free-flying Drosophila using automated model-based tracking for analysis of the wing-body kinematics during voluntary and escape take-offs (Fontaine et al., 2009). However, in both cases, all wings were treated as rigid plates instead of deformable ones because of a lack of spatial resolution, although the importance of deformations in the study of wing kinematics was pointed out in both papers. In another study, three orthogonally aligned high-speed cameras were used to study the wing kinematics and aerodynamics of hovering droneflies (Liu and Sun, 2008). Walker and colleagues used a four-camera photogrammetric system and a combination of manual and semi-automatic tracking of natural features and marker points to reconstruct the wing kinematics and surface topographies of free-flying hoverflies and tethered locusts (Walker et al., 2009b).

Despite some recent quantitative analysis of the wings of tethered (Sunada et al., 2002; Walker et al., 2009a) and free-flying (Bergou et al., 2010; Walker et al., 2010) insects, there is still a lack of literature regarding detailed 3D measurements of wing deformation in both the span-wise and chord-wise directions. This is partially due to the small wing size, fast motion of the wings, and unpredictable movement of free-flying insects. All these factors make high-speed tracking of the details of wing flexion extremely challenging. In most high-speed photogrammetric studies, subjects have been tethered, with few efforts made at capturing true free-flight dynamics. Those who have studied free-flying insects have made assumptions about flexibility such as treating the leading edge as rigid and ignoring span-wise flexibility altogether in their kinematics measurements.

Unsteady effects of wing rotation have previously been established as a key component of force production in insect flight (Dickinson et al., 1999). More recently, several studies have begun to reveal the additional aerodynamic significance of wing deformation details that would not be captured if rigid wing assumptions were made in either insect wing reconstruction or kinematic analysis. For instance, the role of chord-wise camber in improving aerodynamic and mechanical performance has been investigated (Du and Sun, 2008; Du and Sun, 2010; Vanella et al., 2009). In addition, it has been demonstrated that span-wise flexibility, a phenomenon that has been almost completely ignored in the study of insect flight, also plays a role in power consumption (Stanford et al., 2010). Wing flexibility has also been shown to play a role in leading edge vortex stability (Koehler et al., 2011; Zhao et al., 2011). These results highlight the need for insect wing surface reconstruction and deformation analysis methods that capture and utilize all aspects of wing deformation while remaining general enough to be applied to a wide variety of free-flying insects.

Several previous studies have also looked at the mesosurface morphological details, such as corrugation, found in insect wings; however, these studies have focused primarily on static wings. Laser scanning was used to measure the surface roughness of severed insect wings (Tsuyuki et al., 2006). The structure of a dragonfly forewing and hindwing were studied by scanning them with a micro-CT scanner (Jongerius and Lentink, 2010). Corrugation in flapping insect wings was also studied (Walker et al., 2009a). This work used a large number of marker points (approximately 100 per hindwing) for a tethered locust. However, tracking a high number of marker points in free-flight studies is undesirable. Currently, the relationship between 3D surface corrugation and induced airflow remains unclear. This further motivates the need for a simple method for modeling insect wing surface corrugation in free-flight reconstructions.

This paper presents a unified methodology for the reconstruction and analysis of the deformations and surface morphology exhibited by insect wings during free flight. The primary goals of the proposed techniques are to eliminate all rigid wing assumptions during the reconstruction and deformation analysis while minimizing the number of tracked points in the output of the photogrammetry system. Also, it is desirable for the methodology to remain general enough to be applied to a host of different free-flying insects, where the only limitations come from the application of marker points on the wings and the magnification and speed of the cameras, which can be partially overcome by moving to different lens options.

The dragonfly Erythemis simplicicollis (Say 1839) was selected as the test subject for the proposed methods because it is a quad-wing flier whose wings undergo considerable out of phase motion, an aerodynamically relevant characteristic (Lehmann, 2008), and also because its relatively large wings with high aspect ratios result in distinct twist and camber changes while flapping. These characteristics make it one of the most challenging insects to reconstruct digitally. It should also be noted that dragonflies are of particular interest to us because their extreme agility, their ability to fly with damaged wings and their ability to hover for long periods of time have led to their emergence as a model organism for the development of micro air vehicles.

We begin the discussion of our methods with an overview of the high-speed photogrammetry system that was used to capture the dragonfly's motion in multiple 2D image planes and a conceptual look at the projection equations used to convert corresponding points in these images into a 3D coordinate space. Next, a brief explanation of how specimens were selected and prepared for filming is given. A novel template-based hierarchical subdivision surface method for 3D wing reconstruction is then introduced. Specifically, Catmull-Clark subdivision surfaces were chosen as the cubic spline surface representation because of their ability to generate smooth surfaces from meshes of arbitrary topology. Reconstruction accuracy is then validated by projecting the reconstructed wings back into image space and comparing their outlines with expert segmentations of the wings in the original image data.

In the final portion of the paper, the focus shifts to making consistent measurements of the time-resolved wing deformations captured by the reconstruction algorithm. A least-squares plane is proposed as a reference plane comparable to the rigid wing assumptions that are often introduced when quantifying wing kinematics. Metrics for wing camber and twist in the chord-wise and span-wise directions are then defined relative to the aforementioned least-squares plane. An in-depth look at the time history of a wing's deformation during a stroke is presented and detailed visualizations of wing twist and camber are shown at several instants in time. The paper concludes by proposing a novel method to incorporate wing corrugation into the reconstruction by displacement mapping morphological surface details measured from static wings onto the flapping wings.

Object surface geometry has been defined on the macrostructure, mesostructure and microstructure levels based on the amount of detail present (Szirmay-Kalos and Umenhoffer, 2008). In the context of insect wing reconstruction we will use the terms macrosurface, mesosurface and microsurface to refer to the amount of detail captured by an insect wing reconstruction. The macrosurface of a reconstructed wing captures the kinematics as well as the deformations such as twist and camber undergone by the flapping wing. The mesosurface incorporates surface roughness details such as corrugation into the model. The microsurface captures tiny surface details not visible to the human eye, which we are not interested in for the purposes of this study.

The first section (below) details the first portion of the 3D wing surface reconstruction algorithm where the wing macrosurfaces are captured digitally. Dragonflies are first recorded in free flight by a photogrammetry system. Corresponding wing points from multiple calibrated cameras are projected into world space coordinates where subdivision surface wing templates are aligned to them. The resulting reconstructed surfaces are projected back into image space and compared with expert segmentations to evaluate accuracy. The key characteristics of the system as a whole are that it requires relatively few marker points per wing while still capturing wing twist and camber. Later in the Discussion section we propose a second phase of the reconstruction algorithm where mesosurface details, such as wing corrugation, will be incorporated into the model in the future.

The photogrammetry setup used for dragonfly image collection consists of three synchronized Photron FASTCAM SA3 60K high-speed cameras (Motion Engineering Company, Indianapolis, IN, USA) with 1024×1024 pixel resolution. They were aligned orthogonal to each other on an optical table and operated at 1000 Hz with at least a 1/20,000 s shutter speed to capture the dragonfly flight videos. The dragonflies were illuminated by three halogen photo optic lamps (OSRAM, 54428). The cameras were positioned 1.5 m away from the insects, giving a depth of field of 3-4 body lengths in all directions depending on the size of the specimen. Given that the dragonflies flap at approximately 42 Hz, this provides good temporal and spatial resolution. A comparison between the setup used for this study and those used in the most recent 3D reconstructions of insect flight is given in Table 1.

Our photogrammetry setup can be generalized to capture longer flights while still discriminating the marker points. Ignoring camera placement and lens parameters for simplicity, the cameras are limited by pixel resolution. The minimum marker point size is 1 pixel × 1 pixel. Assuming the marker points to be 500μm and the average body length of a dragonfly to be 4 cm, the maximum possible field of view without moving to other camera options is 12.8 body lengths. However, the amount of wing strokes that can be captured at one time is also dependent on the flight path and speed.

The cameras are linked using a hardware tether and synchronized using a master-slave system with a camera response time in the nanosecond range. They are triggered by a laser controller that transmits a 5 V transistor-transistor logic (TTL) signal to the system, which saves a portion of data from pre- and post-trigger. The laser trigger system is a custom-designed circuit that uses the open-source Arduino microcontroller platform to control all three cameras. Common laser diodes are used to provide focused beams of light to opposing light-sensitive resistors. At the intersection of these two laser beams, all cameras are focused, allowing a free-flying insect to trigger the cameras by interrupting the two beams simultaneously. This trigger system removes nearly all human error associated with triggering the cameras and minimizes human contact with the insects being observed. By automatically detecting when the subject is in optimum range, the current photogrammetry system is able to collect data that is consistently in focus. Useable segments of video are identified for reconstruction based on image quality and relevance of maneuver.

Fig. 1 shows a model of the perspective projections in the photogrammetric system used to estimate the 3D world coordinates of a point on a dragonfly's wing by finding its location in multiple projection planes. Generalizing the system, images are recorded with j=1,... n cameras and i=1,... m object points O_i in world coordinate space are mapped to image points I_ij in the corresponding projection planes. The lines connecting each object point and the corresponding image point pass through the camera's perspective center C_j.

The world coordinate system {X,Y,Z} has its origin at an arbitrary point in space, and the image coordinate system corresponding to the jth camera {U_j,V_j,W_j} has its origin at C_j. The coordinates of object point O_i are (x_i,y_i,z_i) in the world frame and the coordinates of the corresponding image point I_ij are (u_ij,v_ij,-d_j) in the image frame. The world coordinates of the perspective center C_j are (x_0j,y_0j,z_0j). The principal axis is the line connecting the perspective center C_j and the corresponding projection plane such that it is parallel to axis W_j of the image frame, and principal point P_j is the point where it intersects the image frame. The principal distance d_j is the distance between points P_j and C_j. The image frame coordinates of the principal point are (0,0,-d_j).

The collinearity equations allow us to determine a ray in the world coordinate system that a point must lie on given its location in the image coordinate system. When two cameras are used, the true world space location of a point can be determined by finding the intersection of the two resulting rays; however, accuracy is increased if more than two cameras are used.

In order to determine several intrinsic and extrinsic parameters of each camera required by the collinearity equations (the rotation matrix R_j, the principal distance d_j and the perspective center C_j), camera calibration is performed for each camera. Camera calibration requires a set of known geometric relationships between a group of target points in the world coordinate frame. The calibration rig used for this is a 3D solid with marked points imprinted along its surface. The device was manufactured using a 3D Systems ProJet HD-3000 (333 Three D Systems Circle, Rock Hill, SC, USA) rapid prototyping machine, capable of printing with a resolution of 0.025 mm. The marker points on the calibration rig are manually digitized and then refined with sub-pixel accuracy based on the neighboring gray values. Given at least 6 marker point correspondences, the DLT algorithm is used for calibration. Experiments showed that incorporating higher order optical and tangential distortion parameters had a negligible effect on the results, so they were removed in order to simplify the calibration process. A more detailed explanation of the DLT method is given elsewhere (Abdel-Aziz and Karara, 1971).

In order to gauge calibration accuracy with respect to the world coordinate system, we first reconstructed the 3D coordinates of the points on our calibration object. The discrepancy between the reconstructed 3D points and the known locations of the 3D points on the calibration object was then calculated. Our tests showed an average discrepancy of 0.1382±0.0684 mm. More information on this and other calibration accuracy metrics can be found elsewhere (Salvi et al., 2002).

This section details the steps taken to generate a 3D reconstruction of a free-flying dragonfly's body and wings based on data gathered from the aforementioned photogrammetry system. In our experience, dragonflies that are tethered to a solid object in any way do significant damage to their bodies while trying to escape. With the lack of detailed metrics to compare tethered and free-flying deformable wing kinematics, we cannot determine to what degree tethered dragonflies adequately capture true in-flight wing kinematics even if the tethered insect appears to be in a typical flight posture.

Free-flying insects present many reconstruction challenges that do not exist when dealing with tethered insects because of the fact that the insect's translation distances and Euler angles are not held constant. Thus, camera angles and focal points in the photogrammetry system cannot be chosen based on a constant desirable position and rotation of the insect's body. Also, marker points placed on the wings and natural landmarks will not pass through approximately the same position in space at the same time during consecutive wing beats. Dealing with quad-winged insects further complicates the process because of the increased likelihood of wing overlap from one or more camera angles.

Individual dragonflies were selected for this study based on the condition of their wings, and flight experiments were started within 1 h of capture. The data presented in this paper are from Eastern Pondhawk (E. simplicicollis) dragonflies, which were taken from a lake near the research lab during the period of August-September 2010 and 2011. The mean body mass of the specimens used to demonstrate our reconstruction was 0.265±0.0005 g, and the forewing and hindwing lengths were 35.03±0.005 mm and 33.21±0.005 mm, respectively. The mid-span chord lengths for forewing and hindwing were 7.39±0.005 mm and 9.21±0.005 mm, respectively.

Prior to being photographed, the forewings and hindwings were each marked in a grid pattern with 15 black ink marker points of approximately 0.5 mm diameter on each forewing and 16 on each hindwing. Marker points were spaced 5.3 mm apart with a spacing accuracy of 0.6 mm in the span-wise direction and 3.6 mm apart with a spacing accuracy of 0.8 mm in the chord-wise direction. The marker points aid in reconstructing the wings in 3D because the subtle natural pattern of wing veins is indiscernible at most time steps. The marker points were applied with a felt-tipped ink pen, adding negligible mass to the insect wings.

The initial 3D wing template models were generated with Catmull-Clark subdivision surfaces by aligning surface points corresponding to the first level of the subdivision surface hierarchy with the marker points on the wings in a top-down image. The dragonfly's body was manually created with Autodesk Maya based on several still images of the dragonfly taken with calibrated cameras, and was assumed to be rigid given how slowly it moves relative to the wings. Fig. 2 shows an initial configuration of a dragonfly's wings and body based on a top-down view prior to takeoff.

Cubic spline curves have already been used to reconstruct features on the surface of a deforming insect wing (Walker et al., 2009b). Our approach is an extension of this concept to the reconstruction of entire wing surfaces in order to further minimize the number of marker points required to capture both wing twist and camber. Catmull-Clark subdivision surfaces (Catmull and Clark, 1978) were chosen as the specific cubic spline surface representation because of their ability to generate smooth surfaces from meshes of arbitrary topology (Stam, 1998). Thus, the placement of marker points on an insect's wings can be based solely on best capturing the true wing deformation within the available resolution, while ignoring surface interpolation limitations.

Once the initial wing template surfaces have been created, marker points are digitized in each image recorded with the high-speed cameras at each time step. The 3D locations of each marker point that can be identified in two or more images are then determined based on the aforementioned projection equations. The vertices on the smooth wing template surfaces corresponding to the first level of the subdivision surface hierarchy are then iteratively aligned to the 3D projections of each marker point until further iterations produce no additional change in the reconstructed wing surface. As marker points are available on both the interior of the wing and the edges, an approximation of the true 3D macrosurface shape of the wings as they bend and twist is captured with the smooth parametric surface representation. At time steps where the wings rotate such that the location of a marker point is not discernible in two or more images, 3D projections of the marker point in question were performed at adjacent time steps and B-spline interpolation was used to estimate the 3D location of the missing points at the unclear time steps. This occurred in 4.8% of marker moments.

The dragonfly's body template mesh was animated by aligning the projection of its outer border to the silhouette of the body in the images; however, the body moves very slowly compared with the wings, so its reconstruction is trivial in comparison. A skilled operator working continuously can reconstruct a complete stroke for one wing in about 1 h. The wing reconstruction process is somewhat labor intensive, but it is currently the only means to attempt reconstruction of a deformable, quad-wing insect in free flight.

Fig. 3 shows the projection of the reconstructed wing template meshes back into image space at a point where multiple wings are undergoing a large amount of deformation. The alignment with the original data is clearly superior to methods that employ rigid wing assumptions. In total, we reconstructed five free-flying dragonflies with this method for at least five and at most nine wing beats, depending how long each dragonfly stayed in view. Analysis of kinematic variation for different flying modes within and between multiple specimens will be presented in separate papers. An animated reconstructed dragonfly can be seen in supplementary material Movie 1.

True negative pixel classifications were ignored so as not to artificially inflate the accuracy due to the large image size. As the expert segmentations are manually traced and not limited to interpolating between a small number of sample points, they are assumed to represent the true wing borders. This accuracy metric measures how close the projected borders of the reconstructed wings match the expert segmentations; however, the vertices on the interior portion of the wings were aligned to the marker points in the images in the same fashion and the same parametric surface interpolates between them, so this metric is a good indicator of the overall accuracy of the 3D reconstruction method. Fig. 4 shows an example of the pixel classifications for a set of reconstructed wings.

Using this accuracy metric, comparisons were made between our reconstructions and rigid wing reconstructions. Rigid wing templates were aligned with the plane generated from the 3D projections of the wing root and the two marker points nearest the 75% chord on the leading edge and the 50% chord on the trailing edge. Table 2 contains mean accuracy measurements for each wing using each reconstruction method. Clearly, the deforming wings align with the expert segmentations more consistently than the rigid wings.

The previously discussed subdivision surface reconstruction method allows us to accurately recreate the macrosurface deformations undergone by free-flying insect wings. This section details a new approach to quantifying the deformations observed in the reconstructed wing macrosurfaces and presents an in-depth analysis of a dragonfly's hindwing as an example of the methodology.

In order to accurately make repeatable measurements of the deformations, such as twist and camber, undergone by a reconstructed flapping insect wing, a universal reference plane must be established. Some studies have attempted to do this by defining wing deformation with respect to a plane that is fixed to several points on the wing or with respect to the straight line between the wing root and tip. However, these methods do not account for span-wise deformations and will vary based on wing size and shape. We address this by defining a least-squares plane as a universal reference plane for each wing that can aid in describing the physical position of the wing at any point in the stroke from a statistical perspective.

The obtained least-squares plane Ω and reconstructed wing surface Π, shown in Fig. 5, are used together to define several wing deformation metrics such as the local angle of twist, the span-wise camber and the chord-wise camber. Points A and B on Ω denote the projections of the user-selected wing root and tip onto Ω. The cross-section plane λ_k can then be defined normal to line segment AB, where the subscript k denotes the percentage of the distance between points A and B. Lower values of k indicate cuts taken close to the root of the wing, and higher values indicate cuts taken near the wing tip.

The camber line S_k is defined as the intersection of the plane λ_k and the wing surface Π. The chord C_k is then the line segment connecting the trailing and leading points of S_k. The line l_k is then defined as the intersection of Ω and λ_k. The local angle of chord-wise twist α^c_k at the cross-section plane λ_k can then be defined as the angle between the chord C_k and line l_k. These definitions are explained visually in Fig. 5.

A similar process is used to define deformation metrics in the span-wise direction. The superscript c denotes a cross-sectional cut in the chord-wise direction and the superscript s denotes one in the span-wise direction. Instead of using points A and B to define a line along which cross-section planes are placed, the projection of the endpoints of C₅₀ onto Ω is used. Thus, the span P_k and local angle of span-wise twist α^s_k can be measured relative to Ω in cuts approximately parallel to the leading edge of the wing.

To illustrate the subtle variations in instantaneous wing deformation captured by our reconstruction method, we have used the aforementioned metrics to perform a detailed analysis of the right hindwing of a dragonfly. In the future, we plan to combine these measurements with CFD results to further explain their causes. The camber lines S_k, cut in chord-wise and span-wise directions, were sampled at 10% increments.

At the start of the downstroke, there is a larger amount of camber present near the wing root (Fig. 6A, t/Γ0-2/8). However, the wing becomes increasingly flat as the measurements progress along the wing span approaching the wing tip. As the stroke progresses, the wing shows only small amounts of camber change near the tip (Fig. 6A, t/Γ3/8). As the wing reaches stroke reversal, it is at the most flattened state (Fig. 6A, t/Γ4/8). Following reversal, the wing shows considerable twisting motion during the upstroke with little camber produced (Fig. 6A, t/Γ5/8-7/8).

In Fig. 6B, span-wise cuts of the right hindwing are shown as a function of chord-wise distance at different time instants during one wing beat. The 10% cuts occur nearest the leading edge and 90% cuts occur nearest the trailing edge of the wing. The maximum span-wise camber occurs near the leading edge during downstroke, due to flexing against the airflow and the wakes formed in the upstroke.

Fig. 7 reveals a more quantitative look at wing camber with the camber-to-chord ratios (Fig. 7A) and camber-to-span ratios (Fig. 7B). The maximum camber height at a cross section of a wing is denoted by h^c_max in the chord-wise direction and h^s_max in the span-wise direction. Thus, the camber-to-chord ratio is defined as h^c_max/C_k or h^s_max/P_k in the span-wise case. This measurement reveals the relative camber throughout the wing as a function of time during the wing beat.

The stroke is divided into downstroke and upstroke with the stroke reversal marked by the dashed line (Fig. 7). Note that stroke reversal does not occur at the half-way point of the cycle, with the downstroke being slightly faster than the upstroke. Fig. 7A shows that chord-wise camber is most prominent near the wing root at the beginning of the downstroke. Combined with the negative span-wise camber shown in Fig. 7B, it suggests a cup-like wing deformation, which is expected as the wing flexes against the airflow.

One particularly interesting feature of the data in Fig. 7A is the two peaks in camber-to-chord ratio on the root half of the wing (10%, 30% and 50% cuts) during the downstroke. The first peak is at the very beginning of the downstroke. As the downstroke progresses, the wing begins to flatten, and then the camber reforms.

It is widely accepted that flapping insect wings are primarily passive structures. Combes and Daniel noted that insect flight muscles are restricted to the wing base, so free-flying insects have less active control over the deformation undergone by their wings than do other flapping fliers such as birds or bats (Combes and Daniel, 2003a). However, Simmons noted that the base of a dragonfly's wing has the musculature and neural control to actively flex and distort itself nearest the root (Simmons, 1977). The presence of multiple local maxima during the downstroke in Fig. 7A could be a result of active camber control near the wing root or the inertial effect due to the wing deceleration or both. Further study involving CFD results will be needed to investigate this phenomenon identified with the more detailed wing reconstructions and measurements.

Fig. 7A also shows that as the downstroke completes, the chord-wise camber is near zero, indicating that the wing is almost completely flat in the chord-wise direction. This is not true in the span-wise direction. At stroke reversal, the span-wise camber closest to the leading edge is near its peak. This can be explained by the inertial loading at stroke reversal, where the leading edge, which is the most dense and rigid portion of the wing, decelerates abruptly. Changing the direction of wing stroke causes the positive camber. This trend is present in the span-wise camber throughout the entire cycle, where the camber nearest the leading edge is caused predominantly by inertial loading in tandem with abrupt accelerations and decelerations at stroke reversal.

Our reconstruction method also captures how insect wings twist while flapping. Fig. 8 shows the local twist angle α_k in both the chord-wise and span-wise directions. It is clear that chord-wise twisting is relatively constant in the upstroke of the right hindwing. In contrast, twist varies near both the wing root and the wing tip during the downstroke. This is most likely due to the faster speed of the downstroke and the incidence of the wing with the wake formed in the previous stroke. To provide more clues as to why the downstroke sees changes in both camber and twist, more wing beat cycles will be analyzed in the future along with force histories of the wing beats.

It can also be noted from Fig. 8B that the twisting axis occurs at roughly 30% of the distance between the leading and trailing edge. At this distance, the twist angle along the span is closest to zero, making it the logical axis about which the wing would twist as a whole. In order to quantify the impact that slight differences in marker point identification in the reconstruction process have on the camber and twist measurements, eight different human operators performed reconstructions of the wing measured in Figs 7 and 8. The differences in the resulting reconstructions were used to generate the error bars in these figures; however, the overall trends were unchanged.

To further elucidate the deformation of the wing, visualizations of the right hindwing were created by mapping the distance from the least-squares reference plane Ω to the color of the wing surface (Fig. 9). When using this method to visualize wing twist, the viewing direction is chosen perpendicular to the least-squares plane with the focal point on the wing centroid X₀ in order to facilitate side-by-side comparisons at different portions of the stroke. The minimum twisted wing shows mainly concavity, whereas the maximum twisted wing shows a more modal display of distance from the plane.

In the case of the right hindwing (Figs 8, 9), the twisting helps to form and hold the camber of the wing before stroke reversal. At the instant of reversal, the twisting and camber diminish, resulting in the least deformed wing surface. As the following upstroke progresses, the local angle of twist changes suddenly (Fig. 8), which induces camber once again. This pattern repeats itself, helping to explain the relationship between twist and camber. It is also worth mentioning that the quantified twist angle and its variation along the span-wise direction can be an indication of passive (Ennos, 1988; Bergou et al., 2007) or active (Weis-Fogh, 1973) wing pitch, once combined with the information of the aerodynamic and inertial forces/torques and associated power (Bergou et al., 2007). Detailed study on the passive or active wing pitch will be left to future work, which in turn can shed light on the causes of wing deformation.

Thus far we have demonstrated a reconstruction method capable of capturing the detailed deformations undergone by free-flying insect wings and several metrics for consistently analyzing the resulting wing surfaces. A photogrammetry system consisting of three high-speed calibrated cameras is used to capture free-flying insects, whose wings have been marked in a grid pattern. The marker points are digitized and projected into a world coordinate system at each recorded time step, where a Catmull-Clark subdivision surface is aligned to them. A least-squares plane is then generated for each reconstructed wing surface at each time step. Chord-wise and span-wise camber and wing twist are then consistently measured relative to this plane at different slices through the wing.

The main strength of the presented methodology is that no rigid wing assumptions are made in either the reconstruction or the deformation analysis. The subdivision surface parameterization was demonstrated to effectively capture wing deformation between marker points, thus requiring fewer overall marker points, which is very desirable when trying to digitize multiple points on overlapping wings. At the same time, it is general enough to allow for arbitrarily placed marker points on wings with varying aspect ratios and natural markings. Similarly, using the least-squares plane to define wing twist and camber adds generality to the methodology so that it can be used effectively regardless of the amount of deformation undergone by the insect's wings.

We zeroed in on a single flap of a single wing in order to identify the subtle deformations occurring at different parts of the wing at different times in the stroke. Multiple visualization methods were demonstrated to identify wing deformations. Images of curves resulting from the chords and spans being projected onto cross-sectional planes convey detailed deformations at single instants in time (Fig. 6). Similarly, mapping wing twist to surface color shows a complete picture of how a wing is twisting at a single time step (Fig. 9). Time histories of chamber to chord ratio and local angle of twist in the span-wise and chord-wise directions were used to track deformation changes at different points in the wing stroke. This allowed us to identify a particularly interesting double peak in the chord-wise camber near the wing root, which suggests that the dragonfly could actively be flexing the muscles at the hindwing root to induce camber. Another possible cause is the inertial effect due to the wing deceleration. This phenomenon will require further validation between many specimens.

Digital reconstruction of free-flying insect wings still has many potential options for improvement. We close our discussion with the proposal of a method to model corrugation in free-flying insect wings, which we plan to explore in more detail in the future.

The 3D wing reconstruction algorithm and deformation metrics discussed thus far have all focused on the macrostructure of the wings. Capturing the information necessary to reconstruct mesosurface details like wing corrugation directly from the output of a high-speed camera system is extremely challenging. Either individual wing veins must be tracked at each time step or a very large number of marker points must be employed. For instance, Walker and colleagues used approximately 100 marker points per tethered locust hindwing in order to capture the most distinct wing veins (Walker et al., 2009b). We believe it is not feasible or desirable to track the very high amount of marker points needed to reconstruct the mesosurface details of free-flying insect wings, especially if one is interested in quad-wing insects.

It is considerably more practical to measure the mesosurface of a static insect wing (Jongerius and Lentink, 2010; Tsuyuki et al., 2006). Thus, our approach is to build on this work in order to model the surface corrugation of reconstructed free-flying insect wings. Instead of tracking the wing corrugation details at each time step of the high-speed camera output, it is captured once from a static wing and then mapped to the reconstructed wings by means of a displacement map.

Displacement mapping is a technique commonly used in computer graphics, which uses a height map to displace the actual geometry of a surface in the direction locally normal to that surface (Fig. 10A,B). In the case of our dragonfly wings, the displacement map (Fig. 10C) used to demonstrate the technique was generated by scaling the geometric profiles described elsewhere (Kesel, 2000) to align with our specimen's wing chord.

Insect wing venation patterns can also be modeled in the final reconstruction in a similar fashion by using texture maps based on the original wing coloration. Dark colored wing veins can be automatically identified as minimum intensity ridgelines, which are extracted from a top-down image of the wings taken before the marker points have been applied. A stylized version of the true wing venation pattern can then be generated based on these ridgelines (Fig. 10D). The texture and displacement are then UV mapped to the wings in their initial static configuration (Fig. 2A, Fig. 10). Because of the fact that the wing geometry is being displaced normal to the initial reconstructed macrosurface, the displacement maps can then be used to project the corrugation and surface roughness onto the already deformed wings at any time step (Fig. 10B,F,G). An ambient occlusion-rendering pass was used to further visually emphasize the subtle corrugation pattern on the wings (Fig. 10E).

In the future we plan to further validate this method; however, the fact that insect wings are largely passive structures suggests that displacement in the direction normal to the reconstructed macrosurface is a reasonable assumption to make when modeling wing corrugation. When compared with the aforementioned challenges with accurately reconstructing wing corrugation for free-flying insects directly from the high speed camera output or the alternative of ignoring it entirely, we believe this is the best approach to take when reconstructing free-flying insect wings. The main benefit of this approach that sets it apart from other methods is that it does not require any additional information beyond the initial marker points to be tracked in the high speed camera data in order to model wing venation and corrugation. This is an extremely important characteristic as we work towards fully automated reconstruction of the deformation and surface corrugation of free-flying insect wings.

 
• A

  wing root projected onto Ω

 
• A_l

  area of a wing surface element

 
• a_t

  reconstruction accuracy at time t

 
• B

  wing tip projected onto Ω

 
• c

  superscript denoting a chord-wise measurement

 
• C_j

  perspective center

 
• C_k

  chord line

 
• d_j

  principal distance

 
• 

  minimized least-squares distance

 
• F

  false positive/negative

 
• h_max

  maximum camber height

 
• i

  object point index

 
• I_ij

  image point

 
• j

  camera index

 
• k

  subscript denoting percentage distance between projected root and tip

 
• l

  reconstructed wing element index

 
• l_k

  intersection of the cross-section and least-squares planes

 
• M

  central moments tensor

 
• n

  unit vector normal to Ω

 
• O_i

  object point

 
• P_j

  principal point

 
• P_k

  span line

 
• R_j

  rotation matrix mapping between world and image space

 
• s

  superscript denoting a span-wise measurement

 
• S_k

  camber line

 
• t

  time step

 
• T

  true positive/negative

 
• U_j,V_j,W_j

  image coordinate system

 
• X,Y,Z

  world coordinate system

 
• X₀

  centroid of an entire wing

 
• X_l

  centroid of a wing surface element

 
• α_k

  local angle of twist

 
• Γ

  single stroke time

 
• χ_k

  cross-section plane

 
• Π

  reconstructed wing surface

 
• Ω

  least-squares reference plane for a wing

We thank Dr Phil Beran, Dr Thomas Wischgoll, Dr Bret Stanford and Dr Aaron McClung for their support and advice.