Predictive saccade in the absence of smooth pursuit: interception of moving targets in the archer fish

The ability to monitor and track moving objects is a characteristic of the visual system of almost every organism (Nilsson, 2009; Schlegel and Schuster, 2008). In primates and other mammals, this ability is facilitated by a dedicated neural circuitry that generates both saccadic and smooth pursuit eye movements (Krauzlis, 2004). This system allows the animal to align its area centralis – the region in the retina optimized for high visual acuity – with a moving object, thus enabling the animal to obtain the visual information needed to perform precise motor actions. In non-mammalian vertebrates, a neural system for the generation of smooth pursuit eye movements has not been discovered and may not exist (Krauzlis, 2004; Lisberger et al., 1987). Despite the lack of smooth pursuit eye movements, many non-mammalian animals still have the ability to intercept fast-moving objects with high precision. Although this ability is widespread, to date little is known about the solutions adopted by the visual systems of these animals for monitoring moving targets. Clearly, understanding this behavior is a prerequisite for studying its underlying neural mechanisms. Furthermore, assuming the mammalian eye tracking system has evolved from such a visual system, we could gain new insight to the evolution of the mammalian visual system.

Most fishes exhibit an optokinetic/optomotor response in which they accurately move their eye/body relative to a moving background (Hermann and Constantine, 1971). However, they do not possess smooth pursuit eye movements (Easter, 1972; Hermann and Constantine, 1971; Land, 1999). Still, some fishes are able to monitor and intercept moving prey successfully (e.g. Lanchester and Mark, 1975; Piccolo et al., 2007; Webb, 1977). Therefore, these fishes are potentially good models for studying strategies for intercepting moving objects in the absence of smooth pursuit. We studied the eye movement strategies of archer fish during interception of targets moving at various velocities. Three characteristics made the archer fish our choice of animal model for this investigation. First, this species provides overt evidence of gaze and visual attention as it shoots down prey on overhanging vegetation by squirting water from its mouth (Luling, 1958; Luling, 1963; Schuster et al., 2006; Vasserman et al., 2010). Second, archer fish can be trained to shoot at artificial targets presented on a computer monitor, which makes it possible to run controlled experiments with complex experimental procedures. And third, based on anatomical studies, Temple and colleagues reported that archer fish retina possesses a well-defined area centralis – a region optimized for high visual acuity (Temple et al., 2010). It has a 1:1 ratio between cone photoreceptors and ganglion cells that provides a spatial resolution at the level of a single cone photoreceptor (Douglas and Hawryshyn, 1990). It is located at the ventro-temporal part of the retina and has a length of ∼1.2 mm of an arc that translates to an angular width of ∼6 deg for an eye with a radius of 4.5 mm. Though not reported here, we confirmed the functional role of this area via behavioral and anatomical experiments. In particular, based on analysis of video recordings from behavioral experiments and based on anatomical localization of the embryonic fissure on the fish retina, we were able to trace light rays from a target that the fish was tracking to their retinal projection (Ben-Simon et al., in press). This projected location was found to coincide with the anatomically reported area centralis. Moreover, we also showed that when the fish needs to track a target that first appears peripherally to this area, it would shift its gaze towards the target to align the target projection on the retina with the center of its area centralis (Ben-Simon et al., in press).

All experiments were conducted in accordance with Ben-Gurion University of the Negev regulations and State of Israel laws on animal care and experimentation. We trained two archer fish, Toxotes jaculatrix (Pallas 1767), to swim into a small region in the water tank and to shoot at a black circular target. The black circular target (1.2 cm diameter) was presented on a white background through a computer monitor (Samsung BX2235 22 in LED monitor, 60 frames s^–1) positioned 57 cm above water level. Stimuli were created using the Psychophysics toolbox in Matlab (Brainard, 1997). We presented the fish with stimuli of moving targets. The target appeared on the monitor above the fish’s head (Fig. 1A). The fish executed a saccade towards the target and moved towards the surface in order to shoot at it. Then, the target moved forward (Fig. 2A). In this manner, when the fish made a saccade, their eyes, which are located on the side of their head, rotated primarily in a plane that was approximately parallel to the fish sagittal plane. Such eye rotations around the axis perpendicular to the center of the pupil were used to obtain gaze angle. We referred to this rotation of the eye as pitch, after one of the three angles that are commonly used to describe rotations (i.e. roll, pitch and yaw).

Four high-resolution high-speed video cameras (SI-1920HD, Silicon Imaging, www.siliconimaging.com, 120 frames s^–1, 1500–800 pixels frame^–1) were used to record behavior. They were positioned facing one side of the tank at a distance of ∼80 cm, 2 cm below the water level. The cameras were assigned three functional modes. One camera monitored the entire scene, i.e. both the fish and the computer monitor (with the stimulus on). The second camera was located ∼45 deg from the tank side, zooming in on the fish eye from the back. The third and fourth cameras, which were used for eye movement measurements, were both facing the front side of the tank to minimize light distortion due to air–glass–water refraction. To image the fish eye at maximal resolution, their field of view (i.e. zoom level) was set close to the diameter of the eye. To ensure that the eye remained in the field of view of the cameras, we trained the fish to track the targets from a small region within the tank (Fig. 2A).

The fish eye rotations are composed of a combination of pitch and non-pitch (i.e. pitch and yaw) rotations. When we measured the fish eye movements, we considered only the pitch component of the eye rotation. In order to verify that the pitch component captures most of the eye rotation, we also measured the non-pitch eye movements. These measurements were based on measuring the displacement in the center of the pupil that was induced by the second saccade. Note that during the short saccadic latency, the body, typically, remained almost static while the eye rotated. If this eye rotation is composed only from pitch eye rotation, the location of the pupil center would not change as a result of the saccade. However, a non-pitch eye movement would induce a displacement of the pupil center. We measured in pixels this displacement based on frames that were captured before and after the second saccade and denoted it by l. To measure the magnitude of the eye rotation that was induced by the pupil displacement, we translated l into a metric unit (1 pixel=0.05 mm) and divided it by the eye radius (5 mm). The arc tangent of this ratio represents the non-pitch rotation of the eye. Considering an accuracy of a single pixel, we receive a resolution of 0.57 deg. In all trials where the fish performed the second saccade, we calculated the contribution of the non-pitch component to the total eye rotation. Out of these trials, throughout the data analysis, we considered only trials in which the contribution of the pitch rotation was >90% of the total eye rotation.

We measured the shot accuracy based on images taken from the first camera, which captured the entire scene 20 ms after shot initiation on each trial. This time frame was chosen because at this stage the trace of the shot still follows a straight line, whereas later on the shot bends and spreads. We fitted the shot trace with a straight line and calculated the angle between that line and the water level, denoted as S. Next we calculated the angle between the fish’s mouth and the trailing edge of the target, denoted as T. Note that our reference on the target is the trailing edge as the fish shot at it when also shooting toward stationary targets that were displaced targets, in a control experiment that we conducted. For accurate shooting, the difference between angles S and T, denoted as shot accuracy, should be 0 at the time of impact 100 ms later. Therefore, we added to T the angular trajectory of the target while the shot is airborne: T′=(T+v×0.1), where v is the target’s angular velocity.

We explored the eye movement strategy that archer fish employ during interception of moving targets. For this purpose we trained two archer fish to shoot at targets moving at different constant angular velocities and we present data collected from both fish. A circular target appeared on a monitor screen and the fish performed an initial saccade toward the target (Fig. 1A). At time t₁, the target started to move at one of four speeds selected at random until the fish shot at it (at time t₃), either after performing a second saccade (at time t₂), or without one (Fig. 1B, upper panel). After a successful shot, as determined by the experimenter and later verified by viewing a video recording of the experiment (with a tolerance angle of ±0.5 deg), the fish were rewarded with a small food pellet.

Throughout the experiments, the fish swam freely while both their body and the target were recorded by a set of high-speed video cameras. These recordings were analyzed offline by custom-written software (see Materials and methods) to measure the fish gaze direction (Ben-Simon et al., 2009) (precision of 0.2 deg), the target direction (precision of 0.1 deg) and the shot direction (s.e.m.=0.5 deg). More specifically, the gaze direction measurements were based on monitoring the rotation of the pigment pattern around the iris, while the target and shot direction were based on monitoring the target position relative to the fish and on the trace of the shot, respectively. An illustration of the target and gaze angles relative to a fish’s eye is presented in Fig. 1B (bottom panel). Another variable we refer to here is the gaze-lag, defined as the difference between the target and gaze angles. Note that the gaze-lag represents the displacement between the projection of the target on the retina and the center of the area centralis, in analogy to retinal eccentricity in foveated animals.

A sketch of (idealized) gaze, target and gaze-lag angles as a function of elapsed time is presented in the inset of Fig. 3A. After the initial saccade, gaze-lag angle is zero as both target and gaze angles are zero. Shortly after, at time t₁, the gaze-lag starts to increase linearly as a result of the target movement until the onset of the second saccade at time t₂. The second saccade, which is an eye saccade only (Ben-Simon et al., 2009) (see Materials and methods), shifts the projection of the target to the other side of the area centralis, creating a negative gaze-lag, after which the gaze-lag increases again until the shot at time t₃. Examples of four time traces of gaze-lag in response to three different target velocities are presented in Fig. 3A. Notice the two time traces that correspond to the slowest tested velocity, and how in one of them the fish did not perform a second saccade before the shot.

On average, the fish initiated earlier and longer saccades in response to faster moving targets (Fig. 3B,C). One feature of these saccades is revealed by examining the ratio G between the saccade amplitude and the gaze-lag just prior to the saccade, a measure known as the saccadic gain (de Brouwer et al., 2001). Note that a gain of G≈1, where the animal essentially compensates for the gaze-lag, is expected in animals with a narrow fovea, whereas animals with wide area centralis but no fovea could tolerate other (and in particular, smaller) (de Brouwer et al., 2002) gain values. Surprisingly, we found that the saccadic gain in the archer fish varied up to ∼2 (P=0, t-test, null hypothesis is the gain equals 1) for fast-moving targets (Fig. 3D), an exceptionally high value that is approximately twice that observed in mammals (de Brouwer et al., 2001; Temple et al., 2010). We further examined whether, as in other animals, these saccades are correlated with the gaze-lag at the time of the saccades. We found no significant correlation between the two variables (Fig. 3E), which indicates a unique eye tracking strategy for moving targets.

In order to understand what triggers the saccade after target motion onset, we measured the gaze-lag versus target angle at the time of shot, all in response to different target velocities (Fig. 4A). We first note that in all experiments the shot occurred at a gaze-lag not greater than ∼6 deg, either with or without a preceding saccade (Fig. 4A, horizontal dashed line). This is also revealed by the probability of a saccade as a function of the total target displacement from its initial position to its position at the time of shot (Fig. 4B), which exhibits a clear threshold: slightly below 6 deg displacement at time of shot there is no saccade, while slightly above 6 deg there is a saccade with a probability of 1. Moreover, as depicted in the scatter plot of Fig. 4A, all shots were divided into two distinct groups – those that were preceded by a saccade and those that were not. The probability of belonging to one of these groups depends on the target velocity, with probability increasing with target velocity (Fig. 4C). This behavior could have been predicted as for slow moving targets (e.g. 8 deg s^–1), the fish would presumably have enough time to shoot the target with a gaze-lag of less than 6 deg (i.e. before the target exits the field of view of the area centralis). But for faster moving targets (e.g. 13 deg s^–1), the target would most probably exceed this threshold before the shot could be completed (see below). Indeed, we found that it is in the best interest of the animals to avoid exceeding the 6 deg gaze-lag limit at shot time, as in the rare events when the fish did shoot with a gaze-lag >6 deg their average shot accuracy declined by ∼1.1 deg as compared with ∼0 deg when the gaze-lag was <6 deg (Fig. 4D). For a typical prey of size 1 cm in diameter and 57 cm away from the surface of the water, this suggests a certain miss of ∼1.1 cm.

We next examined whether the fish execute a saccade before or after the target exits the 6 deg displacement limit. By examining the probability distribution for a saccade as a function of gaze-lag (Fig. 4E) we found that the fish do not wait until the target actually exits the area centralis. Rather, they initiate the saccade much earlier, when the target displacement is on average 2.9 deg (s.e.m.=0.19 deg). Another parameter that depicts the early saccade onset is the time to saccade (t₂; see Fig. 3B), which is always shorter than the time from the saccade to the shot (t₃–t₂), being virtually constant (t₃–t₂≈0.4 s). Clearly, the archer fish are able to predict early on whether the target will leave the area centralis before the shot sequence can be completed, and in such cases it initiates a saccade well before this event occurs. Moreover, the high saccadic gain (Fig. 3D) allows the target to remain long enough on the area centralis for a new shooting sequence to be re-initiated and completed (Fig. 3A).

We investigated the saccadic eye movement strategy the archer fish, vertebrates with no smooth pursuit eye movements, employ during interception of moving targets. We found that the radius of the archer fish area centralis subtends approximately ±6 deg from the center of gaze (Fig. 4A), consistent with previous anatomical observations (Ben-Simon et al., in press; Temple et al., 2010). This substantial radius enables the fish to track slow-moving targets with no eye movements at all. However, for tracking fast targets that are likely to exit the area centralis before the shooting sequence can be completed, the fish perform a saccade early on during interception. This eye movement brings the projection of the target to a new position on the area centralis, typically ahead of its center (Fig. 3A). Thus, the shift in retinal position of the target projection provides the fish with enough time (∼0.4 s, see Fig. 3B) to restart and complete the shooting procedure while the target projection remains within the area centralis. This saccade is predictive in nature, it is performed well before the target passes half the radius of the area centralis (Fig. 4E), and is characterized by exceptionally high saccadic gain, with values as high as ∼2. In cases where the fish made a poor prediction and did not perform a saccade although the target traveled more than the full 6 deg before the shot, the shot accuracy declined by ∼1.1 deg, resulting in a miss (Fig. 4D).

What controls saccade size in the archer fish? Looking at other foveated animals it was found that when these animals eye track moving targets using saccades their saccadic amplitudes typically match the gaze-lag of the target (de Brouwer et al., 2001). However, we found a linear increase in the saccadic gain (Fig. 3D), with a highest average gain value of 2 – twice that found in other foveated animals. In addition, we found no correlation between gaze-lag and saccade amplitude (Fig. 3E). This may indicate on the presence of an alternative mechanism that controls saccade amplitude.

To explore this possibility we tested different variables as possible determents of the saccade amplitude and found there is a correlation (R²=0.43) between saccade amplitude and target velocity (Fig. 3C). This might indicate that the archer fish indeed uses an alternative mechanism. However, one should keep in mind that this issue is not resolved and requires further studies.

To date, most research on tracking and interception of moving targets has been performed on animals that possess smooth pursuit eye movements. Understanding how this problem is handled in both fish and other systems may shed new light on the evolution of visual tracking in general, and on our understanding of the neural mechanisms underlying the predictive saccade behavior.

What parts of the fish brain preform the computation needed to carry out the predictive saccade? Most research investigating brain structures involved in predictive saccades has been conducted on humans (Gagnon et al., 2002; Milea et al., 2007; O’Driscoll et al., 2000; Pierrot-Deseilligny et al., 2004). For example, O’Driscoll and colleagues reported that predictive saccades in humans are associated with activation of subjects’ visual areas in the cortex, the superior colliculus and the cerebellum (O’Driscoll et al., 2000). Examining research related to the fish brain, we find many neural systems that differ from that of mammals. The main structure devoted to vision is the optic tectum, which in mammals is called the superior colliculus. In addition, computations made in the mammalian telencephalon are apparently made in other areas of the fish brain. Finally, fish have a cerebellum structure that has preserved much of its anatomical structure and functions during evolution (Rodríguez et al., 2005). In fishes, the optic tectum replaces some processing functions of a cortex, such as encoding of target direction and velocity (e.g. Masseck and Hoffmann, 2008) and in conjunction with the cerebellum is engaged in controlling and generation of eye movements (Pérez-Pérez et al., 2003), while the cerebellum and cerebellum-like structures were reported to be engaged in predicting temporal aspects of certain stimuli (Bell et al., 2008; Rodríguez et al., 2005). Based on these studies, we suggest that in fishes the cerebellum and optic tectum may be primarily responsible for processing sensory-motor information related to the generation of these predictive saccades.

Predictive saccade behavior has been observed in humans under certain extreme visual conditions. Shalom and colleagues found that subjects playing the Breakout computer game fixated on a ball while it was bouncing from a wall and subsequently shifted their gaze direction to an anticipated position of the ball (Shalom et al., 2011). This strategy enabled the players to place the paddle in the anticipated ground position of the ball. In addition, Land and McLeod reported that a predictive saccade is performed by professional cricket batsmen when they are challenged with fast flying balls thrown by the bowler (Land and McLeod, 2000). Specifically, the batsman gazes at the ball in the bowler’s hand, before it leaves the hand. However, instead of tracking the ball once it leaves the bowler’s hand, the batsman shifts their gaze direction towards a predicted point where the ball will bounce off the ground, and does so well before the bounce itself (Land and McLeod, 2000). Interestingly, the characteristics of the predictive saccades in professional batsman are similar to those in the archer fish (Fig. 3B). As target angular velocity (before the bounce) increases, the batsman initiates earlier and longer saccades.

Although many predator vertebrates with an area centralis employ eye movements during their prey-catching behavior it is important to note that successful prey catching is also applicable in vertebrates with no area centralis and eye movements (Ewert, 1976; Roth, 1976). For example, Ewert (Ewert, 1976) has showed that the optimum velocity for target interception in the toad starts at 30 deg s^–1 and reaches a value as high as 60 deg s^–1. However, these animals typically do not intercept slowly moving targets (e.g. <0.5 deg for Salamandra salamandra) and have low spatial resolution.

We would like to thank G. Glusman and R. Marom for technical assistance, and the generous support of the Zlotowski center for neuroscience research, the Frankel Fund, and the Paul Ivanier Robotics Center at Ben-Gurion University.