EyeLoop: An Open-Source System for High-Speed, Closed-Loop Eye-Tracking

EyeLoop: An Open-Source System for High-Speed, Closed-Loop Eye-Tracking Simon Arvin Rune Nguyen Rasmussen Keisuke Yonehara

Eye-trackers are widely used to study nervous system dynamics and neuropathology. Despite this broad utility, eye-tracking remains expensive, hardware-intensive, and proprietary, limiting its use to high-resource facilities. It also does not easily allow for real-time analysis and closed-loop design to link eye movements to neural activity. To address these issues, we developed an open-source eye-tracker – EyeLoop – that uses a highly efficient vectorized pupil detection method to provide uninterrupted tracking and fast online analysis with high accuracy on par with popular eye tracking modules, such as DeepLabCut. This Python-based software easily integrates custom functions using code modules, tracks a multitude of eyes, including in rodents, humans, and non-human primates, and operates at more than 1,000 frames per second on consumer-grade hardware. In this paper, we demonstrate EyeLoop’s utility in an open-loop experiment and in biomedical disease identification, two common applications of eye-tracking. With a remarkably low cost and minimum setup steps, EyeLoop makes high-speed eye-tracking widely accessible.
12 9 2021 779628 10.3389/fncel.2021.779628 1 10.3389/crossmark-policy frontiersin.org true Lundbeckfonden http://dx.doi.org/10.13039/501100003554 Lundbeckfonden http://dx.doi.org/10.13039/501100003554 Lundbeckfonden http://dx.doi.org/10.13039/501100003554 Novo Nordisk Fonden http://dx.doi.org/10.13039/501100009708 Carlsbergfondet http://dx.doi.org/10.13039/501100002808 European Research Council http://dx.doi.org/10.13039/501100000781 https://creativecommons.org/licenses/by/4.0/ 10.3389/fncel.2021.779628 https://www.frontiersin.org/articles/10.3389/fncel.2021.779628/full https://www.frontiersin.org/articles/10.3389/fncel.2021.779628/full J. Eye Mov. Res. Andersson 3 1 2010 Sampling frequency and eye-tracking measures: how speed affects durations, latencies, and more. 10.16910/jemr.3.3.6 Sci. Adv. Banks 1 e1500391 2015 Why do animal eyes have pupils of different shapes? 10.1126/sciadv.1500391 PLoS One Cahill 3 e2055 2008 The optokinetic reflex as a tool for quantitative analyses of nervous system function in mice: application to genetic and drug-induced variation. 10.1371/journal.pone.0002055 Surg. Neurol. Int. Chen 2 82 2011 Pupillary reactivity as an early indicator of increased intracranial pressure: the introduction of the neurological pupil index. 10.4103/2152-7806.82248 Behav. Res. Methods Instrum. Comput. Cornelissen 34 613 2002 The eyelink toolbox: eye tracking with MATLAB and the psychophysics toolbox. 10.3758/bf03195489 Neuron Costa 89 8 2016 More than meets the eye: the relationship between pupil size and locus coeruleus activity. 10.1016/j.neuron.2015.12.031 eLife de Gee 9 e54014 2020 Pupil-linked phasic arousal predicts a reduction of choice bias across species and decision domains. 10.7554/eLife.54014 J. Vis. Exp. de Jeu e3971 2012 Video-oculography in mice. J. Neurosci. Ehinger 35 7403 2015 Predictions of visual content across eye movements and their modulation by inferred information. 10.1523/JNEUROSCI.5114-14.2015 Br. J. Ophthalmol. Ellis 65 754 1981 The pupillary light reflex in normal subjects. 10.1136/bjo.65.11.754 Dev. Ophthalmol. Fetter 40 35 2007 10.1159/000100348 Vestibulo-ocular reflex. Neuron Grosenick 86 106 2015 Closed-loop and activity-guided optogenetic control. 10.1016/j.neuron.2015.03.034 Numerically Stable Direct Least Squares Fitting of Ellipses. Halır 1998 bdhammel/least-squares-ellipse-fitting: Initial Release. Hammel 2019 Biol. Cybern. Juhola 53 67 1985 Effect of sampling frequencies on computation of the maximum velocity of saccadic eye movements. 10.1007/bf00337023 Comput. Stat. Data Anal. Kanatani 55 2197 2011 Hyper least squares fitting of circles and ellipses. 10.1016/j.csda.2010.12.012 J. Neurophysiol. Kretschmer 118 300 2017 Comparison of optomotor and optokinetic reflexes in mice. 10.1152/jn.00055.2017 Philos. Trans. R. Soc. Lond. B Biol. Sci. Lee 290 169 1980 The optic flow field: the foundation of vision. 10.1098/rstb.1980.0089 Science Lucas 299 245 2003 Diminished pupillary light reflex at high irradiances in melanopsin-knockout mice. 10.1126/science.1077293 Clin. Exp. Optom. Markwell 93 137 2010 10.1111/j.1444-0938.2010.00479.x Intrinsically photosensitive melanopsin retinal ganglion cell contributions to the pupillary light reflex and circadian rhythm. Cold Spring Harbor Laboratory. Mathis 457242 2018 On the inference speed and video-compression robustness of DeepLabCut. 10.1101/457242 Neuron McGinley 87 1143 2015 Waking state: rapid variations modulate neural and behavioral responses. 10.1016/j.neuron.2015.09.012 Curr. Biol. Meyer 30 2116 2020 Two distinct types of eye-head coupling in freely moving mice. 10.1016/j.cub.2020.04.042 Front. Neuroinform. Muller 9 11 2015 Python in neuroscience. 10.3389/fninf.2015.00011 Nat. Protoc. Nath 14 2152 2019 Using DeepLabCut for 3D markerless pose estimation across species and behaviors. 10.1038/s41596-019-0176-0 Vision Res. Nyström 92 59 2013 Post-saccadic oscillations in eye movement data recorded with pupil-based eye trackers reflect motion of the pupil inside the iris. 10.1016/j.visres.2013.09.009 Nat. Commun. Rasmussen 11 831 2020 10.1038/s41467-020-14643-z A segregated cortical stream for retinal direction selectivity. Curr. Biol. Rasmussen 31 1165 2021 Binocular integration of retinal motion information underlies optic flow processing by the cortex. 10.1016/j.cub.2020.12.034 Neurosci. Res. Sakatani 49 123 2004 PC-based high-speed video-oculography for measuring rapid eye movements in mice. 10.1016/j.neures.2004.02.002 Proceedings of the Symposium on Eye Tracking Research and Applications ETRA ’12 Świrski 173 2012 10.1145/2168556.2168585 Robust real-time pupil tracking in highly off-axis images J. Exp. Biol. Theobald 213 1366 2010 Dynamics of optomotor responses in Drosophila to perturbations in optic flow. 10.1242/jeb.037945 Front. Neurol. Wang 9 1029 2018 Arousal effects on pupil size, heart rate, and skin conductance in an emotional face task. 10.3389/fneur.2018.01029 Rev. Sci. Instrum. White 81 103108 2010 Faraday rotation data analysis with least-squares elliptical fitting. 10.1063/1.3470126 J. Vis. Yaramothu 18 2 2018 10.1167/18.6.2 Effects of visual distractors on vergence eye movements. Neuron Yonehara 89 177 2016 Congenital nystagmus gene FRMD7 is necessary for establishing a neuronal circuit asymmetry for direction selectivity. 10.1016/j.neuron.2015.11.032 Front. Integr. Neurosci. Zeki 9 21 2015 Area V5-a microcosm of the visual brain. 10.3389/fnint.2015.00021 Data_Sheet_1.docx 10.3389/fncel.2021.779628.s001 https://www.frontiersin.org/articles/10.3389/fncel.2021.779628/supplementary-material/10.3389/fncel.2021.779628.s001 Image_1.TIF 10.3389/fncel.2021.779628.s002 https://www.frontiersin.org/articles/10.3389/fncel.2021.779628/supplementary-material/10.3389/fncel.2021.779628.s002 Video_1.AVI 10.3389/fncel.2021.779628.s003 https://www.frontiersin.org/articles/10.3389/fncel.2021.779628/supplementary-material/10.3389/fncel.2021.779628.s003 Video_2.AVI 10.3389/fncel.2021.779628.s004 https://www.frontiersin.org/articles/10.3389/fncel.2021.779628/supplementary-material/10.3389/fncel.2021.779628.s004 Video_3.AVI 10.3389/fncel.2021.779628.s005 https://www.frontiersin.org/articles/10.3389/fncel.2021.779628/supplementary-material/10.3389/fncel.2021.779628.s005