Sergei Gepshtein is a scientist specializing in perceptual psychology and sensorimotor neuroscience. Much of his research dwells on the interface between two aspects of visual perception: the entry process called "early vision" and the constructive process called "perceptual organization."

• Early vision is essentially a gate — or filter — that controls what information penetrates the visual system. The filter is plastic. It rapidly changes in accord with your optical environment and in service of your intentions.

• The information that enters the visual system must be organized: grouped, segmented, layered, or else it remains outside of your awareness. Research of this process of perceptual organization is an offshoot of the Gestalt movement in psychology, which became a mature science only recently, a century after the Gestalt movement was born.

Sergei studies computational principles and biological mechanisms underlying these processes, in particular how visual systems organize information for the perception of motion and change.

He also studies sensorimotor integration, which is a field of research concerned with how we use visual and haptic information to guide action. A key questions is how vision helps us to plan actions prospectively, for many steps ahead, in view of the dynamic nature of the environment, its varying risks and uncertainties.

As a scientist and a principal investigator at the Salk Institute for Biological Studies, he uses experimental and computational methods to characterize neuronal mechanisms of sensation, perception, and action.

As a member of the board of directors of the Academy of Neuroscience for Architecture, and as founding member of the Worldbuilding Institute (originally 5D Institute), Sergei is increasingly engaged in translational and applied research, which includes design and analysis of the built environment and sensory media.

In 2017, he founded the Center of Spatial Perception & Concrete Experience (SPaCE) at the University of Southern California’s School of Cinematic Arts, for which he serves as the director, with the goal to develop new forms of physical, immersive, and environmental “spatial media,” including augmented reality, virtual reality, and beyond.

As an educator, Sergei helps to develop and teach new "art-science" curricula connecting scientific and design disiplines at the University of Southern California (Los Angeles), the New School for Architecture and Design (San Diego), and the University of California (San Diego).

• Neural mechanisms of visual sensitivity
  National Institutes of Health (2019-2024)
  
  One of the most important questions in neuroscience today concerns the mechanisms by which sensory neurons give rise to perceptual experience. There are many ways to address this question, which have long populated the field of visual neuroscience. Prominent among them is the study of visual selectivity. Observers are highly sensitive to some visual stimuli and less sensitive to others. Visual neurons are also highly selective: Each responds to a limited range of stimuli along several stimulus dimensions. We aim to understand how the selective pattern of neuronal responses accounts for the observer's selective perceptual experience and discriminative capacity. This understanding is achieved through experiments that first evaluate patterns of neuronal selectivity for visual stimuli that vary in their spatial and temporal properties. Second, these data, in combination with manipulations of stimulus context, are used to develop a novel mechanistic account of neuronal selectivity based on activity within cortical visual circuits stabilized by inhibition. Finally, to understand how neuronal selectivity underlies perceptual experience, direct comparisons will be made between physiological measures of neuronal selectivity and behavioral measures of perceptual selectivity, assessed concurrently under identical conditions. The experiments will yield an unprecedented body of comprehensive data regarding the spatiotemporal tuning of the primate visual system. These data will be used to further understanding of the mechanisms of sensory processing and will provide insights into pathologies of vision caused by trauma, disease and developmental disorders of the brain.
  

• Enhancing eyewitness performance by optimizing context
  The Laura and John Arnold Foundation (ongoing)
  Project site at Open Science Framework
  
  Eyewitness identification has a long history of use as a form of forensic evidence by law enforcement and the courts in the USA and elsewhere. It has played a valuable role in both convictions and acquittals. Despite this value, our society has been confronted with egregious failures of this type of evidence, leading to conviction of innocent people. Of particular note are findings from the era of post-conviction DNA profiling, during which over 350 people, many serving extended prison sentences, have been exonerated based on DNA evidence. In approximately 70% of these cases, the original erroneous convictions resulted from testimony of eyewitnesses. The consequences of erroneous convictions based on flawed eyewitness testimony are profound. Not only have we imprisoned innocent people while the guilty remain at large, but these outcomes risk undermining public trust in the criminal justice system. This project seeks to improve the performance of eyewitnesses using a basic science approach derived from traditional research on human sensory processing and memory.
  

• Improving wayfinding and reducing challenging behaviors in persons with dementia using adaptive visual technologies
  National Institutes of Health (ongoing)
  
  An estimated 5.4 million Americans are currently suffering from Alzheimer's disease. Besides affecting the person with the condition, Alzheimer's disease also presents a public health concern, due to the significant cost required to care for persons with dementia (PWD) and to the mental and physical stress placed upon caregivers. There is an urgent need for products and programs that improve the quality of life of PWD and reduce challenging behaviors. We develop an adaptive visual technology to improve wayfinding and reduce challenging behaviors in PWD by automatically presenting PWD with customized messages on displays within memory-care facilities.
  

• Sensorimotor optimization in dynamic environments
  ONR MURI Center: From brain to behavior
  
  The human cortex creates rich representations of the world. These representations are based on learning, which often proceeds self-supervised. This kind of learning - often dubbed "unsupervised" - is commonplace in naturalistic settings and it it critical to humans in novel complex environments. Unsupervised spatial learning is based on internal representations which allow for a flexible acquisition of knowledge, situational awareness as well as readiness to act appropriately. The overall objective of this project is to understand the basic brain and behavioral processes underlying this type of learning and training. Sergei Gepshtein and his colleagues are developing new computational models and experimental paradigms for research of action and decision making, to improve our understanding of how subjects act in rapidly changing environments, under risk and uncertainty.

• Vision science for dynamic architecture
  Harold Hay Grant Research Program
  Academy of Neuroscience for Architecture
  
  This project is a collaboration between several disciplines: systems neuroscience and sensory psychophysics (Sergei Gepshtein at the Salk Institute for Biological Studies), production design and world building (Alex McDowell at USC World Building Media Lab), architectural and urban design (Greg Lynn at UCLA Department of Architecture and SUPRASTUDIO). Using the computational and experimental tools developed at the Salk Institute, we conduct a series of studies at USC and UCLA. The goal is to reveal the spatial organization of spaces generated by built environments.
  
  

• Neural mechanisms underlying adaptive optimization of visual sensitivity
  National Institutes of Health
  
  One of the fundamental tenets of sensory biology is that sensory systems adapt to environmental change. It has been argued that adaptation should have the effect of optimizing sensitivity to the new environment. To make this premise concrete and precise, the proposed research builds on a normative theory of visual motion perception, which argues that the visual system will adapt optimally by balancing stimulus and measurement uncertainties. This theory makes predictions about visual spatiotemporal sensitivity as a function of environmental statistics: Adaptive optimization should be manifested as a change in spatiotemporal sensitivity for an observer and for the underlying motion-sensitive neurons. We test these predictions by measuring effects of adaptation on visual sensitivity.

• Gestalt detection
  National Science Foundation
  
  When we look at the world, how does the nervous system know which parts of the visual input belong to the same object and which do not? The process known as perceptual grouping takes elements of the visual input and combines them into what we experience as a visual scene that contains objects, people, plants, shadows, and so on. Most of the time perceptual grouping is involuntary but it can come under voluntary control. For this reason, the study of perceptual grouping is a part of the larger effort toward understanding consciousness. Although phenomena of perceptual grouping are an essential foundation of perception, they are often described using a list of qualitative "principles," such as proximity, similarity, and good continuation, that are vague and unquantified. The goal of this project is to clarify some of the fundamental processes of perceptual grouping, starting with simple visual patterns that allow one to study one force of perceptual grouping at a time. The individual forces of grouping are then combined using more complex visual patterns, aiming to derive general quantitative laws of perceptual grouping. This study involves the interaction of geometric factors (such as proximity between elements of visual displays) and intensive factors (such as the luminance and contrast of the elements) in perceptual grouping. The laws of combination of grouping factors are compared with the laws of combination of other sensory cues, which have been intensively studied in the perception of visual depth and in multisensory integration.

• Gepshtein, Pawar, Kwon, Savel'ev, Albright (2022)
  Spatially distributed computation in cortical circuits
  Science Advances 8 (16) Article abl5865 [PDF] [+]

  The traditional view of neural computation in the cerebral cortex holds that sensory neurons are specialized, i.e., selective for certain dimensions of sensory stimuli. This view was challenged by evidence of contextual interactions between stimulus dimensions in which a neuron’s response to one dimension strongly depends on other dimensions. Here we use methods of mathematical modeling, psychophysics, and electrophysiology to address shortcomings of the traditional view. Using a model of a generic cortical circuit, we begin with the simple demonstration that cortical responses are always distributed among neurons, forming characteristic waveforms which we call "neural waves." When stimulated by patterned stimuli, circuit responses arise by interference of neural waves. Results of this process depend on interaction between stimulus dimensions. Comparison of modeled responses with responses of biological vision makes it clear that the framework of neural wave interference provides a useful alternative to the standard concept of neural computation.
  
  Tagline: Investigating interference of neural waves helps to overcome limitations of the traditional view of cortical computation.
  
  Gepshtein S, Pawar AS, Kwon S, Savel'ev S & Albright TD (2022). Spatially distributed computation in cortical circuits. Science Advances 8 (16), 1-19. doi: 10.1126/sciadv.abl5865.

• Proietti, Gepshtein (2021)
  Architectural proportion from an empirical standpoint
  Journal of Interior Design 47 (1) 11-29 [PDF] [+]

  Proportion has long been central to theories of design and architecture before its role was questioned in the mid-twentieth century. In spite of this scrutiny, no definitive conclusion has been reached on the question of how proportion affects experience of the built environment. Here, an architect and a scientist attack this question from an empirical standpoint. They argue that, as a starting point, it is important to understand under which conditions proportions are perceptible by the flesh-and-blood person who moves freely through the built environment and observes it under various distances and angles. The authors begin by surveying the work of two forerunners of empirical study of proportion: The Russian architect and educator Nikolai Ladovsky, whose “psychotechnical” approach was inspired by scientific psychology, and the Dutch architect Hans van der Laan, who developed the theory of plastic number concerned with proportions of three-dimensional objects, and whose approach was strikingly similar to scientific approaches. Following an analysis of informal studies by Ladovsky and Van der Laan, the authors examine implications of such studies for education in architecture and interior design, and then describe two approaches to investigating architectural proportion formally, by methods of sensory psychophysics.
  
  Proietti T & Gepshtein S (2021). Architectural proportion from an empirical standpoint. Journal of Interior Design, 47 (1), 11-29. doi: 10.1111/joid.12210.
  
  

• Gepshtein, Wang, He, Diep, Albright (2020)
  A perceptual scaling approach to eyewitness identification
  Nature Communications 11 Article 3380 [PDF] [+]

  Eyewitness misidentification accounts for 70% of verified erroneous convictions. To address this alarming phenomenon, research has focused on factors that influence likelihood of correct identification, such as the manner in which a lineup is conducted. Traditional lineups rely on overt eyewitness responses that confound two covert factors: strength of recognition memory and the criterion for deciding what memory strength is sufficient for identification.
  
  Here we describe a lineup that permits estimation of memory strength independent of decision criterion. Our procedure employs powerful techniques developed in studies of perception and memory: perceptual scaling and signal detection analysis. Using these tools, we scale memory strengths elicited by lineup faces, and quantify performance of a binary classifier tasked with distinguishing perpetrator from innocent suspect. This approach reveals structure of memory inaccessible using traditional lineups and renders accurate identifications uninfluenced by decision bias. The approach furthermore yields a quantitative index of individual eyewitness performance.
  
  Gepshtein S, Wang Y, He F, Diep D & Albright TD (2020). A perceptual scaling approach to eyewitness identification. Nature Communications, 11, Article 3380..

• Gepshtein, Snider (2019)
  Neuroscience for architecture: The evolving science of perceptual meaning
  Proceedings of the National Academy of Sciences, USA 116 (29) 14404-14406 [journal link] [+]

  "Besides its traditional reliance on the tacit knowledge of timeless practices of construction, architecture relies largely on theories and findings of other areas of research and knowledge, instead of possessing an independent theoretical foundation of its own. During the past decades, architecture has been viewed from various theoretical perspectives, provided by, for instance, psychology, psychoanalysis, structural linguistics and anthropology as well as deconstructionist and phenomenological philosophies, just to name a few." -Juhani Pallasmaa
  
  Today this sweeping array of theories and perspectives is being expanded again by the disciplines allied under the loosely defined umbrella of neuroscience. Several families of concepts, paradigms, and methods from the neurosciences appear to be perfectly suitable for investigating what the architect would call the "human response to the built environment." These overlapping families include systems neuroscience and affective neuroscience, sensorimotor psychophysics and experimental phenomenology, to mention only some of the contenders. It is far from clear, however, which ideas will stick and what shape they will take in the new context. Just as it has happened to many prior imports to architecture, hard-won scientific knowledge may remain a foreign entity within the living body of architecture, supplying occasional metaphor and inspiring freewheeling speculation. Or the sciences may retain their natal rigor and invigorate architectural theory and practice by helping architects to test some old ideas and possibly rid their discipline of unbuttoned preconceptions, some of which had already been subjected to incisive analytical scrutiny...
  
  Gepshtein S & Snider J (2019). Neuroscience for architecture: The evolving science of perceptual meaning. Proceedings of the National Academy of Sciences, USA 116 (29) 14404-14406.

• Gepshtein (2019)
  On some paradoxes of current perceptual theories
  In Bianchi I & Davies R (Eds.) Paolo Bozzi's Experimental Phenomenology, 56-63 [publisher link] [+]

  Let us recall that the philosophical discipline of phenomenology and the scientific discipline of experimental phenomenology have a common origin. This origin is the descriptive psychology introduced by Franz Brentano and promulgated by the "Brentano circle" to the threshold of a new approach in the philosophy of mind and a new approach in empirical psychology. The aspiration of descriptive psychology was to develop a "science of mental phenomena." This aspiration became figurative in philosophical phenomenology and literal in experimental phenomenology, but the common origin left its mark on both disciplines. From the outset, their common goal was to investigate the human mind from an adamantly first-person perspective.
  
  Let us also recall that the scientific study of the mind was the goal of another discipline that made no commitment to the first-person perspective. This discipline is sensory psychophysics, conceived just before Brentano's descriptive psychology and dedicated to investigating mental phenomena from the third-person perspective of natural science...
  
  Gepshtein S (2019). On some paradoxes of current perceptual theories. In Bianchi I & Davies R (Eds.) Paolo Bozzi's Experimental Phenomenology, 514-527.

• Pawar, Gepshtein, Savel'ev, Albright (2019)
  Mechanisms of spatiotemporal selectivity in cortical area MT
  Neuron 101 (3) 514-527 [journal link] [+]

  Cortical sensory neurons are characterized by selectivity to stimulation. This selectivity was originally viewed as a part of the fundamental "receptive field" characteristic of neurons. This view was later challenged by evidence that receptive fields are modulated by stimuli outside of the classical receptive field. Here we show that even this modified view of selectivity needs revision. We measured spatial frequency selectivity of neurons in cortical area MT of alert monkeys and found that their selectivity strongly depends on luminance contrast, shifting to higher spatial frequencies as contrast increases. The changes of preferred spatial frequency are large at low temporal frequency and they decrease monotonically as temporal frequency increases. That is, even interactions among basic stimulus dimensions of luminance contrast, spatial frequency and temporal frequency strongly influence neuronal selectivity. This dynamic nature of neuronal selectivity is inconsistent with the notion of stimulus preference as a stable characteristic of cortical neurons.
  
  Pawar AS, Gepshtein S, Savel'ev S & Albright TD (2019). Mechanisms of spatiotemporal selectivity in cortical area MT. Neuron 101 (3), 514-527.

• Zharikova A, Gepshtein S & van Leeuwen C (2017)
  Paradoxical perception of object identity in visual motion
  Vision Research 136 1-14. [pdf] [+]

  In the course of perceptual organization, incomplete optical stimulation can evoke the experience of complete objects with distinct perceptual identities. According to a well-known principle of perceptual organization, stimulus parts separated by shorter spatial distances are more likely to appear as parts of the same perceptual identity. Whereas this principle of proximity has been confirmed in many studies of perceptual grouping in static displays, we show that it does not generalize to perception of object identity in dynamic displays, where the parts are separated by spatial and temporal distances.
  
  We use ambiguous displays which contain multiple moving parts and which can be perceived two ways: as two large objects that gradually change their size or as multiple smaller objects that rotate independent of one another. Grouping over long and short distances corresponds to the perception of the respectively large and small objects. We find that grouping over long distances is often preferred to grouping over short distances, against predictions of the proximity principle. Even though these effects are observed at high luminance contrast, we show that they are consistent with results obtained at the threshold of luminance contrast, in agreement with predictions of a theory of efficient motion measurement. This is evidence that the perception of object identity can be explained by a computational principle of neural economy rather than by the empirical principle of proximity.
  
  Zharikova A, Gepshtein S & van Leeuwen C (2017). Paradoxical perception of object identity in visual motion. Vision Research, 136, 1-14.

• Gepshtein, Albright (2017)
  Adaptive optimization of visual sensitivity
  Journal of the Indian Institute of Science 97 (4) 423-434 [pdf] [+]

  Sensory systems adapt to environmental change. It has been argued that adaptation should have the effect of optimizing sensitivity to the new environment. Here we consider a framework in which this premise is made concrete using an economic normative theory of visual motion perception. In this framework, visual systems adapt to the environment by reallocating their limited neural resources. The allocation is optimal when uncertainties about different aspects of stimulation are balanced. This theory makes predictions about visual sensitivity as a function of environmental statistics. Adaptive optimization of the visual system should be manifested as a change in sensitivity for an observer and for the underlying motion-sensitive neurons. We review evidence supporting these predictions and examine effects of adaptation on the neuronal representation of visual motion.
  
  Gepshtein S & Albright TD (2017). Adaptive Optimization of Visual Sensitivity. Journal of the Indian Institute of Science, 97 (4), 423-434.

• Snider, Lee, Poizner, Gepshtein (2015)
  Prospective optimization with limited resources
  PLoS Computational Biology 11 (9) e1004501 [link] [+]

  The future is uncertain because some forthcoming events are unpredictable and also because our ability to foresee the myriad consequences of our own actions is limited. Here we studied how humans select actions under such extrinsic and intrinsic uncertainty, in view of an exponentially expanding number of prospects on a branching multivalued visual stimulus.
  
  A triangular grid of disks of different sizes scrolled down a touchscreen at a variable speed. The larger disks represented larger rewards. The task was to maximize the cumulative reward by touching one disk at a time in a rapid sequence, forming an upward path across the grid, while every step along the path constrained the part of the grid accessible in the future. This task captured some of the complexity of natural behavior in the risky and dynamic world, where ongoing decisions alter the landscape of future rewards. By comparing human behavior with behavior of ideal actors, we identified the strategies used by humans in terms of how far into the future they looked (their "depth of computation") and how often they attempted to incorporate new information about the future rewards (their "recalculation period").
  
  We found that, for a given task difficulty, humans traded off their depth of computation for the recalculation period. The form of this tradeoff was consistent with a complete, brute-force exploration of all possible paths up to a resource-limited finite depth. A step-by-step analysis of the human behavior revealed that participants took into account very fine distinctions between the future rewards and that they abstained from some simple heuristics in assessment of the alternative paths, such as seeking only the largest disks or avoiding the smaller disks. The participants preferred to reduce their depth of computation or increase the recalculation period rather than sacrifice the precision of computation.
  
  Snider J, Lee D, Poizner H & Gepshtein S (2015). Prospective optimization with limited resources. PLoS Computational Biology, 11 (9): e1004501. doi:10.1371/journal.pcbi.1004501.
  
  

• Gepshtein, Lesmes, Albright (2013)
  Sensory adaptation as optimal resource allocation
  Proceedings of the National Academy of Sciences, USA 110 4368-4373 [pdf] [+]

  Visual adaptation is expected to improve visual performance in the new environment. The expectation has been contradicted by evidence that adaptation sometimes decreases sensitivity for the adapting stimuli, and sometimes it changes sensitivity for stimuli very different from the adapting ones. We hypothesize that this pattern of results can be explained by a process that optimizes sensitivity for many stimuli, rather than changing sensitivity only for those stimuli whose statistics have changed. To test this hypothesis, we measured visual sensitivity across a broad range of spatiotemporal modulations of luminance, while varying the distribution of stimulus speeds. The manipulation of stimulus statistics caused a large-scale reorganization of visual sensitivity, forming the orderly pattern of sensitivity gains and losses. This pattern is predicted by a theory of distribution of receptive field characteristics in the visual system.
  
  Gepshtein S, Lesmes LA & Albright TD (2013). Sensory adaptation as optimal resource allocation. Proceedings of the National Academy of Sciences, USA 110 (11), 4368-4373.
  
  

• Jurica, Gepshtein, Tyukin, van Leeuwen (2013)
  Sensory optimization by stochastic tuning
  Psychological Review 120 798-816 [preprint] [+]

  Jurica P, Gepshtein S, Tyukin I & van Leeuwen C (2013). Sensory optimization by stochastic tuning. Psychological Review 120 (4) 798-816. doi: 10.1037/a0034192.
  
  Individually, visual neurons are each selective for several aspects of stimulation, such as stimulus location, frequency content, and speed. Collectively, the neurons implement the visual system's preferential sensitivity to some stimuli over others, manifested in behavioral sensitivity functions. We ask how the individual neurons are coordinated to optimize visual sensitivity. We model synaptic plasticity in a generic neural circuit, and find that stochastic changes in strengths of synaptic connections entail fluctuations in parameters of neural receptive fields. The fluctuations correlate with uncertainty of sensory measurement in individual neurons: the higher the uncertainty the larger the amplitude of fluctuation. We show that this simple relationship is sufficient for the stochastic fluctuations to steer sensitivities of neurons toward a characteristic distribution, from which follows a sensitivity function observed in human psychophysics, and which is predicted by a theory of optimal allocation of receptive fields. The optimal allocation arises in our simulations without supervision or feedback about system performance and independently of coupling between neurons, making the system highly adaptive and sensitive to prevailing stimulation.
  
  

• Gepshtein (2010)
  Two psychologies of perception and the prospect of their synthesis
  Philosophical Psychology 23 217-281 [pdf] [+]

  Gepshtein S (2010). Two psychologies of perception and the prospect of their synthesis. Philosophical Psychology 23 (2), 217-281.
  
  Two traditions have had a great impact on the theoretical and experimental research of perception. One tradition is statistical, stretching from Fechner's enunciation of psychophysics in 1860 to the modern view of perception as statistical decision making. The other tradition is phenomenological, from Brentano's "empirical standpoint" of 1874 to the Gestalt movement and the modern work on perceptual organization. Each tradition has at its core a distinctive assumption about the indivisible constituents of perception: the just-noticeable differences of sensation in the tradition of Fechner vs. the phenomenological Gestalts in the tradition of Brentano. But some key results from the two traditions can be explained and connected using an approach that is neither statistical nor phenomenological. This approach rests on a basic property of any information exchange: a principle of measurement formulated in 1946 by Gabor as a part of his quantal theory of information. Here the indivisible components are units (quanta) of information that remain invariant under changes of precision of measurement. This approach helped to understand how sensory measurements are implemented by single neural cells. But recent analyses suggest that this approach has the power to explain larger-scale characteristics of sensory systems.
  
  

• Nikolaev, Gepshtein, Gong, van Leeuwen (2010)
  Coherence intervals in electrical brain activity and perceptual organization
  Cerebral Cortex 20 365-382 [pdf] [+]

  Nikolaev AR, Gepshtein S, Gong P & van Leeuwen C (2010). Duration of coherence intervals in electrical brain activity in perceptual organization. Cerebral Cortex 20 (2), 365-382.
  
  We investigated the relationship between visual experience and temporal intervals of synchronized brain activity. Using high-density scalp electroencephalography, we examined how synchronized activity depends on visual stimulus information and on individual observer sensitivity. In a perceptual grouping task, we varied the ambiguity of visual stimuli and estimated observer sensitivity to this variation. We found that durations of synchronized activity in the beta frequency band were associated with both stimulus ambiguity and sensitivity: the lower the stimulus ambiguity and the higher individual observer sensitivity the longer were the episodes of synchronized activity. Durations of synchronized activity intervals followed an extreme value distribution, indicating that they were limited by the slowest mechanism among the multiple neural mechanisms engaged in the perceptual task. Because the degree of stimulus ambiguity is (inversely) related to the amount of stimulus information, the durations of synchronous episodes reflect the amount of stimulus information processed in the task. We therefore interpreted our results as evidence that the alternating episodes of desynchronized and synchronized electrical brain activity reflect, respectively, the processing of information within local regions and the transfer of information across regions.
  
  

• Gepshtein, Kubovy (2007)
  The lawful perception of apparent motion
  Journal of Vision 7 8:9 1-15 [pdf] [+]

  Gepshtein S & Kubovy M (2007). The lawful perception of apparent motion. Journal of Vision 7 (8):9, 1-15.
  
  Visual apparent motion is the experience of motion from the successive stimulation of separate spatial locations. How spatial and temporal distances interact to determine the strength of apparent motion has been controversial. Some studies report space-time coupling: If we increase spatial or temporal distance between successive stimuli, we must also increase the other distance between them to maintain a constant strength of apparent motion (Korte's third law of motion). Other studies report space-time tradeoff: If we increase one of these distances, we must decrease the other to maintain a constant strength of apparent motion. In this article, we resolve the controversy. Starting from a normative theory of motion measurement and data on human spatiotemporal sensitivity, we conjecture that both coupling and tradeoff should occur, but at different speeds. We confirm the prediction in two experiments, using suprathreshold multistable apparent-motion displays called motion lattices. Our results show a smooth transition between the tradeoff and coupling as a function of speed: Tradeoff occurs at low speeds and coupling occurs at high speeds. From our data, we reconstruct the suprathreshold equivalence contours that are analogous to isosensitivity contours obtained at the threshold of visibility.
  
  

• Gepshtein, Tyukin, Kubovy (2007)
  The economics of motion perception and invariants of visual sensitivity
  Journal of Vision 7 8:8 1-18 [pdf] [+]

  Gepshtein S, Tyukin I & Kubovy M (2007). The economics of motion perception and invariants of visual sensitivity. Journal of Vision 7 (8):8, 1-18.
  
  Neural systems face the challenge of optimizing their performance with limited resources, just as economic systems do. Here, we use tools of neoclassical economic theory to explore how a frugal visual system should use a limited number of neurons to optimize perception of motion. The theory prescribes that vision should allocate its resources to different conditions of stimulation according to the degree of balance between measurement uncertainties and stimulus uncertainties. We find that human vision approximately follows the optimal prescription. The equilibrium theory explains why human visual sensitivity is distributed the way it is and why qualitatively different regimes of apparent motion are observed at different speeds. The theory offers a new normative framework for understanding the mechanisms of visual sensitivity at the threshold of visibility and above the threshold and predicts large-scale changes in visual sensitivity in response to changes in the statistics of stimulation and system goals.
  
  

• Trommershäuser, Gepshtein, Maloney, Landy, Banks (2005)
  Optimal compensation for changes in task relevant movement variability
  Journal of Neuroscience 25 7169-7178 [pdf] [+]

  Trommershäuser J, Gepshtein S, Maloney LT, Landy MS & Banks MS (2005). Optimal compensation for changes in task relevant movement variability. Journal of Neuroscience 25 (31), 7169-7178.
  
  Effective movement planning should take into account the consequences of possible errors in executing a planned movement. These errors can result from either sensory uncertainty or variability in movement planning and production. We examined the ability of humans to compensate for variability in sensory estimation and movement production under conditions in which variability is increased artificially by the experimenter. Subjects rapidly pointed at a target region that had an adjacent penalty region. Target and penalty hits yielded monetary rewards and losses. We manipulated the task-relevant variability by perturbing visual feedback of finger position during the movement. The feedback was shifted in a random direction with a random amplitude in each trial, causing an increase in the task-relevant variability. Subjects were unable to counteract this form of perturbation. Rewards and penalties were based on the perturbed, visually specified finger position. Subjects rapidly acquired an estimate of their new variability in <120 trials and adjusted their aim points accordingly. We compared subjects' performance to the performance of an optimal movement planner maximizing expected gain. Their performance was consistent with that expected from an optimal movement planner that perfectly compensated for externally imposed changes in task-relevant variability. When exposed to novel stimulus configurations, aim points shifted in the first trial without showing any detectable trend across trials. These results indicate that subjects are capable of changing their pointing strategy in the presence of externally imposed noise. Furthermore, they manage to update their estimate of task-relevant variability and to transfer this estimate to novel stimulus configurations.
  
  

• Banks, Gepshtein, Landy (2004)
  Why is spatial stereoresolution so low?
  Journal of Neuroscience 24 2077-2089 [pdf] [+]

  Banks MS, Gepshtein S & Landy MS (2004). Why is spatial stereoresolution so low? Journal of Neuroscience 24 (9), 2077-2089.
  
  Spatial stereoresolution (the finest detectable modulation of binocular disparity) is much poorer than luminance resolution (finest detectable luminance variation). In a series of psychophysical experiments, we examined four factors that could cause low stereoresolution: (1) the sampling properties of the stimulus, (2) the disparity gradient limit, (3) low-pass spatial filtering by mechanisms early in the visual process, and (4) the method by which binocular matches are computed. Our experimental results reveal the contributions of the first three factors. A theoretical analysis of binocular matching by interocular correlation reveals the contribution of the fourth: the highest attainable stereoresolution may be limited by (1) the smallest useful correlation window in the visual system, and (2) a matching process that estimates the disparity of image patches and assumes that disparity is constant across the patch. Both properties are observed in disparity-selective neurons in area V1 of the primate (Nienborg et al., 2004).
  
  

• Gepshtein, Banks (2003)
  Viewing geometry determines how vision and touch combine in size perception
  Current Biology 13 483-488 [pdf] [+]

  Gepshtein S & Banks MS (2003). Viewing geometry determines how vision and touch combine in size perception. Current Biology 13 (6), 483-488.
  
  Vision and haptics have different limitations and advantages because they obtain information by different methods. If the brain combined information from the two senses optimally, it would rely more on the one providing more precise information for the current task. In this study, human observers judged the distance between two parallel surfaces in two within-modality experiments (vision-alone and haptics-alone) and in an intermodality experiment (vision and haptics together). We find that the combined size estimates are finer than it is possible with either vision or haptics alone. Indeed, the combined estimates approach statistical optimality.
  
  

• Gepshtein, Kubovy (2000)
  The emergence of visual objects in space-time
  Proceedings of the National Academy of Sciences, USA 97 8186-8191 [pdf] [+]

  Gepshtein S & Kubovy M (2000). The emergence of visual objects in space-time. Proceedings of the National Academy of Sciences, USA 97 (14), 8186-8191.
  
  It is is natural to think that in perceiving dynamic scenes, vision takes a series of snapshots. Motion perception can ensue when the snapshots are different. The snapshot metaphor suggests two questions: (i) How does the visual system put together elements within each snapshot to form objects? This is the spatial grouping problem. (ii) When the snapshots are different, how does the visual system know which element in one snapshot corresponds to which element in the next? This is the temporal grouping problem. The snapshot metaphor is a caricature of the dominant model in the field (the sequential model) according to which spatial and temporal grouping are independent. The model we propose here is an interactive model, according to which the two grouping mechanisms are not separable.
  
  

• COMPLETE LIST