Sergei Gepshtein

Center for Neurobiology of Vision
The Salk Institute for Biological Studies
10010 North Torrey Pines Road
La Jolla, CA 92037, USA

858.453.4100 ext 1014 | office
858.546.8526 | fax
sergei (at) salk.edu


Overview

Sergei Gepshtein is a scientist specializing in perceptual psychology and sensorimotor neuroscience. Much of his research dwells on the interface between two aspects of visual perception: the entry process called "early vision" and the constructive process called "perceptual organization."

  • Early vision is essentially a gate — or filter — that controls what information penetrates the visual system. The filter is plastic. It rapidly changes in accord with your optical environment and in service of your intentions.
  • The information that enters the visual system must be organized: grouped, segmented, layered, or else it remains outside of your awareness. Research of this process of perceptual organization is an offshoot of the Gestalt movement in psychology, which became a mature science only recently, a century after the Gestalt movement was born.

Sergei studies computational principles and biological mechanisms underlying these processes, in particular how visual systems organize information for the perception of motion and change.

He also studies sensorimotor integration, which is a field of research concerned with how we use visual and haptic information to guide action. A key questions is how vision helps us to plan actions prospectively, for many steps ahead, in view of the dynamic nature of the environment, its varying risks and uncertainties.

As a scientist and a principal investigator at the Salk Institute for Biological Studies, he uses experimental and computational methods to characterize neuronal mechanisms of sensation, perception, and action.

As a member of the board of directors of the Academy of Neuroscience for Architecture, and as founding member of the Worldbuilding Institute (originally 5D Institute), Sergei is increasingly engaged in translational and applied research, which includes design and analysis of the built environment and sensory media.

In 2017, he founded the Center of Spatial Perception & Concrete Experience (SPaCE) at the University of Southern California’s School of Cinematic Arts, for which he serves as the director, with the goal to develop new forms of physical, immersive, and environmental “spatial media,” including augmented reality, virtual reality, and beyond.

As an educator, Sergei helps to develop and teach new "art-science" curricula connecting scientific and design disiplines at the University of Southern California (Los Angeles), the New School for Architecture and Design (San Diego), and the University of California (San Diego).

Projects

  • Neural mechanisms of visual sensitivity
    National Institutes of Health (2019-2024)

    One of the most important questions in neuroscience today concerns the mechanisms by which sensory neurons give rise to perceptual experience. There are many ways to address this question, which have long populated the field of visual neuroscience. Prominent among them is the study of visual selectivity. Observers are highly sensitive to some visual stimuli and less sensitive to others. Visual neurons are also highly selective: Each responds to a limited range of stimuli along several stimulus dimensions. We aim to understand how the selective pattern of neuronal responses accounts for the observer's selective perceptual experience and discriminative capacity. This understanding is achieved through experiments that first evaluate patterns of neuronal selectivity for visual stimuli that vary in their spatial and temporal properties. Second, these data, in combination with manipulations of stimulus context, are used to develop a novel mechanistic account of neuronal selectivity based on activity within cortical visual circuits stabilized by inhibition. Finally, to understand how neuronal selectivity underlies perceptual experience, direct comparisons will be made between physiological measures of neuronal selectivity and behavioral measures of perceptual selectivity, assessed concurrently under identical conditions. The experiments will yield an unprecedented body of comprehensive data regarding the spatiotemporal tuning of the primate visual system. These data will be used to further understanding of the mechanisms of sensory processing and will provide insights into pathologies of vision caused by trauma, disease and developmental disorders of the brain.
  • Enhancing eyewitness performance by optimizing context
    The Laura and John Arnold Foundation (ongoing)
    Project site at Open Science Framework

    Eyewitness identification has a long history of use as a form of forensic evidence by law enforcement and the courts in the USA and elsewhere. It has played a valuable role in both convictions and acquittals. Despite this value, our society has been confronted with egregious failures of this type of evidence, leading to conviction of innocent people. Of particular note are findings from the era of post-conviction DNA profiling, during which over 350 people, many serving extended prison sentences, have been exonerated based on DNA evidence. In approximately 70% of these cases, the original erroneous convictions resulted from testimony of eyewitnesses. The consequences of erroneous convictions based on flawed eyewitness testimony are profound. Not only have we imprisoned innocent people while the guilty remain at large, but these outcomes risk undermining public trust in the criminal justice system. This project seeks to improve the performance of eyewitnesses using a basic science approach derived from traditional research on human sensory processing and memory.
  • Improving wayfinding and reducing challenging behaviors in persons with dementia using adaptive visual technologies
    National Institutes of Health (ongoing)

    An estimated 5.4 million Americans are currently suffering from Alzheimer's disease. Besides affecting the person with the condition, Alzheimer's disease also presents a public health concern, due to the significant cost required to care for persons with dementia (PWD) and to the mental and physical stress placed upon caregivers. There is an urgent need for products and programs that improve the quality of life of PWD and reduce challenging behaviors. We develop an adaptive visual technology to improve wayfinding and reduce challenging behaviors in PWD by automatically presenting PWD with customized messages on displays within memory-care facilities.
  • Sensorimotor optimization in dynamic environments
    ONR MURI Center: From brain to behavior

    The human cortex creates rich representations of the world. These representations are based on learning, which often proceeds self-supervised. This kind of learning - often dubbed "unsupervised" - is commonplace in naturalistic settings and it it critical to humans in novel complex environments. Unsupervised spatial learning is based on internal representations which allow for a flexible acquisition of knowledge, situational awareness as well as readiness to act appropriately. The overall objective of this project is to understand the basic brain and behavioral processes underlying this type of learning and training. Sergei Gepshtein and his colleagues are developing new computational models and experimental paradigms for research of action and decision making, to improve our understanding of how subjects act in rapidly changing environments, under risk and uncertainty.
  • Vision science for dynamic architecture
    Harold Hay Grant Research Program
    Academy of Neuroscience for Architecture

    This project is a collaboration between several disciplines: systems neuroscience and sensory psychophysics (Sergei Gepshtein at the Salk Institute for Biological Studies), production design and world building (Alex McDowell at USC World Building Media Lab), architectural and urban design (Greg Lynn at UCLA Department of Architecture and SUPRASTUDIO). Using the computational and experimental tools developed at the Salk Institute, we conduct a series of studies at USC and UCLA. The goal is to reveal the spatial organization of spaces generated by built environments.

  • Neural mechanisms underlying adaptive optimization of visual sensitivity
    National Institutes of Health

    One of the fundamental tenets of sensory biology is that sensory systems adapt to environmental change. It has been argued that adaptation should have the effect of optimizing sensitivity to the new environment. To make this premise concrete and precise, the proposed research builds on a normative theory of visual motion perception, which argues that the visual system will adapt optimally by balancing stimulus and measurement uncertainties. This theory makes predictions about visual spatiotemporal sensitivity as a function of environmental statistics: Adaptive optimization should be manifested as a change in spatiotemporal sensitivity for an observer and for the underlying motion-sensitive neurons. We test these predictions by measuring effects of adaptation on visual sensitivity.
  • Gestalt detection
    National Science Foundation

    When we look at the world, how does the nervous system know which parts of the visual input belong to the same object and which do not? The process known as perceptual grouping takes elements of the visual input and combines them into what we experience as a visual scene that contains objects, people, plants, shadows, and so on. Most of the time perceptual grouping is involuntary but it can come under voluntary control. For this reason, the study of perceptual grouping is a part of the larger effort toward understanding consciousness. Although phenomena of perceptual grouping are an essential foundation of perception, they are often described using a list of qualitative "principles," such as proximity, similarity, and good continuation, that are vague and unquantified. The goal of this project is to clarify some of the fundamental processes of perceptual grouping, starting with simple visual patterns that allow one to study one force of perceptual grouping at a time. The individual forces of grouping are then combined using more complex visual patterns, aiming to derive general quantitative laws of perceptual grouping. This study involves the interaction of geometric factors (such as proximity between elements of visual displays) and intensive factors (such as the luminance and contrast of the elements) in perceptual grouping. The laws of combination of grouping factors are compared with the laws of combination of other sensory cues, which have been intensively studied in the perception of visual depth and in multisensory integration.

Publications

Selected publications
  • Gepshtein, Pawar, Kwon, Savel'ev, Albright (2022)
    Spatially distributed computation in cortical circuits
    Science Advances 8 (16) Article abl5865 [PDF] [+]
  • Proietti, Gepshtein (2021)
    Architectural proportion from an empirical standpoint
    Journal of Interior Design 47 (1) 11-29 [PDF] [+]
  • Gepshtein, Wang, He, Diep, Albright (2020)
    A perceptual scaling approach to eyewitness identification
    Nature Communications 11 Article 3380 [PDF] [+]
  • Gepshtein, Snider (2019)
    Neuroscience for architecture: The evolving science of perceptual meaning
    Proceedings of the National Academy of Sciences, USA 116 (29) 14404-14406 [journal link] [+]
  • Gepshtein (2019)
    On some paradoxes of current perceptual theories
    In Bianchi I & Davies R (Eds.) Paolo Bozzi's Experimental Phenomenology, 56-63 [publisher link] [+]
  • Pawar, Gepshtein, Savel'ev, Albright (2019)
    Mechanisms of spatiotemporal selectivity in cortical area MT
    Neuron 101 (3) 514-527 [journal link] [+]
  • Zharikova A, Gepshtein S & van Leeuwen C (2017)
    Paradoxical perception of object identity in visual motion
    Vision Research 136 1-14. [pdf] [+]
  • Gepshtein, Albright (2017)
    Adaptive optimization of visual sensitivity
    Journal of the Indian Institute of Science 97 (4) 423-434 [pdf] [+]
  • Snider, Lee, Poizner, Gepshtein (2015)
    Prospective optimization with limited resources
    PLoS Computational Biology 11 (9) e1004501 [link] [+]
  • Gepshtein, Lesmes, Albright (2013)
    Sensory adaptation as optimal resource allocation
    Proceedings of the National Academy of Sciences, USA 110 4368-4373 [pdf] [+]
  • Jurica, Gepshtein, Tyukin, van Leeuwen (2013)
    Sensory optimization by stochastic tuning
    Psychological Review 120 798-816 [preprint] [+]
  • Gepshtein (2010)
    Two psychologies of perception and the prospect of their synthesis
    Philosophical Psychology 23 217-281 [pdf] [+]
  • Nikolaev, Gepshtein, Gong, van Leeuwen (2010)
    Coherence intervals in electrical brain activity and perceptual organization
    Cerebral Cortex 20 365-382 [pdf] [+]
  • Gepshtein, Kubovy (2007)
    The lawful perception of apparent motion
    Journal of Vision 7 8:9 1-15 [pdf] [+]
  • Gepshtein, Tyukin, Kubovy (2007)
    The economics of motion perception and invariants of visual sensitivity
    Journal of Vision 7 8:8 1-18 [pdf] [+]
  • Trommershäuser, Gepshtein, Maloney, Landy, Banks (2005)
    Optimal compensation for changes in task relevant movement variability
    Journal of Neuroscience 25 7169-7178 [pdf] [+]
  • Banks, Gepshtein, Landy (2004)
    Why is spatial stereoresolution so low?
    Journal of Neuroscience 24 2077-2089 [pdf] [+]
  • Gepshtein, Banks (2003)
    Viewing geometry determines how vision and touch combine in size perception
    Current Biology 13 483-488 [pdf] [+]
  • Gepshtein, Kubovy (2000)
    The emergence of visual objects in space-time
    Proceedings of the National Academy of Sciences, USA 97 8186-8191 [pdf] [+]

Unfolding

Double-click the blue markers [+] for further detail.

[+] Solid field of visibility: first model and test
[+] Prospective optimization and visual representation
[+] MT neurons have different tuning at contrast threshold and above
[+] Visual adaptation by stochastic tuning
[+] Invariants and variability of visual sensitivity
[+] Adaptive estimation of spatiotemporal contrast sensitivity
[+] Perception and action in immersive worlds
[+] Oxford handbook of computational perceptual organization

Press & public events

Solid field of sensitivity: perceptual structure of immersive space
Keynote at VI International Conference on Spatial Cognition | Rome, September 10, 2015
WEAVING MERCURY: Art and Science in Perception
Livestream panel at FMX conference on animation, effects, games and transmedia | Stuttgart, May 5, 2015
Seen and unseen: Could there ever be a "cinema without cuts"?
Scientific American Blogs | April 29, 2014
How the movies of tomorrow will play with your mind
Pacific Standard | April 29, 2014
The visual system as economist: neural resource allocation in visual adaptation
Medical Xpress | April 1, 2013
Despite what you may think, your brain is a mathematical genius
ScienceNewsline | April 10, 2013
Brain waves challenge area-specific view of brain activity [video 1 2]
KU Leuven | March 20, 2013