Comments and reactions to the posts are welcome! Simply click on the "comment" line below each post to see previous comments or on the pencil icon to add a new one!

Beware of fashion...

Reading through yet another study (Huebner and Gegenfurtner, 2012) using natural images as stimuli, I started thinking about their contribution to our understanding of visual processes. Using natural images has undeniably shown that current models of visual processing are still incomplete. Indeed, most model based predictions of neuronal responses to natural images are poor, including for cells in primate primary visual cortex (V1), a cortical area considered by many to be relatively “well-understood”.
Knowing that a model is inaccurate or incomplete and gaining insight into the causes of its failures are, however, two very different things. In my opinion, using “natural” stimuli has hardly, if at all, contributed to the latter. I do believe that the most important results concerning the functional properties of visual neurons have been obtained, and will for a while continue to be, using simple stimuli. Sinusoidal gratings, for example, allow us to use the powerful tools of linear system theory , to systematically study failures of simple linear models, and to uncover the important non-linearities in visual signal processing.

Why is it then that the number of studies using natural images has grown so much in recent years (by my counting, only 2 papers used them in 1990, they were 62 in 2010, and there are already 62 half-way through 2012)? I think it became a fashion due to the increasing ease with which these stimuli can be stored, displayed and manipulated, and to the erroneous notion that using “natural” stimuli would bring us closer an understanding of what neurons do outside of the lab, in “nature”. Such a notion would make us  believe that Galileo Galilei should not have performed   his inclined plane experiments to study mechanics. Instead, he should have saved the trouble by simply observing falling bodies in nature, thereby gaining simultaneous insight into the laws of motion, inertia, acceleration, gravitation and turbulent  air resistance. Well, the inclined plane experiments are now considered seminal in mechanics and a monument of reductionist science, the fabled dropping of objects from the tower of Pisa apparently never  even occurred...

It may be appealing to use complex stimuli to study the complexity of nature, it may even become very fashionable, but the complexity may be so high that it masks the underlying processes. Similar thoughts occur to me when I hear people criticizing such old fashioned methods as extracellular single cell recordings. It is of course much more fashionable to use optogenetics or two-photon calcium imaging. The main question for me is whether we have learned everything we can learn from the old methods using simple stimuli. New methods are great but it doesn’t mean old ones should be abandoned…

- D. Kiper


Unknown said...

Hi Dan,

It could be possible that visual cortex processes "simple" and "complex" stimuli in different ways... I'm thinking of some of Yves' recent work, where complex stimuli effectively change the linearity of receptive fields. I agree that natural stimuli introduce a degree of difficultly in analysing the neural responses they provoke, but it could be that simple stimuli mask some aspects of cortical processing.


Unknown said...

I think we are saying the same thing: complex stimuli can reveal the existence of non-linearities (i.e. as in Yves' work). But they don't help much to characterize them. In principle, I don't think cortex would treat simple and compley stimuli differently, how would it decide ?

Sylvia said...

Several studies using natural movies have shown that neurons give qualitatively different responses to such stimuli compared to artificial ones: To mention 2 examples, neurons in V1 respond with higher temporal precision to movies than gratings [1] and larger cell populations are far less driven by natural stimuli than by drifting gratings, which cause overwhelming but also quickly adapting activity levels [2]. Although such studies do not explain mechanisms underlying neural responses, they show us the limits of simple stimulus studies and point us into directions of future research. After we gained some understanding for the processing of simple stimuli, which admittedly are useful to solve some mysteries, we can now explore the mechanisms the brain developed to process the stimuli we encounter naturally. That the brain does tune in for the processing of natural stimuli showed a close look on activity distributions during spontaneous activity [3]. In contrast to young and naïve animals, the spontaneous activity of neural populations in adult animals matches the activity arising during the presentation of natural movies but not that during artificial gratings. This was interpreted to reflect the prior probability of stimulus statistics, which is then used to compute the posterior of the presented stimulus. I think these insights would be impossible to gain from the sole use of simple stimuli and also do contribute to an understanding of the visual system. Specifically studies of the fine temporal dynamics of single neurons and neural populations in response to stimuli the brain adapted to can, in my opinion, shed light on the relevant non-linearities in neural computations and the adaptational processes of the brain during short intervals and during development.
I see, however, the danger of fashion and the resulting lack of solid thinking. Forcing old models like the n-th revision of linear-nonlinear models onto the new data and using analysis tools developed for responses to artificial stimuli, like reverse-correlation, might not lead to enlightenment. In many cases, responses to natural stimuli alone are studied, e.g. in much of the sparseness literature and surround suppression studies. Comparing effects of simple versus natural stimuli in the same neurons (as the mentioned studies did) seems more fruitful to me – at least at this stage, at which we have difficulties to tame the complexities of natural stimuli.

PS: To my understanding, the Huebner & Gegenfurtner study [4] investigates mechanisms of memory and the effects of conceptually versus purely visually similar items on performance in humans. They don’t primarily aim at explaining vision, and using familiar scenes to use common knowledge of certain concepts doesn’t strike me as an overuse of ‘complex’ stimuli.

[1] Herikstad R, Baker J, Lachaux J-P, Gray CM, Yen S-C (2011) Natural Movies Evoke Spike Trains with Low Spike Time Variability in Cat Primary Visual Cortex. Journal of Neuroscience 31:15844–15860.
[2] Onat S, König P, Jancke D (2011) Natural Scene Evoked Population Dynamics across Cat Primary Visual Cortex Captured with Voltage-Sensitive Dye Imaging. Cerebral cortex (New York, NY : 1991) 21:2542–2554.
[3] Berkes P, Orban G, Lengyel M, Fiser J (2011) Spontaneous Cortical Activity Reveals Hallmarks of an Optimal Internal Model of the Environment. Science 331:83–87.
[4] Huebner GM, Gegenfurtner KR (2012) Conceptual and Visual Features Contribute to Visual Memory for Natural Images. PLoS ONE 7(6): e37575. doi:10.1371/journal.pone.0037575

Kevan Martin said...

It is interesting that the Tony Movshon et al school come out of
Cambridge - perhaps strongly influenced by Fergus Campbell and John Robson, yet their neighbour was Horace Barlow who was taking a very different line - influenced by Fred Attneave and the ecologists of vision - where the features of the natural world were the issue. This was also the direction taken by Jerry Lettvin in his studies of the frog's retinotectal system and by James Gibson in psychophysics. Gibson was of course strongly criticised by David Marr, whose own computer science approach to vision actually led us nowhere.

The problem is a bit that we look for a missing key under the lamplight -where we can see, not where the key actually is. We continue to use simple stimuli because that seems have got us somewhere, even if painfully slowly and not very far. However, the present generation of mouse visualists - difficult to call them physiologists - don't seem to have even attended the INI Computational Vision course for all they understand of the principles behind linear analyses, so with the mouse visual system we really are still in the Dark Ages. For me - if we don't
work with natural stimuli then we won't get any better at understanding what they do to cortex, so there is a case for doing studies like Sylvia's where she looks at both linear and natural stimuli and tries to understand how the same circuits deal with both.


Unknown said...

I think Sylvia made very good points. It is clear to me that studying responses to natural images and particularly comparing them to the simple, traditional stimuli can be very instructive and provide insights into what the visual cortex is "tuned" to.

Studies of extrastriate temporal visual areas, for example, would not have yielded anything useful if cells had only been stimulated by gratings, bars or edges. This goes a little bit further than my statement claiming that natural images studies only show the limits of linear models, or models including simple non-linearities.

On the other hand, it is in fact no surprise that visual cortical neurons can have highly complex properties and have very sophisticated stimulus specificities... My main point is that the really interesting question is "how do they do that"? What is the nature of the computations they perform? How should we minimally modify our current models to account for these responses? My impression is that these questions have not, and are unlikely to be soon, answered by studies using natural images, simply because natural images are incredibly complex themselves.

A second point that I find disturbing is the claim that cortical neurons respond "qualitatively" differently to simple stimuli than natural images. Where is the border between a natural and a non-natural stimulus? Who tells the cortex in which mode it should operate? Linear system analysis was mentioned: Fourier decomposition shows that natural images are just made of the sum of a large number of simple sine waves. How many does it take to make the image more like a natural image than a simple one?

Using the superposition of only two sine wave gratings (masking, plaids, etc) has helped to reveal interesting non-linearities and to refine the simple linear models used previously. With just two components in the stimulus, complex computations have been revealed and subsequently *understood*. Adding further components can certainly reveal additional properties, albeit with rapidly (exponentially?) increasing complexity. Is it wise to jump ahead and use complex stimuli that do not use a few, but thousands of components?

In short, I do recognize the usefulness of natural images to show what the cortex has evolved to analyze best, but I doubt they are useful to understand the underlying mechanisms.