Keywords

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Introduction

It was with considerable surprise that, 30 years later, in examining the literature of modern psychology I found that the particular problem with which I had been concerned had remained pretty much in the same state in which it had been when it first occupied me. It seems, if this is not too presumptuous for an outsider to suggest, as if this neglect of one of the basic problems of psychology were the result of the prevalence during this period of an all too exclusively empirical approach and of an excessive contempt for ‘speculation’. It seems almost as if ‘speculation’ (which, be it remembered, is merely another word for thinking) had become so discredited among psychologists that it has to be done by outsiders who have no professional reputation to lose. But the fear of following out complex processes of thought, far from having made discussion more precise, appears to have created a situation in which all sorts of obscure concepts, such as ‘representative processes’, ‘perceptual organization’, or ‘organized field’, are used as if they described definite facts, while actually they stand for somewhat vague theories whose exact content requires to be made clear. Nor has the concentration on those facts which were most readily accessible to observation always meant that attention was directed to what is most important. Neither the earlier exclusive emphasis on peripheral responses, nor the more recent concentration on macroscopic or mass processes accessible to anatomical or electrical analysis, have been entirely beneficial to the understanding of the fundamental problems.

– Friedrich Hayek, Preface to The Sensory Order: An Inquiry into the Foundations of Theoretical Psychology (1953).

In 1920, a 21-year-old Friedrich Hayek (later to become the famous economist and winner of the 1974 Nobel Prize in Economic Sciences) wrote one of the first explicit proposals linking the coordinated activity of neural assemblies to consciousness and the representation of percepts in the brain (Hayek 1991). Though Hayek would devote the majority of his adult life to economic theory,Footnote 1 he would, some three decades later in 1953, publish an extended book on those same ideas in The Sensory Order: An Inquiry into the Foundations of Theoretical Psychology (Hayek 1999).Footnote 2 The general “problem of theoretical psychology” that Hayek introduced in The Sensory Order was to first describe what, and then explain how, physical states of the brain give rise to sensory perception. To satisfy these criteria he postulated a mechanism for how the collective action of individual neurons could carry out a highly complex hierarchical classification function and how such aggregate activity binds sensory primitives to represent percepts—a defining problem still fundamental to modern neuroscience. By recasting the problem of perceptual representation in terms of classification, Hayek made a great leap forward in suggesting a specific framework of neural processing that accounts for our subjective experience. The mechanistic descriptions offered by Hayek point to unparalleled insightfulness at the conceptual level, ultimately bridging the gap between the seemingly ineffable psyche and the algorithmic framework of computation.

Theoretical (and often philosophical) work has continued in the decades since Hayek’s work, but perhaps the most progress has been in identifying biophysical signals that correlate to different behavioral and psychological states. Most typically, electrical activity, as measured via electroencephalography (EEG) or fluctuations of magnetism assayed via magnetoencephalography (MEG) gathered from the scalp of humans, has been shown to correlate with behavioral and psychological states. An offspring of such studies is the well-known framework of the neural correlates of consciousness (or NCC), i.e., the minimal set of neural events and mechanisms jointly sufficient for a specific conscious percept. The NCC framework, first proposed by one of the discoverers of DNA structure and Nobel prize winner, Francis Crick, and his colleague Christof Koch, was suggested as a scientific framework in which to study consciousness (Crick and Koch 1990, 2003). Generally, the study of consciousness can be separated into studying “contents” and “level.” The contents of consciousness refer to those perceptual objects that a subject is aware of, for instance, when a subject reports being aware of a tree in their visual field. Level, on the other hand, refers to the continuum spanning from dreamless sleep to normal waking life.

The use of NCC, studying both contents and level, has yielded a fruitful but extremely nuanced list of candidate signals that correlate (in varying degrees and with varying evidence) with consciousness and other related subjects, like attention and decision-making. Due to the necessary use of noninvasive techniques in humans, these signals are often found using EEG or imaging techniques such as functional magnetic resonance imaging (fMRI). Alternatively, in a clinical setting, human patients that have to undergo brain surgery (e.g., to treat epilepsy) live days with intracranial depth electrodes implanted in their brains recording extracellular voltage time-series, allowing neuroscientists to work with them and study how cognitive processing is related to neural signals. Thus, when measured with EEG, MEG, or depth electrodes, the NCC usually consist of modulations in amplitude of these extracellular signals (alongside their timing) or modulations of oscillatory power in certain frequency bands. When measured with fMRI, blood-oxygen level-dependent (BOLD) signals are used as a proxy for neural activity and to find spatial locations of activation mainly in the primate brain. Despite the immense advances in this kind of research, they have taken place largely independent from more theoretical concerns, like those discussed by Hayek.

To understand psychological phenomena, neuroscience must find mechanistic explanations for how these signals reflect, support or even constitute the conscious mind, ultimately explaining theoretical concerns through an understanding of the function of neurons and the circuits they compose. Moreover, we will show how an investigation of the details of physiology and anatomy of the brain can drive the creation of experimentally testable psychological theories. Importantly, neuroscience is now at a point where biophysical and anatomical details can be used to close the gap between experimental neuroscience, psychology and theoretical concerns. In this chapter we introduce and discuss the tight relationship between abstract theoretical concerns, detailed physiological and anatomical facts, and population signals often used in psychophysics experiments. Although much work has been done to find explanations that relate signals to psychological phenomena, it is important to realize that it is the physiology and anatomy of neurons and the networks they create that actually compute and perform tasks in the brain. In other words, within the framework presented herein, the neural substrate of psychology is cells and their networks and not (directly) extracellular fields, oxygenation levels, or frequencies in certain bandwidths (though alternative ideas exist; Hameroff 1994; McFadden 2002; Pockett 2012). Thus, theories of consciousness and perception acknowledge that the signals mentioned are proxies for the activity of cells and their networks. The method is thus easily described by a triumvirate of areas of study (in no particular order) related to each other as shown in Fig. 1. We will quickly introduce these three concepts and then delve into them more concretely in the subsequent sections of this chapter.

Fig. 1
figure 1

Signals correlated to conscious perception and theoretical concerns can be connected by considering the biophysics of signals and the computations they perform. Theory concerns itself with what it means in terms of computation and algorithm to consciously perceive something. Signals refer to the population level measurements found in the psychophysics literature (e.g., EEG, fMRI, ECoG). The underlying biophysics of these signals can be uncovered using the tools of experimental neuroscience, and then the computational functionalities of networks made from those biophysics can be explored to bridge theory and signals

First are the empirically reported signals that correlate with psychological phenomena. As discussed, these can include signatures of the EEG, anatomical locations found via fMRI, extracellular recorded spiking of cells in the thalamocortical system, and power spectrum analysis in different bands. Second are the theoretical considerations regarding psychological phenomena. These include questions regarding computational and functional concerns; for example, what does it mean in terms of a general algorithm to attend to something or represent a conscious percept? Answers to these questions are often given using some mathematical framework, for instance Bayesian inference (Knill and Pouget 2004; Ma et al. 2006; Yuille and Kersten 2006), predictive coding (Carandini and Ringach 1997; Rao and Ballard 1999), integrated information theory (Oizumi et al. 2014), or the free-energy principle (Friston 2010), or they can take a more conceptual form such as neural Darwinism (Edelman 1993), global workspace theory (Baars 2005), or indeed the ideas of Hayek and their modern extensions like the cognit (Fuster 2003, 2006).

Bridging the empirical signals and theoretical concerns are the biophysical mechanisms. One natural area of study arises in elucidating the physiological underpinnings of signals that correlate to specific psychological states. For instance, given a specific EEG amplitude occurring over the visual cortex, which networks, cell types, transmembrane currents, etc., contribute to that signal? Because these anatomical and physiological details are the substrates of neural computation, we can then delve into the computational role these physical mechanisms play. These questions connect high-level (macro-scale) theory, low-level (micro-scale) biophysical details, and mid-level (meso-scale) psychophysical signals.

In this chapter we explore how distinct biophysical processes connect between signals and psyche. Specifically, using the physiology and anatomy of pyramidal neurons in the neocortex, we explore a mechanism for perceptual binding. Notably, we focus exclusively on the contents of conscious perception. It is important to state at the onset that the connections presented herein are just one of a set of plausible frameworks for understanding how the different scales studied by neuroscientists connect to each other. This chapter is meant not to present the final word on how to comprehensively think about the micro-, meso-, and macro-scales in neuroscience as they relate to consciousness but, instead, to present, by way of example, one possible path to bridge these multiple concerns. Importantly, the task of finding the relationship between biophysics, network computation, theory, and psychology is still very much an open area of study.

Signals of Conscious Perception

What processes in the brain accompany and support conscious perception? In the attempt to answer this question, scientists and clinicians have carried out more than a century’s work, often under the area of study called psychophysics, to find measurable signals in the brain that correlate to consciousness. In particular, we discuss the evidence for three such neural signatures: (1) late extracellular signals, (2) distributed information sharing in the cortex, and (3) long-range feedback connections within the cortex. As we will see, the boundaries between these topics are often overlapping but have been studied in an independent enough manner to discuss individually (though not necessarily independently). Notably, given that many of these subjects are discussed in other chapters of this book, we review a number of perceptual correlates rather succinctly in order to relate them to the more general framework discussed in the introduction of this chapter.

Late Extracellular Signals

In 1964, Haider et al. (1964) used scalp electrodes to record extracellular signals from humans during a simple detection task. Dim flashes of light were shown to the subjects, who were asked to report perception of these stimuli. When comparing the averaged extracellular signature of seen and unseen trials, a significant difference was found in the amplitude of a negative wave occurring approximately 160 ms after the signal onset, with the amplitude of the negative wave being positively correlated to perception. These visual results were later reproduced in the auditory cortex (Spong et al. 1965).

Similar conclusions were formed in a series of papers in the 1980s and 1990s. Cauller and Kulics performed a go/no-go discrimination task on forepaw stimulation in monkeys (Kulics and Cauller 1986, 1989). They compared the extracellular signal in the somatosensory cortex and found that an early positive component (called P1, occurring about 50 ms after the stimulus) correlated well with the signal strength whereas a later negative component (called N1) correlated with the behavioral report of the signal (interpreted as the conscious perception). In a later study using depth electrodes, the laminar structure of these signals was examined using current source density analysis. Interestingly, the early P1 signal was found to be attributal to a current sink in layer 4, whereas the later N1 signal was attributed to a current sink in layer 1. Later work also showed that the later N1 signal was absent during sleep and anesthesia (Cauller and Kulics 1988).

More recent psychophysical work, using a spectrum of masking techniques, has suggested a variety of different extracellularly recorded signals that might correlate with consciousness. Two of the most plausible seem to be the Visual Awareness Negativity (VAN; Koivisto et al. 2008) and the p3b (also known as p300 or late potential). Discussion of whether these signals correlate with consciousness itself, or with pre- or post-conscious events, is ongoing (for reviews see Koivisto and Revonsuo 2010; Railo et al. 2011). The p3b is a signal occurring in a largely all-or-none fashion from 300 to 400 ms after stimulus onset (Fig. 2a), but it can occur earlier based on expectation (Melloni et al. 2011).Footnote 3 The VAN (Fig. 2a) shows a more graded response than p3b and occurs from 100 to 200 ms after the stimulus, but it has been shown to occur as late as 400 ms under specific stimulus conditions. One study asked subjects to report the subjective awareness of a change in a visual stimulus. EEG signals in aware and unaware trials from the occipital lobe were compared (Fig. 2a). Both the p3b (referred to as P3 in their figure) and the VAN can be seen to clearly signify the difference in awareness (Koivisto and Revonsuo 2007). We will not review all the differences between these signals and all the evidence for their correlation (or absence of correlation) to conscious perception here, but suffice it to say, there seems to be an NCC in a late signal occurring at least 100 ms after the stimulus onset, extracellularly measurable from the scalp. The VAN is particularly interesting as the timing of this signal corresponds to the timing of the signals measured in the Haider et al. (1964) study as well as the Kulics and Cauller work discussed above.Footnote 4 As argued below, the VAN or p3b might even correspond to recent measurements in behaving rodents.

Fig. 2
figure 2

(a) EEG signals taken from occipital sites during a change blindness task. On the left are averaged responses from trials where the subject was aware or unaware of the change. On the right is the difference between aware and unaware trials. Data from Koivisto and Revonsuo (2003), figure from Koivisto et al. (2007). (b) The subthreshold membrane potential of a mouse L2/3 pyramidal neuron during a whisker stimulus task. Behavioral hits and misses are shown in black and red. There are two epochs of depolarization, with the late epoch correlating to the behavioral output. Figure from Sachidhanandam et al. (2013). (c) Weighted symbolic mutual information between EEG sites in control (CS), minimally conscious (MCS), and vegetative (VS) patients. As the distance between sites increases, the differences in wSMI become more and more significant between the different conscious states. Figure from King et al. (2013). (d) Phosphene report after TMS stimulation in area V5 followed by V1, after a time delay shown on the x-axis. When V1 stimulation followed V5 stimulation within ~50 ms, phosphene report was abolished. Figure from Pascual-Leone and Walsh (2001)

One of the main advantages of primate experiments is the relatively direct knowledge of what the subjects’ perception is, though of course this advantage is offset by more limited access to physiological properties. Rodent experiments have been used as a model organism for cortical physiology at the synaptic, single-neuron (including dendrites), and small network level. Recent genetic tools (e.g., cre-lines, opsins) have made the mouse a preferred animal in cellular and systems neuroscience, despite the relative difficulty in establishing complex behavioral tasks and inferring perceptual state. By establishing measurable (often population or indirect) signals in primates, experimentalists are now able to find analogous signals in the rodent cortex as they attempt to establish links between behavior and perception. One recent example is from Sachidhanandam et al. (2013) (Fig. 2b). In this experiment, mice were trained to report a whisker stimulus during whole-cell patch recording of single pyramidal neurons in the barrel cortex. Two periods of depolarization were found. The first, occurring within 50 ms of stimulus onset, correlated well with stimulus strength. The second signal, occurring 50–400 ms after stimulus onset, correlated well with the behavioral report. Taking advantage of the animal preparation, optogenetics was used to silence pyramidal neurons during both the early and late epochs. Both types of inhibition abolished the behavioral report. In a control experiment, inactivation of the forepaw somatosensory cortex (and not the whisker cortex) had no effect on performance. These experiments established a causal influence of the late depolarization specifically in the whisker cortex for the perception of whisker deflection.

Taken together, these findings suggest a potential NCC in a late (~150 ms) signal that originates in the upper layers of the neocortex.

Distributed Processing in the Cortex

How distributed is the cortical representation for a given conscious percept? What are the necessary and sufficient conditions related to the communication between different areas of the brain and representation of such percepts? Here we review the evidence pointing to the distributed nature of cortical percepts.

Perhaps the earliest work hinting at the distributed mode in which the cortex operates was given by the pioneering physiologist Flourens, who sought to test the theory of localized function in the brain made popular by phrenologists like Gall and Spurzheim around the turn of the nineteenth century.Footnote 5 Flourens removed different parts of the brain in rabbits and pigeons and assessed a range of behavioral abilities. Although he was able to ascribe differences in function between the cerebellum and cerebrum, for instance, he was unable to relate different parts of the cerebrum to different cognitive and memory-dependent behaviors, ultimately positing that memory and cognition were highly distributed throughout the cerebrum (Flourens 1842).

Alongside medical results from the injured soldiers of WW1 (Goldstein 1942) and a number of famous case studies (Harlow 1999), this line of study was continued a century later by Lashley. In this body of work (Lashley 1929, 1950), Lashley aimed to study the relationship between cerebral damage and cognitive behavior, wanting to more quantitatively explain results in human patients with cortical damage who had their visual discrimination assessed by using more invasive experiments in rodents, very similar to those of Flourens. In this work, rats were trained to run through a maze. Upon removing varying volumes of cortex in different areas, rats were reintroduced into the maze, and their ability to complete the maze was assessed. Lashley found that the maze-running ability was related to the volume, but importantly not the location, of the cortical lesion. He thus posited that the ability to run through the maze was not contained in any specific local part of the cerebrum but was, instead, distributed among the entirety of the cortex.

One caveat of the work presented so far is that it is often not explicitly testing the distributed nature of a conscious percept per se but instead a more general cortex-dependent behavior. More recently, psychophysical experiments in humans have suggested that widely distributed cortical activity is associated with conscious perception, whereas activity more localized to the primary sensory areas is not. Using intracortical EEG, Gaillard et al. (2009) used a masking paradigm to compare conscious and unconscious extracellular signatures. They found that conscious perception of the stimulus was associated with widely distributed voltage deflections sustained across the cortex, increased beta (12–20 Hz) synchrony across the cortex, as well as gamma (30–60 Hz) power. The timing of these changes was late, occurring most obviously 300 ms after stimulus presentation (this was interpreted as being the p3b, though significant differences could be measured starting at 200 ms). Other similar studies showed that more localized gamma band activity relegated to the visual cortex accompanied conscious perception (Fisch et al. 2009), though follow-up studies argued that these signals were related more to pre- or post-conscious processing (e.g., decision making and report; Aru et al. 2012) than with conscious perception itself, a general weakness of the contrastive method (Aru et al. 2012; de Graaf et al. 2012; Tsuchiya et al. 2015).

Two recent studies used mathematical concepts related to information sharing across the cortex to successfully quantify the amount of consciousness in patients. King et al. (2013) used weighted symbolic mutual information, a novel measure of information sharing, between pairs of EEG recording sites (Fig. 2c). Importantly, in comparing this information measure using different distances between electrodes, it was found that differences between different levels of consciousness (e.g., vegetative vs. minimally conscious vs. healthy) were most significant for mid- to long-range distances, implicating information sharing between far-away parts of cortex in consciousness. Casali et al. (2013) used TMS evoked potentials to assess the amount of integration and differentiation distributed across the scalp EEG of patients. Importantly, this method was able to accurately and quantifiably assess the level of consciousness in patients undergoing anesthesia, sleep (Massimini et al. 2005), and varying degrees of brain injury. Similar results were more recently shown by Sarasso et al. (2015) by comparing propofol and xenon anesthesia, which induce states of unconsciousness with no dreams, to dream-inducing ketamine anesthesia. In propofol and xenon anesthesia, integration and differentiation measures were found to be low, whereas in ketamine, these same measures were high. These two studies show that the concept of long-range distributed information sharing is not only a qualitatively useful correlate of consciousness but is also quantifiable and workable in a medically applicable setting. Similar studies using transfer entropy measures have been used to study anesthesia in rats (Imas et al. 2005).

How distributed the representation for a conscious percept needs to be is a matter of ongoing debate. For visual perception, it is quite clear that V1 is generally necessary but not in itself sufficient to support a conscious content (Blake and Fox 1974; Crick and Koch 1995; Cumming and Parker 1997; Gawne and Martin 2000; Rees et al. 2002), though it is unclear if information processing needs to reach extrastriate areas or the most frontal regions or the entirety of cortex. Whatever the case, long-range communication in the cortexFootnote 6 between at least several centimeters in a human (or on the order of a millimeter in the mouse) is a necessary condition for representation of a conscious percept.

Feedback Processing

A separable but not completely independent area of study from the distributed nature of processing in the cortex is the study of feedback processing of extrastriate areas or frontal regions to primary visual cortex. Here, the data in any one study do not often explicitly implicate feedback processing but are instead interpreted to be feedback from considerations like timing and anatomy.

The timing of extracellularly measured potentials that correlate to consciousness, like the VAN discussed previously, suggests that they might have their origin in long-range feedback connections from other areas of cortex. The sensory driven, feedforward step of information processing follows a stereotyped succession of cortical areas and is completed in ~100 ms (Lamme and Roelfsema 2000). Indeed, many theories of consciousness rest on this fact, and some even go so far as to equate recurrent processing with consciousness (Lamme 2006). Experiments using TMS and other stimulation techniques have tested the causal influence of late, presumably long-range feedback processing, on perception. Multiple studies using different sensory paradigms have now shown interruption of perception by TMS over V1 during two distinct time periods, the early one interpreted to be the feedforward sweep and a later one (>200 ms) interpreted to be a feedback sweep (Heinen et al. 2005; Juan and Walsh 2003). Additionally, phosphenes induced by TMS over V5 (an extrastriate visual area) can be reduced by a lagging TMS pulse over V1, presumably interrupting the feedback of information from V5 to V1 (Fig. 2d; Pascual-Leone and Walsh 2001).

Another line of evidence comes from single cell recordings, showing that cells in the cortex continue spiking past initial feedforward activity. Many cells in macaque V1 have been found to possess dynamic orientation tuning, having precise tuning to one orientation starting at around 50 ms and then inverting at 120 ms (Ringach et al. 1997). Network simulations have shown that feedback, but not feedforward, networks can recapitulate these dynamic tuning curves (Carandini and Ringach 1997). Furthermore, single unit recordings have shown the early firing of cells codes tuned for the general category (e.g., face), whereas later spiking, ~165 ms, was tuned for specific identity (Sugase et al. 1999). Finally, inactivation of higher areas of cortex (e.g., area MT) greatly altered the response properties of cells in lower areas (e.g., V1 and V2), where feedback axons project (Nowak and Bullier 1997).

A host of studies using a technique called backwards masking might also be explained by the need for feedback processing in consciousness. In backwards masking, a target stimulus is followed, after ~50 ms, by a mask (Breitmeyer and Ogmen 2000). The subject is not aware of the target stimulus, even though on trials without a mask the target is consciously perceived. One explanation for this phenomenon is that, while the feedforward information flow through the cortex is preserved, the feedback signals conflict with the mask, rendering the target unconscious. A similar effect is found in patients with V1 lesions. These so-called “blindsight” patients retain the ability to perform forced choice tasks even though they can no longer consciously perceive visual stimuli into the affected visual field (Weiskrantz 1986). Although the exact neural underpinnings of blindsight are unknown, one candidate mechanism implicates the largely intact feedforward sweep in the retained information processing capabilities and the disturbed feedback processing in the absence of consciousness (Lamme 2001). Feedback processing has also been implicated in “contextual modulation,” which is the altering of cellular responses by changes of the stimuli outside of their classical receptive field. Interestingly, blindsight of stimulus that would normally create contextual modulation abolishes such modulation (Zipser et al. 1996), as does anesthesia (Lamme et al. 1998).

Biophysical Foundations of Signals Associated with Conscious Perception

The aforementioned relationships between conscious perception and a number of characteristic signals and signatures point to the importance of understanding the neural substrate of these signals. Such understanding bridges the gap between the underlying cellular biophysics, the network effects, and the high-level behavioral readouts. To gain insights into the signals associated with conscious perception, it is important to understand the underlying physics, in terms of the physical laws governing the generation of these signals as well as the neural origins that brings them about.

We first present the physics underlying electric measurements in the brain (‘Biophysics Related to Electric Measurements’). We have chosen to specifically focus on electric signals and measurements such as the VAN as they have produced the largest body of evidence in terms of psychophysics of conscious perception. (Later in this chapter we also present other methods that have impacted or will potentially critically impact the field.) In a next step, we introduce the most significant cellular contributors of electric activity in brain matter as a means to understand which processes (synapses, cells, circuits, etc.) contribute to these signals (‘Biological Electric Field Contributors’). Finally, we present the most prominent methods and technologies used to monitor brain activity (‘Monitoring Neural Activity’).

The previous section featured results using several different types of electrical measurements, including EEG (Koivisto and Revonsuo 2010), single unit recordings (Sugase et al. 1999), and depth electrodes to compute the power of different frequency spectrum (Aru et al. 2012), as well as both local field potential (LFP) and current source density (CSD) recordings (Kulics and Cauller 1986). These techniques as well as others used in the field of neuroscience will be presented. Additionally, the biophysical underpinnings of the late current sink in layer 1 (Kulics and Cauller 1986) that correlates to conscious perception is discussed.

Biophysics Related to Electric Measurements

Charge transfer across the membrane of all structures in brain matter such as neurons, glial cells, etc., induces so-called extracellular sinks and sources that, in turn, give rise to an extracellular field, i.e., a negative spatial gradient of the extracellular voltage (V e ) measured in comparison to a distant reference signal. The physics governing such events are described by Maxwell’s equations. In their simplest form, Maxwell’s equations of electromagnetism dictate that V e depends on the transmembrane current amplitude (I), the conductivity of the extracellular medium (σ) and the distance between the location of the ionic flux and the recording. Specifically, when assuming a so-called point-source (i.e., when a localized current injection occurs within an electrically conductive medium), the relationship between the aforementioned variables and the resulting V e is (Fig. 3a):

Fig. 3
figure 3

Biophysics of extracellular signatures and conductivity of the extracellular medium. (a) Illustration of V e calculation in a population through the superposition of contributions from all compartments in all cells. Individual compartment contributions are primarily determined by their transmembrane currents and distances from the electrode. (b) Charge transfer elicited across the membrane (dark region) of a long, small diameter cable gives rise to an extracellular field. The extracellular potential close to the cable was calculated using the line-source and the cylinder-source approximation. The difference between the two approximations is very small (they overlap). (c) Simulated location dependence of the extracellular action potential (EAP) waveform of a pyramidal neuron. The peak-to-peak voltage range is indicated in this simulation by the color of each trace. EAPs are calculated at the location of the start of each trace. EAP amplitude decreases rapidly with distance. (d) Experimentally obtained values of components of the conductivity tensor in the frog (Rana) and toad (Bufo) cerebellum as a function of depth. (e) In vivo measurements of impedance as a function of cortical depth in monkey. (f) Microscopic measurements of the relationship between intracellular and extracellular spike signals in rodent slice. Whole-cell patched neurons are brought to spike (blue line) and a proximally positioned extracellular silicon probe with eight contacts is used to record the elicited extracellular voltage transients (red line). At the initiation time of the spike, the extracellular negativities (red) associated with the intracellular spikes attenuate with distance from the soma (see also panel c), with the attenuation occurring per the point-source approximation. Figure contributions are from (a, c) Schomburg et al. (2012), (b) Holt and Koch (1999), (d) Nicholson and Freeman (1975), (e) Logothetis et al. (2007), (f) Anastassiou et al. (2015)

$$ {V}_e(d)=\frac{I}{4\pi \sigma d} $$

Based on the point-source equation, one can note the following: first, there is an inverse relationship between distance d and the amplitude of the resulting voltage deflection V e , i.e., the farther away to recording site is from the location of the current point-source, the larger the attenuation of the amplitude of the V e -deflection; the stronger the point-source I, the larger the V e -deflection; finally, the conductivity of the extracellular medium critically impact propagation of the signals from the point-source to the recording site.

Notably, when the source is not limited to a point but instead possesses physical extent, the approximation needs to be re-formulated accordingly to account for such physical extent. For example, when charge transfer takes place along the elongated, cable-like morphologies of neurons, it gives rise to a spatially distributed extracellular source not compatible with the aforementioned point-source expression. Probably the most prominent such approximation accounts for the field induced by a linear, one-dimensional (line) source of infinitesimally small diameter. The line source approximation (LSA) makes the simplification of locating the transmembrane net current for each neurite on a line down the center of the neurite. By assuming a line distribution of current, V e is described via a two-dimensional solution in cylindrical coordinates. For an elongated current source of length Δs, the resulting V e (r, q) is given by:

$$ {V}_e\left(r,q\right)=\frac{1}{4\pi \sigma }{\int}_{-\Delta s}^0\frac{I}{\Delta s\sqrt{r^2+{\left(q-s\right)}^2}}ds=\frac{I}{4\pi \sigma \Delta s} log\left(\frac{\sqrt{q^2+{r}^2}-q}{\sqrt{l^2+{r}^2}-l}\right) $$

where r is the radial distance from the line, q the longitudinal distance from the end of the line, and l = Δs + q is the distance from the origin of the line. Holt and Koch (1999) analyzed the accuracy of the LSA and found it to be highly accurate except at very close distances (i.e., about 1 μm) to the cable (see also Rosenfalck 1969; Trayanova and Henriquez 1991; Fig. 3b). The LSA has been the primary method of calculating extracellular voltages arising from transmembrane currents (Gold et al. 2006, 2009; Holt 1998; Holt and Koch 1999; Pettersen and Einevoll 2008; Fig. 3c).

Notably, the aforementioned relationships assume that the extracellular medium in the brain is described via electrostatics and not by much more elaborate elements of electrodynamics. Furthermore, a widespread assumption is that the extracellular medium is isotropic and homogeneous. What evidence exists for such claims to be made? It turns out that this question has remained unresolved, with a number of studies reporting an anisotropic and homogeneous σ (Nicholson and Freeman 1975; Logothetis et al. 2007) (Fig. 3d, e) to strongly anisotropic and inhomogeneous (Goto et al. 2010; Hoeltzell and Dykes 1979; Ranck 1973) and, finally, even of capacitive nature (Bédard and Destexhe 2009; Bédard et al. 2004; Gabriel et al. 1996).

Part of the difficulty in determining the properties of σ, especially at the local, microscopic scale, has to do with the inhomogeneity of the brain as a structure. In that sense, the questions to be answered are where, in what species, in what frequency band and at what spatial scale should σ be measured. The danger is that measuring σ over larger volumes leads to possibly quite different results (attributed to averaging) than recording σ over tens of μm. Moreover, measuring σ within distances of tens micrometers, i.e., the relevant spatial scale for signals related to spiking, poses significant technical challenges given the large number of sites (both for current injection and voltage recording) that need to be positioned within μm-distances and the resulting tissue deformation/damage.

Recently, detailed whole-cell patch recordings of excitatory and inhibitory neurons in rat somatosensory cortex slices were performed in parallel to positioning a silicon probe in the vicinity of the patched somata, allowing concurrent recording of intra- and extracellular voltages (Anastassiou et al. 2015). Using this experimental setup, the authors characterized biophysical events and properties (intracellular spiking, extracellular resistivity, temporal jitter, etc.) related to extracellular spike recordings at the single-neuron level. It was shown that the extracellular action potential (EAP) amplitude decayed as the inverse of distance between the soma and the recording electrode at the time of spike (Fig. 3f). The spatial decay of the EAP-amplitude at the spike time was very close to the prediction of the point-source approximation: at the spike time, transmembrane charge transfer was still spatially localized (close or at the axon initial segment), resulting effectively in a point-source. Even fractions of a ms after the spike time, the relationship between the EAP-amplitude and distance was shown to become more intricate as more extended sections of the cellular morphology acted as sources, leading to more complex superposition rules (e.g., based on the LSA). On that limit, various contributions of a cell’s different compartments need to be accounted for. Interestingly, in the same experiments, a time lag was observed at the extracellular spike waveform with increasing distance of the electrode location from the cell body with respect to the spike time at the soma. While such time lags could be explained by the presence of a non-ohmic extracellular medium, the authors showed that they were actually attributed to the spatial propagation of the action potential along the neural morphology, i.e., backpropagating action potentials. Finally, this study demonstrated that different cortical layers exhibited different conductivity, with the conductivity of layer 4 being higher than the conductivity of layer 2/3 and 5, i.e., an observation in line with the finding that layer 4 possesses a higher density of neurons compared to layers 2/3 and 5.

Do these observations hold in vivo? A number of experimental studies have appeared offering compelling insights into the physics of the extracellular medium. Nicholson and Freeman (1975) studied the conductivity profile in the cerebellum of bullfrogs using current injections through micropipettes and concluded that it is anisotropic, homogeneous, and purely ohmic, with later measurements by Logothetis et al. (2007) confirming these observations (Fig. 3d, e). Yet, others found the extracellular medium to be strongly anisotropic and inhomogeneous (Hoeltzell and Dykes 1979; Ranck 1973) or even of capacitive nature (Gabriel et al. 1996; Bédard et al. 2004; Bédard and Destexhe 2009). In a more recent study, Goto et al. (2010) used extracellular recordings to measure the conductivity profile along the entire somatosensory barrel cortex in rodents using depth multi-electrode recordings and reported that radial and tangential conductivity values varied consistently across the six neocortical laminas. Thus, they showed that the electric properties of the extracellular medium in the living animal were anisotropic and inherently inhomogeneous, agreeing with the in vitro findings of Anastassiou et al. (2015). Importantly, in their work Goto and colleagues provided evidence that (at least for frequencies less than 500 Hz) σ can be assumed to be purely ohmic. Based on the aforementioned, the temporal characteristics of the extracellular field and signals like the VAN are not due to extracellular medium properties but, instead, solely attributed to cellular functioning.

Biological Electric Field Contributors

Given the aforementioned biophysics dictating how transmembrane currents are generated and propagated in brain matter, what cellular processing gives rise to these electric signals? Here we present the most important contributors of the extracellular field. In principle, any charge transfer along the membranes of the neural morphology elicits extracellular sinks and sources, as will be discussed below. (For a more thorough treatise, the interested reader is pointed to Buzsáki et al. 2012; Einevoll et al. 2013.)

Synaptic Activity

In physiological situations, synaptic activity and postsynaptic currents, in particular, are often the most prominent sources of extracellular current flow. While the majority of individual synaptic connections induce fairly small extracellular signals (e.g., Bazelot et al. 2010; Glickfeld et al. 2009), thousands of synapses are present along a single neuron’s morphology (e.g., a rat layer five pyramidal neurons has approximately 10,000 synapses along its processes). Thus, even if the individual contribution of such postsynaptic events is fairly small, the fact that thousands of them may become co-activated within a small time increment suggests a substantial overall effect (Fig. 4a). Furthermore, the time constant of synaptic events can vary substantially: while the time constant of fast excitatory AMPA- and inhibitory GABA subtype A-receptors ranges approximately 1–15 ms (Hille 1992), excitatory NMDA and inhibitory GABA subtype B-receptor dynamics can be particularly slow (i.e., 50–300 ms; Pérez-Garci et al. 2006) and, as such, may readily contribute to slow bands of the electric signal (Elul 1971; Logothetis and Wandell 2004).

Fig. 4
figure 4

Main contributors of extracellular signals. (a, top) Postsynaptic currents influence extracellular voltage recordings. Overlay of events elicited by single action potentials of an interneuron and the resulting distribution of unitary postsynaptic amplitudes in rodent hippocampus. (a, bottom) The distributed nature of sinks and sources induced by postsynaptic currents. A single excitatory synapse (solid circle) is activated along an apical branch and its impact is propagated along the entirety of extracellular space due to the spatially distributed morphology of the excitatory neuron and the presence of passive return currents. (b, top left) Spike triggered average EAP waveform of a layer 5 pyramidal neuron with the intact EAP waveform (red) and when the EAP negativity is missing (black; window of 0.6 ms around spike initiation time is substituted by a spline). (Top right) Extracellular traces composed using the L5 pyramid EAP waveform (red: using the intact EAP waveform; black using the de-spiked EAP waveform). As observed, the typical EAP-related negativity is missing whereas the remainder of the waveform is attributed to slow afterpotential currents. (Bottom) Mean spectral density as a function of temporal frequency for the intact and de-spiked EAP waveform of the L5 pyramid (green: no spiking; red: 1 Hz; black: 8 Hz; blue: 30 Hz). As observed, the effect of spike afterpotentials (even in the absence of the salient EAP-negativity) can impact voltage recordings in frequencies as low as ~20 Hz. (c) Impact of Ca-dependent dendritic spikes on extracellular voltage recordings. A computational model of a layer 5 pyramidal neuron in the presence (left) and absence (right) of the Ca-hot-zone (location shown by the arrow) is used to emulate the electric field produced by a single neuron. The simulated depth LFP for the two cases is shown by the traces. The presence of the Ca-hot-zone and elicitation of a Ca-spike give rise to a strong, long-lasting event in the superficial regions of the cortex. Figure contributions are from (a, top) Bazelot et al. (2010), (a, bottom) Lindén et al. (2010), (b) Anastassiou et al. (2015), (c) simulations by A.S. Shai and C.A. Anastassiou

The influx of cations when excitatory synaptic input impinges along the neural membrane from the extracellular into the intracellular space gives rise to a local extracellular sink. To achieve effective electroneutrality within the time constants of relevance for systems neuroscience, the extracellular sink needs to be balanced by an extracellular source, that is, an opposing ionic flux from the intracellular to the extracellular space, along the neuron. In this case, the counter-flux is termed passive or return current. It follows that such passive return currents do not only depend on where synaptic input impinges along the neural morphology but also on the actual morphological features of the neuron itself. For example, impinging inputs in one area of the elongated morphology of pyramidal neurons gives rise to passive return currents along the same neuron (Fig. 4a). On the other hand, the symmetric location of the dendrites of inhibitory basket cells does not allow the formation of such strong passive return currents due to cancellation effects, even when these neurons receive strong synaptic input (Pettersen and Einevoll 2008; Reimann et al. 2013). Depending on the location of the sink current(s) and its distance from the source current(s), a dipole or a higher-order n-pole is formed.

Beyond the location-related aspects of synaptic input, another important factor crucially dictating the characteristics of extracellular electrophysiology signals is input correlation, i.e., the amount of synaptic input impinging along a neuron or neural population in a unit of time. Typically, enhanced input correlation is manifested in larger V e and local field potential (the lowpass filtered part of V e or LFP) amplitude, even if such intuition is not always warranted (see below). Beyond the extracellular LFP magnitude, an additional feature of electrophysiology recordings impacted by input correlation is the spatial extent or spread. For example, for uncorrelated input the majority of the extracellular voltage signal measured by an electrode originates from neurons within a lateral distance of approximately 200 μm (Katzner et al. 2009; Xing et al. 2009; Lindén et al. 2011; Reimann et al. 2013). Notably, such low input correlation results in the independence of the region size generating the LFP from neural morphology and the spatial distribution of the synapses. In the presence of more considerable input correlation, the picture changes drastically: pyramidal neurons with their extended spatial morphologies as well as their synaptic specialization tend to dominate the extracellular field. Moreover, correlated synaptic inputs give rise to correlated neural membrane sources that result overall in stronger LFP amplitude. Yet, the degree of LFP amplitude enhancement depends on the spatial separation between impinging synaptic currents and return currents—for substantial separation (i.e., spatially inhomogeneous input along the extended dendritic arbor), the LFP amplitude enhancement becomes significant whereas for smaller separation (i.e., spatially homogeneous input along more compact dendritic arbor), LFP enhancement becomes weaker. Such interdependence between neural morphology features, location of synaptic inputs, input correlation, etc., can putatively explain the disparate length scales encoded by extracellular recordings (Kreiman et al. 2006; Liu and Newsome 2006; Katzner et al. 2009; Xing et al. 2009).

Thus far, we have mostly considered chemical communication via dedicated synapses that are prevalent in brain tissue. Another component whose impact on population dynamics and, as a result, the extracellular voltage remains unaccounted for is electrical synapses, which provide a low-resistance pathway between neurons permitting the direct transmission of electrical signals. Gap junctions (GJs), the morphological correlate of electrical synapses, have been used as a proxy for electrical coupling and to infer electrically coupled network architectures. Numerous studies have revealed such networks of electrically coupled neurons in many mammalian brain structures. In cortex, extensive coupling has been reported primarily between inhibitory parvalbumin-positive (PV) interneurons and between somatostatin (SST)-expressing neurons. Such PV- and SST-expressing inhibitory neurons critically contribute to many aspects of ensemble encoding in the mammalian brain (Hu et al. 2014), with one of their most prominent roles being balancing excitation and shaping rhythmic activity. In addition, PV interneurons, the most populous among inhibitory cortical cell types, shape cortical ensemble activity, both during gamma (Szabadics et al. 2001; Traub et al. 2001) and during other rhythms and events such as hippocampal theta or sharp waves. Given that proximally located SST and PV neurons are connected both via chemical and electrical synapses (for a recent review, see Pereda 2014) in the developing and in the developed neocortex (Connors et al. 1983), can GJs alter extracellular electric fields? Because ions passing through GJs do not enter the extracellular space, it follows that GJ themselves contribute neither to the extracellular current flow nor to the extracellular field explicitly. On the other hand, because GJs contribute to the functioning of inhibitory cells and cell populations altering, for example, their spiking characteristics, they can have an implicit effect on field activity that hitherto has remained unexplored.

Active Membrane Currents

Most neurons produce brief action potentials or spikes that travel along their axons and give rise to synaptic currents at the synapses. It is through the propagation of such electric activity from one neuron to its post-synaptic targets that information is generated and processed within neural populations. Action potentials are produced through active ionic membrane mechanisms allowing the exchange of ions such as Na+, K+ and Ca2+ across the membrane. Specifically, fast, Na+-dependent spikes and spike afterpotentials generated at the axon initial segment and somata of neurons give rise to the strongest currents across the neuronal membrane, detected as ‘unit’ or ‘spike’ activity in the extracellular medium. Although Na+-spikes generate large-amplitude and transient (typically lasting 0.5–1 ms) V e deflections proximal to the soma with a cascade of ionic mechanisms, spike- and spike afterpotential-associated fields remain local (Fig. 3c). The fact that spikes typically last less than a few ms has led to the assumption that they only contribute to extracellular unit activity whereas not appreciably to slower signals such as the LFP or the scalp-recorded EEG like the VAN. Yet, synchronously elicited action potentials (e.g., due to increased spike correlation) from many proximal neurons can contribute substantially to slower bands of extracellular recordings (Anastassiou et al. 2015; Belluscio et al. 2012; Schomburg et al. 2012; Taxidis et al. 2015; Zanos et al. 2011). In addition, it has been shown that spikes give rise to slower, smaller-amplitude afterpotential currents. These spike afterpotentials have recently gathered much attention with studies showing that they can impact bands as low as 20 Hz (Fig. 4b; see also sections below).

Another type of active membrane current is constituted by Ca-spikes and Ca-related signals. Decades of work, mostly in vitro, have revealed that the dendrites of cortical pyramidal neurons support a variety of nonlinear signals such as so-called NMDA spikes, Ca-spikes, Na-spikelets and backpropagating action potentials. Of particular interest are the temporally extended NMDA spikes and dendritic Ca-spikes. With regards to NMDA spikes, basal, oblique, and apical tuft dendrites of cortical pyramidal neurons receive a high density of glutamatergic synaptic contacts. The synchronous activation of 10–50 such neighboring glutamatergic synapses triggers a local dendritic regenerative potential, NMDA spike/plateau, that is characterized by significant local amplitude (40–50 mV) and an extraordinary duration (up to several hundred milliseconds). Notably, the conductance of the glutamate-dependent NMDA receptor (NMDAr) channel is also dependent on voltage, giving the NMDAr its spiking ability. NMDArs are found on the thin tuft dendrites of pyramidal neurons, such as those that reside in layer 1, and have been shown to support spatially localized (~30 μm) all-or-none spiking events, due to the glutamate binding requirement. Given the electrotonic distance to the spike initiation zone, single NMDA-spikes do not, in general, cause somatic action potential output. Yet, the effect of such NMDA spikes depends on the location where they take place: if occurring in the apical tuft, they have the ability to substantially depolarize the entire tuft region whereas, if occurring closer to the cell body, they can depolarize the soma in a fashion similar to an UP-state. In addition, it has been shown that a distributed set of multiple NMDA-spikes across the dendritic tuft has the ability to cause action potentials during in vivo sensory stimulation (Palmer et al. 2014).

Dendritic Ca-spikes are nonlinear events mainly attributed to large conductance, high-voltage activated channels along the pyramidal dendrites that mediate a sustained Ca2+-influx in a variety of dendrites (Larkum et al. 1999, 2009; Shai et al. 2015). The apical dendrite of pyramidal neurons has a main bifurcation that occurs in L2/3 or L1 and contains a high density of voltage-dependent Ca-channels. This “hot spot” of Ca-channels, alongside other nonlinear channels, supports a relatively slow but large all-or-none depolarizing current known as the Ca-spike. Lasting for ~20–100 ms in in vitro conditions, and possibly longer in vivo, the Ca-spike has the ability to depolarize a pyramidal neuron for an extended period of time. These Ca-spikes can be triggered with a variety of mechanisms: by strong synaptic drive, by a triplet of back-propagating action potentials or via an extracellular stimulus.

Due to the location of the channels responsible for NMDA and Ca-spikes, these signals are well-suited to being controlled and evoked by inputs into the dendritic arbors of excitatory neurons. One such cortical pathway is long-range connections into layer 1. Indeed, channel-rhodopsin-assisted mapping techniques have shown that higher-order areas send strong-direct excitatory input into the apical dendrites of pyramidal neurons (Fig. 5; see also Yang et al. 2013). What are the functional consequences of NMDA and Ca-spikes in the dendrites of a pyramidal neuron? In vitro experiments have shown that the Ca-spike can integrate with a backpropagating action potential to elicit a spike burst (i.e., a multitude of somatic spikes elicited within a few tens of milliseconds) at the soma (Larkum et al. 1999). It is difficult to precisely control the amount and timing of synaptic inputs into spatially segregated areas of a single neuron experimentally, though some efforts deserve recognition (Jarsky et al. 2005). Modeling approaches present themselves as useful tools to explore the possible functional roles of a complicated mixture of linear and nonlinear channels across the dendritic membrane, as well as their interactions with large barrages of synaptic input (e.g., Shai et al. 2014).

Fig. 5
figure 5

Monitoring the electrical activity of the brain. (a, top) The traces of wide-band recordings (left) and 30 Hz highpass filtered (right) hippocampal CA1 shown together with their independent components (obtained with independent component analysis). (Bottom) Two-dimensional voltage (LFP) and CSD maps of the three main CA1 independent components from a rat with an electrode array spanning the transverse axis of the hippocampus (seven shanks spaced 300 mm; one shank shown on the left) indicates activation of different projections (CA1pyr pyramidal layer, rad radiatum, lm lacunosum moleculare). (b) Electrocorticography (ECoG) records indicating periods of behavior-relevant slow oscillations (orange) and spindles (gray). (Bottom) Intraoperative ECoG recordings in human patients using new technologies have the ability to detect spiking. Highpass filtered traces from a novel 64-grid electrode containing spiking activity (black traces). Below, sample spike waveforms are shown. (c) Simulation of an individual neuron (layer 5 pyramidal injected with intracellular somatic current by a pipette: intracellular somatic spiking shown in blue is detected in the extracellular space by a proximal electrode (red; part of a silicon depth electrode) as well as by the ECoG strip electrode (simulating the same layout as the one in panel b)). The spike-triggered average ECoG signal from the middle of the ECoG strip is shown (right). (Bottom) The spike triggered average ECoG field for two cell types extending to superficial layers: a layer 23 pyramidal (left) and a layer 5 pyramidal neuron (right). While the amplitude of the spiking ECoG signature is very similar, the spatial extent is markedly different. (d, left) A large-scale, biophysically realistic model of thousands of reconstructed and interconnected neocortical layer 4 (red) and layer 5 (green) pyramidal neurons emulating a patch of deep cortical volume. The population model was used to study the extent to which active membrane conductances impact the extracellular LFP and CSD signals. (Right) Two scenarios were instantiated: passive-only membranes and active ones. The simulated LFPs and CSDs show the result of these simulations (top: passive-only; bottom: active) with the spatiotemporal characteristics of the LFP and CSD being markedly different. (e, left) Hippocampal model of the CA1 region consisting of reconstructed excitatory neurons capturing the various projections during sharp wave ripples accounts for the extracellular signals during such events. (Right) Replay sequences during sharp waves yield consistent LFP patterns in the ripple (150–200 Hz) bandwidth. As observed, simulations point to the spatiotemporal patterned activity that is also observed in the same band in vivo, reflecting the spiking activity of cell assemblies activated during sharp waves. Figure contributions are from (a) Schomburg et al. (2014), (b) Khodagholy et al. (2015), (c) simulations by C.A. Anastassiou and A.S. Shai, (d) Reimann et al. (2013), (e) Taxidis et al. (2015)

After creating a detailed multi-compartmental model of a L5 pyramidal neuron based on a combination of previous modeling work (Hay et al. 2011) and dual soma and dendrite patch clamp recordings in V1, Shai et al. (2015) imposed barrages of dendritic and somatic excitatory synapses onto a single cell. The results of this simulation showed that the coincident input of perisomatic and apical input elicited a burst of high-frequency action potentials at the soma, whereas only perisomatic or apical input in isolation would not. Furthermore, this effect was dependent on the Ca hot-spot around the bifurcation point. This finding was further summarized in a simple abstract model whereby input into dendrites of the abstracted cell would modulate, in a thresholded manner, the input-output relationship between perisomatic input and output frequency. This “composite-sigmoid” model captured the complicated interaction of perisomatically elicited backpropagating action potentials (mediated by nonlinear sodium channels in the apical trunk) and the tuft elicited Ca-spike, in a compact form, thus elucidating the computational structure of a single pyramidal neuron.

In particular, the Ca-spike, being a relatively slow and large-amplitude signal in the superficial layers, may constitute a candidate for NCC. Multi-compartmental modeling is well suited to disentangling the potential contributions of different cell types of channels to extracellular signals (e.g., Reimann et al. 2013; Fig. 4c). Although more thorough investigations are needed, both in terms of modeling and experiments, to establish the role of the Ca-spike in the superficial extracellular signature, these simulations show that the Ca-spike is indeed a plausible mechanism for these signals (Fig. 6).

Fig. 6
figure 6

A mechanism of coincidence detection via feedback into layer 1. (a) A top view of the mouse brain showing the anterior cingulated cortex (ACA, a frontal region) and primary visual cortex (V1). (b) The anterograde projections of ACA axons into V1 show a clear striation in layer 1 (green fluorescence). Subcellular channel-rhodopsin-assisted mapping (sCRACM) on a layer 5 pyramidal neuron (red) shows strong excitatory input into the apical tuft dendrites. (c) 100 tuft and 175 basal NMDA/AMPA synapses are distributed randomly across the apical tuft and basal dendrites of a multi-compartmental L5 pyramidal neuron model. All synapses are randomly and uniformly elicited in time across 100 ms. In the following panels, somatic traces are in black and dendritic (location shown by the red arrow in c), are in red. (d) Simultaneous tuft and basal inputs trigger a burst of somatic action potentials and a dendritic Ca2+ spike, whereas (e) basal inputs alone evoke only a single somatic spike. (f) Apical tuft inputs alone do not evoke somatic spiking. (g) Reducing Ca2+ channel conductance by 50 % during tuft and basal input gives rise to a single somatic spike. (h) When applying a 200 pA hyperpolarizing DC current to the soma, the subthreshold response of the tuft and basal inputs are similar to the case with Ca2+conductances reduced shown in (i), even though the suprathreshold (b, c) cases are remarkably different. (a) Taken from the Allen Institute Brain Explorer. (b) Experiments performed by Adam Shai, but also see Yang et al. (2013), for similar results. (c–i) Taken with permission from Shai et al. (2015)

Non-neural Contributors (Glia, etc.)

During the last two decades, glial cells have been shown to be of great significance for brain signaling (Volterra and Meldolesi 2005) while also possessing active ionic conductances that result in fairly slow but prominent transmembrane processes being activated during neural activity (Perea and Araque 2007). Electrically passive astrocytes coexist with others that show voltage-dependent currents such as inwardly rectifying or delayed, outwardly rectifying K+ or both types (D’Ambrosio et al. 1998). Given the abundance of glia in brain tissue, how do these contribute to the extracellular electric field (Wang et al. 2006)? Can certain LFP or EEG bands (such as the slow 0.1–1 Hz band) be influenced by glial and astrocytic transmembrane activity? Such questions are also related to the link between LFP activity, the blood oxygen-level dependent (BOLD) signal and the overall metabolic demands of specific brain areas. Interestingly, the BOLD signal, which has been linked to neural as well as astrocytic activity, has been found to correlate preferentially with specific LFP bandwidths.

Monitoring Neural Activity

In this section of the chapter we present the most prominent methods of monitoring brain activity. We separate this section into two parts: a part on monitoring spatially local brain activity and a part on methods used to monitor spatially extended (even whole-brain) activity. While local monitoring can offer superior spatiotemporal resolution from identified signal sources, spatially diffuse monitoring offers insights from multiple brain regions, as discussed previously, such distributed processing has been often implied to be a cornerstone for the formation of conscious percepts.

Local Monitoring

Local monitoring refers to measurements of neural activity within fairly small volumes of brain tissue. Historically, the most prominent of these local monitoring techniques have been extracellular voltage recordings either from a single location (e.g., via a metal wire) or multiple locations (multiple wires bundled together or multiple contacts manufactured along the length of a silicon shank; Buzsáki 2004). The exact sampling volume of such extracellular electrodes remains the object of investigations. In the past, various distances have been suggested ranging from a few tens to a few hundreds of micrometers. Here we present and succinctly discuss the most prominent methods (also beyond extracellular recordings) that allow monitoring neural activity within similar volumes.

Action potential elicitation of neurons proximal to a recording electrode is typically reflected in the approximately 1 ms long and 50–200 μV deep negativities of the extracellular voltage time series. These rapid and spatially localized V e -deflections reflect membrane currents in the axonal and perisomatic region (Fig. 3b, c) but are also impacted by more distributed currents such as backpropagating action potentials traveling along the dendritic arbor of a neuron. An important aspect of monitoring spiking activity in the extracellular space is the inherent, activity-dependent variability of the EAP waveform. Indeed, the EAP amplitude of hippocampal pyramidal neurons can vary as much as 60 % during a high-frequency (approximately 200 Hz) burst (Buzsáki et al. 1996) for example, within a place field (Harris et al. 2001). These features, together with artefactual sources of variability from electromyogram contamination or hardware sources, pose challenges for spike waveform-based clustering and classification of neurons in vivo. Specifically, the most salient features of EAPs, the EAP negativity attributed to fast sodium- and potassium-dependent currents and the immediately proceeding EAP positivity attributed to slower potassium (but also calcium) currents, can substantially vary as a function of spike frequency. In general, the spike variability of the EAP amplitude is more pronounced compared with intracellular spike variability (Anastassiou et al. 2015; Henze et al. 2000) and is non-monotonic as a function of spike frequency. The temporal EAP waveform features, such as halfwidth or decay time of spike repolarization, vary more reliably with the intracellular waveform than the amplitude (Anastassiou et al. 2015; Barthó et al. 2004). It follows that, while EAP recordings are fairly straightforward to obtain compared to more elaborate methodologies, can be performed in deeper brain structures and offer superior temporal resolution, their disadvantage lies in the difficulty of the procedure involved in separating spikes and EAP waveforms originating from different neurons, a process often referred to as spike clustering.

When the wideband signal from extracellular recordings is lowpass filtered (typically below 300 Hz), the resulting time series is referred to as the local field potential (LFP). The LFP has been studied extensively in the past as, in contrary to highpass part of the recordings, it reflects electrogenesis from a spatially more distributed region (Fig. 5a). What the length scale of the LFP is has been a debated topic and remains a vibrant field of science (e.g., Katzner et al. 2009; Xing et al. 2009; Ray and Maunsell 2011; Lindén et al. 2011; Reimann et al. 2013). What is clear is that the spatial extent of the LFP is not a static feature depending on multiple factors such as cytoarchitecture, input correlations (see above), etc. Hitherto, the LFP has been considered to mainly reflect postsynaptic and associated return currents (even if recently more LFP-contributors have been identified); as such, it is uniquely positioned to measure input into and output from a particular brain region (e.g., Mitzdorf 1985; Colgin et al. 2009; Einevoll et al. 2013; Logothetis et al. 2007; Buzsáki et al. 2012; Schomburg et al. 2014; Taxidis et al. 2015).

A method used in conjunction with LFP recordings is the so-called current source density (CSD) analysis. The CSD analysis is a particularly useful tool in deciphering the location of the extracellular current sources and sinks giving rise to the LFP (Fig. 5a). CSD per se represents the volume density of the net current entering or leaving the extracellular space (Nicholson and Freeman 1975; Mitzdorf 1985; Buzsáki et al. 2012). Unfortunately, it is not possible to conclude from the CSD analysis alone whether, for example, an outward current close to the cell body layer is due to active inhibitory synaptic currents or reflects the passive return current of active excitatory currents impinging along the dendritic arbor. Such insights have to be gathered from complementary information such as the cytoarchitecture of the brain region under investigation, its anatomy, projecting pathways, etc. Even so, CSD analysis can point to regions of interest to be studied more elaborately.

Conventionally it has been thought that spiking currents cannot affect temporally slower signals such as the LFP or the EEG due to the rapid, approximately 1-ms transient sodium/potassium charge transfer giving rise to the stereotypical intracellular positivity (or extracellular negativity). Lately this view has been challenged by a number of studies showing that neural spiking can affect electric signals at much lower frequencies than the typical time scales suggested by action potentials (Belluscio et al. 2012; Zanos et al. 2011; Ray and Maunsell 2011; Schomburg et al. 2012; Reimann et al. 2013; Anastassiou et al. 2015). What part of the EAP waveform can impact power at slow bands of extracellular recordings? This has been the focus of a few studies (e.g., Zanos et al. 2011; Belluscio et al. 2012; Anastassiou et al. 2015). In a recent one, the authors performed so-called “de-spiking,” i.e., the procedure of substituting a window of 0.6 ms before and after spike initiation time with a different (non-spiking) time series in the extracellular voltage time series (Belluscio et al. 2012), in experiments where both the intracellular and extracellular spikes were monitored concurrently (Anastassiou et al. 2015). This resulted in EAP waveforms lacking the typical spike negativity but containing the characteristic afterpotential repolarization. Performing spectral analyses of the de-spiked time series led to a surprising conclusion: spike afterpotential currents of pyramidal neurons can impacted the spectrum of recorded signals as low as 20 Hz, i.e., bands hitherto solely related to synaptic processing (Fig. 4b). Importantly, when the same analyses using the EAP waveform from basket cells was performed, the outcome was very different: spiking of these neurons minimally contributed to spectral power under 100 Hz and, even then, did so only for elevated spike frequencies. The lack of impact of basket cell spiking to LFPs under 100 Hz was attributed to their temporally narrow EAP waveform as well as the lack of long-lasting depolarizing currents (compared to pyramidal neurons). The study concluded that the effect of EAPs in such low frequencies was attributed to the slower, smaller amplitude repolarization typically attributed to slower potassium- and calcium-dependent currents difficult to distinguish in vivo.

Electrocorticography (ECoG) is the intracranial recording of electrophysiological signals using electrodes and multi-electrode arrangements (grids) from the surface of the brain after craniotomy and has been used for decades to monitor (and sometimes perturb) cortical activity. Specifically, ECoG recordings have conventionally been used to record slow signals (similar to the LFP) related to brain states or evoked activity, though spiking activity has been difficult to detect (Fig. 5b). In that sense, ECoG has been largely used as a spatially distributed monitoring method much related to electroencephalography and magnetoencephalography (see below). Yet, very recently, advances in technology and materials have for the first time allowed robust recording of cortical spiking (Khodagholy et al. 2015) using ECoG (Fig. 5b, c), rendering the possibility of concurrent monitoring of intra- and inter-cortical processing in terms of spiking and slower activity from the brain surface.

Beyond electric recording methodologies, optical imaging techniques capturing electric or ionic activity in neurons have flourished over the past decade or so. Specifically, voltage changes can also be detected by membrane-bound voltage-sensitive dyes or by genetically expressed voltage-sensitive proteins (Siegel and Isacoff 1997; Grinvald and Hildesheim 2004; Akemann et al. 2010). Using the voltage-sensitive dye imaging (VSDI) method, the membrane voltage changes of neurons in a region of interest can be detected optically, using a high-resolution fast-speed digital camera, at the excitation wavelength of the dye. A major advantage of VSDI is that it directly measures localized transmembrane voltage changes, as opposed to the extracellular potential. A second advantage is that the provenance of the signal can be identified if a known promoter is used to express the voltage-sensitive protein. Limitations are inherent in all optical probe-based methods (Denk et al. 1994); for VSDI these include interference with the physiological functions of the cell membrane, photoxicity, a low signal-to-noise ratio and the fact that it can only measure surface events.

Calcium imaging has emerged as a promising technology for observing hundreds to thousands of neurons within a micro-circuit with both high spatial resolution and precise localization to specific brain regions. The technique works by introducing calcium-sensitive indicators into neural populations of interest and then imaging these neurons in vivo through a light microscope. These fluorescence measurements are interpreted as a proxy for the underlying neural spiking activity, as there is a biological relationship between elicited action potentials and changes in calcium concentration; a spike causes increases in [Ca2+], which gradually decays due to cell buffering and other extrusion mechanisms. A major advantage of Ca-imaging is that, in combination with genetically modified cre-animals, it offers the ability to record activity from different cell types. In addition, fluorophore kinetics have been drastically reduced so that, in principle, single-spike resolution is obtainable in a limited volume. On the other hand, a major problem arises when intending to monitor spiking activity in larger volumes; instead, what is recorded is a noisy and temporally sub-sampled version of the spiking activity, which in some cases can be orders of magnitude slower than the underlying neural dynamics. Even so, technology advances are continuously offering indicators with faster response times and increased signal-to-noise ratio.

Finally, a method recently revamped as a test bed for understanding the origin and functionality of signals is computational modeling. The first model to link intra- and extracellular voltages was the work of Pitts (1952) describing the extracellular negativity appearing as a result of spiking. Accordingly, the first simulations shedding light into the LFP signal were the pioneering work by Shepherd and Rall explaining the LFP recordings in the olfactory bulb of rabbit from first principles (Rall and Shepherd 1968). Since that time, a number of significant contributions have been made with respect to the neural underpinning of brain signals, where more involved computational models have been employed, for example, accounting for different cell types, varying ratio of excitation and inhibition, etc.

A caveat of simulations typically used to study brain functioning and recreate brain signals is that they have remained somewhat too conceptual. Neurons are typically taken as point-like processes with rules of connectivity imposed upon such nodes. While such simulations have proven informative with regards to analyzing network dynamics (Koch 2004), signals related to electric field activity are induced by the multitude of synaptic and membrane conductances activated along the intricate, three-dimensional morphology of neurons (see also previous sections) and are critically impacted by factors such as the alignment of dendrites and other neural processes, input impinging along these processes, etc. (see above; Buzsáki et al. 2012). Thus, the use of point neurons, while informative for illuminating computational principles, either presumes or even fully neglects the primary means by which such effects are mediated, that is, ionic fluxes along the neural membrane and the extracellular medium. These restrictions are by no means limited to models of electric activity (Fig. 5d, e). For example, a similar lack of understanding is combined with models attempting to replicate Ca-imaging response. In this case, limitations do not arise from the lack of morphology features anymore but instead from the lack of understanding and accurate representation between intracellular Ca-dynamics and the resulting fluorescence signal.

The recent rise in computational power and advances in parallelization have allowed larger, more realistic models to be implemented. Such models carry the potential of being able to link subcellular and cellular biophysics with locally measured signals such as cortical spiking, LFPs, Ca-imaging, etc. For example, morphologically detailed and biophysically realistic single-neuron (Gold et al. 2006; Druckmann et al. 2007; Hay et al. 2011) and population models (Pettersen and Einevoll 2008; Lindén et al. 2011; Schomburg et al. 2012; Reimann et al. 2013; Taxidis et al. 2015) have offered considerable insights into extracellular spiking and LFP signals. Even more recently, large-scale simulation programs combining unprecedented level of detail have been initialized promising to unravel novel insights into a plethora of brain signals (e.g., Markram et al. 2015).

Spatially Distributed Monitoring

Spatially distributed monitoring relies on the same biophysical principles as local monitoring, yet the inability of measuring highly localized sinks and sources due to spatially undersampling renders the origins of electric activity spatially equally diffuse.

Electroencephalography (EEG) is one of the oldest and most widely used methods for the investigation of the electric activity of the brain (Niedermeyer and Lopes da Silva 2005; Nunez and Srinivasan 2006). The scalp electroencephalogram is a spatiotemporally smoothed version of the ECoG (though the impact of the skull on the recorded signal needs also to be accounted for) or LFP, integrated over an area of 10 cm2 or more. Under most conditions, it has little discernible relationship with the firing patterns of the contributing individual neurons, largely due to the distorting and attenuating effects of the soft and hard tissues between the current source and the recording electrode. The recently introduced ‘high-density’ EEG recordings, in combination with source modelling that can account for the gyri and sulci (as inferred from structural MRI imaging) of the subject, have substantially improved the spatial resolution of EEG (Nunez and Srinivasan 2006; Ebersole and Ebersole 2010).

Magnetoencephalography (MEG) uses superconducting quantum interference devices (SQUIDs) to measure tiny magnetic fields outside the skull (typically in the 10–1000 fT range) from currents generated by the neurons (Hämäläinen et al. 1993). Because MEG is non-invasive and has a relatively high spatiotemporal resolution (~1 ms, and 2–3 mm in principle), it has become a popular method for monitoring neuronal activity in the human brain. An advantage of MEG is that magnetic signals are much less dependent on the conductivity of the extracellular space than EEG. The scaling properties (that is, the frequency versus power relationship) of EEG and MEG often show differences, typically in the higher-frequency bands, that have been attributed to capacitive properties of the extracellular medium (such as skin and scalp muscles) that distort the EEG signal but not the MEG signal (Dehghani et al. 2010).

Functional magnetic resonance imaging (fMRI) is an imaging technique that monitors oxygenation levels of blood flow in the brains of animals and humans. Specifically, the BOLD contrast has been used as a proxy for neural activity, though the exact relationship between neural processing and the output signal is a complicated one (Logothetis and Wandell 2004). A number of pivotal studies have appeared over the years relating the BOLD signal with depth LFP recordings rather than spiking (Logothetis et al. 2001; Logothetis and Wandell 2004; Nir et al. 2007; Schölvinck et al. 2010). The main advantage of fMRI is that it can be applied in a brain-wide fashion, allowing for whole-brain associations, and it is non-invasive. At the same time, the temporal sampling rate is fairly slow (typically fractions or a few Hz) and the voxel size of the signal acquisition is considerable (e.g., from fractions to a few mm).

Linking spatially distributed measurements with the biophysics and workings of networks and circuits all the way to single-cell and synaptic contributions typically measured via local measurements has remained a challenge, mainly due to the multiple spatiotemporal scales involved requiring simultaneous monitoring at all levels. While such monitoring is difficult to pursue in humans, recent advances in sensing technology have allowed performing it in other animals, particularly rodents. For example, as mentioned earlier, recent advances in material and technology have allowed simultaneous measurement of spiking, LFPs and ECoG in rodents (but also humans), offering the possibility to link between micro-, meso- and macroscopic electric signals (Khodagholy et al. 2015). In similar fashion, the relationship between the BOLD fMRI signal has been studied in conjunction with spiking and LFP measurements (e.g., Logothetis et al. 2001; Nir et al. 2007; Whittingstall and Logothetis 2009) and, recently, by engaging specific neural population via optical perturbation (Lee et al. 2010).

Computational modeling has the ability to link across scales and relate microscopic with meso- and macroscopic observables. Yet, at the level of distributed brain circuits, detailed representations of each circuit and its elements—such as synapses or single-neuron morphologies—becomes prohibitive. Even so, more abstract models of neural processing, such as circuits consisting of leaky-integrate-and-fire units, have provided many insights into the functioning of distributed brain circuits during sleep and wakefulness (Hill and Tononi 2005), the perception-action cycle (Eliasmith et al. 2012), etc. With regards to conscious perception, modeling has been employed in attempts to link between the various signals and neural dynamics during tasks. In an important study, Dehaene and colleagues (2003) used a neural network model to investigate mechanisms underlying visual perception typically giving rise to activity patterns such as sustained activity in V1, amplification of perceptual processing, correlation across distant regions, joint parietal, frontal, and cingulate activation, band oscillations, and the p3b waveform. The neural network model indicated that access awareness (the step of conscious perception) is related to the entry of processed visual stimuli into a global brain state that links distant areas, including the prefrontal cortex, through reciprocal connections and thus makes perceptual information reportable by multiple means. This study is an excellent example of the kinds of insights computational modeling can offer towards relating signals linked to conscious processing with underlying neural processing in distributed areas.

From Cellular Biophysics to Systems and Computations Associated with Conscious Perception

In the previous section, we reviewed a single-cell mechanism for spike bursting via the dendritic Ca-spike of pyramidal neurons, whose extracellular signature is a plausible candidate for a late superficial current sink. Cortical layer 1 is unique in that it is extremely sparse, and the vast majority (upwards of 90 %; Hubel 1982) of the synapses there are from long-range inputs rather than from the local circuit. Importantly, the pyramidal neurons whose dendrites support Ca-spikes are precisely those neurons that make long-range connections themselves, both cortically (feedforward, horizontal, and feedbackFootnote 7) and subcortically. What computational role could be played by such a physiological and anatomical setup?

One intriguing possibility, which we will call Association by Apical Amplification (AAA), was described by Matthew Larkum (2013). AAA takes a largely bottom-up approach, starting from the detailed physiology of pyramidal neurons and the anatomy of long-range connections in the cortex. Of particular importance is the laminar structure of long-range feedforward and feedback axons in the cortex. There is now ample evidence that feedforward connections strongly innervate the basal dendrites of layer 5 pyramidal neurons with excitatory synapses. Feedback axons innervate layer 1, where the dendritic tufts of pyramidal neurons reside. As discussed previously, the physiology of layer 5 pyramidal neurons allows for a coincidence detection mechanism, whereby concurrent excitatory input into both the basal and apical tuft dendrites causes a high frequency burst.

Additionally, the local inhibitory circuit consists of a number of different cell types that can generally be classified into distinct groups based on their specific effects on either the somatic or apical areas of the pyramidal neuron. For instance, neurogliaform cells in layer 1 metabotropically inhibit voltage-gated calcium channels in the apical dendrites (Palmer et al. 2012; Pérez-Garci et al. 2013), whereas single bouquet cells in layer 1 disinhibit the apical dendrite via their inhibitory effects on layer 2/3 inhibitory cells (Jiang et al. 2013). SST-positive inhibitory neurons are known to directly inhibit the apical dendrites, whereas PV-positive inhibitory neurons directly inhibit the basal dendrites, affording these groups of neurons distinct computational roles in the regulation of pyramidal neuron output (Royer et al. 2012; Shai et al. 2014). In this way, inhibition of the apical dendrites by neurogliaform cells or SST-positive interneurons can act as a form of gain control, regulating the frequency of firing in pyramidal neurons. Alternatively, inhibition of neurogliaform cells, for instance via cholinergic action (though under certain conditions, acetylcholine can have opposite effects; Brombas et al. 2014), can bias pyramidal neurons to high frequency firing.

Taken together, pyramidal neurons and the local inhibitory circuit that surrounds them are well suited to associate feedback and feedforward information streams (Fig. 7). That association, signaled via a high-frequency spike burst in a pyramidal neuron, is then communicated to other areas of the brain, including other areas of cortex. For instance, a pyramidal neuron receiving feedforward orientation information from V1 and motion information via feedback from V5 can bind these two information streams. These associated signals can then contribute, via their influence on the apical or basal dendrites of far-away pyramidal neurons, to other associations. The single-cell mechanism through which concurrent basal and tuft excitatory input creates spike bursting has been named the BAC mechanism (Larkum 2013). In this way, the BAC mechanism causes high-frequency burst firing, as is often observed in vivo (de Kock and Sakmann 2008; Buzsáki and Mizuseki 2014), whereas input into only the basal dendrites will only cause tonic low-frequency firing (Fig. 5). Long-range input can also robustly regulate the BAC mechanism indirectly by recruiting the effect of the different cell types in local inhibitory circuit.

Fig. 7
figure 7

Association by apical amplification (AAA) connects physiological and anatomical details to network level computation and perceptual representation in the cortex. (a) As shown in Fig. 6, input into the basal dendrites of a cell causes steady low-frequency firing in a pyramidal neuron. This feedforward input into the basal dendrites, when combined with feedback input into the apical tufts, causes high frequency burst firing. In the scheme of AAA, feedforward input into the basal dendrites carries sensory information from the periphery, while feedback input into the apical tufts carries predictive information about the stimulus. (b) The parallel feedforward/feedback interactions in multiple areas acts as a selection mechanism to choose which pyramidal neurons are in a state of high frequency firing, ultimately binding different aspects to represent the percept, in this case a tiger (figure from Larkum 2013)

Importantly, low-frequency firing is still available as a unit of information transfer in cases where excitatory input exists into the basal dendrites in the absence of excitation in the apical tufts or when the BAC mechanism is inhibited. These different modes of firing (low-frequency vs. high frequency bursting) can have substantially different influences postsynaptically (Buzsáki and Mizuseki 2014; Lisman 1997). For instance, different short-term plasticity mechanisms act as filters, allowing only certain frequencies to effectively communicate with downstream neurons (Markram et al. 1998; Tsodyks and Markram 1997; Tsodyks et al. 1998). There is evidence that presynaptic bursts cause postsynaptic potentials with substantially greater efficacy (>90 %) than single action potentials (~40 %; Lisman 1997). In this way, the coincident excitatory input into a pyramidal neuron, representing the association of information from different areas of cortex, can create a unique signal that has markedly different influence on other cortical areas than the integration of a purely feedforward (basal dendrite) input.

AAA importantly serves as a concrete mechanism linking details of electrophysiology and anatomy to larger-scale concepts like perceptual binding and the representation of conscious percepts. The connection between forming a bound representation and consciousness are matters of current debate and require some discussion. The topic of binding is generally separated into two issues (Revonsuo 1999). The first concerns the grouping of sensory inputs to form distinct objects (e.g., combining color, shape, and motion to represent a tiger); the second concerns the inherent grouping of the phenomenal mind to give a single conscious experience (a property called the unity of consciousness). The difference between these two sets of concerns is also contentious. One way to think of the difference between these two issues is that the first relates to computational function whereas the second relates to phenomenology. Another interpretation is that the first relates to defining distinct objects and the second refers to arranging those objects into some kind of unified structure. This interpretation holds great relevance to the ideas discussed in the final section. Yet another interpretation is that there is no real difference between the two sets of concerns or even that the unity of consciousness is illusory (Dennett and Kinsbourne 1992). For the purposes of this chapter, it is important to point out that for AAA to be a mechanism of consciousness, it assumes that the functional substrate of phenomenology is the binding of sensory information to form unified wholes.

Let us take into account the different facts that are put together here. The most basic starts with the distribution of nonlinear channels in the apical dendrites of pyramidal neurons, which support nonlinear regenerative spiking, and acts as a mechanism for high frequency burst firing in those cells. Anatomically, these cells make long range connections, both in feedforward pathways where they synapse onto the basal dendrites of other pyramidal neurons, and in feedback pathways, where they synapse into layer 1, and they can act to manipulate the apical dendrites of pyramidal neurons. The extracellular signature of such manipulation, in particular, the dendritic Ca spike, is a large current sink in the upper layers. Psychophysics experiments have found that such a signal correlates to conscious perception. In terms of cortical computation, the association of feedforward and feedback signals might act to bind different aspects of a percept together, though the exact details of such a process at the network level remain elusive. In the next section we discuss candidate theoretical frameworks that might be able to describe such a process in cortex.

Towards a (Unifying) Theory/Framework of Conscious Perception

Before delving into the details of theoretical considerations, it will be useful to quickly review what has been covered in this chapter so far. We began by looking at psychophysical results describing signals that correlate to conscious perception. In particular, late extracellular voltage signals that occur in the superficial layers as well as distributed information processing between different cortical regions were presented as candidate NCC. From there we considered the physical origins of these extracellular signals, residing in the transmembrane currents brought about by the electrical structure of dendrites and synapses. Dendrites of pyramidal neurons, supporting highly nonlinear NMDA and Ca-spiking, were presented as a likely origin for late extracellular signals in the superficial layers. Next, we asked what computational role such an electrogenic structure could play in terms of single neuron processing of synaptic inputs, and we discussed how pyramidal neurons and their dendrites act as coincidence detectors between inputs into the basal and apical dendrites and additionally have powerful mechanisms to regulate such a coincidence mechanism. Importantly, the output of this single cell mechanism is given by a nonlinear increase in the frequency of action potential outputs, in the form of a burst at 100 Hz or greater. As discussed elsewhere (Larkum 2013) the network implication of such a single cell mechanism is a general principle by which pyramidal neurons distributed across the cortex can be associated with each other, ultimately serving as the physical representation of any given conscious percept.

This series of connections—from psychology to signals, signals to neural biophysics, from biophysics to single cell computation, and single cell computation to network level computation—is built upon more than a century of work in a variety of fields. Still, the connections between these levels of understanding require substantial amounts of work to be sufficiently fleshed out before becoming widely agreed-upon scientific fact. Instead, what has been presented so far should be understood as an attempt to combine results from psychology to physiology in a coherent and testable framework. The testability of this framework is of special import, as this requires (in the best case) taking the somewhat ineffable topic of consciousness into the realm of neurons and their functions.

As an important part of that project, a number of theoretical (and often mathematical) frameworks emerged attempting to describe the abstract underpinnings of representation and consciousness in the brain, ultimately providing a description for what it means, in terms of algorithm or function, to create a representation or to be conscious. In the subsection that follows, we will discuss some of these frameworks and explore how they might be related to the ideas mentioned so far. This discussion will not be an in-depth review but will instead feature a largely conceptual overview. Importantly, the discussion that follows should not be interpreted as arguing for an equivalence between these various theories. Instead, what follows is a discussion of the potential areas of conceptual overlap between seemingly disparate ideas and how they might be brought together, at least at certain points of conceptual intersection.

We will frame this section with Friedrich Hayek’s contributions to theoretical psychology, most explicitly given in his 1953 work The Sensory Order: An Inquiry into the Foundations of Theoretical Psychology. The reasons for this are multifold. First, Hayek’s contributions mark a stark departure from multiple theoretical frameworks of that time, for instance behaviorismFootnote 8 and the theory of psycho-physical parallelism,Footnote 9 ultimately arriving at the modern understanding of the role of the brain in perception. Second, as we will see, there are direct conceptual parallels between his ideas and many of the more mathematically rigorous modern ideas. Third, Hayek’s work in theoretical psychology is underappreciated, especially given both its breadth and depth. We will see that Hayek’s work provides a conceptual framework that suggests overlap between a number of modern theoretical ideas and AAA (Fig. 8d). With regards to AAA, the main point here is the connection between computation at the single cell level (e.g., as discussed, coincidence-detection, association) and more network-level implications. This link is what Hayek explores.

Fig. 8
figure 8

Hayek’s types of classification and their relationship to Integrated Information Theory. In Hayek’s theory of cortical function, neurons perform a classification function by grouping presynaptic cells that have similar postsynaptic effects together. (a) In simple classification, classes are defined via their different effects on different cells. Here neuron X defines a class {r,s}, because each of that class causes neuron X to fire. Similarly, neuron Y defines a different class {t}. In the conceptual framework of integrated information theory, these “differences that cause a difference” (i.e., the groups {r,s} and {t} each cause different cells to fire) confer the network with high differentiation but not high integration. (b) In hierarchical classification, simple classification occurs in multiple stages. This allows the network to create classes of classes, and, importantly, to classify the relationships between different classes. For example, each of neurons W, X, Y, and Z defines a class made up of three cells. The cells postsynaptic to W, X, Y, and Z require two simultaneous inputs to fire, signified by the dotted lines. This defines {W&X}, and {Y&Z} as two groups. The neuron R defines a group {W&X,Y&Z}. In this way, the neuron R requires any one of the three cells in groups W and any one of the three cells in group X, or any one of Y and any one of Z, to fire. In this way, the cell R is said to fire to the relationship between W and X or to the relationship between Y and Z. Because each of these relationships similarly causes R to fire, these relationships are thus the same. (c) In multiple classifications, neurons can be in multiple classes, and different classes can have overlapping members. In this way, neuron r is in group X and in group Y, and neuron s is in groups X, Y, and Z. In terms of information theory, this type of classification confers the network with integrated information, since neurons r and s have distinct, but semi-overlapping, causal effects. Thus the network has “differences that cause a difference” but also causal dependencies. (d) A conceptual network of the connections between different aspects of biophysics, signals, and theory

Hayek’s foundational idea is quite straightforward. He posited three orders: (1) the external world (which he called the physical order), (2) the brain (which he called the sensory order), and (3) the mind (which he called the phenomenal order), and he focused his efforts on understanding the relationship between the three. In Hayek’s formulation, the state of the brain has an isomorphic correspondence with that of the mind. The structure of the psychological realm, for Hayek, was relational (e.g., psychological objects are defined relative to other psychological objects), and as such, that structure of relationships that make up the psyche had to be recapitulated in the structure of the neural network and its activity. This strict correspondence contrasts with the correspondence between the outside world and the structure of the brain (and thus the mind), which is imperfect, as shown by the existence of sensory illusions. The problem for Hayek was then to describe how the relational network that is the psyche can be encoded in the structure and activity of a neural network, given the computational properties of single neurons that make up that network. Although this might seem trivial to today’s standards, it cannot be overstated how important this development was, especially given prevailing ideas at the time. In the end, we will see that Hayek’s solution comes in a form that is in many ways remarkably similar (though missing the details of biophysics and anatomy that remained uncovered until the 1990s) to the ideas of AAA, Integrated Information theory, and Predictive Coding and discuss their connections. For Hayek the main questions were:

  1. 1.

    How can a relational network be encoded in the structure and activity of a neural network?Footnote 10

  2. 2.

    How are the relations between objects in the outside world learned and encoded (imperfectly) in the neural network of the brainFootnote 11?

The answers to these questions came by positing that a foundational computation the brain performs is classification.Footnote 12 Hayek described types of classification of increasing complexity (Fig. 8). Simple classification is the sorting of externally different objects into one of a set of different classes by virtue of their differing effects. One example of this is a machine that sorts balls of even diameter into a bin marked A and balls of an odd diameter into a bin marked B. The machine is said to have classified each ball into either group A or B. Simple classification of this sort can describe simple reflexes, which act to group external stimuli by the behaviors that are produced, often by a chain of very few neurons. Hierarchical classification Footnote 13 occurs when successive acts of classification occur in successive stages. In this way, the groupings that occur in a previous stage become the objects to be grouped in the next stage. Multiple classification allows for stimuli to be in multiple groups at once and also for multiple stimuli to be classified differently than when they occur individually.Footnote 14

It is this classification, carried out by the activity of postsynaptic neurons (as a function of presynaptic activity and the structure of anatomical connections), that builds up a system of relations. Here, we already see a conceptual overlap with some modern ideas. For instance, Buzsáki’s (2010) reader concept is a framework for defining cell assemblies by virtue of postsynaptic effects (e.g., by collective effects on reader neurons). Similarly, an important aspect of integrated information theory, which will be discussed more below, is the defining of causal groups as differences that make a difference, in other words, defined by their causal postsynaptic effects (Oizumi et al. 2014). There are even mathematical theories of computation in dynamical systems, which have not been created or even used in thinking about neural systems, that use the same conceptual idea, such as epsilon machine reconstruction (Crutchfield 1994), and could potentially be used to analyze network function.

In the simplest case of classification, two neurons that individually cause the same postsynaptic effect are seen by the network as being equivalent, that is, as being in one class. Thus, the position of these two neurons in the entire system of relationships is the same. Different neurons will in general have varying degrees of overlap in their postsynaptic effects, making it possible to talk about varying degrees of similarity with respect to their position in the system of relations. In this way, Hayek spoke of the postsynaptic activity representing the common attributes of presynaptic impulses that bring about that postsynaptic effect, though he preferred to say that the postsynaptic activity constitutes the attribute, rather than represents it. This was to make the ontological point that these neural systems are what the common attributes actually are and that they do not exist outside of the material actions of the neural network. In other words, the contents of consciousness have a one-to-one correspondence not only with the activity of neurons but also in the structure of the network in which that activity exists. Importantly, this theory differed radically from contemporaneous theories where the qualitative aspects of the mind were somehow attached to the properties of electrical signals themselves. Here, instead, we see the beginnings of an understanding of the psyche that has at its core relations and information: “it is thus the position of the individual impulse or group of impulses in the whole system of connections which gives it its distinctive quality.” (Hayek 1999).

Indeed, it is important to point out that there are two separable aspects of this scheme. The first is the (simple) classification of different signals by their differing effects (“to respond differently to different impulses”). In this way, if each of a group of cells causes the firing of a postsynaptic cell A, and each of a different group of cells causes the firing of a different cell B, then the network has classified these groups of cells into two distinct classes. This alone, however, does not make up a system of relations, because so far we have only described distinct attributes, A and B, with no real relationship between them. The second aspect is then that of putting those attributes in a relationship with one another. This is where multiple classification comes in. By way of example, this process occurs when a postsynaptic cell requires the concurrent input of any of a member of class A alongside any of a member of class B, or the concurrent input of any member of class C and any member of class D. In such a case, we can say that the postsynaptic cell responds to the relationship between A and B, which is the same relationship as between C and D.

These two processes have been put to quantitative work in a modern theory of consciousness, called integrated information theory (IIT), proposed by Giulio Tononi (2008). We will not describe the theory in all of its conceptual and mathematical detail here. For our purposes, it is important to point out the conceptual overlap with Hayek’s ideas of classification, even though the two theories start from a very different set of considerations. The two concepts necessary for Hayek’s scheme to set up a network of relations, that of setting up distinct attributes by virtue of them having distinct postsynaptic effects and that of relating these attributes to each other by virtue of their overlapping (classifying classes) and diverging (being in multiple classes at once) inputs onto postsynaptic cells, can be conceptually reformulated into the language of information theory. In an information theoretic framework, the setting up of distinct postsynaptic effects (simple classification) confers a high entropy to the network, and thus a high informational content (information here can be estimated as the negative logarithm of the number of different potential states of the system). On the other hand, this information needs to somehow be put in a relational network. This is done by the cooperative effects of different classes both postsynaptically and on each other (multiple classification). This informational dependency is called integration in IIT. Importantly, for a system to have both high information content and high integration, and thus high integrated information, the system must simultaneously have dependencies between different attributes to put them in a relation with one another, but not so much dependency as to erase distinctions between different attributes. Because of a set of principled reasons, IIT posits that systems with high integrated information are conscious.

The exact physiological underpinnings of the thalamocortical system that might give the brain high amounts of integrated information, and thus consciousness, are still illusive. However, by considering the similarities between integrated information and classification, a way forward is seen whereby specific network and physiological structures are found to be plausible candidates. In particular, we will find that, by considering increasingly complex structures of classification (and by extension increasingly complex amounts of integrated information), a hierarchical network of feedforward and feedback interactions can work as the substrate for the representation of conscious percepts.

After the general description of classification and how it can be used to set up a series of relationships, Hayek goes on to find implications for this idea in terms of the structure of the cortex and how it might act to build representations. He begins by considering the simplest of automatic reflexes, which performs a simple classification by grouping sensory inputs by the movements they produce. The evolution of the brain led to these pathways of these reflexes, often carried out by a small number of nerve cells from the periphery to the afferent, branching off and sending axons to higher areas of the brain.Footnote 15 This allows the brain to receive information about both the state of the periphery and the actions that the organism is about to take. Unlike pure afferent information, information in the higher centers is available to be used for multiple classification, which can eventually send out motor commands.

Hayek posited a hierarchical scheme whereby the cortex would perform classification in successive layers and could even perform classifications on relationships themselves, thus providing a highly complex and structured substrate for the psyche. As classifications continue on up the hierarchy, classes become more general and abstracted (classes of classes of classes, and classes of relations between classes, etc.). In the case of the evolution of more complicated control of motor responses, the higher levels can thus act to represent and control more general groups or motor commands. Importantly, sensory input comes into an already active network and thus interacts not only with the anatomical structure of the network but with the activity already present in the network. Hayek describes the type of information processing that feedforward and feedback connections might serve in such a case:

The position of the highest centres [of the brain] in this respect is somewhat like that of the commander of an army (or the head of any other hierarchical organization), who knows that his subordinates will respond to various events in a particular manner, and who will often recognize the character of what has happened as much from the response of his subordinates as from direct observation. It will also be similar in the sense that, so long as the decision taken by his subordinates in the light of their limited but perhaps more detailed observation seems appropriate in view of his more comprehensive knowledge, he will not need to interfere; and that only if something known only to him but not to his subordinates makes those normal responses inappropriate will he have to overrule their decisions by issuing special orders.

In this way, certain cells (or groups of cells) in the brain act by comparing their knowledge with what they receive from sensorium, only interfering in the network when there is a mismatch. A framework for neural computation, called predictive coding, is the mathematical description of such a process. The predictive coding framework posits that the brain uses an efficient coding scheme to represent the external world. In particular, this idea posits that natural redundancies in the external environment acting on the sensory apparatus are not explicitly represented in the brain, and instead what is represented is the deviation of the sensory drive from what is predicted. Rao and Ballard (1999) have used this idea to explain the tuning properties of cells in the retina, LGN, and V1. Importantly, this framework puts an emphasis on efficient coding in the brain, something that Hayek did not consider. Despite this, we will see that the biophysical mechanism in which feedforward and feedback signals interact to represent sensory perceptions is conceptually consistent with the predicting coding framework.

In the parlance of predictive coding, feedback signals, from higher to lower levels in the hierarchy, convey predictions of the activity of the lower levels to which they project, that is, predictions of general classes of motor commands given the sensory input. In turn, cells compare predictions with information from lower levels and send error signals forward in the hierarchy. In this way the predictions are continually refined. The diction here becomes conceptually important. A restatement of the processes of refining predictions via error signals representing the comparison of prediction and feedforward sensory driven information puts the ideas regarding network level computation discussed earlier in the chapter squarely in the framework of predictive coding. Indeed, a comparison is biophysically nothing more than the local integration of feedforward and feedback signals that occur in a single pyramidal neuron (and influenced by the surrounding local circuit). A prediction is the feedback activity that predisposes specific neurons in lower cortical areas to varying degrees of activity (or inactivity). An error signal is then the result of the integration of feedforward and feedback signals, which are then broadcast to higher areas of the hierarchy. Feedback activity, and its robust control over the output of pyramidal neurons via dendritic nonlinearities (NMDA and Ca-spikes), serves here as a physiological mechanism in which these kinds of computations might be carried out in the brain. Importantly, NMDA and Ca-spikes are physical mechanisms that can be both monitored and exquisitely manipulated in experiment and thus provide a way to test hypotheses about how the cortex implements predicting coding.

In thinking of the further evolution of cortex, Hayek posits that there is fundamentally no difference between the increased control of more complicated motor responses (for instance, being able to account for context) and the representation of complex percepts. Indeed, the addition of more and more layers alongside more complicated forms of classification in the network allows for the network to form a map of the outside world. In this way the role of convergent fibers to higher levels of the hierarchy confers the binding of different attributes into more abstract attributes, whereas divergence confers the distribution of common attributes to different categories (Hayek 1999; Fuster 2003). Associations between different attributes are given by connections that predispose but do not on their own elicit activity in a postsynaptic cell or group of cells. Associations of this type can occur in any direction of the hierarchy. An important aspect of such a distributed network is that higher levels of the hierarchy can, through feedback connectivity to lower cortical areas, act to predispose certain cells in the lower areas to fire. Hayek describes the process through which multiple categorization and multiple associations interact to create a dynamic and ongoing selection of categories at multiple levels of the hierarchy:

The different associations attaching to individual impulses… will often not only not be convergent but even conflicting; and not all the representations which will form part of the following [ie. postsynaptic effects] of the elements of the complete situation will be capable of simultaneous realization, or would produce a significant new pattern if they did. Since from each element of the structure of connected fibers impulses can pass in a great variety of directions, the initial stream of impulses would merely diffuse and dissipate itself if the overlapping of the following [ie postsynaptic effects] did not determine a selection of some among the many potential paths on which they might travel.

This type of selection, which occurs on account of multiple associations interacting with each other, is consistent with the network level computation that follows from the single-cell biophysics of pyramidal neurons discussed. The main single cell computation in AAA is that of coincident detection or, more generally stated, of integration from multiple axonal pathways. Individually, these pathways bias, but do not cause, the neuron to fire a burst of high-frequency action potentials. The pyramidal neuron thus acts as a highly nonlinear classifier of the thousands of excitatory and inhibitory inputs (and in reality even neuromodulatory inputs) that impinge on the basal and tuft dendrites. These classifying neurons then send their own long-range axons to many cells in far away areas, establishing hierarchical and multiple classification. In direct analogy to what Hayek discussed, it is the collective action of this process that works to select which pyramidal neurons are active in different areas of the cortex and which act to form the bound representation of percepts in the brain.

The ideas presented in this section are all active areas of research. The connections between these topics (Fig. 8d) range from scientific fact (e.g., NMDA and Ca-spikes) to plausible speculation (the connection between the single cell BAC mechanism and network level binding), or are even philosophical in nature (the relationship between consciousness and binding). In the coming decade, it will be important to establish exactly where, in both mathematical and physiological foundations, these ideas overlap and differ. At the very least, Hayek’s stream of thought suggests that there are connections waiting to be uncovered. Ultimately, understanding the cortical network implications of single cell and local network computation would be made easier if a more direct connection between ideas like AAA, which explicitly take into account physiological and anatomical details of the type that are experimentally measurable and readily manipulated, and the more theoretical ideas of network computation like predictive coding and IIT was better understood.