A Cross-Modal Perspective on the Relationships between Imagery and Working Memory

Abstract

Mapping the distinctions and interrelationships between imagery and working memory (WM) remains challenging. Although each of these major cognitive constructs is defined and treated in various ways across studies, most accept that both imagery and WM involve a form of internal representation available to our awareness. In WM, there is a further emphasis on goal-oriented, active maintenance, and use of this conscious representation to guide voluntary action. Multicomponent WM models incorporate representational buffers, such as the visuo-spatial sketchpad, plus central executive functions. If there is a visuo-spatial "sketchpad" for WM, does imagery involve the same representational buffer? Alternatively, does WM employ an imagery-specific representational mechanism to occupy our awareness? Or do both constructs utilize a more generic "projection screen" of an amodal nature? To address these issues, in a cross-modal fMRI study, I introduce a novel Drawing-Based Memory Paradigm, and conceptualize drawing as a complex behavior that is readily adaptable from the visual to non-visual modalities (such as the tactile modality), which opens intriguing possibilities for investigating cross-modal learning and plasticity. Blindfolded participants were trained through our Cognitive-Kinesthetic Method (Likova, 2010a, 2012) to draw complex objects guided purely by the memory of felt tactile images. If this WM task had been mediated by transfer of the felt spatial configuration to the visual imagery mechanism, the response-profile in visual cortex would be predicted to have the "top-down" signature of propagation of the imagery signal downward through the visual hierarchy. Remarkably, the pattern of cross-modal occipital activation generated by the non-visual memory drawing was essentially the inverse of this typical imagery signature. The sole visual hierarchy activation was isolated to the primary visual area (V1), and accompanied by deactivation of the entire extrastriate cortex, thus 'cutting-off' any signal propagation from/to V1 through the visual hierarchy. The implications of these findings for the debate on the interrelationships between the core cognitive constructs of WM and imagery and the nature of internal representations are evaluated.

Keywords: drawing; fMRI; primary visual cortex V1; visual imagery; visuo-spatial sketchpad; working memory.

Figures

Figure 1

Experimental design. (A) Drawing was investigated in a three-phase paradigm consisting of a memory-guided drawing task, abbreviated as “MemoryDraw” (MD), plus two control tasks: a motor-control and negative memory-control task “Scribble” (S), and a task of perceptual exploration and memorization of the model to be drawn “Explore/Memorize” (E/M). Each task’s duration was 20 s, with 20 s null intervals interposed between the tasks, the whole 140 s trial sequence being repeated 12 times in each scanning session using a new image for each repeat. (B) Raised-line drawings of realistic faces and objects were presented as templates to be explored by the subject using her left hand. The quality of the reproductions was assessed by a masked rating procedure, based on recognition and similarity to the templates (examples of reproduction are shown in Figure 3).

Figure 2

A subject on the scanner bed operating our novel multimodal MRI-compatible drawing system. The plexiglass gantry supports a drawing tablet while a fiber-optic drawing stylus captures and records the drawing movements with high precision. The motion-capture information synchronized with the fMRI allows the effect of behavioral events to be analyzed to high precision.

Figure 3

Examples of blindfolded drawings of the vase with a flower, the face profile, and the boot (the corresponding templates shown in Figure 1B are easy to recognize: the first and the second in the top row, and the first in the bottom row). Remarkably, the post-training drawings, recorded in the scanner by the motion-capture system show a lot of specific detail, which makes them readily recognizable as specific examples of their category, although they were drawn without visual or tactile input (i.e., with eye-hand coordination eliminated), but were guided solely by tactile-memory.

Figure 4

BOLD activation and deactivation in non-visual memory drawing in the blindfolded. Post-training group responses from the MD task are derived according to the GLM described in Section “Materials and Methods,” and projected on inflated representations of the lateral left (A) and right (B), and medial left (C) and right (D) hemispheres in MNI brain coordinates. Dark-gray, sulci; light gray, gyri. (A,B) A non-occipital network of temporal, parietal, and frontal regions is activated (yellow-orange coloration), together with strong deactivation (blue-cyan coloration) in a network that corresponds broadly to the default-mode network (except for the occipital lobe portion). (C,D) Both medial views show massive activation along the calcarine sulcus (CaS) corresponding to V1 surrounded by deactivation, which extends throughout the lateral regions of the visual cortex. Activation is shown down to −1 < z < 1; the scale bar indicates the color-coding for the respective z-score levels. Note that, interestingly, the medial CaS activation spreads to the same eccentricity in both hemispheres.

Figure 5

(A)MD flat-maps centered on the occipital pole. ROIs for the retinotopic hierarchy are indicated by colored outlines, with hMT+ and LOC based on functional localizers. The post-training MD map shows a “triad” of three activation regions (orange-yellow coloration). Note in particular that the (non-stimulated visually) primary visual cortex, V1, forms an unusual isolated “island” of activation surrounded by a “sea” of suppression in the adjacent retinotopic areas. The other two activated regions seen on the flat map are the caudal intraparietal sulcus (cIPS) dorsally and an additional locus at the occipitotemporal border (LOtv). (B) Average response amplitude with standard errors for blindfolded memory-guided drawing in a group of six subjects, showing positive signal in the triad of areas – primary visual area V1, cIPS, and LOtv; these three “islands” of positive activation are separated by strong deactivation in both the ventral and the dorsal extrastriate areas. Error bars represent 1 standard error of the means.

Figure 6

The MD task shows the predominant training effect in the primary visual cortex. A voxel-wise comparison, projected on inflated representations of the posterior left (LH) and right (RH) hemispheres, with orange-yellow coloration showing the average pre/post increase in BOLD activation for MD in the CaS (V1) region.

Figure 7

Comparison of the activation pattern across the three tasks. (A) Average time-courses of BOLD activity (black curve) in V1 for the sequence of the E/M, MD, and S task intervals (white bars); the colored curves are the best fits of the model predictions for the three tasks to the time course; the four dark-gray bars indicate the 20 s null intervals separating the three tasks. (B) Bar-graphs for the estimated V1 activation for each task; the activation levels refer to the beta weights for the event types in the GLM. Dotted lines and the error bars represent confidence intervals for two different forms of statistical comparison of the activation levels. Dotted lines represent the 99% “zero” confidence interval, within which the activations are not significantly different from zero. Error bars are 99% “difference” confidence intervals designed to illustrate the t-test to assess the significance of the differences between pairs of activation levels in each figure, i.e., amplitude differences are not significant unless they exceed the confidence intervals for both compared activations. (C) Average time-courses of BOLD activity in the deactivated regions surrounding V1. (D) Bar-graphs of estimated activation for each task for the deactivated regions presented in (C). Conventions in (C,D) are as in (A,B), respectively.

Figure 8

The proposed re-conceptualization of the visuo-spatial sketchpad as an amodal-spatial sketchpad. Modified schematic of the main modules of Baddeley’s classic model of working memory including the visuo-spatial sketchpad (after Baddeley, 2003), where the added “Amodal-Spatial Sketchpad” block depicts our re-conceptualization of the visuo-spatial sketchpad as being accessible to any sensory modality (from Likova, 2012).