The idea of writing this book arose several years ago, evolving from my interest in how audio-visual media, especially television information programs, influence the way their viewers understand their content. While summarising an overview of psychological, sociological and media studies theories as well as the results of my studies on understanding television broadcast, I noticed two characteristic tendencies (Francuz, 2002). First and foremost, it seemed to me that audio-visual media analysts focus much more often on the textual (verbal) element of the broadcast than on the visual data. I also concluded that they refer much more often to the content of media broadcast rather than to its structure.
The analytical approach, according to which the meaning of the audio-visual message can be found almost exclusively in words contained in it, while the meaning of an image – if it is taken into account at all – is treated as a marginal and insignificant addition to the message conveyed, represents a substantial misunderstanding of the medium. Images and words are equally important in terms of conveying meaning and they determine the sense of audio-visual communication. What is more, the marginalization of the role of the image in creating the meaning of audio-visual communication leads to a failure to see the complex semantic interactions occurring between the meaning of words and the meaning of accompanying images. Moreover, their foundation, or form, is often ignored in studies of audio-visual media.
Both those findings drew my attention to the image as a carrier of meaning, and in particular, to the association between the structure of an image and the meaning it conveys. In the first version of this book, my aim was to describe the relationship between the content of audio-visual communication, which consists of both the meaning of the image (video) and the meaning of wording (audio) and the structure of both methods of encoding meaning. Specifically, my aim was to describe all these relations in the context of the results of studies on the neurophysiological correlates of visual and audial perception and perceptual integration. In the first decade of the twenty-first century, studies on the subject were in abundance, because cognitive neuroscience was developing in an uncannily dynamic way. While studying the literature on the subject, I quickly came to realize that in this version, it would be virtually impossible to carry out the project in a reasonable time frame. The rapidly increasing number of reports from studies was not the only problem; the high level of doubt regarding the reliability of all the results published in them was of greater importance, something that is, incidentally, typical for every new field of science.
I, therefore, decided to limit the scope of my work to the description of the neurocognitive basis of image perception. At this point, however, I had to make some other decisions restricting the scope of the problem for the same reasons I had to limit my work only to visual analysis of media content. At first, I decided not to discuss matters related to the motion of objects in a visual scene.
The primary reason for this was the fact that motion in films, television, or theatre stages complicates the process of forming the communication of meaning in an incomparably greater way than motionless forms enclosed in single, still frames.
The second reason was the fact that audio-visual media analysis which considers only the visual aspect may lead to artifacts, as the basis for this type of messages to be formed is precisely the interactive influence of both pathways of meaning encoding. Taking mobile visual communications as an example, we would find that their scope would have to be limited only to certain silent films from the beginning of the twentieth century (and, for that matter, only those that were entirely stripped of their textual element), pantomime or dance (excluding music), or television reports of no comment type, which do not contain commentaries expressed in words, but are filled with all sound effects, i.e., ones. To put it briefly, contemporary culture has created relatively few mobile media communications which are entirely deprived of text and audio elements.
Finally, studying movement in visual communication leads to the necessity of undertaking deepened studies on the neurocognitive correlates of seeing apparent movement, optical illusions associated with movement and actual movement, both of the objects in a visual scene and the observer, which is an issue that is both fascinating and immensely complicated.
The second and last fundamental limitation that I imposed on this work is the decision not to discuss the matters related to the three-dimensional space of the visual stage. First and foremost, three-dimensionality is very closely associated with movement, both in the sense of the subjective experience of seeing and its neurobiological basis. Sculptures or architecture, perceived in the real three-dimensional space can be observed not only using the stereoscopic perception but also from unlimited points of view, which produces entirely new interpretation opportunities, compared to observing them in a photograph or a painting. Therefore, studying three-dimensional communications imposes the necessity of taking the location of the observer on the stage as well as their movement and multiple characteristics related to their posture into account. The situation is additionally complicated by the problem of perception of three-dimensional stage, generated by new 3D technologies. As it is in the case of seeing movement in a film, which is based on entirely different principles than seeing movement in real life, the experiences of three-dimensional space in a 3D cinema and in the real world use different mechanisms of vision.
Having introduced all these limitations to the original version of this monograph, I stood before a flat image, and I realized that from neurocognitive perspective it was the most appropriate subject of studies to start with. Several facts convinced me on this point.
First and foremost, flat images – created by painters, photographers and draftsmen – have been produced by man for more than 30 thousand years. The number of examples of images understood this way is essentially uncountable, which produces limitless opportunities for illustrating issues related to seeing these images.
Secondly, I realized that flat images are the subjects of advanced studies in various fields of knowledge, such as history of art, anthropology, aesthetics, philosophy as well as social and biological sciences. Such studies are conducted not only by scientists, but also by the creators of the images. I also noticed that the number of biologically-orientated researchers showing interest in the visual arts has rapidly increased due to the fact that the ways of picturing reality by artists of almost all epochs constitute an inspiration for studies on visual perception in general as well as its neurocognitive foundations. There is a reason why artists are sometimes referred to as ‘unconscious neurobiologists’ (Ramachandran and Hirstein, 1999). Art historians, anthropologists and even aesthetes have also noticed the new perspectives that the contemporary neuroscience opens before the traditional methods of analysis.
Thirdly, when I wanted to illustrate the effects related to seeing during my lectures on perception, I often preferred to use the examples of pieces of visual art rather than the stimuli created artificially for the purposes of psychological and neurocognitive studies. In those pieces of art, I quite often noticed glimpses of ingenious and surprising intuition related to the things which artists saw and perpetuated in their canvas.
The fourth aspect is the fact that studies on the neurocognitive foundations of image perception constitute a combination of my two greatest interests. On the one hand, they are professional interests in perception, in particular of visual perception, on the other an interest in art as a hobby. I have been developing these interests somewhat independently. The idea of writing this book motivated me to use my knowledge on both domains within one monograph. The opening of the Psychoneurophysiological Laboratory by the Department of Experimental Psychology of the Lublin Catholic University run be me, in 2008, was also an additional motivator. It opened up the possibility of conducting independent experimental research on media by means of oculographic and electroencephalographic techniques.
Writing by means of text and image
In the preface to the twelfth edition of his book The Story of Art (2005) Ernst Gombrich wrote, ‘From the very beginning, the book was supposed to speak of art, using both words and images, enabling the readers to look at the illustration under discussion without turning the page, if it was only possible’ (p. 10). This sentence became something of an idée fixe to me while I was writing this book. Its first parts indicate the fundamental rule of writing about images, both using text and images. The idea is not obvious for many authors and publishers of this type of monographs who view text as the most important source of knowledge, including the knowledge of images. In particular, the implementation of the idea of making use of images as conveyors of meaning equal to text is still distant in the case of academic works, both books and articles, in which the space for illustrations is limited to a minimum. It is a display of exactly same tendencies referred to above. Perhaps financial considerations could be of importance here as well.
Be that as it may, in my opinion, the image and the space it occupies in a publication, especially one devoted to images, is like the text and the space of the paragraph it occupies. Images quite often contain much more useful information than even the richest text devoted to describing things that can be simply seen. This is how I approach the illustrations presented in this book. The implementation of the rule of writing using text and image is not easy and requires constant control of the space of the page, on which fragments of text and images are placed in a logical sequence, based on a principle similar to that upon which the author orders his thoughts in subsequent paragraphs. It is not always successful in the case of images that cannot be broken into two halves, so that the second half can be moved to the next page.
The second thought from that quotation by Ernst Gombrich concerns content control, the so-called centerfold, namely two pages of an open book that are connected at the spine of the book. The control of the centerfold pages consists in distributing the text and illustrations on them in such a way that the pages do not have to be turned in order to see the things that are is being discussed. This is another limitation that I imposed on myself while writing this monograph. Maintaining this principle was just as difficult as maintaining the previous one, but I did my very best to follow it.
If I wanted to comply with both principles I had to break the text and the illustrations on subsequent pages of the book on my own. At least at this stage of project implementation I could not really imagine the necessity of continuously making agreements with the publisher regarding how much text I should remove and how much text I should add on every page so that an illustration of an optimal size could be placed under a given paragraph and, in addition, within the same centerfold. Indeed, the classic printing technique is a true challenge for the author who tries to make the content and the form of its presentation conform.
The next rule that I was trying to follow concerned the appropriate choice of possible high quality of the illustrations. Carrying out this assumption took almost as much time as writing the texts the illustrations were accompanied by. At this point, I would like to thank my co-worker, Paweł Augustynowicz, for helping me prepare over 30 illustrations. Many of them were developed on a three-dimensional model of the brain and the eye produced by AnatomiumTM – 3D Digital Human Brain. Without Paweł’s help, I would not have been able to prepare all the illustrations that are included in the book.
The illustrations presented in the book are classified into several categories, which include patterns, two and three-dimensional models, photographs and drawings from various scientific publications, painting reproductions and graphic depictions of the results of own studies. All illustrations, which are presented with information regarding their source, have been added to the text by means of quotation. If they graphically illustrate the results of own studies, they also contain information regarding the procedure of their creation. It particularly concerns the images that were processed using Adobe Photoshop in order to reveal their interesting structural properties, such as, for example, edges of the object depicted in them or separation of colors used for their painting.
I created some images illustrating the results of the oculographic studies by combining two images with the use of the Adobe and OGAMA 4.3 or iViewX programs. Work on a dozen or so illustrations resembled an attempt to separate, for instance, a piece of cerebral cortex using the technique of cytochrome oxidase in order to be able to see its internal structure. All the illustrations that presented the effects of processing their content using computer programs were developed using the same algorithm, which enables their comparison in the book as well as providing interested readers with an opportunity to create illustrations on their own.
The method that I readily used in this book is analogy. I believe that at the present stage of development of the interdisciplinary field which is, after all, a meeting point of distant sciences such as history of art, media sciences and neuroscience along with cognitive psychology, prolific analogies, which will allow the discipline to develop, unrestrained by the methods of the above fields, are needed. I am particularly grateful to Professor Adam Biela for drawing my attention to this aspect of the narration that I applied. I am also grateful for his suggestions based on his reading of the entire manuscript.
Some words of gratitude
The first official readers of this book were its editorial reviewers: Prof. Aneta R. Borkowska of the UMCS (Maria Curie-Skłodowska University), a clinical psychologist and neuropsychologist, and Prof. Dobrosław Bagiński, a specialist in visual communication. Both of them, on the basis of two very different points of view, drew my attention to several important matters, clarification of which enabled me to articulate my thoughts better.
Among the many remarks I received from my reviewers, there is one I would like to dwell on for a bit longer. The manuscript of this book was given to them under the title of A Neurocognitive Theory of Images. It seemed to me that the title was appropriately adjusted to the book’s content. After I wrote the preface I realized that, due to the numerous limitations which I presented in it, I could not refer to the content of the monograph as theory, in the sense of a coherent concept system in the field of cognitive neuroscience. It is, however, a work that is clearly aimed at developing such a concept. For this reason, the previous title has become a subtitle and was additionally complemented with the expression: TOWARDS a Neurocognitive Theory of Images.
The person who helped me formulate the new title was Professor Dobrosław Bagiński, with whom I often talked about the issues discussed in the book. During one of such discussions he unexpectedly brought forward the word imagia to define the magic tricks that the brain uses in the process of seeing. Closer look at the word revealed some incredibly accurate references to the problems explored in this monograph. Its name clearly refers to the Latin word imāgō, which denotes image. Unlike the English or French-sounding words image or imagé, the word imagia sounds more familiar and brings to mind associations with the word magic the Latin origin of which was, in turn, associated with the magical arts (ars magica). This book is partly about the art and magic of paintings, and about the magic tricks of the visual system, which – like an illusionist – conjures up new knowledge on the way the world looks. Finally, there is the prefix, i-, which locates the associations in the area of the Internet, the most powerful contemporary medium, and forms a new word – iMagic – derived from iPhone, iTunes as well as iBook, iPerfumy, iTV etc. Furthermore, the word imagia has one more advantage – it is short. For all those reasons the book was entitled imagia.
I would like to take this opportunity to also thank the Rector of the Lublin Catholic University, Professor Antoni Dębiński, whose short fiat paved my way to publishing an obviously costly book in full color at the University’s publishing house. I would also like to thank my co-workers, postgraduate students and the employees of the Department of Experimental Psychology who undertake research projects concerning art perception with great commitment. I would also like to express my gratitude to Mr Paweł Jaskanis, the director of the Museum of Palace in Wilanów, Warsaw, and Mrs Elżbieta Grygiel, the head of the Department of Social Communication at the Wilanów Museum for their kindness and help in organizing the material for a research on beauty and for making a documentary.
Last but not least, my greatest gratitude goes out to my wife, Grażyna. Had she not taken on the sizable burden of nearly all the household chores, this book would have never been written. She also found enough time to read it from cover to cover, and to provide linguistic correction (the responsibility for all errors and oversights that may still appear, however, remains with me). Either way, this book is dedicated to her.
Piotr Francuz, 10 June 2013