Ðóñ Eng Cn Translate this page:
Please select your language to translate the article


You can just close the window to don't translate
Library
Your profile

Back to contents

SENTENTIA. European Journal of Humanities and Social Sciences
Reference:

The Barrier for Robots. Subjective Experience as a Magical Phenomenon

Subbotsky Eugene

Doctor of Psychology

Professor, the department of Psychology, Lancaster University, Great Britain

CR0 2GG, Velikobritaniya, London oblast', g. Croydon, ul. Saffron Square, 11

subbotsky@me.com
Other publications by this author
 

 

DOI:

10.7256/1339-3057.2016.3.21104

Received:

16-11-2016


Published:

01-12-2016


Abstract: The paper discusses problems that arise regarding the relationships between the brain and subjective experience. Can robots create in humans a full-scale illusion of subjective experience? Can a person tell the difference between simulated consciousness and authentic subjective experience? What is better: to enjoy life in the world of illusions, or to live the life of hardships in the world of true reality? The analysis has shown that even if the computer technologies in the future became incommensurably more powerful than the technologies of today, we can’t expect that people with the help of computers, not to mention computers in their own right, would be able to create an authentic copy of human subjective experiences. The reason is that subjective experience is a magical phenomenon. This phenomenon is not determined by physical causes, cannot be logically deduced from more general premises, and therefore, cannot be simulated on a full-scale. The altered states of consciousness such as hallucinations or virtual reality are imitations and not authentic subjective experiences. A person immersed in these altered states of the mind is aware, during the altered states of shortly after, that they are nothing but illusions. Neuroscience and cybernetics will be creating increasingly complex interfaces between machines and subjective reality of consciousness, but the gap between simulations of mental processes and authentic subjective experience will never be bridged.


Keywords:

consciousness, subjective experience, computer models, simulated subjective experience, magical thinking, authentic subjective experience, brain mechanisms, double-consciousness, free will, physical reality

Problem

Within yourself, keep life in hold:

Your soul is a whole world

Of thoughts of mystery and charm,

They will be sunk in daily hum,

And scattered by the sun’s rays, rude:

Hark to their song, and just be mute.

Fedor Tyutchev (Translated by Yevgeny Bonver [1] )

In the 1999 film “The Matrix”, written and directed by the Wachowski Brothers, a fictional world of the future is depicted in which robots subdued the human population by turning people into the source of energy [2]. In order to secure that a human body, which lays in a capsule clad in a net of wires, worked faultlessly as a source of power the machines created simulated reality – The Matrix - by stimulating the person’s brain. In this artificial reality the person is immersed into the illusory dream world, which looks identical with the normal everyday world. The film could be interpreted as a modernist version of Greek philosopher Plato’s theory, which suggests that a real world is an illusion. According to this theory, a person is chained in a cave and can only see shadows of the “real things” thrown by the fire on the wall of the cave opposite him or her. These shadows people take for the real world.

And so, a prisoner of The Matrix computer programmer Neo is immersed in the world of simulated subjective reality and takes this reality seriously. Only assistance from the real world made Neo learn the truth and discover the real world. In the film the real world looks unappealing: Dead cities demolished by machines, hostile for life ecology, the underground city-sanctuary in which people who had managed to escape from The Matrix found a shelter, and food that is devoid of color, taste and aroma. A person who managed to move from the world of The Matrix into the real world faces a dilemma: To remain in this devastated and colorless world or to return into the more attractive and pleasurable world of illusions.

In the film, artistic fantasy is skillfully waved into the fabric of real problems of the human being. Can robots simulate human subjective experience? Is it really the case that a person, who is plunged into the world of simulated subjective experience is unable to tell this world from the real world? What is better: To enjoy life in the illusory world or to move into the true world in order to fight for happiness but live a life of hardships and deprivation?

These are the questions that I would like to discuss in this paper, in the context of recent psychological studies on magical thinking in modern people.

Magical thinking and the magical world

My interest in magical thinking emerged over 30 years ago, while I was observing children playing games of pretend. The feature that surprised me was how easy it was for a 5-year-old «player» to overcome the most insurmountable obstacles and solve most difficult problems. «A tiger is chasing you, what are you going to do?» - I ask a child, and the child answers «I'll run away from it». «And if there is a precipice in front of you?» «I'll jump over». «And if the precipice is very wide?» «I will make a bridge and run over». In play a child can pretend to be a bird, fly to another planet, read the mind of a wizard, erect or demolish a castle with the help of a magic wand. In other words, in play the child «thinks magically». Simply put, magical thinking is the kind of thinking that embraces creatures and events, which violate known laws of physics, biology and psychology. Physical objects that appear from nothing; demons, which sort out fast molecules from slow ones; animals, which can speak human languages; gods and spirits, which can read human minds and be in several places simultaneously - these are examples of magical creatures and events [3].

But isn't yesterday's magic today's science? As British science fiction writer Arthur Clark said, any sufficiently advanced technology is indistinguishable from magic [4]. Indeed, just three hundred years ago such events as instant transfer of a visual image over long distances and flying in the sky would look magical, but today they are science. American physicist Michio Kaku in his book «Physics of the Impossible» forecast that in the future even more advanced «magical» devices would arrive, which allow time travel and teleportation [5].

The parallel between magic and advanced technology is tempting, yet it is not entirely accurate. There is one important difference between a magical action and an action of an even the most advanced technical device. In the world of classical physics everything – from an atom to a human being – is a complex mechanism. The objects that classical physics study are soulless and mindless machines. By contrast, in the world of magic all objects are animated; in this world there is no difference between living and non-living things, there are no dead objects; every object, every entity has a soul and a mind of its own. Every entity can be spoken with – all you need is to know the language. There is a fundamental difference between magical communication and physical interaction – the same difference that distinguishes an invitation to sit down from making one sit down by brute force. Physical interaction is based on one of the four known fundamental forces – gravitational, electromagnetic, strong nuclear and weak nuclear [6], whereas magical communication, just like social communication, is based on semantics. For instance when we ask a person at a table to pass a saltcellar, we initiate the person’s action, which cannot be reduced to physical energy that we spend on speaking the words. In other words, there are important features that distinguish a magical event from a natural causal event.

One of these counterintuitive features is that in the magical world mind can change matter (“mind over matter” magic). The ancients believed that if a person wanted a certain event to occur (e.g., that it started raining) and asked gods for it by performing certain prayers and rituals, then the wanted event would really happen. It is easy to see the difference between magical “mind over matter” actions and actions of modern devices that react to electromagnetic waves of the brain (e.g., motor prosthetics or brain-computer interface [7]). The principle on which neuroprosthetics are based looks magical but this resemblance is only superficial. Neuroprosthetics do not decode the meaning of a thought but react to the neuronal signals that accompany the thought, thus converting these signals into a movement (e.g., switching on the motor of an artificial limb or shifting a cursor on a computer screen). In principle, neuroprosthetics is a highly advanced and sophisticated version of a remote control device. On a remote control device we press buttons by a finger, whereas on a neuroprosthetic device a disabled person “presses the buttons” by intentionally producing appropriate neuronal impulses. This kind of interaction is still physical, not semantic. It is not that a receiving end of this interaction – a computer or a prosthetic limb - has a mind of its own and “understands” what a disabled person thinks or wants; these devices blindly react to the physical force – electromagnetic waves produced by a neuronal signal that correlates with the thought or the wish. By contrast, at a receiving end of a magical incantation or a prayer there is another mind, which considers the plea and makes a decision about whether to grant or reject it.

Another distinctive feature of a magical event is that this event doesn’t conform to conservation laws (e.g., conservation of energy, matter or momentum) (“something from nothing” magic). A wizard or a genie can create a mountain, a castle or even a city by waving a magic wand, a cat can vanish into nothing with the last thing visible being its grin.

Finally, in the magical world different objects can have magical connection to each other that cannot be causally explained (“participation” magic). The term «participation» was coined in the beginning of the XX-th century by French anthropologist Lucien Levy-Bruhl [8]. Participation implies that an object that was in contact with a person (for instance, the person's bunch of hair or a piece of clothes) retains a magical connection with this person, which works instantly and at a distance. A similar participation exists between objects that resemble each other (e.g., a person and the person’s photograph or a doll called by the person's name). If a wizard burns the bunch of hair or punctures the doll with a knife while simultaneously chanting a certain spell, then the person is expected to get ill and die. Participation can also exist between two objects that are totally unrelated one to the other. For example a character of Slavic folklore Koschei cannot be killed by targeting his body. His soul is hidden separate from his body inside a needle, which is in an egg, which is in a duck, which is in a hare and so on and so forth. To kill Koschei, the hero must find and break the needle.

It is worth to note that these three features of the magical world are interconnected: “Mind over matter” and “participation” magical events don’t conform to conservation laws, and “something from nothing” magical event can happen as a version of “mind over matter” (e.g., a wizard creates an apple from thin air) or “participation” (e.g., speaking a magic spell that includes the words "the mountain move" can indeed make the mountain move).

For most people magical thinking unfolds in their imagination. Indeed, in our dreams we can travel back in time, speak to our diseased relatives, see magical transformations of people into animals and vice versa. Many of us enjoy watching movies with a magical content, reading books about wizards and shamans, studying ancient myths, attending museums in which paintings with magical creatures and events are on display. Because these “games with magic” are confined to the realm of our fantasy and labelled as entertainment, art or dreams, they peacefully coexist with our belief in science. Science rejects magic as a fallacious type of thinking, but is prepared to tolerate magic as long as magic does not trespass from the realm of imagination into the realm of physical reality.

Yet, even in the domain of physical reality magical events happen. One of such events is an event that we call random. Indeed, scientists are talking of random events all the time, but what is a random event? It is commonly accepted that a random event is not an effect of a certain casual process, but then it must be a miracle – a “something” that appeared from nothing. And if a random event is an effect of a certain cause, or a bunch of causes, then the event is not random. In other words, by definition a random event is a magical event of the “something from nothing” type, which doesn’t conform to conservation laws. That is why scientists are prepared to allow random events to happen in everyday life, but not in the fundamental laws of nature. As Einstein once put it "God doesn't play dice". We call a result of a dice roll a random event, but only mean that the randomness comes from our inability to account for atmospheric noise and all the tiny variations in the effort of a throwing hand. But that doesn’t mean that this result is random in the fundamental sense of the word, i.e. it can not be causally explained “in principle”. If we had a supercomputer, which could take into account all the forces that affect this individual dice throw, then we would be able to exactly predict the result. Às it stands, a “random” event in the everyday life is only “pseudo-random”.

Like it prohibits randomness, classical physics also forbids a direct effect of the mind on results of experiments. Of course, an experimenter may affect experimental results in many ways, but these «experimental biases» only happen due to inaccuracies of the experimental procedure. What is impossible is the magical «mind over matter» effect, e.g., affecting a result of throwing a coin by just thinking hard about the desired outcome. Indeed, if the experimenter's mind could directly affect a result of an experiment, this would undermine the objectivity principle, which is fundamental for science [9]. However, the emergence of quantum mechanics in the XX-th century shattered the aforementioned prohibitions, thus opening a gate for magical effects to enter physical reality. It tuned out that randomness is an unavoidable element of quantum processes, and the observer's consciousness plays a definitive role in what the observed physical reality is [10].

But if magical events occur in the domain of physical reality, could they also be real in the domain of human subjective experience?

What color is a magnet? Physical world and subjective experience

Blue sky covered by light patches of thin clouds, a quiet splash of clear ocean waves licking the white sand of a tropical island, tender, as if cut out of yellowish marble, petals of a tea-rose, delicate taste and aroma of a cream cake – this is how the real world in its best appears to our senses. But those of us who studied physics and chemistry know that all the aforementioned sensations are noting but illusions. In the real world there are no such things as “green” or “blue”, “sweet” or “sour” – there are only electromagnetic waves of certain wavelengths that enter our eyes and molecular structures of food that stimulate taste buds in our mouth. Reflected from magnolia leaves, electromagnetic waves get into color receptors of our eyes and are transformed into nerve impulses, which proceed further into our brain. Having passed our optic nerves, the impulses are sucked into the unimaginably complex network of neurons of our brain’s visual cortex and, lo and behold, we see green color.

This transformation of physical processes into subjective phenomena, or qualia, is a miracle indeed, which remains unexplained by science. What existed prior to that moment - light, reflected from magnolia leaves, electrical impulses in receptors of the eyes, optic nerves and neurons of the brain – all of these physical structures were connected one to the other by a causal chain, which can be traced down step by step. And suddenly this chain is broken and there appears something entirely different - the experience of greenness. Colors, tastes, odors, sensations of heavy and light, big and small, short and long, quick and slow – the whole world of subjective experience is a magical phenomenon, if by magical phenomena we understand the events that can not be deduced from the known laws of physics, biology and psychology. Like in The Matrix film, we are faced with the dilemma: Which of the two realities - the reality of subjective experiences or the reality of physical processes that underlie these subjective experiences - is primary reality, and which is secondary, simulated reality?

It would be logical to assume that primary reality of the two aforementioned realities is the one that defines the other reality. Indeed, can we change colour of magnolia leaves or taste of a cream cake by just thinking hard about changing them? No, we cannot. By contrast, we can easily accomplish the above changes by placing the magnolia leaf under a colour filter and by changing molecular composition of cream that covers the cake. Not only sensations, but a lot more complex subjective structures, such as temperament and personality, can be affected by altering material structures. For instance, Huntington’s disease causes dementia and the lack of coordination in people aged between 35 and 44 years; but it can also affect personality, causing manifestations of anxiety, depression, aggression, egocentrism, and worsen addictions, such as gambling, alcoholism and hyper sexuality. And all these disorders result from a mutational change of just a single gene among tens of thousands [11]. Invisible to the naked eye molecules of drugs, neuromodulators, hormones and viruses, when they enter the brain or body, can change animal and human behaviour. If a capsule of an antidepressant - the neurophysiologist Martha Farah asks, - can help us effortlessly overcome problems in the everyday life, then aren’t personality and temperament nothing but the effects of the structure of our bodies? And if this is the case, then is there anything at all in a human being that is not the effect of the structure of our bodies? [12]

Indeed, taken on their own, magnetic and electric fields don’t have colour, molecules and their combinations don’t taste, genes have nothing in common with human personality, and chemical substances used in antidepressants have nothing to do with our mood and behaviour. And yet manipulations with these structures (electromagnetic field, molecules and genes) can change subjective experiences. It appears that the dilemma “What stands in the beginning: biological structures or subjective experience?” is decisively solved in favour of biological structures. But let’s not jump to conclusions.

First of all it turns out that biological structures are connected to subjective experiences in a very loose way, which allows a wide range of variations. To start with, it is well known that chemical substances affect different people differently. For instance, effects of antidepressants depend on the individual structure of gene 5-HTT, which is responsible for production and transfer of serotonin [13]. Às for the link between our genes and our behaviour, this link is even more flexible. Cases like Huntington disease, when human behaviour can be affected by changes in a single gene, are rare exceptions. Most diseases depend on a combination of dozens or even hundreds of genes. Besides, the way a certain combination of genes affects human behaviour depends on a number of environmental factors. For example, scientists established hundreds of genes that are linked to schizophrenia, yet these genes’ role is mediated by environment; most often the unfavourable combination of genes results in schizophrenia in immigrants, especially in those whose culture and appearance differ most from those of indigenous population [14]. There are genes that predispose a person to depression, but whether these genes would or would not fire depends on the person’s life experience. If a person’s life experience is positive and a number of traumatic events is low, the person is unlikely to develop a depressive psychological condition, even if he or she has a genetic predisposition to depression [15].

It turns out that in order to write a computer program for even most elementary subjective experiences, such as depression, one has to take into account the unique combination of genes and environment. We can imagine the vastness of the information about various combinations of genetic and environmental factors that robots in The Matrix would need in order to simulate full-scale subjective experiences of a person, with all of these experiences’ colours and sounds, tastes and odours, happiness and unhappiness, hope and beliefs, love and hatred, sufferings and joy, communication and loneliness. The robots would also have to simulate the person’s unique personality, temperament, memories of his or her past life, knowledge of languages, information the person has about his or her country and society, and literally all the bulk of knowledge that a modern educated person holds in his or her mind.

“And so what?” – a reader might ask. – “The robots of The Matrix have almost unlimited capabilities to create the unlimited number of combinations of genes and environmental influences. If the robots were capable of taking the upper hand over their creators – humans, they would certainly be able to simulate full-scale human subjective experience”. If the reader’s argument is correct, then human subjective experience has nothing to do with magic. In this case, subjective experience is simply a product of a complex combination of genetic and environmental factors, in the same way as the “ability” of a spaceship to fly in space is a product of combination of the spaceship parts. Indeed, none of the spaceship parts taken separately can reach space, but all of the parts put together in the right combination can move astronauts into the orbit. So, let us consider the reader’s question more closely.

Brain and thinking: A producer or a receiver?

There are two types of complex systems in a human organism: those, which produce, and those, which receive. For example, liver produces cholesterol, thyroid gland produces thyroxin. Liver and thyroid gland are biological plants responsible for production of hormones. By contrast, an eye receives light, and transforms electromagnetic rays into nerve impulses that proceed into the brain through optical nerves. The question is to what of these two types does the brain belong in regard to subjective experiences? Is the brain a producer or a receiver of subjective phenomena?

Às far as it concerns sensations and perceptions, the answer is clear: Because our perceptual organs (eyes, ears, taste buds) are tightly teamed with the brain, they can be viewed as parts of the brain which are “brought forward” beyond the skull and towards external reality. In this regard the brain is a receiver of sensory stimulation. But how about thinking, personality and emotions? Today many researchers share the view that thinking is the brain’s function. For example, Francis Crick, one of the scientists who discovered the structure of DNK, writes “"You," your joys and your sorrows, your memories and your ambitions, your sense of personal identity and free will, are in fact no more than the behavior of a vast assembly of nerve cells and their associated molecules. As Lewis Carroll's Alice might have phrased: "You're nothing but a pack of neurons."” [16, p.3]. In this view, our brain produces thinking in a way similar to the way our liver produces cholesterol. But there is an alternative view on the relationships between thinking and the brain.

Neurophysiologist David Eagleman brings the following example. Imagine that you are a bushman in the Kalahari desert, who found a portable radio in a heap of sand. Trying various buttons, you suddenly hear human voices. You start to explore how these magical voices could possibly come out of the mysterious box. You open the box, manipulate various wires, and see some regularities to emerge. You discover that every time you break the contact between a green wire and a base, the voices disappear, but when you restore the contact, the voices are back. By manipulating other wires, you learn how to increase of decrease volume, create and remove noises, and so on. In the end, you come to the conclusion that the voices is a product of the enigmatic box. You can even create a theory of how the wires in the box produce the voices. At that, you don’t know anything about electromagnetism, about a radio station that is located thousands miles away and is presently broadcasting the news or Mozart’s symphony into the ether [17]. By analogy, we can consider the relationships between the brain and complex forms of subjective experiences, such as thinking, emotions and free will.

The brain does not produce thinking or emotions, but receives these subjective experiences in the way the radio receives the symphony. In reality the “symphony of subjective experiences” is being played and conducted by an entity that we cannot see, like we cannot see the radio waves. Without the brain we wouldn’t hear the “music of subjective experience”, but the brain is a receiving organ, whereas subjective experience is generated by an “orchestra” invisible for us. So, what is the brain: a producer or a receiver? If it is a producer, then subjective experience is a mirage, an illusion, which accompanies the work of the brain. And what if the brain is a receiver? When a certain part in the receiver fails and the receiver stops functioning, disappears the sound but not the music. The “music of thinking and emotions”, which is unique to each individual human being, keeps playing in the magical ether of subjective reality, and can be accessed by you again once the “radio of the brain” is fixed. Perhaps, this music is what we call “the soul”. According to Eagleman, this exotic hypothesis doesn’t contradict current knowledge of how the brain functions. Science can present no arguments that prohibit viewing the brain as the receiver of the “waves of subjective experience”.

And now suppose that the “symphony of subjective experience”, which the human brain receives, is produced by robots of The Matrix. Could the robots create the “symphony” so complex that a dreaming person would not be able to tell this symphony from real life?

Robots and the magic of subjective experience

Back in the 1950-th Canadian neurosurgeon Wilder Penfield was looking for the way of easing the fits of patients suffering from particularly hard cases of epilepsy. He used to open the patient’s temporal lobe of the brain and stimulate the cortex by low-voltage electric current [18]. Penfield registered a new phenomenon, which he named “double-consciousness”. The patients, who were under local anaesthesia and in full consciousness, reported having two parallel but separate currents of subjective experience. One of the currents was artificially created by the electric stimulation but seemed to the patient completely authentic, and the other was elicited by the stimuli coming from the current environment in the operating theatre. At that, the patients could unmistakably tell the artificially created subjective experience from the real one. For example, Penfield recorded an occasion in which "a young South African patient lying on the operating table exclaimed, when he realized what was happening, that it was astonishing to him to realize that he was laughing with his cousins on a farm in South Africa, while he was also fully conscious of being in the operating room in Montreal” [19]. Using results of his pioneering studies, Penfield developed the cortical “homunculus map”, which showed cortical localisation of motor and sensory zones connected to the actions of limbs and other organs. One might assume that robots of The Matrix, having an immeasurably more perfect “homunculus map” were indeed able to create in people the full-scale illusion of real life. But there is a problem.

It wasn’t difficult for Penfield to interpret his patients’ reports of their subjective experiences, because he knew from his own subjective experience what subjective experience was. But did the robots of The Matrix know this? As we know, a person’s subjective experience is strictly private, it cannot be directly shared with another person. One can share one’s subjective experience only through verbal or nonverbal (i.e., through gestures or facial expressions) reports. But however hard one tries to put in words even the simplest sensation, one is unable to transfer the most important part of it – the authentic quality of this experience, or the “qualia”. Trying to pass the authentic subjective experience through words is the same as trying to explain to a person who was born blind what blue colour is. So, for the robots of The Matrix to be puzzled by the problem of simulating human subjective experience, they had to have subjective experience of their own. But subjective experience is exactly what robots cannot have without overcoming still another barrier – the barrier that separates living entities from non-living things. In other words, to have subjective experience the robots have to become living creatures.

Indeed, today we don’t have any proof that a non-animate object, however complex, can have subjective experience. As for the mechanisms of the emergence of life, modern science knows about these mechanisms no more than it knows about how subjective experience emerged. According to Oparin-Haldane hypothesis, life on Earth originated in the oxygen-poor primeval atmosphere that resulted from volcanic activity. Under the impact of lightning and ultraviolet light organic molecules were synthesized; these molecules later changed into more complex molecules and finally into primitive colloid aggregates called “coacervates” [20]. In 1953 American scientists Stanley Miller and Harold Urey managed to synthesize in a glass flask simple amino acids, which are parts of protein molecules. They did this by firing continuous electrical sparks in the vapour containing water, methane, ammonia and hydrogen [21]. However, the emergence of life on Earth remains a mystery. Subsequent studies have shown that the primitive atmosphere on Earth contained a significant amount of oxygen, and this would impede the synthesis of amino-acids. American biochemist Jonathan Wells argues that even if scientists were able to create all of the chemical compounds a living cell contains, they would not be able to create the cell [22]. The emergence of life on Earth, and in the universe for that matter, remains a mystery.

So, subjective experience in its simplest forms, as the feeling of hunger or thirst, can only appear in a living entity. A stone, a river, a planet, a computer – any non-living thing, however complex it might be, is indifferent to its condition. The river doesn’t care whether it flows within its usual banks or flooded the surrounding fields. Only a living entity cares about anything. Only a person can ask the question of what caused the flood. Without living entities, without a human being there is no causality in the universe, and there is no the universe as we know it for that matter. The ability to have a wish or a drive, to experience joy or pain, to feel the need to explain things is impossible to find in non-living things. This miraculous ability – to be alive and to have subjective experience as a definitive feature of being alive – is a supernatural ability, which is brought into the inanimate universe from elsewhere.

But when and how did subjective reality managed to “put a claw” into indifferent non-animate matter, to “subdue” this apathetic reality? When and how for the first time the non-living atoms and molecules suddenly started moving not by the order of physical laws, but by the order of subjectibe reality that existed outside these laws? Perhaps, like subjective experience, life is a magical event and appeared in the universe not because, but against the laws of nature. If so, then it is understandable that all of the scientists’ efforts thus far to recreate, or even merely comprehend the origin of life failed. Perhaps, death is a magical event as well. May be, when a person dies, he or she does not turn into a mixture of atoms and molecules dissipated in the vastness of the universe, but passes up to another level of existence, leaving his or her dead body behind?

We don’t know the answers to these questions. Perhaps we’ll never know. But whatever the answers, one thing is clear: Even the most advanced robots can only imitate living systems, though the imitations can be impressive. It is possible that robots can be programmed to repair themselves and even to create copies of themselves thus imitating a self-replicating living cell. But they would only do this because they had been programmed. What the robots can’t have in principle is subjective experience. But this means that the robots will never be able to simulate subjective experience in a human being. Just like ourselves, people, would be unable to simulate full-scale subjective experience in an alien creature living in conditions totally different from those we have on Earth.

But let’s ask the question: Even if robots of The Matrix were able to simulate the full-scale subjective experience in humans, why would the robots need to do that? For what purpose would the robots spend so much effort on creating the Matrix when a person’s body can be used as a source of energy without the person being in a conscious state of the mind? Indeed, a person who is in a comatose state can be alive for years, even dozens of years [23]. There could only be one reason for the robots: For their purposes they needed real living people and not just the people’s animated bodies.

And this raises another question: Why do people need subjective experiences in order to be full-scale human beings?

Who moves a finger? Free will as a magical action

With all the diversity of my subjective phenomena (sensations, perceptions, thinking, emotions, actions) the most quintessential phenomenon is my sense that all of these phenomena belong to me. But what am “I”? Of course, my name, gender, age, family status, nationality, race – all of this is me, but these are only external objective properties. My “I” also includes my temperament, my personality, my desires, my thoughts and my feelings – but even those properties are not the most essential ones. The most essential property of my “I” is my feeling of “authorship” or “agency” – the feeling that my actions and my thoughts are not imposed on me by someone else, but originate inside myself. The notion “I am an agent of my actions” means that I am a centre of creation, in which something very real (e.g., emotions, thoughts and actions) are created from nothing. People know very well which of their thoughts and actions they created from inside of their own mind and which they borrowed from someone else. Only those thoughts and actions that we produce from inside of our mind is our real subjective “I”. But as we know, creation of something from nothing is a magical event, which violates the principle of physical causality and the law of energy conservation. In philosophy this magical property of the human mind is sometimes called “freedom of choice”, and sometimes “free will”.

“But behind every choice – a reader might say – stands a certain motive or a wish; for instance, when I choose cola instead of mineral water, I give up to my longing for a sweet drink. Certainly, one can’t act against his or her drives and wishes”. Yes one can, but we shouldn’t conflate an action of free choice with an action of free will. What is an action of free choice in a nutshell? An action of free choice is the ability to choose from a number of options. Usually this opportunity turns up when our options are arranged not in a hierarchical order (like a choice between life and death) but are spread in the horizontal plain and allow us enough time to think and discuss “pros” and “cons”. Having considered the options we finally make a decision: To buy this or another brand of shoes, to order fish or meat in a restaurant, to go for this or another tour, etc. Obviously, the action of free choice only seems to be free, but in reality is determined by our needs. Even so, determination of our actions by our needs is different from physical causal determination. Physical causality works through the known physical forces, such as gravity or electromagnetism. By contrast, determination by needs conforms to the magical causation through participation. I submit to my needs because I don’t distinguish my needs from myself; my need for food or love is a part of me. Yet my needs are only a part of my conscious “I” and determine my actions only partially. The other part of my conscious “I” is my voluntary decision to accept the demands of my needs, with the right to “veto” these demands. Under normal circumstances of everyday life the right of veto is rarely used. But when a situation arises in which my private needs get in conflict with my moral consciousness, the action of free choice may grow into the action of free will.

So, what is the action of free will? The action of free will is the ability to choose an option, which stands at the bottom of the hierarchically arranged “survival scale” wired in us by evolution, e.g., to starve to dearth for the cherished idea or to be honest when there is an opportunity to lie and get rich without undesirable side effects. An action of free will works against the survival instinct; this action doesn’t have a motive except maintaining the awareness of being fair and right. The action of free will breaks the psychological causal chain “interest – wish – action”. But is the action of free will indeed independent from the brain mechanisms, or is it only an illusion?

In search of an answer, American physiologist Libet made a series of interesting experiments. In one of these experiments participants were asked to press a button by a finger, or simply to move a finger when they wish. On top of that the participants were instructed to remember a position of a running dot on a tableau of the device measuring time in milliseconds at the moment “when they only start feeling the wish to make the move”. It turned out that the awareness of the wish to make an action and the action were separated by the time period of approximately 200 milliseconds. This result wasn’t surprising. What was surprising however was that an electric potential in the motor cortex of the participants’ brain (called “readiness potential”) appeared 300 milliseconds prior to the awareness of the intention to make the movement. It looked as though the brain made a decision to make the movement prior to the moment when our conscious “I” made this decision [24]. If the action of free will is a conscious decision to make the action, then the freedom appears to be an illusion. It looks as though an action of free will is simply a delayed awareness of the decision which had already been made by our brain [25]. But let’s look at this experiment more closely.

First of all, we notice that the participants’ actions in Libet’s experiment were not the actions of free will; the participant’s actions “to move a finger” were conditioned by their motive to follow the experimenter’s instruction. This motive in its turn was conditioned by the participants’ consent to participate in this experiment, which was also conditioned by some other motive (i.e., to get paid), etc. In other words, Libet studied not the action of free will, but the action of free choice between two options: To move or not to move the finger at this particular moment of time. Às everybody knows from his or her own experience, the action of free choice is not a momentary event but is stretched over time. Usually, when we have a range of options, we hesitate for some time before we make a decision, and when the decision if finally made, the awareness of this also comes gradually. For example, in the case of Libet’s participants, they may have hesitated for some time on “Do I want to move my finger “now” or don’t I?” Finally, when the decision comes “I certainly want to move my finger Now”, the participant registers the position of the dot, and after that moves the finger. So, the “readiness potential” of the participant’s brain may have occurred not prior to the participants conscious decision, but prior to the participants’ “final decision” to proceed with the action. In reality, the readiness potential may have accompanied the participants’ period of hesitations, when the participants were aware that they want to make the action but yet were unsure they wanted to do it “now”. The above considerations undermine the conclusion that Libet’s experiment testifies to the “brain’s priority” in the action of free choice. Even to a lesser extent Libet’s experiment can be used to understand the action of free will. The action of free will occurs in situations of an existential choice, such as a choice of compassion versus egotism, honesty versus dishonesty, moral integrity versus corruption. Studying the action of free will requires a special type of psychologycal experiments [26][27][28][29]. These experiments showed that for both adults and children making the action of free will is hard but possible.

But suppose that the action of free will, and the whole subjective experience for that matter, is indeed an illusion that reflects, with some delay, decisions made by the brain. Let’s investigate if the assumption “the brain is a decision maker” is free from a logical contradiction. Indeed, the question arises of why we need the illusion of agency if this illusion is a useless reflection of the “brain work”. Would it not be more economical if a person acted like a zombie? If evolution created the illusion of subjective experience, then subjective experience must be needed for something. One answer to this question is that humans need subjective experience for cognition and exploration of the world.

In the aforementioned book, David Eagleman writes ”Consciousness is a long-term planner, the CEO of the company, while most of the day-to-day operations are run by all those parts of her brain to which she has no access” [17, p.70]. Indeed, any training, for example, learning sport skills or driving skills, starts with a conscious effort that eventually becomes automatized and in no need of conscious control. By analogy, an action of free will is only needed in new and unexpected situations. For instance, when you are running down the stony hill you don’t think about where to put your foot; your foot automatically finds the right position. But now you run into a crack in the ground that you can’t jump over; immediately you consciousness is switched on and starts looking for the way around the obstacle.

And now let us apply the same argument not to skills but to our knowledge about the brain. In the present time the notion that people think “with their brains” is so common that it almost became a “habit”. Yet Greek philosopher Aristotle, who lived in the IV century BC, was of the opinion that thinking is the function of the heart, and the brain is a machine for cooling the blood [30]. Our knowledge about the brain, nervous system, neuronal chains, etc., which today looks unquestionable and even self-evident, at some earlier times was a subject for observation, analysis and conscious decision. In order to come to the conclusion that subjective experience is an illusion reflecting decisions made by the brain, people had to first use their subjective experience for creating theories about the brain. Clearly the argument that the brain is a decision maker runs into a circle: To discover that the brain is a decision maker which creates the illusion of subjective experience we need a support of our subjective experience.

Primacy of subjective experience in relation to concepts of physics and physiology is also obvious from the fact that subjective phenomena don’t cease to exist after their illusionary status is "explained" by science. For millennia people believed that the Sun revolves around the Earth, until Copernicus explained this illusion by the Earth’s rotation around its axis. Nevertheless, the illusion is still there: In the everyday life, we speak of sunrises and sunsets. Explaining sensations of taste and odour by chemistry and physiology did not change the fact that salt is salty and sugar is sweet. To the luck of specialists on physiology of colour vision, their theoretical explanations of how colour vision works doesn’t deprive them from the pleasure of enjoying the sensations of blue sky and green grass. By analogy, psychologists and philosophers who think that the sensation of the freedom of action is an illusion don’t cease to feel themselves responsible for their actions. In other words, subjective experience is independent from a rational explanation of the physical mechanisms that this experience accompanies. This means that subjective experience must have special properties, which distinguish it from physical reality. Let’s consider some of these properties.

Properties of subjective experience

The first and foremost property of subjective experience that distinguishes it from physical objects is that it makes us human. Any physical part of our body is a means to the end; we need our tongue for speaking, out heart for pumping the blood around our body, our legs for walking, etc. This does not apply to our subjective experience. It is not that a person needs subjective experience in order to accomplish something; a person is his or her subjective experience. Without a person’s subjective experience there is no the person. And what is there? There is what we call “a body”. But again, “a body” is a perceived phenomenon and a scientific concept; for both of these manifestations of “the body” we need subjective experience, either in the form of perception or in the form of thinking. From this it follows that subjective experience is not an illusion, it is a magical phenomenon, which cannot be explained by logic and science, but can give birth to logic and science. Subjective experience is a gift to human beings, and to all living creatures for that matter. A person can accept this gift with gratitude, or refuse to accept it. Having accepted the gift, we can develop or spoil it, but we cannot create subjective experience.

“And what about the fact that, by stimulating parts of human cortex, it is possible to create the illusionary stream of consciousness, like Penfield’s experiments showed?” – a reader might ask. Yes, artificial creation of subjective phenomena is possible but we shouldn’t forget that a person who experiences artificially created subjective phenomena can distinguish these phenomena from the parallel real ones, just like we distinguish dreams from reality or a fake from a real thing. But most important, artificially created subjective phenomena are as inaccessible for rational explanation as are the authentic subjective phenomena. We understand how the stimulation of the cortex by electric current is converted into the person’s experience of “laughing with his cousins on a farm in South Africa” no more than we understand how light waves of a certain frequency are converted in the sensation of green color. Phenomena such as free will, intentionality, voluntary action, creative insight, and other manifestations of subjective experience are inexplicable in terms of physical causality; they are cases of the magic “something from nothing” type. These phenomena are accompanied by certain processes in the brain, but they cannot be deduced from the brain processes in causal or logical ways.

“Fine – the reader goes on – but what about subconscious? Isn’t subconscious a part of subjective experience which is inaccessible to conscious awareness? And can we not deduce conscious subjective experiences from our subconscious?” Indeed, our subconscious memory, thinking and feelings are parts of our subjective experience. Psychologists and philosophes often use subconscious subjective experience in order to “scientifically” explain phenomena that are not yet fully explained by science. Dreams, hypnotic states, hallucinations, telepathy, telekinesis, Freudian “complexes”, Jungian “archetypes”, and many other unexplained phenomena are relegated to the department of subconscious. “Explaining” unexplained phenomena by placing them into the subconscious sounds scientifically plausible and gives these phenomena a legal status in the modern world. But what subconscious subjective experience really is nobody knows. Is subconscious subjective experience a source of conscious subjective experience, or is it a “wormhole” through which magical phenomena of subjective experience filter into our conscious mind? In any case, subconscious subjective experience is as mysterious and irreducible to the brain functions as is conscious subjective experience.

The second property that differs subjective experience from physical reality is that a person’s subjective experience is not accessible for direct observation from the outside. Indeed, physical objects we first register in perception: this is a stone, this is a river, this is a trace of an alpha particle. Having perceived the objects, we compare them one with the other, measure them and estimate in figures. Having done the measurements, we create scientific concepts of physical objects and proceed with establishing causal connections between these concepts via four known physical forces: gravitation, weak and strong nuclear, and electromagnetism. By contrast, we cannot see what the other person is seeing or thinking, we can only infer the other person’s subjective experiences from the person’s behavior. We know that every person has to believe in something, e.g., in god, science, or materialism. The person’s beliefs reveal themselves through the person’s verbal or nonverbal behaviors. By carefully observing and analysing these behaviors, we can study the person’s beliefs and then use these beliefes as “carrot and stick” to influence the person. This means that it is possible to study subjective experience by objective methods; it is also possible to influence people’s subjective experiences, but not through tampering with their brains. A person’s subjective experience can be influenced via manipulation with his or her beliefs, desires or perceptions. Religious leaders, politicians, psychotherapists, advertising specialists, teachers and artists routinely influence people’s subjective experiences without intervening into their brains. As far as it concerns interventions in the brain functioning, these interventions affect not subjective experiences as such, but the process of the brain’s functioning. A parallel can be drawn between the work of a neurosurgeon and the work of a radio engineer who is fixing the radio. Without normal functioning of its mechanisms the radio won’t play music. But the engineer can’t make the fully fixed radio play the music if there is no music in the ether for the radio to play.

The third property that makes subjective experience different from objects of physical reality is that, whereas a person’s subjective experience is inaccessible for observation from the outside, it can be accessed by the person “from inside”. The fact that a person has his or her subjective experience inside his or her mind opens a unique opportunity to study this experience in a different way from the way we study physical objects. Indeed, as I mentioned above, the first step in studying physical objects is to register them in observation through perception. By contrast, our subjective experience can’t be registered in our perception. We see a tree, but not our perception of the tree. We cannot perceive our thought, our voluntary decision, or our feeling of love, but we can register these subjective experiences through self reflection and give them names. In the realm of subjective experience a perceptual image of an object, a thought about the object and the object’s name are connected via associative participation, or, to use a term from quantum physics, “entangled”. For instance, the image, the thought and the name of a rose flower are not the same, but it is hard to separate one from the others; where there is the image, there is always the thought and the name, and vice versa. More than that, one kind of thoughts and images can trigger another kind of thoughts and images by the same principle of associative participation. For example, while walking in a park we suddenly catch the aroma of a cherry tree blossom, and this olfactory sensation may trigger thoughts and memories about the events in our childhood, and travelling further along the associative chain bring us to the most unexpected thoughts and images. The study of subjective experience “from inside” is most skillfully conducted not by scientists, but by writers; Marsel Proust’s masterpiece “In Search of Lost Time” is one of the most known examples.

Finally, one can distinguish subjective experiences from physical objects by the way people interact with each other. Physical objects interact via aforementioned four fundamental physical forces and the interaction conforms to the laws of physical causality and energy conservation. By contrast, interaction between subjective experiences of two people conforms to the law of associative participation. For instance, by saying our thoughts out loud we address a partner and tune the partner’s subjective experience to similar thoughts and images. Suppose, a husband, when walking with his wife on the shore of the Red Sea, says “ A beautiful sunset, isn’t it?”, and the wife answers “Do you remember our trip to San Diego? There were sunsets like that there as well”. Likewise, when a matematician writes a formula e=mc2 on a blackboard, his or her colleagues sitting in the lecture theatre understand that the talk is about mass-energy equivalence and not about the theory of evolution by natural selection. Clearly, the exchange by communicative messages cannot be reduced to the exchange by light and sound waves that the speakers produce; the content of the communication follows the magical law of associative participation.

Conclusion: The impossibility of the Matrix

I started this paper by asking the questions: Can robots simulate human subjective reality? Is it really the case that a person who is plunged into the world of simulated subjective experience is unable to tell this world from the real world? What is better: Enjoying life in the illusory world or struggling for happiness in the true world full of hardships and austerities?

In the second half of the XX-th century, at the peak of computer euphoria, there appeared the idea that it is possible to simulate all of the universe by converting every atom in a series of ones and zeros. Because a person is a part of the universe, in this cosmic program every individual would occupy his or her humble place. “There is no way – American physicist Frank Tipler writes - for the people inside this simulated universe to tell that they are merely simulated, that they are only a sequence of numbers being tossed around inside a computer and are not in fact real" [31, p.181]

How do we know that we are real and not a simulation in some gigantic computer – Tipler asks? And answers: We don’t know, but it doesn’t matter. Likewise, it doesn’t matter whether the universe is real or merely a simulation. All that matters is whether it is possible to create an abstract program that is capable of simulating the whole universe. There is a concept “data compression” in algorithmic information theory [32]. According to this concept, creating a computer program makes sense only if the program is shorter than a simple description in digital code of the process that this program codes. If making the program shorter than a simple description is not possible, then it is simpler and faster to deal with the simple description than with the program. Accordingly, Tipler’s hypothesis might be reworded as follows: Can an algorithmic program code the whole universe, and if it can, then would this program be shorter than the description of the universe in digital code?

These questions bring us back to the problem of the relationships between knowledge and subjective experience. What was in the beginning? If we accept the view that knowledge is a model of perceived things and their interactions, then knowledge must exist in two realties at once: In the visible reality of perceived phenomena and in the invisible reality represented by signs, numbers and scientific theories. But both of the above realities (perceptual phenomena and abstract meanings) by definition imply the existence of subjective experience. From this we can infer that subjective experience precedes knowledge. First, we have the subjective experience of an object (e.g., a tree) in the form of sensations and perceptions, next we become aware of our knowledge about the object (e.g., that this is a plant). It becomes clear that concepts such as “physical laws” and “computer programs” can only be developed by a person who has subjective experience and is aware of his or her own existence.

Amazingly, as computer programs got more complicated and compressed, the fact that these programs are products of human subjective experience started to slip away from scientists’ view. There appeared an increasing number of attempts to present subjective experience as a computer simulation. Some scientists began to ignore the fact that such attempts involve a vicious circle. There is also a mathematical proof of the impossibility of this kind of “self simulation”. Simply put, according to Gödel's theorem of incompleteness “Anything you can draw a circle around cannot explain itself without referring to something outside the circle – something you have to assume but cannot prove.” [33]. For instance, it is impossible to decide whether the statement “this sentence is false” is true or false while remaining within the rules of formal logic at the same time. The same applies to the universe. Gödel's theorem proves that it is impossible to simulate the universe from inside the universe. But is it possible to do from the outside?

The answer depends on who the programmer is. If the programmer is a living entity with subjective experience infinitely more powerful than human subjective experience, then it could be possible. The Bible is written exactly about such a “programmer”. But computers cannot simulate human subjective experience. Being non-living things, they don’t know what subjective reality is. In his new book “The Future of the Mind” Michio Kaku defines consciousness “from a physicist’s point of view” as “the process of creating a model of the world using multiple feedback loops in various parameters [such as temperature, space and time] in order to accomplish a goal [such as finding shelter, mates or food]” [34]. Letting his fantasy loose, Kaku writes about the possibility in the future to separate human consciousness from a human body and feed it into a computer.

But before speaking about the future, it is useful to look back at the past. And looking at the past we see that the first “model” of our visible world was the invisible world of spirits [35]. Computers can’t possibly invent the world of spirits because, being non-living things, they are immortal. Only living mortals who dream of the afterlife could create the world of spirits and benefit from their creation. It shows that even the consciousness “from the physicist’s point of view” could only emerge in living entities. But this puts a fundamental limitation to the potential capacities of computer technologies. With all the practical benefits of such technologies, one can’t expect that people with the help of computers would ever be able to simulate a full-scale human subjective realty. Even less one can expect that computers on their own would manage to do this.

Having said this, I didn’t mean to diminish achievements of modern computer technologies in the domain of brain-computer interface. These achievements are impressive. A functional magnetic resonance imaging (fMRi) can approximate which areas of the brain are involved in solving certain tasks. By scanning electric potentials of the brain it is possible to teach a disabled person to “mentally” control a wheelchair or a computer cursor. It is possible that in the future advanced methods of analysis of brain’s electric potentials might establish correlations between patterns of these electric potentials and subjective images we see in our dreams. But all these achievements won’t change the fact that subjective experience and electrical signals of the brain exist in different realms. There will ever be a “neutral strip” of the unknown between subjective experience and computer simulated virtual reality. Rewording a known paradox, one might say that if human subjective experience were so simple that it could be simulated by computers, humans would not be so clever to be able to create these computers.

To conclude, current studies on magical thinking suggest that subjective experience is a magical phenomenon of “something from nothing” type. In the realm of subjective experience, the magical law of participation holds sway. Subjective experience was a basis from which modern science and formal logical thinking grew; an implication of this fact is that subjective experience cannot be explained in terms of physical causality or formal logic. Subjective experiences can be studied, both objectively and “from the inside”, but studying subjective experiences require special methods different from the methods used in physics and physiology. Robots of The Matrix, originally created by humans on the basis of formal logic, with all their enormous computational power could never be able to simulate human subjective experience. The barrier that prevents robots from doing this is the necessity to be alive. Being non-living things, robots are hopelessly devoid of the magical gift of subjective experience.

What do we need to know this for? For not wasting our time and effort on chasing unattainable goals. Also, for not confusing studies of human subjective experiences with studies of inanimate objects in natural sciences. Or perhaps, for a better realisation of how unique and irreplicable life and human beings are in this universe. Finally, for the understanding that the explanatory power of physical sciences, though enormous, still has its limits.

Às for the question of whether it is better to enjoy life in the illusory world of simulated reality (if someone would ever be able to simulate a full-scale human subjective experience) or live a difficult life in the true world, I agree with The Matrix creators: This is a matter of personal preference.

References
1. http://www.poetryloverspage.com/yevgeny/tyutchev/silentium.html
2. https://en.wikipedia.org/wiki/The_Matrix
3. Subbotsky E. (2004). Magical thinking: Reality or illusion? The Psychologist, 17, 6, 336-339.
4. Clarke, A.C. (1962). Hazards of prophecy: The failure of imagination. In Arthur C. Clarke «Profiles of the future: An enquiry into the limits of the possible». A collection of works. New York: Harper & Row.
5. Kaku, M. (2008) Physics of the impossible. A Scientific exploration into the world of phasers, force fields, teleportation, and time travel. New York: Doubleday.
6. https://en.wikipedia.org/wiki/Fundamental_interaction
7. https://en.wikipedia.org/wiki/Neuroprosthetics
8. Lévy-Bruhl, Lucien. The Primitive Mentality (English translation). Paris: Alcan, 1923.
9. https://en.wikipedia.org/wiki/Objectivity_(science)
10. Subbotsky, E. (2016). Miracles in law. Magical underpinning of physical universe. SENTENTIA. European Journal of Humanities and Social Sciences (accepted)
11. http://en.wikipedia.org/wiki/Huntington%27s_disease
12. Farah, M. (2005). Neuroethics: The practical and the philosophical. Trends in Cognitive Science, 9, 34-40.
13. http://www.eurekalert.org/pub_releases/2006-10/jaaj-eoc092806.php
14. Selten, J. P., Cantor-Craae, E., & Kahn, R.S. (2007). Migration and schizophrenia. Current Opinion in Psychiatry, 20, 111-15.
15. Caspi, A., Sugden, K., Moffitt, et. al. (2003). Influence of life stress on depression: Moderation by a polimorphism in the 5-HTT gene. Science, 301, 386.
16. Crick, F. (1994). The astonishing hypothesis: The scientific search for the soul. London: Simon & Schuster.
17. Eagleman, D. (2012). Incognito. Edinburgh: Canongate Books.
18. http://en.wikipedia.org/wiki/Wilder_Penfield
19. Penfield, W. (1975). The mystery of the mind. Princeton, New Jersey: Princeton University Press.
20. https://en.wikibooks.org/wiki/Structural_Biochemistry/The_Oparin-Haldane_Hypothesis
21. https://en.wikipedia.org/wiki/Miller–Urey_experiment
22. Uells, D. (2012), Anti-Darvin. Moskva: Al'pina Biznes Buks.
23. http://en.wikipedia.org/wiki/Coma#cite_note-18
24. Libet, B. (1999). Do we have free will? Journal of Consciousness Studies, 6, 47-57.
25. Wegner, D. (2002). The illusion of conscious will. Cambridge, MA: MIT Press.
26. https://en.wikipedia.org/wiki/Stanford_prison_experiment
27. http://www.lancaster.ac.uk/staff/subbotsk/Shaping%20moral%20action.pdf
28. https://en.wikipedia.org/wiki/Milgram_experiment
29. http://www1.appstate.edu/~kms/classes/psy2664/moral.pdf
30. http://en.wikipedia.org/wiki/History_of_neuroscience
31. Tipler, F. J. (1989). The Omega point as Eschaton: Answers to Pannenberg's questions for scientists. http://www.geocities.ws/theophysics/pdf/tipler-omega-point-as-eschaton.pdf
32. https://en.wikipedia.org/wiki/Data_compression
33. https://www.perrymarshall.com/articles/religion/godels-incompleteness-theorem/
34. Kaku, M. (2014). The future of the mind: The scientific quest to understand, enhance, and empower the mind. New York: Doubleday Books.
35. Subbotsky, E. (2016). Art as a window into the supernatural. SENTENTIA. European Journal of Humanities and Social Sciences (accepted).