Theories & Models of Emotion
Emotional Descriptions, Theories, & Models
Many theories have been developed describing the coursing and emergence of emotions. James Hillman (1962) described the functions of emotions as having psychological transformative aspects of general energy, which ascribe importance, value, significance, and meaning to a given attribute. Nico Frijda (2008) stated that emotions were defined by appraisals and motivations (preceding their expression). They were spontaneously “triggered by events as appraised” (p. 71) and were shaped by regulatory processes (Frijda, 1986, p. 6).
Richard Lazarus (1991) said emotions stemmed from automatic and conscious evaluative judgments that centered on either beneficial or harmful experiences in one’s relationships with others in one’s environment.
Oatley & Johnson (1987) described emotions as being pivotal in the mental and social lives of humans and as having both integrative and important roles (and functions) in subjective experience. Accordingly, emotions mediated changes in physiological reactions and in the selecting of intentional actions and responses to interpersonal interactions.
Edmund T. Rolls (2005) said emotions were states that were initially elicited from rewards and punishers. A reward was something or an aspect of a relationship that an animal or person was willing to work in order to acquire potential sense of well-being (ref: Lazarus, 2006); a punisher was something or an aspect of a relationship or a negative outcome from which one worked hard to avoid or to escape. Positive emotions were therefore elicited when reward was received; negative emotions (like anger, frustration, and sadness) arose when reward was either denied or terminated or replaced by something that was perceived as aversive or painful. (The roles for appraisals as important, intermediary processes between reward/punishment status and emotion were sorely overlooked in E.T. Roll’s conceptualization of emotion.)
Jerome Kagan (2007) noted that emotional states served to support sustained attention and avoidance learning. Emotions, as such, supported the expression of drives for goal persistence. Kagan also saw a role for certain emotions in triggering later unethical behaviors and long term memory retrieval.
Mascolo, Harkins, & Harakal (2000) assessed emotional experience as having hierarchal and multiple component processes underlying their expression. These component processes were identified as the following: appraisals (like Nico Frijda (2008) noted above), which were generated by motivational and sensory-perceptual aspects and later representations, physiological states, which were identified as central (CNS) and autonomic nervous system (ANS) components along with other bodily sensations, and action/behavioral systems, which involved both involuntary (e.g. facial expressions) and voluntary, deliberative behaviors. All these constructs mutually stimulated as well as regulated one another to modulate responses of the other. Mascolo, Harkins, & Harakal’s component process system was context-sensitive, i.e. processes were dynamic and demonstrated an intra and interdependent, inseparable interplay with one another and were also interactive with and reactive to the state of conditions in the environment. For instance, when emotionally aroused by an aggressive reaction, coping appraisals modulate CNS, ANS, and bodily arousal to inhibit action potential for behavioral aggression. As a result of coping processes, bodily arousal (e.g. heart rate) is reduced, and social interaction can proceed with little change or incident. Perceptual input can elicit primary appraisals, which can then produce CNS, ANS, and bodily arousal, emotional responses of sadness, and later facial grimacing of sadness and crying as well as behavioral withdrawal.
In Mascolo, Harkins, & Harakal’s (2000) conceptualization, emotional experiences were viewed as self-organizing, pattern-forming, and attractor-seeking (i.e. tending toward organizing input by feature similarity or associative processes). Emotion’s self-organizing role was judged as important and critical for subsequent memory recognition and retrieval processes. Mascolo, Harkins, & Harakal provided a conceptualization, which recognized the many different components in the human social experience, their interdependence, interaction, and regulation of one another.
Ahn & Picard (2006) recognized the significant role that emotions, intrinsic and extrinsic rewards, and motivational roles played in learning and in decision-making. They identified extrinsic rewards as stimulus salience and intrinsic rewards as likening them to the "wanting circuitry.”
Their affect model, like Malfaz & Salich's model below, a′ θ = Pr (v′ θ | c ', c, aθ, d), was also influenced by the valuation of reward, R0 , and the outcome response of not receiving it, i.e. the feeling bad state with breached expectation, (v′θ = 1 ) and the outcome response of receiving it, i.e. the feeling good state with obtaining it, (v′θ = 2 ). Accordingly, affective probabilities were represented by a = {aθ} = (a0, a1, ∈ a | Α |) .
However, the remainder of their model, evaluated decision-making.
Malfaz & Salichs (2004) sought to develop an emotional and reactive computer model for the development of an autonomous robot. This model of emotion was embedded into their Automatic-Deliberative Architecture, which had tow different levels, the preliminary and deliberative levels. The preliminary level, or automatic level, was evidenced during low-level modules and involved sensors and actuators. The deliberative level was required during reasoning, decision-making, and the execution of orders. Both architectural levels were stored in short and long-term memory modules and were interactive and propelled by drives. Drives were implicitly evidenced in the nature and the degree and intensity of an organism’s energy, affect, health, perspective or agenda, and entertainment.
An emotion, like happiness, was generated when well-being was either sustained or newly-generated. The emotion of sadness was elicited when something “bad” had happened to the organism and well-being was either threatened, reduced or eliminated. Anger was produced when well-being was reduced in response to an environmental event’s insult and in response to a dramatic loss of well-being.
Malfaz & Salichs (2006a) referenced the need for their motivational model, which sought to monitor the internal state of an agent’s equilibrium (i.e. of positive well-being). In this model, when levels of well-being differed from an ideal model, an error signal occurred and was embodied in a drive. Environmental stimuli served to elicit and release drive-induced behavior. The inner derived drive strength interacted with environmental stimulus strength. If the drive demonstrated a decreasing trend, then environmental (arousing) stimuli needed to elicit increasing (reward) value to trigger the agent’s motivational behavior and action potential. In situations involving a high drive, a mildly (arousing and rewarding) stimulus is needed to stimulate motivational behavior. (I think a more delineated interpretation is that the generated error signal reflects the occurrence of a discrepancy between what was expected to happen in acquiring well-being and what actually happened (loss of well-being and anxiety. In response to the motivation (or desire) to return to a state of equilibrium and positive well-being, a drive is generated to propel the organism’s energy to work toward developing, selecting, and implementing strategies that could return the organism to a state of equilibrium and well-being. It is therefore motivation that propels the generation of the drive and the desire for well-being that propels motivation).
Per Malfaz & Salichs (2006a) the following algorithm describes the relationship between both these components, Mi = Di + wi , , where the drive is impacted by the intensity of an external stimulus's value or imputed weight (wi). The ideal value for all drives is 0. A value of zero is applied when a drive decreases its weighted value. With stimulus presentation, the value of a wi is 1.
Heightened motivational intensity can be elicited in two ways, one, intrinsically by a high drive, or two, extrinsically by an arousing stimulus. Therefore, if Di ≤ Ld then Mi = 0; if D i > Ld then Mi = 1 is applied (Malfaz & Salichs, 2006a). If a drive level has decreased, then the inherent motivation valuation is zero. If a drive level has increased, then the motivation valuation is one. Decreasing drive is therefore suggestive of decreasing motivation for well-being; increasing drive is suggestive of increasing motivation for well-being.
According to Malfaz & Salichs (2006a) well-being is a function of both values of drives (Di ) and personality (ai ) factors. (I think that motivation is a function of the state of well-being, i.e. states of satiation and equilibrium will diminish the motivation for well-being; state deficits in equilibrium will increase efforts and motivation for acquiring well-being. The nature of well-being is a function of personality, as personality features determine what in the environment or internal self is capable of instilling a sense of positive well-being. Drives are a function of motivation, i.e. when motivation is reduced (in response to equilibrium) so is the need to generate drives and to develop behavioral strategies to achieve well-being. When motivation for well-being is heightened (in response to disequilibrium) so is the need to generate drives and to develop behavioral strategies to achieve well-being.)
According to Malfaz & Salichs (2006a), WBideal is an ideal value and is valued at 100. The value of well-being and its ideal are evaluated at each step. The variation in well-being is calculated by the current value of well-being less the value of well-being from the prior step. Therefore: Wb = Wbideal - Σiai • Di. Where well-being is well-being ideal less cumulative summations of personality factors and the drive state at time interval (i).
Behavioral selection emanates from all known actions that are capable of satisfying motivational goals. Δ Wb ≥ Lh → Happiness and Δ WB ≤ Ls → Sadness, where Lh > 0 and Ls < 0 represent minimal variations of well-being. When an agent's well-being increases to a certain level, the emotion of happiness is elicited. On the other hand, sadness is elicited when well-being decreases to a certain level (Mafaz & Salichs, 2006b).
These concepts can be embodied in the following. Happiness is experienced when positive well-being reaches a certain level, i.e. as the state approaches the ideal state of well-being, lims →maxWBideal = Happiness.
On the other hand, sadness is experienced when well-being is reduced to a certain threshold that is well below the well-being ideal, lims → minWBideal = Sadness.
Both emotions are conveyed similarly in a "reinforcement learning algorithm" format. Motivational goals therefore optimize the well-being of the agent when well-being rises above (exceeding equilibrium) or drops below (disequilibrium) a certain level .
Salichs & Malfz (2006) related their motivational model to concepts from the Q-learning model. According to this model, Q-values of state and action pairs (Q ( s,a )) interact optimally, where an agent learns iteratively, on a trial-and-error basis and where expected discounted reinforcement trials shape the future execution of an agent's actions (a) and states (s). The agent uses experience to enhance its policy development by integrating current knowledge acquired during a task with that which had been previously learned (Yang & Gu, 2005). However, unlike Q-learning's reliance on Markov Decision Processes (MDP), this motivational model assumed that the agent's environment is dynamic (not stationary) and involves other adaptive external agents. In a muli-agent system as such, joint actions (between two agents) elicit the next agent's state and action. The state of an agent (or robot) is a culmination of all inner states at any point in time (Sinner) and different state objects (Sobj) that the agent has encountered (including external agents). Therefore, S = Sinner × Sobj1 × Sobj2· · · depicts that the state of an agent in relation to each object experienced is represented accordingly, s ∈ S inner × Sobj1.
On the other hand, the emotion of fear is generated when something bad is about to happen that has the capacity for decreasing well-being. According to Salichs & Malfaz (2006) when one is fearful (or more accurately, acutely anxious) one anticipates the worst case scenario. When one is afraid (or more accurately, anxious) the worst case scenario is embodied in the following algorithm, Qobjworst ( s,a ), which is generated after each action production (as a response to a fearful object or event). The fear sequence update would proceed as follows. Qobjworst ( s,a ) = min (Qobjworst ( s,a ), r + γ maxa∈Aobj ( Qobj ( s',a )))
Where Aobj is the set of all actions to the fearful object (a is the optimal action for facilitating return to well-being), s' is the new state that occurs in response to having taken (a) action, r is reinforcement or reward (valuation that is capable of eliciting well-being), and γ is a discount factor. Fear mobilizes the agent's selection of actions that can reduce fear expression or Qobjfear ( s,a), as opposed to a stimulus or object that maximizes the emotion of fear and fear expression before action is taken, Qobjworst.
Therefore, Qobjfear ( s,a ) = β Qobj ( s,a ) + (1 - β )Qobjworst ( s,a ), the fear reducing strategy has been developed from the worst case scenarios during problem solving. The fear-reducing action is embodied accordingly af = arg max Qobjfear ( s,a ), as this action has been selected due to its maximal effect at reducing fear.
Markov Decision Processes might approach algorithm development a bit differently, where T is a transition, A is a finite discrete set of available actions, π is the policy driving the selected action that the agent can take, and S is the state at any point in time. R is a real number and reflective of the agent's response.
T = S × A → R.
Likewise, the fear reducing strategy (Qπfear) is typically developed from worst case scenarios during problem solving. This can also be described as follows.
Qπfear ( s,a ) = ∑ Qπfeart-1( s,a ) + γ ∑ rt+1 − Qπtfear ( s,a ).
The optimal strategy for reducing fear and increasing well-being is the sum of all prior and future discounted return (or anticipated well-being) less the present fear related state.
According to Salichs & Malfaz (2006) the emotion of fear is also expressed as a drive, Dfear. The minimum acceptable value of the worst result when the agent fails to do anything" can be reflected accordingly, Qobjworst (s, Nothing) < Lfear . This can also be noted as follows, lims → Qobjworst, reflecting the lowest tolerable threshold or state for fear expression.
And any tolerable level or higher for fear is the safe level, Qobjworst (s, Nothing) > Lsafe . This can also be reflected as follows, lims → Qobjfear.Some agents have a higher tolerance for experiencing fear; when reaching a limit or certain threshold, the emotion of fear is expressed.
References
Ahn, H. & Picard, R. W. (2006, April). Affective-cognitive learning and decision making: The role of emotions. The 18th European Meeting on Cybernetics and Systems Research (EMCSR 2006). Vienna, Austria.
Frijda, N.H. (2008) The psychologist’s point of view. In: M. Lewis, J.M. Haviland-Jones, L. Feldman-Barrett (Eds.) Handbook of emotion-3rd Edition. New York: Guilford.
Frijda, N.H. (1986). The emotions: Studies in emotion and social interactions. New York: Cambridge University Press.
Hillman, J. (1962). Emotion. Illinois: Northwestern University Press.
Hudlicka, E. (2004). Two sides of appraisal: implementing appraisal and its consequences within a cognitive architecture. Architectures for modeling emotion: cross disciplinary foundations. Paper from the 2004 AAAI Spring Symposium.
Kagan, J. (2007). What is emotion? Binghamton, New York: Vail-Ballou Press.
Lazarus, R.S. (1991). Cognition and motivation in emotion. American Psychologist, 46(4), 352-367.
Lazarus, R.S. (2006). Stress and emotion: A New Synthesis-2nd Edition. New York: Springer Publishing.
Malfaz, M. & Salichs, M.A. (2004, July). A new architecture for autonomous robots based on emotions. The Fifth IFAC Symposium on Intelligent Autonomous Vehicles. Lisbon, Portugal.
Malfaz, M. & Salichs, M.A. (May, 2006a). Emotion-based learning of intrinsically motivated autonomous agents living in a social world. International Conference on Development and Learning. ICDL5. Bloomington, Indiana, USA.
Malfaz, M. & Salichs, M.A. (August, 2006b). Using emotions for behaviour-selection learning. The 17th European Conference on Artificial Intelligence. ECAI, Riva del Garda, Italy.
Mascolo, M.F., Harkins, D., & Harakal, T. (2000). The dynamic construction of emotion: varieties in anger. In: M. D. Lewis & I. Granic (Eds.) Emotion, development, and self-organization. United Kingdom: Cambridge University Press.
Oatley, K. & Johnson-Laird, P.N. (1987). Towards a cognitive theory of emotions. Cognition & Emotion, 1, 29-50.
Rolls, E. T. (2005). Emotion explained. New York: Oxford University Press.
Salichs, M.A. & Malfaz, M. (April, 2006). Using emotions on autonomous agents: the role of happiness, sadness, and fear. Integrative approaches to machine consciousness, part of AISB 2006: Adaptation in Artificial and Biological Systems. Bristol, England.
Yang, E. & Gu, D. (April, 2005). A survey on multiagent reinforcement learning towards multi-robot systems. Proceedings of IEEE Symposium on Computational Intelligence and Games, Colchester, Essex.