1. A first appraisal is conceptualized for assessing the nature of initial sensory, perceptual stimuli.  As a person focuses on sensory stimuli, sensory input, a person perceives (listens to, observes, feels, tastes, smells) the relationships and interplay between sensory variables, the meaningfulness of these relationships to the self, the difficulty or ease at interpreting a stimulus’s meaningfulness, the rate of change between stimulus variables, the introduction of new stimulus variables, how stimulus variables dynamically interact with one another, how certain stimulus variables dominate and how others recede, the location of all or certain stimulus variables, etc.  This is embodied here in the perceptual (sensory) appraisal, a^p, symbol. The perceptual, sensory appraisal process is receptive, i.e. it is a response to incoming perceptual information.

2.  A second appraisal is developed for assessing and conceptualizing the potential for future returns (rewards) and future sense of well-being that is associated with the stimulus.  When a person focuses and conceptualizes on the benefit for taking a potential future action, the person is engaging appraisal processes for answering a question like, “What is in it for me?”  If there is a perceived benefit for initiating a future action, whether it be extrinsically delivered (as in task-related reward) or intrinsically derived (as in experiencing a sense of accomplishment in performing or finishing a task), the person will be more apt to perform the required activity to work toward and earn that reward. Such reward appraisal processing is largely implicit, is characterized by its momentary, fleeting nature, and is often forgotten minutes after experience.  The second appraisal corresponds to Sutton & Barto’s r_t+1 and is embodied here in the reward/return appraisal, a^r_t+1 , symbol. The reward, well-being seeking appraisal process is productive, i.e. it is a thought process that conceptualizes the motivational state of the organism. The a^r_t+1 symbol often drives the motivation underlying goal development, drives one's sense of anticipation, and underlies later expressed goal-directed behavior.

3. A third appraisal is developed for assessing different alternative strategies that could compose the strategic action plan in order to achieve certain task goals and potential reward outcome.  Strategies are developed to support the mapping out of future action that will satisfy task requirements.  Problem-solving as such evaluates what would happen (in the environment task or interaction) if such and such move or action was taken.  Would this action be effective?  What modifications would need to be made in reaction to the response-outcome? What would the outcome be then?  The third appraisal relates to Sutton & Barto’s policy, π, and is embodied here in the appraisal policy and a^π _t+1 symbol. The appraisal representing strategic action planning process is productive, i.e. it is a thought process that conceptualizes, and with compilation and selection, prepares the organism for later behavioral selection and action. Productive-type of appraisals typically accompany some type of planning for obtaining a rewarding stimulus and the resulting sense of well-being that is associated with the rewarding stimulus, whether that planning is for experienced relief from discomfort, for comfort, soothing, recognition, validation, relief from threat, achieving correctness and accuracy in alignment with an internal or external model, affection, self-determination, achievement, well-being, etc. The a^π_t+1 symbol often drives goal and strategy development as well as one's later expressed goal directed behavior.

Appraisal items of 1-3, a^p , a^r_t+1, and a^π_t+1 are embodied in Sutton & Barto’s policy state, V^π(s) . They are all appraisals (A) that are immediately and spontaneously linked with cognitive and appraisal responses to perceptual stimuli in response to the task at hand or interpersonal interactions. As a result these types of appraisals are reflective of both initial sensory receptive and cognitively productive appraisal processes. Their interaction can be described accordingly.

Where:

a^p , a^r_t+1,  a^π_t+1 ∈   A and

A : a^p →  a^π_t+1 and   A : a^r_t+1 →  a^π_t+1

Thus: A : a^p + a^r_t+1 →  a^π_t+1

Where both perceptual and reward motivation appraisals are mapped onto the strategy developing appraisal thus describing their relationship with one another.

Temporal Difference-Mediator of Primary-Secondary Appraisal Processes

The selection of an appraisal for strategy-action planning is often followed by an associated selected behavioral response to meet environmental or task challenges.  Upon and after behavioral implementation, the environment or task responds.  This response is the environmental outcome.  The outcome includes a change in the state of environmental stimuli and, when available, provides reward delivery, which not only provides incentive for response persistence and continuation, but also feedback on the correctness and accuracy of the prior selected behavioral strategy.

A person’s or organism’s responses to an environmental outcome are many.  Responses occur in rapid succession; oftentimes it is difficult to discern the temporal order and progression of responses, such as physiological, cognitive, and emotional components and expressions.  The person/organism typically has appraisal responses to an outcome and this is elaborated below.

4. A fourth appraisal, the reward confirming appraisal,  is developed in response to the environment’s response to the correctness of a person’s prior implemented action (cited in item 3); it is manifested in return or reward feedback (or lack of feedback).  Sometimes the reward outcome is explicit and evidenced in task-related generated external reward or return; at other times the reward outcome (or lack of it) is intrinsic, i.e. it is solely perceived by the participant, and evidenced in a match to expectation and correctness or mismatch to expectation and error.  In the perspective of the producing participant, the return provides feedback on the accuracy of the previously selected action strategy that had been implemented and also provides an incentive for continuing.  The reward confirming appraisal is symbolized by a^r_t. The appraisal for the reward-confirming appraisal process is receptive (i.e. responsive to environmental reward input) and describes what is occurring.  It is not a productive thought process as is the anticipated future appraisal, a^r_t+1. The a^r_t symbol also triggers physiological arousal, whether that arousal be positive or negative in nature.

5. Another appraisal is elicited that answers the following questions. Did the environment’s reward or return response match to that which was expected and mapped out in the action strategy, a^π _t+1?  How was it different?  How was it similar? If there was a difference, is this difference acceptable?  As a result of the delivered or omitted reward, what are the implications for the selected action strategy?  Should it be changed?  Should it be abandoned?  This type of problem-solving activity is a continuation of that noted in item 3 and as a result is embodied in the a^π _t symbol. The a^π _t symbol also mobilizes further cascades of expressed reactions and triggers physiological arousal, whether that arousal be positive or negative in nature.

Appraisal items of 1-5, a^p , a^r_t+1 , a^π_t+1 , a^π _t, and a^r_t  are embodied in Sutton & Barto’s policy state, V^π(s).  They share two aspects of an individual’s experience, namely they are receptive and productive appraisals that are immediately and spontaneously linked with responses to perceptual stimuli with the task at hand or interpersonal interactions and are involved in problem-solving responses that are critical for underlying later action or behavior, respectively. Therefore:

All Appraisal Processes = A

And, a^p , a^r_t+1, a^π _t+1, a^r, a^π _t ∈  A

And a^p, a^r_t+1, a^π _t+1, a^r _t, a^π_t ⋅⋅⋅ aⁿ → A

A is composed of a full sequence of both receptive and productive appraisal processes noted above. The parts (aⁿ ) are elements of and become and transition to the whole (A), which accounts for all cognitive activity relating to the task or event.

The latter two appraisals, appraisals 4 and 5, are elicited in response to the outcome, i.e. the match/mismatch between the appraisal representing the initial anticipatory strategy action plan, a^π_t+1, and the environmental response or outcome, a^π_t. This is reflected as V^π(s_t+1)(a_t+1) - V^π(s_t) according to Sutton & Barto's nomenclature and temporal difference model. The temporal difference algorithm is a representation of the goodness-to-fit of a reward expectation model. In review from the prior section, the temporal difference algorithm is a representation of the goodness-to-fit of a reward expectation model.

Τ^DIFF = (β -1) ( V^π(s_t+1 , a_t+1) - V (s_t)

The expectation model's accuracy (i.e. the trueness of one's previously selected strategy and action to the strategy required by the task or relationship) is confirmed with a goal-outcome match, reward/return acquisition, and a temporal difference product that is equal to zero, suggesting no further revisions to the goal for reward are necessary.  Because the product is a function of a match, it is represented as  f,  f  (0) = (0, 0) , suggesting that there is no error and other responses to the goal-outcome match.

When the expectation's model accuracy is being questioned and a mismatch between expectation/goal and outcome exists, the temporal difference exceeds zero.  Because the product is a function of a goal-outcome mismatch, it is represented as  f,  f  (1) = (0, 1) , suggesting that there is a disparity and other responses to the goal-outcome mismatch.  These representations also suggest the need for goal revision, acceptance, or goal abandonment.

6. A sixth appraisal is generated for conceptualizing the outcome’s meaning on the organism’s relationship with the environment. Typically a person or organism has a cognitive response to the source of an outcome. This response can describe how attractive or aversive the source is considered to be. This response will answer what the outcome’s meaningfulness is about the source. The process for assessing the outcome’s impact about the source is captured in the symbol of a^other .

7. Another appraisal is elicited conceptualizing the outcome’s relevance to the organism. What is the meaningfulness of the environment’s response (of reward delivery or omission) to the person? What is the impact of receiving confirmation and the sense of well-being associated with the approval and acceptance process? What is the effect of not receiving confirmation or return and the loss of sense of well-being associated with disapproval or lack of acceptance for what one has done? The process for examining the correctness of one’s response and the outcome’s impact on the organism is embodied in a self-appraisal and the self appraisal symbol of a^self.

Where:

All Appraisal Processes = A

And,

a^p, a^r_t+1, a^π_t+1, a^r_t, a^π_t, a^other, a^self ∈  A

And

a^p, a^r_t+1, a^π_t+1, a^r_t, a^π_t, a^other, a^self ⋅⋅⋅ aⁿ →  A

And

Τ^DIFF  =  (β -1) ( V^π(s_t+1 , a_t+1) - V (s_t) where, anticipatory responses can be dissociated from actual outcome in the same way that ….

Τ^DIFF  =  (β -1) (a^p, a^r_t+1, a^π_t+1) - (a^r_t, a^π_t )  =  a^other + a^self  = a^self-other

Τ^DIFF  =  a^other +  a^self   =  Identity of self and other. When integrated into the internal working model it would be reflected accordingly,

Im =  ∑ _i=1 a^self-other_t-1 + a^other_t + a^self_t

Im = ∑ _i=1 a^other + a^self = Self-Concept

Representation of the Match and Mismatch

The above algorithm, a^p , a^r_t+1, a^π _t+1 ∈  A , symbolically represents processes for strategy development that are required for acquiring future reward/return. Its temporal difference, a^r_t, a^π _t, and later expressing mental product, a^self-other, components are symbolically in primary appraisals.

Primary sensory appraisals are reactive in nature to environmental outcomes and mobilize sensory conceptualizations and visceral, neurohormonal, and autonomic bodily reactive responses.  Two different primary appraisals , which are associated with self-concept, will be elaborated on below.  When cumulatively and repeatedly expressed personally relevant primary appraisals become incorporated into an internal working model of self and other during social interactions.  The first primary appraisal to be discussed, appraisal of others, describes the impact of another on the self.  Repeated like experiences of another later become the basis of one's expectations of how another person will react to the self during social interactions.

Social Reward Motivation

Behavior is guided by reward motivation, i.e. sensations of well-being and of being good. Sometimes one aims for tangible sources of feeling good, at other times intangible sources. Different social reward motivations were discussed in an earlier portion of this website, Early Social Reward Motivation, Differentiating Reward Motivation, Reward Motivation, etc. There are different timetables where different social reward motivations are expressed in different manners, depending upon the developmental maturity of the person. There are different social motivations that can be classified and delineated in detail (e.g. motivation to be loved, to be accepted, comforted, wanted, etc.). These same social motivations can also be generalized, classified, and encompassed by one word, social motivation lovable-affiliation. The designations of love-affiliation, control over outcomes, achievement, validation, and sense of well-being are such generalizations that encompass other like characterizations.

Sources: (1) Goulding & Goulding, 1979; (2) Goulding & Goulding, 1989; (3) Shapiro, 2001; (4) Diener, 2000.

When, T^DIFF = a^other = 0 = He/she wants me and loves me.

When, T^DIFF = a^other = 1 = He/she doesn't want me and doesn't love me.

And when the cumulative sum of all appraisals of others has been negative ∑ _i=1 a^other-neg = 1 = he/she never wants me or never needs me, then ∑ _i=1 a^other-neg = Im_neg, wherein the internal model is negative and characterized by negative self-esteem and a sense that others have been and will be unresponsive in providing love and nurturance.

Appraisals of others assessments are often accompanied by or followed by self appraisals. Self-appraisals cumulatively expressed later morph into one’s self concept and internal working model.

Examples of self-generated primary sensory appraisals are reflected in the following table.

Sources: (1) Goulding & Goulding, 1979; (2) Goulding & Goulding, 1989; (3) Shapiro, 2001; (4) Diener, 2000.

We can also apply the other-appraisal model to the self-appraisal model noted below.

Likewise, when the T^DIFF = a^self = 0  the "I" statement is I am loved and wanted and the self statement is  a^self-pos.  When T^DIFF = a^self-pos = 0 = I am lovable and wanted, the sum of like experiences,  ∑ _i=1 a^self-pos = 0 = I am wanted; I am loved, cumulatively shapes into a positive internal working model, ∑ _i=1 a^self-pos = Im_pos.  This internal model is positive and emits positive self-esteem and a sense that the self is worthy of being loved by others.

This is depicted in the pictorial representation below, i.e. ∑ _i=1 a^self-pos = 0. The reader will notice that with event, Τ₁^DIFF= (β -1) (a^p, a^r_t+1, a^π_t+1) - (a^r_t, a^π_t ) = a₁^other + a₁^self = a₁^self-other, and event, Τ₂^DIFF = (β -1) (a^p, a^r_t+1, a^π_t+1) - (a^r_t, a^π_t ) = a₂^other + a₂^self = a₂^self-other, and so on, one's immediate response to an event becomes farther and farther removed with time, and, with cumulative processing of long term memory, i.e. the sum of all temporal differences, ∑ _i=1Τ_n^DIFF = (β -1) (a^p, a^r_t+1, a^π_t+1) - (a^r_t, a^π_t ) = a_n^other + a_n^self = a_n^self-other , and with each succeeding event experience, each event becomes even more conceptualized and generalized onto all environmental sources. This would be depicted as ∑ _i=1Τ_n^DIFF = a_n^self-other = 0, when positive.

We can also apply the other-appraisal model to the self-appraisal model noted below.

Likewise, when the T^DIFF = a^self = 1  the "I" statement is I am unlovable and unwanted and the self statement is  a^self-neg.  When T^DIFF = a^self-neg = 1 = I am not lovable and am not wanted, the sum of like experiences,  ∑ _i=1 a^self-neg = 1 = I am not wanted; I am not loved, cumulatively shapes into a negative internal working model, ∑ _i=1 a^self-neg = Im_neg.  This internal model is negative and emits negative self-esteem and a sense that the self is not worthy of being loved by others.

This is depicted in the pictorial representation below, i.e. ∑ _i=1 a^self-neg = 1. The reader will also notice that the sum of all temporal differences, ∑ _i=1Τ_n^DIFF = (β -1) (a^p, a^r_t+1, a^π_t+1) - (a^r_t, a^π_t ) = a_n^other + a_n^self = a_n^self-other = 1 , and with each succeeding event experience, each event becomes even more conceptualized and generalized onto all environmental sources. This would be depicted as ∑ _i=1Τ_n^DIFF = a_n^self-other = 1, when negative.

This internal model can also be summarized in a matrix with an embedded ∑ _i=1Τ_n^DIFF = (β -1) (a^p, a^r_t+1, a^π_t+1) - (a^r_t, a^π_t ) = a_n^other + a_n^self = a_n^self-other, whether the findings result in consistent, cumulative match of 0 or result in persistent reoccurring mismatch of 1.

Formulation

When T^DIFF = 0 and T^DIFF = (a^p, a^r_t+1, a^π_t+1) - (a^r_t, a^π_t) = 0 and f , f ( 0 ) = ( 0 , 0 ), the appraisal formulated for a^self-other is positive.

This is accompanied by ∑_i=1(a^p, a^r_t+1, a^π_t+1) - (a^r_t, a^π_t) = 0

Thus yielding an Im = 0, which is positive and free of stress-related arousal and mismatch error. This also suggests little or no need for changing initial goal planning associated with a^r_t+1 and a^π_t+1.

When T^DIFF = 1 and T^DIFF = (a^p, a^r_t+1, a^π_t+1) - (a^r_t, a^π_t) = 1 and f , f ( 1 ) = ( 0 , 1 ), the appraisal formulated for a^self-other is negative.

This is accompanied by ∑_i=1(a^p, a^r_t+1, a^π_t+1) - (a^r_t, a^π_t) = 1

Thus yielding an Im = 1, which is negative and characterized by and fraught with stress-related arousal and mismatch error. This also suggests a significant need for changing initial goal planning associated with a^r_t+1 and a^π_t+1. The stress-related arousal and mismatch error signal the need for a goal change, which is mediated by generated emotion.