More about 360 degree feedback

Overview

Multisource feedback, also known as 360 degree or multi-rater feedback, refers to appraisals of a person, usually a manager, that are derived from two or more distinct categories of individuals, such as supervisors, peers, subordinates, customers–from either within or outside the organization–and suppliers (Dalessio, 1998). Indeed, in many instances, the participants are also encouraged to evaluate their own behavior and performance.

Since the early 1990s, multisource feedback has become increasing popular. In the US, an estimated 90% of the Fortune 1000 firms have introduced some form of multisource assessment (Atwater & Waldman, 1998). These assessments are primarily utilized to facilitate the development of managers and leaders. Although not as frequent, this information can also affect the pay and promotion of individuals (Atwater, Waldman & Brett, 2002).

Although sometimes effective, many factors affect the utility of multisource feedback. For example, if managers exhibit elevated levels of anxiety and neuroticism, they are seldom motivated to adjust their behavior in the aftermath of this process (Smither, London, & Richmond, 2005); the feedback is almost futile. Furthermore, if managers perceive themselves unfavorably, they are more likely to respond to feedback that aligns with their perception of themselves (Bono & Colbert, 2005).

Consequences of multisource feedback

Many scholars and practitioners maintain that multisource feedback offers a variety of benefits (see Antonioni, 1996; Bernardin & Beatty, 1987; Hoffman, 1995; London & Beatty, 1993; Morgeson, Mumford & Campion, 2005). This process:

Consequences to work performance

Despite these optimistic assertions, the majority of studies indicate the impact of multisource or 360 degree feedback on management performance is modest. To illustrate, Smither, London and Reilly (2005) conducted a meta-analysis of 24 longitudinal studies. In general, improvements in the performance of managers, as evaluated by subsequent multisource feedback, were small in size. The assessments derived from subordinates, often called direct reports in this context, and supervisors revealed only a small improvement in performance, with d=.15. The assessments derived from peers revealed a d of only .05 (see also Smither, Brett, & Atwater, 2008).

Nevertheless, past studies might have underestimated the benefits of multisource feedback. In particular, after individuals rate a manager, they form more comprehensive perceptions of this person. These perceptions might then bias the attention, appraisal, and memory of individuals. They might become more inclined to direct their attention, skew their interpretations, or bias their memory towards behaviors that align with these preconceptions (Smither, London, Vasilopoulos, Reilly, and Millsap, & Salvemini, 1995). Genuine changes in the manager might be overlooked (see also Buda, Reilly, & Smither, 1991; Smither, Reilly, & Buda. 1988).

Furthermore, the improvements might not be immediate but unfold over several years. Walker and Smither (1999), for example, did not uncover any improvements in ratings between the first and second years. Steady improvements from the second to fourth years were observed, however.

Despite this more favorable perspective, some doubts remain. No evidence has yet been accrued to show, at least convincingly, that multisource feedback does enhance indices of company performance (cf., Seifert, Yukl, & McDonald, 2003).

Consequences to interpersonal behavior

Multisource feedback can affect the interpersonal behavior of managers. After they receive negative feedback, for example, managers sometimes demonstrate less commitment and support to their subordinates (Atwater, Waldman, Atwater, & Cartier, 2000).

Consequences to attitudes and emotions

Multisource feedback, especially if unfavorable, can obviously provoke a host of negative attitudes or emotions. Negative feedback, for example, has been shown to elicit anger or discouragement, which can curb subsequent improvement and performance (Brett & Atwater, 2001).

Factors that moderate the effects of multisource feedback

Prior level of performance

The benefits of multisource feedback may depend on the performance of participants even before the process was implemented. As Smither, London, Vasilopoulos, Reilly, and Millsap, and Salvemini (1995) showed, if the participants are average or below average in performance, multisource feedback seems to improve subsequent behavior. Nevertheless, when the feedback is especially negative, defensive responses might be evoked, and change is also limited (Kluger & DeNisi, 1996). Furthermore, if participants are above average in performance, multisource feedback does not seem to improve, and even tends to impair, subsequent performance.

Accessibility of feedback

Generally, feedback that is more accessible–that is readily recalled weeks or months after the information is presented–should be more likely to shape the subsequent behavior and performance of the participant. Feedback that is not accessible and cannot be readily recalled may not affect behavior (Smither, Brett, & Atwater, 2008).

Smither, Brett, and Atwater (2008) examined some of the factors that influence the accessibility of feedback and hence the subsequent improvement in performance. First, participants are more likely to recall positive, rather than negative, feedback. Second, participants are more inclined to recall feedback that relates to their performance orientation–but not as likely to recall feedback that concerns their capacity to develop subordinates. Third, participants were more inclined to recall feedback that alluded to specific, tangible behaviors rather than global, abstract traits. Finally, participants were more likely to recall feedback from supervisors and subordinates than from peers.

Smither, Brett, and Atwater (2008) did not, however, obtain definitive evidence that recall of feedback did affect subsequent performance. That is, whether the feedback was recalled was not significantly related to improvements in performance. Conceivably, some other factors might have obscured this association. To illustrate, if self efficacy is limited–or the attribute is not perceived as malleable–recall of negative feedback could provoke defensive responses and even curb change.

Characteristics of participants

Some of the characteristics of recipients–their temperament and motivations, for example–can affect the utility of multisource feedback. That is, positive attitudes towards change and feedback enhance the efficacy of this process (Smither, London, & Reilly, 2005). These positive attitudes encompass the belief that change is possible and desirable as well as the assumption that feedback is valuable (Smither, London, & Reilly, 2005).

Furthermore, the capacity and motivation of recipients to regulate their behavior also influences the utility of multisource feedback. If these individuals set appropriate goals, and have developed the capacity to implement their intentions, this feedback process is more likely to enhance subsequent behavior (Smither, London, & Reilly, 2005).

Emotional stability or neuroticism could also influence the utility of performance appraisal systems. In one study, when leaders report elevated levels of emotional stability, rather than frequent and prolonged experiences of anxiety, they seemed more motivated to apply the feedback they received (Smither, London, & Richmond, 2005).

Characteristics of organizations

Although not examined definitively, the culture of organizations could also affect the efficacy of multisource feedback. According to Moravec, Gyr, and Friedman (1993), for example, this initiative is more effective if:

McCarthy and Garavan (2007) examined some facets of organizations that might influence responses to multisource feedback. As this study showed, when individuals perceived the feedback initiative as fair and just–and when they felt the workplace interventions at the organization tend to be effective–they were more inclined to perceive this process as productive and thus accept the feedback. If managers are skeptical about whether changes tend to be sustained in organizations, they withhold effort, and seldom embrace feedback.

Consistency across raters

Sometimes, all the raters–the subordinates, supervisor, peers, and customers, for example–provide similar feedback to one another and to the participants themselves. In other instances, some pronounced discrepancies are observed. These discrepancies affect the utility of multisource feedback.

When participants rate themselves less favorably than do their subordinates, for example, change is often stifled (Johnson & Ferstl, 1999). Nevertheless, these participant—relative to managers who rate themselves more favorably than do their subordinates–often perform more effectively overall (Atwater, Ostroff, Yammarino, & Fleenor, 1998; Van Velsor, Taylor & Leslie, 1993). Because their performance is already more proficient, improvements might be curtailed.

Nevertheless, the effect of these discrepancies on subsequent behavior and improvement depends on the core self evaluations of individuals (Bono & Colbert, 2005)–the extent to which their self esteem is elevated, self efficacy is strong, locus of control is internal, and emotional stability is high. In particular, if individuals perceived themselves favorably, they were more committed to development goals after they received discrepant information. That is, in these individuals, feedback that diverged from their own perceptions enhanced motivation to change (Bono & Colbert, 2005). In contrast, if participants perceived themselves unfavorably, they were more committed to development goals after they received feedback that aligned with their perceptions of themselves (Bono & Colbert, 2005). Such discrepancies could evoke negative emotions, elicit defensive reactions, and undermine the confidence of managers who adopt unfavorable perceptions of themselves.

Conceivably, regulatory focus–that is, the extent to which individuals orient their attention to future aspirations or immediate duties–could affect the impact of these discrepancies and explain some of these findings. Leonardelli, Lakin, and Arkin (2007), for example, showed that individuals who orient their attention to immediate duties seek information that confirms their perceptions of themselves. Feedback that diverges from self reports, therefore, might elicit anxiety. Individuals who orient their attention to future aspirations seek information that enhances their perception of themselves. Negative feedback will elicit dejection.

Other factors that could moderate the benefits of multisource feedback

A variety of other factors or initiatives could affect the efficacy of multisource feedback. This process might be beneficial if subordinates can discuss their feedback with the participant (Walker & Smither, 1999). Furthermore, this feedback is beneficial if, in the aftermath of this process, participants feel the need to change (Smither, London, & Reilly, 2005).

Construction of multisource feedback programs

Morgeson, Mumford and Campion (2005) summarize a series of principles that practitioners should apply when they design a multisource feedback program. Specifically:

Multisource feedback should, at least initially, be introduced merely to facilitate development. Only once managers are familiar with this technique should the feedback ever be used to inform decisions about pay or promotion. The number of items should range from 40 to 60, although longer instruments are sometimes administered Items should depict tangible behaviors, such as “Frequently communicates praise to employees” Some scholars argue that six point scales should be applied, ranging from strongly disagree to strongly agree, for example. Raters should receive some training on how to evaluate individuals effectively; common biases such as halo effects, central tendency, and other distortions should be discussed (Antonioni, 1996) Generally, most scholars maintain that ratings should be anonymous Participants should be instructed to rate themselves, partly to prevent the common assumption that all of the responses were anticipated After they receive a report, the participants should be granted an opportunity to share their feedback with raters and seek suggestions–an activity that significantly, albeit modestly, enhances improvement over time (Smither, London, Reilly, Flautt, Vargas, & Kucine, 2004)

Identification of the key facets to measure

A variety of approaches have been applied to uncover the facets, attributes, qualities, competencies, or dimensions that need to be assessed. Smither, London, Vasilopoulos, Reilly, and Millsap, and Salvemini (1995), for example, interviewed 10 senior managers.

In contrast, London, Wohlers, and Gallagher (1990) conducted a more comprehensive process, interviewing employees at all levels. A committee, representing many constituencies, developed the items. A pilot study, in which the items were assessed by many employees, was then conducted. Furthermore, London and Beatty (1993) maintained the dimensions could be extracted from job analyses but then extended by reflecting upon the strategic direction of the organization.

Key facets that are often measured

Previous studies tend to examine overlapping dimensions (Antonioni, 1994; Johnson & Ferstl, 1999; Mount, Judge, Scullen, Sytsma, & Hezlett, 1998; Smither, London, Vasilopoulos, Reilly, and Millsap, & Salvemini, 1995; Walker & Smither, 1999). Common dimensions, according to Morgeson, Mumford and Campion (2005), relate to:

Reporting of feedback

Typically, participants receive a report that presents the average rating they received on each item. Sometimes, the ratings of each category, such as subordinates and peers, are presented separately (London, Wohlers, & Gallagher, 1990). In addition, discrepancies between self and other ratings are highlighted

To ascertain whether these ratings are favorable or unfavorable, norms may be reported. That is, the average ratings of other managers, in the same industry, nation, or organization, may be presented.

Furthermore, a list of recommendations might be presented. These recommendations might be derived from the comments or remarks of raters. Alternatively, these recommendations might be generated by consultants after an analysis of the ratings.

Alternative designs

Usually, a series of questions are presented, such as “This managers considers the opinions of employees”. Raters specify the extent to which they agree or disagree with these items on a numerical scale.

Nevertheless, alternative designs have been developed. According to Good and Coombe (2009), for example, raters tend to develop extensive mental representations of managers, called relational schemas, that entail information that is not always accessible to conscious awareness. These representations evoke a specific emotional response and elicit particular inclinations or behaviors. Good and Coombe (2009) attempted to create a measurement system that captures these implicit representations.

To illustrate, rather than a numerical scale, a row of schematic faces is presented, ranging from upset to happy. Raters then imagine three distinct events: receiving an email from the target participant, conversing during a meeting with this person, and conversing informally with this person. They consider the feeling that is evoked by each event, specifying the face that corresponds to this emotion. This process is assumed to represent implicit rather than conscious representations (Good & Coombe, 2009).

References

Source: The Psychlopedia (http://www.psych-it.com.au/Psychlopedia/article.asp?id=373) Created by Dr Simon Moss