Cymeon Research

A NEW MULTI-DIMENSIONAL AND INTERNATIONAL INSTRUMENT TO AUDIT EMPLOYEE PERCEPTIONS

Adrian Furnham

The aim of this research was to develop a robust, multi-dimensional, fully psychometrised questionnaire able to be used in a wide variety of organisations and various different countries for both descriptive and predictive purposes.  Presently available categorical and dimensional questionnaire measures were described and evaluated and a brief but critical review of the organisational climate literature executed.  A study is described in which two population groups, over 200 British employees and over 300 European employees at all levels, doing a variety of jobs in a large multi-national transportation company, completed a 108 item multi-dimensional questionnaire which required them to rate each question on two scales - performance and importance.  Cronbach’s Alpha indicated that most dimensions had highly satisfactory internal reliability and correlational analysis suggested many of the scales were closely related.  These factors were correlated with various other personal (age, sex, years with the company )  and organisational features (location, level, job).  The potential application of this measure to other occupational settings was discussed, along with its potential use in international analysis.

INTRODUCTION

This paper concerns the development of a climate survey  that can be used in different organisations and different countries.  The concept of organisational climate is usually attributed to Lewin (1951) with his field theory motivation (Bonoma & Zaltman, 1981 ).

The concept became popular in the industrial and organisational literature particularly in the 1960’s and 1970’s with the book of Litwin and Stringer (1968)  and the two major reviews of Forehand and Gilmer (1964) and James and Jones (1974).  The topic remains one not only of considerable theoretical speculation and research (La Follette, 1975; Qualls & Puto 1989, Kozlewski & Doherty 1989), but also disagreement (Jackofsky & Slocum, 1988; Payne, 1990).

Attempting to define or operationalize the concept, many researchers quote Forehand and Gilmer (1984 )  Who noted:  “Organiational Climate is the set of characteristics that describe an organisation and that (a) distinguish one organisation from another (b) are relatively enduring over a period of time and (c) influence the behaviour of people in the organisation.”

However, the concept proved ambiguous, nebulous and controversial.  The main problems in the conceptual clarification concern whether climate should be conceived of in terms of the objective (physical or structural) features of the organisation or the subjective (perceptual)  reactions to the organisation.  Hence Guion (1973) argued that a perceived climate concerned both the attributes of an organisation and those of the perceiving individual and that as most often conceived climate was simply an alternative label for affective responses to organisation, like job satisfaction.  James and Jones (1974) suggested the psychological climate be used to emphasise the fact that it is the aggregated cognitive interpretations of an organisational work-force which arise from experience in the organisation and provide a representation of the meaning inherent in the organisational features, events and processes (Schneider,1983 a,b; Kozlowski and Farr, 1988).

An important but related issue concerns the amount of consensus within an organisation concerning the perceived climate.  Pace and Stern (1958) suggested a two-third agreement but Guion (1973) has argued that it should be 90% for the concept of climate to be invoked.  Payne (1990) has argued that the concept of organisational climate is invalid because people in different parts of the organisation have radically different perceptions of the organisation (hence the perception is not shared) and that where perceptions are consensually shared, in small groups, they are not representatives of the climate of the whole organisation.  Thus for Payne (1990) it is possible to have departmental but not organisational climates.

This conceptual muddle has become worse with the introduction of the concept of corporate or organisational culture (Schein,1990) defined as:

a.  A pattern of basic assumptions.

b.  invented, discovered, or developed by a given group.

c.  as it learns to cope with its problems of external adaptations and internal integration.

d.  is to be taught to new members as the, (f) correct ways to perceive, think, and feel in relations to those problem”

However, there are as many, if not more, problems associated with the concept of corporate culture as there are of climate.  One way to circumvent, rather than overcome, the conceptual issues is to talk of employee perceptions rather that culture and climate.  Naturally, employee perceptions differ within an organisation as a function of seniority, department, etc. and those  perception influence and are influenced by organisational behaviours.  But because the term climate has been used in the past it shall be returned here to examine the current literature.

The second major theoretical problem concerns the effect of climate (or employee perception) on organisational behaviour.  Climate may be conceived of as an independent variable, as for instance in the work of Campbell, Dunnette, Lawler and Weick (1970), it is assumed that organisational climate itself directly influences (causes) various work outcomes both positive like productivity, satisfaction, and motivation, and negative like absenteeism, turnover and accidents.  Other have considered climate a dependent outcome variable that is the result, and not the cause of, organisational structure and process.  In this sense climate may be a useful index of organisation’s health but not a causative factor of it.  A third and perhaps more common approach has been to see climate as a moderator variable in that climate may be the indirect link between two organisational outcomes.  Thus climate may be the moderator variable between job satisfaction and productivity.  Various untested but heuristically satisfying models consider climate as one of a number of powerful moderator variables (Litween and Stringer, 1968).  Finally, some researchers believe that climate is epephenomenal, neither a direct cause or effect variable but one that emerges in some form in all organizations with no influence on it.  Employee perception is then interesting but no directly relevant to the functioning of the organisation.

There are many models which use the concept of climate (Litwin and Stringer, 1968, Bonoma and Zaltman, 1981) but very few specify the exact relationship between climate and other organisational processes.  Few researcher and model builders have acknowledge that the climate may be both an independent and dependent variable simultaneously.  Few studies have tested any longitudinal path-analytic models to find out what major factors influence climate and which are influenced by it; thus this seems an important and relevant theoretical and empirical avenue to pursue.  A third major problem in the area concerns the issue of measurement of climate or employee perception.

There are numerous ways of measuring organisational climate.  The first is categorical, which attempt to classify organisations into pre-existing theoretical types.  The second id dimensional. which are thought to capture or fully describe the organisational climate.

The first or categorical approach has not been very popular or successful.  Examples of this approach can be seen in the work of Ginsberg (1978), who described three basic climates (inception, post-entrepreneurial and bureaucratic) and Halpin and Croft (1962) who felt climates could be categorised as either open autonomous, controlled, familiar, paternal or closed.  Although this approach has attracted a certain amount of research (Hall 1971)  its limitation are those of all typologies -- lack of fine discriminability, inappropriate categories, and most importantly the idea that organisational climates are multi-dimensional and should be measured on various salient albeit related, dimensions.

A number of dimensional organisational climate measures exist.  Litwin and Stringer’s (1968) 50 items Organisation Climate Questionnaire (Form B) is designed to measure nine characteristics reflecting the degree of organisational emphasis on Structure, Responsibility, Reward, Risk Warmth Support, Standard, Conflict and Identity.

Several additional measures have often been utilised in psychological/organisational research.  for example, House and Rizzo (1972) developed the Organisation Description Questionnaire.  Taylor and Bowers (1972) popularised the University of Michigan Survey of Organisations.  The survey has 22 items designed to measure organisational climate.  Similarly, Payne and Pheysey (1971) offered a Business Organisation Climate Index which is a refinement of Stens’s (1967) Organisational Climate Index.  Other Measures were also developed by Jones and James (1979), Halpin and Croft (1963) and Pritchard and Kurasick (1973).

A major problem with many of these earlier measures was their poor psychometric properties -poor internal reliability (ie. low alpha’s ) and little or no validity.  This study sets out specifically to devise a psychometrically sound Employee Perception  Questionnaire (useful in corporate Audits) that have six basic criteria.  The first is that the questionnaire should be parsimonious yet comprehensive  in that all the salient dimensions of climate  are measured but that the questionnaire is not over-long and redundant.  Second, it was planned that the questionnaire should be reliable showing most importantly internal reliability.  Third, the questionnaire should be valid, (in the sense that it measured what it said it measured)  able to discriminate between different parts o fan organisation and able to reveal important and sensitive differences.  Fourth, the questionnaire should travel well in that it could be used in different countries so that comparisons could be made, in order to audit similar and/or different companies in different cultures or the same organisation in different organisations and different (international) countries.  Fifth, most climate or employee perception questionnaires required respondents to rate the extent to which they agree or disagree that such a “Climate” situation/dimension to the organisation.  I is the aim of this questionnaire to get respondents to rate both the truthfulness and salience of each dimension to the individual.  Having a rating of importance or salience, as well as agreement, offers on e way of assessing the validity of the  questionnaire for  the organisation although it should be pointed out that there is limited evidence of variation on measures of importance.  Sixth and finally, the questionnaire should produce a measure that can be used to highlight international differences within and between multi-nationals where appropriate.

It was intended to produce a measure that is comprehensive particularly as it could be used as a before and after measure to evaluate the efficacy of certain subject measures of structural variables and that “social structures are designed to produce certain patterns of behaviour and belief”.  This measure is concerned exclusively with personal belief and behaviours (See Table1) which inevitably reflect the organisational structure.  It would be impossible in devising a sensitive and comprehensive measure completely to separate the two.  Hence the term Employee Perception Questionnaire.

METHOD

SUBJECTS

Sample 1.  This consisted of 204 British subjects working in the South East of England.  They all worked for a large American-based airline.  110 were male and 94 female.  There were 84 under 35 years of age, 86 between 36 and 50 and 34 over 50.  187 were full time and 17 part time.  70 had been in the organisation less than 5 years, 48 between 5 and 15 years and 86 over 15 years.  Of all the 43 held management grades and 161 non-management grades.  They did a wide variety of jobs from Secretarial to Engineering.

Sample 2  This consisted of 345 European subjects, fully competent in English, working in seven European countries.  All were employees of the same airline.  177 were male and 168 female.  There were 118 under 35 years of age, 158 between 36 and 50, and 69 over 50.  In all 239 were full-time, 107 had been with the organisation less than 5 years, 65 between 5 and 15 years and 173 over 15 years.  Of the total, 67 held management grades 244 non-management grades.  Like the above they did a wide variety of jobs from Secretarial, Senior Management, to Engineering.

Questionnaire

A 108 item questionnaire was devised after extensive piloting.  A review of the literature in academic and applied fields suggested that a number of dimensions should be measured.  Items were written for each dimension.  These were shown to Personnel Directors, Management. Consultants, and Applied Psychology Academics for their opinions.  Many changes were made: some dimensions were added other removed, and still others were either collapsed or sub-divided.  In the end 14 dimensions remained, with a total of 108 questions.

Each person was asked to respond to 108 questions concerning the organisation on two different scales, for a total of 216 individual responses.  They were first asked to indicate on 7-point Likert scales how much they agreed/disagreed with each statement; then they were asked to indicate how important/unimportant they felt each was in the organisation.  It is important to note:

1.  As all the questions were described positively, a high agree score (ie. 4 or above) indicates the employees satisfaction with the performance of that aspect of the company.  Where scores fall below 2.50, this suggests the employees as a whole seemed less than satisfied.

2.  Similarly a high importance score (5 or above) suggests that employees believed the feature that the item was referring to was an important aspect of the workings of the company.  Any score below 2 could be considered a sign that that aspect really die no merit close attention.

3.  Looking at the scores together, simple matrix occurs.

This leaves four different courses of action given the results.

1.  Ignore:  those questions referring to behaviours with low agreement performance and importance can be ignored because although performance was low so was importance.  This dimension of climate is of no consideration.

2.  Consider:  Where performance is high (ie good) but the importance is low it suggests that employees see certain things done well which are really not very important.  Those need to be considered by change agents as they may represent misguided effort.

3.  Celebrate:  Where performance is high and importance is high one can celebrate the fact that important issues are being perceived as being done well.  These factors/behaviours need to be maintained.

4.  Fix it:  the major course for action occurs where the issues are considered high in importance but performance is low.  It is these items that most warrant most attention particularly the very low performance, high importance.

Two things need to be pointed out with this scale.  Firstly all the questions were “expressed positively” which of course makes it liable to a acquiescence response set.  However, this was absolutely necessary as it becomes impossible for many respondents to rate the importance/salience of faced the same problem.  However, pilot wok suggested that an acquiescence response set was not operating to any significant degree.  Secondly the questionnaire is multi-dimensional in the sense that is assesses different facets of the organisation as perceived by the employees.  They may, or may not be inter-related and independent.

PROCEDURE

The questionnaire was given to individuals in their place of work along with a pre-paid envelope to send their replies to an independent outside consultant of analysis.  The response rate varied as varied as a function of the part of the organisation and the country of response varying from about 40 % to 80%.  All employees were given extensive group feedback about 2 months after the survey was conducted.

IMPORTANCE

                                                            LOW                    HIGH

                                    HIGH              CONSIDER          CELEBRATE

PERFORMANCE

                                    LOW               IGNORE                 FIX

RESULTS

A. Reliability

A major criterion of the success of any questionnaire is its reliability.  Cronbach Alpha’s were calculated for both the British and European subjects for both agreement and importance scales.

Table 2 shows the alphas, which are consistently high with few exceptions for both British and European subjects.  Interestingly, the alphas for the agreement and importance scales were virtually identical.  Overall these reliability figures appear to be better tan any other measure of climate.  The fact that the alphas are almost identical across the eight nations sampled (one British and seven European) attests to the use fullness of this questionnaire in different countries.

B. Intercorrelations

Previous work has shown that frequently the dimensions of climate are significantly correlated.  Hence various correlational matrices were calculated.

The intercorrelations in Table 3 are interesting for three reasons.  First most corrections are positive and low - those over .50 are underlined suggesting that the various dimensions of climate are modestly related.  Second, whereas some dimensions of climate are modestly related.  Second, whereas some dimensions.  (e.g.  Commitment and Morale)  seem relatively highly correlated with all other dimensions the opposite is true for other dimensions (H:ClimateRelationships).  Third, correlation between significant and occasionally negative.  Fourth, the correlation matrix was calculated for both populations (British and European)  and very similar results occurred suggesting a coherent structure across cultures.

C. Correlates

The scores on the 14 dimensions for both rated performance (aggreement) and importance were completed with a variety of demographic variables.  The results for sex, age and seniority are shown in Table 4, although various others were also examined, such as department, country, and job function. 

Table 2

Cronbach alpha’s for each of the 14 climate dimensions for both agreement and importance.

                                    British               (N=204)                     European         (N345)

 .showing complete consensus in the relative importance of climate variables.  Second, although there were some correlates of sex (four for the British, four for the Europeans) and some of age (five for the British, two for the Europeans) they were few in number and no clear pattern was discernible.  Third, the correlates of seniority were systematic, substantial and very similar across the two population groups.  The more senior the person the higher they are rated all of the variables.  Fourth, there were some interesting differences between the correlates of the two national groups.  For instance “Teamwork and Support” was correlated with all three demographic variables in the British sample but not at all in the European group.  Equally “Client/Passenger relationships” were correlated with sex and seniority among the European but not among the British.

Table 3

Inter-correlation between the 14 dimensions of climate rated on agreement (showing British N=204).  Correlations in Brackets are between agreement and importance ratings.

The issue of organisational climate remains one that is constantly researched and hotly debated (Jackofsky and Slocum, 1988; Payne, 1990).  This study set out to devise and validate a new multi-dimensional measure of organisational climate. Perhaps the most fundamental question to be asked is why devise a new measure given that a number already exist?  There are a number of answers to this: first, many of those that exist have either had insufficient psychometric assessment or else the reliability and validity statistics available are only modestly impressive.  Second, some of the dimensions of climate measured by other scales seem to reflect a lack conceptual clarity as to what the nature of climate really is.  Finally, some scales, developed about 20 years ago appear to have lain dormant in the literature and would probably require extensive updating.

The new employee perception questionnaire reported in this study demonstrated satisfactory internal reliability given both the number of questions per scale and the heterogeneity of the respondents.  Predictably the Co-efficient alphas were on average higher in the ratings of importance than agreement showing that variance of response on important was lower than on agreement.  As has been found with other climate surveys the various dimensions of climate had modest positive correlations (Jackofsky and Slocum 1988) which suggests that employee perceptions of the organization are not as differentiated as some researchers suppose.

The issue of validity was with in two ways.  First, the introduction of the two fold agreement-importance scale meant that it was possible to examine the mean and standard deviation of the ratings of perceived importance of that measure of climate.  This yields face validity evidence.  The mean or the various dimension was nearly always over 5.00 (on a 7 point scale)  and the SD less than 1.00 for both British and European samples which suggests that overall all respondents believed the questions salient.  Second a set of ANOVAS were computed in an attempt to examine systematic differences between individuals who completed the questionnaire.  Sex, age location, job function and nationality yielded minimal differences but seniority did (See Table 4).  This finding certainly throws light on the debate between Jackofsky and Slocum (1988,1990) and Payne (1990) concerning the  definition of climate.  That is if there were demonstrably nosystematic differences in the perceived climate between employees in different places and departments (both within and between countries)  it could not be argued that employees did not share perceptions of the organisational climate.  One difference which did emerge however is quite predictable and expected - namely between seniority and perceived climate.  The results showed a completely clear pattern:  while there were no differences in the perceived ratings o f importance of the various dimensions, over half of the dimensions revealed significant differences in the agreement scale, showing that the more senior the person the more they expressed satisfaction and contentment with the prevailing climate.  This is in accordance with previous findings (Jackofsky and Slocum 1988)

The definition of organisational climate has lead to considerable debate and something of a proliferation of measures.  This is due to both conceptual and pragmatic reasons.  Conceptually there remains no agreement upon easily operationalizable definition of climate that is both prescriptive and proscriptive in what should ( and can ) be measured.  Also because of the different needs and structures of various organisations, different measures with different dimensions become salient.  Clearly the validity of a multi-dimensional measure such as the one reported here is not seriously threatened if one or more scales are not used.  Yet it is important that if any additional scales are used that their psychometric properties should be assessed.  Futuremore it maybe that no “off-the-shelf” questionnaires should be used I corporate assessment of employee perception unless and until they have piloted for appropriateness.  This  may involve minor modifications of language, taking out inappropriate items or dimensions and adding others.

This paper highlights one other very important issue for the measurement of organizational climate:  namely the issue of cross-national measurement.  Multi-national may be very interested in comparing the climate of their different “national” companies.  Highly, centralised multinationals may attempt to impose a “head office” climate or culture on their subsidiary companies with differing success depending on the “fit” between the national/regional culture and that of the founder organisation require change.  This albeit modest study appeared to indicate that the differences between this American owned and directed airline’s office climate in Britain and Europe was not fundamentally different: a fact that may cause either despair or celebration depending on management’s view.

REFERENCES

Bonoma, T, & Zaltman, G (1981). Psychology for Management. Boston: Kent

Campbell, J, Dunnetter M, Lawler, E & Weick K. (1970).  Managerial behaviour

Performance and effectiveness New York: McGraw-Hill.

Forehand, G, &Von Gilmer, B(1964).  Environmental variations in studies of organisational behaviour.  Psychological Bulletin, 62, 362-381.

Guion, R. (1973).  A note on organisational climate.  Organisational Behaviour and Human Performance, 9,120-125.

Ginberg,L. (1978).  Strategic planning for work climate modification. Amacon

Hall, J. (1971).  A comparison of Halpin and Groft’s Organizational Climate and Likert and Likert’s Organizational systems.  Administrative Science Quaterly, 16,586-396

Hall, D, &Mansfield, R.(1975).  Relationship of age and seniority with career variables of engineers and scientists.  Journal of Applied Psychology, 56, 388-396.

Halpin A&Croft, D (1963) The organizational climate of schools. Chicago: U of C Press.

House R & Rizzo, J (1972).  Toward the measure of organizational practices:  Scale development and validation.  Journal of Applied Psychology, 56, 388-396.

Jackosky, E & Slocum, J (1988). a longitudinal study of climates.  Journal of Organizational Behaviour.  9, 319-334. 

James, L, & Jones A (1974).  Organizational Climate: A review of theory and research.  Psychological Bulletin, 12, 1096-1112

Jones, A, & James, L (1979) Psychological climate: Dimensions and relationships of individual and aggregated work environment perception.  Organizational Behaviour and Human Performance, 23, 201-250.

Kozlowski, S, & Farr, J (1988).  An integrative model of updating and performance.Human Performance, 1, 5-29.

Kozlowski S & Doherty M (1989).  Integration of climate and leadership:

Examination of neglected issue.  Journal of Applied Psychology

La Follette, W. (1975.  Field theory in the Social Sciences.  New York: Harper

Litwin G & Stringer,R (1968)  Motivation and organizational Climate.  Boston HUP

Muchinsky, P. (1977).  Organizational communication: Relationship to organizational Climate and job satisfaction.  Academy of Management Journal 20,592,607

Pacem, C & Stern G (1958) An approach to the measurement of the psychological characteristics of college environments.  Journal of Educational Psychology 49, 269-277.

Payne, r, (1990) Madness in our method.  A comment on Jackofsky and Slocum’s paper.  A longititudinal study of climates.  Journal of Organizational Behaviour, 11, 77-80

Payne, R & Pheysey, D (1971).  CG Stern’s organizational climate index Organizational Behavour and Human Performance 18,45-62.

Qualls, W & Puto, C (1989) Organizational climate and decision framing.  Journal of Marketing Research  26,179-192.

Schein E (1990).  Organizational Culture.  American Psychologist 45,109-119

Schneider B (1983) Interactional psychology and organizational behaviour.  In

L Cummings & B Staw ( Eds), Research in organizational behaviour.  Greenwich, CT: JAI Press.

Schneider, B (1983B)  Work climates:  An interactionist perspective. In N. Feimer & E Geller (Eds), Environmental psychology: Directions and percepectives.  New York: Praeger

Stern E. (1967) People in context.  Unpublished Report: University of Syracuse

Taylor, J, & Bowers, D (1972)  Survey of organizations: A machine scored standardized questionnaire instrument, University of Michigan.

TABLE 4

Pattern of Significant differences across sex, ages, and seniority for Agreement and Importance Scales (British and European Subjects)

Return back to product page