« PreviousContinue »
same major work behaviors may be cal or important work behavior(s) of grouped together for validity studies, the job(s), or through a demonstration in order to obtain an adequate sample of the relationship between measures These guidelines do not require a user of performance in training and measto hire or promote persons for the ures of job performance. Measures of purpose of making it possible to con relative success in training include but duct a criterion-related study.
are not limited to instructor evalua(2) Analysis of the job. There should tions, performance samples, or tests. be a review of job information to de- Criterion measures consisting of paper termine measures of work behavior(s) and pencil tests will be closely reor performance that are relevant to viewed for job relevance. the job or group of jobs in question. (4) Representativeness of the sample. These measures or criteria are rele. Whether the study is predictive or vant to the extent that they represent concurrent, the sample subjects critical or important job duties, work should insofar as feasible be reprebehaviors or work outcomes as devel- sentative of the candidates normally oped from the review of job informa available in the relevant labor market tion. The possibility of bias should be for the job or group of jobs in quesconsidered both in selection of the cri. tion, and should insofar as feasible interion measures and their application. clude the races, sexes, and ethnic In view of the possibility of bias in groups normally available in the relesubjective evaluations, supervisory vant job market. In determining the rating techniques and instructions to representativeness of the sample in a raters should be carefully developed. concurrent validity study, the user All criterion measures and the meth- should take into account the extent to ods for gathering data need to be ex which the specific knowledges or skills amined for freedom from factors which are the primary focus of the which would unfairly alter scores of test are those which employees learn members of any group. The relevance on the job. of criteria and their freedom from bias Where samples are combined or are of particular concern when there compared, attention should be given are significant differences in measures to see that such samples are comparaof job performance for different ble in terms of the actual job they pergroups.
form, the length of time on the job (3) Criterion measures. Proper safe where time on the job is likely to guards should be taken to insure that affect performance, and other relevant scores on selection procedures do not factors likely to affect validity differenter into any judgments of employee ences; or that these factors are includadequacy that are to be used as crite- ed in the design of the study and their rion measures. Whatever criteria are effects identified. used should represent important or (5) Statistical relationships. The critical work behavior(s) or work out- degree of relationship between seleccomes. Certain criteria may be used tion procedure Scores and criterion without a full job analysis if the user measures should be examined and can show the importance of the crite computed, using professionally acceptria to the particular employment con able statistical procedures. Generally, text. These criteria include but are not a selection procedure is considered relimited to production rate, error rate, lated to the criterion, for the purposes tardiness, absenteeism, and length of of these guidelines, when the relationservice. A standardized rating of over- ship between performance on the proall work performance may be used cedure and performance on the critewhere a study of the job shows that it rion measure is statistically significant is an appropriate criterion. Where per- at the 0.05 level of significance, which formance in training is used as a crite- means that it is sufficiently high as to rion, success in training should be have a probability of no more than properly measured and the relevance one (1) in twenty (20) to have occurred of the training should be shown either by chance. Absence of a statistically through a comparsion of the content significant relationship between a seof the training program with the criti. lection procedure and job performance should not necessarily discourage (8) Fairness. This section generally other investigations of the validity of calls for studies of unfairness where that selection procedure.
technically feasible. The concept of (6) Operational use of selection pro fairness or unfairness of selection procedures. Users should evaluate each se cedures is a developing concept. In adlection procedure to assure that it is dition, fairness studies generally reappropriate for operational use, in quire substantial numbers of employ. cluding establishment of cutoff scores ees in the job or group of jobs being or rank ordering. Generally, if other studied. For these reasons, the Federal factors reman the same, the greater enforcement agencies recognize that the magnitude of the relationship the obligation to conduct studies of (e.g., coorelation coefficent) between fairness imposed by the guidelines performance on a selection procedure generally will be upon users or groups and one or more criteria of perform- of users with a large number of perance on the job, and the greater the sons in a job class, or test developers; importance and number of aspects of and that small users utilizing their job performance covered by the crite own selection procedures will generalria, the more likely it is that the pro- ly not be obligated to conduct such cedure will be appropriate for use. Re- studies because it will be technically liance upon a selection procedure infeasible for them to do so. which is significantly related to a cri- (a) Unfairness defined. When memterion measure, but which is based bers of one race, sex, or ethnic group upon a study involving a large number characteristically obtain lower scores of subjects and has a low correlation on a selection procedure than memcoefficient will be subject to close bers of another group, and the differreview if it has a large adverse impact. ences in scores are not reflected in difSole reliance upon a single selection ferences in a measure of job performinstrument which is related to only ance, use of the selection procedure one of many job duties or aspects of may unfairly deny opportunities to job performance will also be subject to members of the group that obtains the close review. The appropriateness of a lower scores. selection procedure is best evaluated (b) Investigation of fairness. Where in each particular situation and there a selection procedure results in an adare no minimum correlation coeffi- verse impact on a race, sex, or ethnic cients applicable to all employment group identified in accordance with situations. In determining whether a the classifications set forth in section selection procedure is appropriate for 4 above and that group is a significant operational use the following consider- factor in the relevant labor market, ations should also be taken into ac- the user generally should investigate count: The degree of adverse impact of the possible existence of unfairness the procedure, the availability of for that group if it is technically feasiother selection procedures of greater ble to do so. The greater the severity or substantially equal validity.
of the adverse impact on a group, the (7) Overstatement of validity find greater the need to investigate the ings. Users should avoid reliance upon possible existence of unfairness. techniques which tend to overestimate Where the weight of evidence from validity findings as a result of capital other studies shows that the selection ization on chance unless an appropri- procedure predicts fairly for the group ate safeguard is taken. Reliance upon in question and for the same or similar a few selection procedures or criteria jobs, such evidence may be relied on in of successful job performance when connection with the selection procemany selection procedures or criteria dure at issue. of performance have been studied, or (c) General considerations in fairthe use of optimal statistical weights ness investigations. Users conducting for selection procedures computed in a study of fairness should review the one sample, are techniques which tend A.P.A. Standards regarding investigato inflate validity estimates as a result tion of possible bias in testing. An inof chance. Use of a large sample is one vestigation of fairness of a selection safeguard: cross-validation is another. procedure depends on both evidence of validity and the manner in which the (f) Continued use of selection proce. selection procedure is to be used in a dures when fairness studies not feasiparticular employment context. Fair ble. If a study of fairness should otherness of a selection procedure cannot wise be performed, but is not techni. necessarily be specified in advance cally feasible, a selection procedure without investigating these factors. In may be used which has otherwise met vestigation of fairness of a selection the validity standards of these guideprocedure in samples where the range lines, unless the technical infeasibility of scores on selection procedures or
resulted from discriminatory employ-. criterion measures is severely restrict- ment practices which are demonstrated for any subgroup sample (as com- ed by facts other than past failure to pared to other subgroup samples) may conform with requirements for validaproduce misleading evidence of unfair- tion of selection procedures. However, ness. That factor should accordingly when it becomes technically feasible be taken into account in conducting
for the user to perform a study of fairsuch studies and before reliance is
ness and such a study is otherwise placed on the results.
called for, the user should conduct the (d) When unfairness is shown. If un
study of fairness. fairness is demonstrated through a
C. Technical standards for content showing that members of a particular
validity studies.-(1) Appropriateness group perform better or poorer on the
of content validity job than their scores on the selection
studies. Users procedure would indicate through
choosing to validate a selection proce
dure by a content validity strategy comparison with how members of
should determine whether it is approother groups perform, the user may
priate to conduct such a study in the either revise or replace the selection
particular employment context. A seinstrument in accordance with these guidelines, or may continue to use the
lection procedure can be supported by selection instrument operationally
a content validity strategy to the with appropriate revisions in its use to
extent that it is a representative assure compatibility between the prob
sample of the content of the job. Seability of successful job performance
lection procedures which purport to and the probability of being selected.
measure knowledges, skills, or abilities (e) Technical feasibility of fairness
may in certain circumstances be justistudies. In addition to the general con
fied by content validity, although they ditions needed for technical feasibility
may not be representative samples, if for the conduct of a criterion-related
the knowledge, skill, or ability measstudy (see section 16, below) an inves
ured by the selection procedure can be tigation of fairness requires the fol
operationally defined as provided in lowing:
section 14C(4) below, and if that (i) An adequate sample of persons in knowledge, skill, or ability is a neceseach group available for the study to
sary prerequisite to successful job perachieve findings of statistical signifi formance. cance. Guidelines do not require a user A selection procedure based upon into hire or promote persons on the ferences about mental processes basis of group classifications for the cannot be supported solely or primarpurpose of making it possible to con ily on the basis of content validity. duct a study of fairness; but the user Thus, a content strategy is not approhas the obligation otherwise to comply priate for demonstrating the validity with these guidelines.
of selection procedures which purport (ii) The samples for each group to measure traits or constructs, such should be comparable in terms of the as intelligence, aptitude, personality, actual job they perform, length of commonsense, judgment, leadership, time on the job where time on the job and spatial ability. Content validity is is likely to affect performance, and also not an appropriate strategy when other relevant factors likely to affect the selection procedure involves validity differences; or such factors knowledges, skills, or abilities which should be included in the design of the an employee will be expected to learn study and their effects identified.
on the job.
(2) Job analysis for content validity. and is a necessary prerequisite to perThere should be a job analysis which formance of critical or important work includes an analysis of the important behavior(s). In addition, to be content work behavior(s) required for success- valid, a selection procedure measuring ful performance and their relative im- a skill or ability should either closely portance and, if the behavior results approximate an observable work bein work product(s), an analysis of the havior, or its product should closely work product(s). Any job analysis approximate an observable work prod. should focus on the work behavior(s) uct. If a test purports to sample a and the tasks associated with them. If work behavior or to provide a sample work behavior(s) are not observable, of a work product, the manner and the job analysis should identify and setting of the selection procedure and analyze those aspects of the its level and complexity should closely behavior(s) that can be observed and approximate the work situation. The the observed work products. The work closer the content and the context of behavior(s) selected for measurement the selection procedure are to work should be critical work behavior(s) samples or work behaviors, the strongand/or important work behavior(s) er is the basis for showing content vaconstituting most of the job.
lidity. As the content of the selection (3) Development of selection proce- procedure less resembles a work bedures. A selection procedure designed havior, or the setting and manner of to measure the work behavior may be the administration of the selection developed specifically from the job procedure less resemble the work situand job analysis in question, or may ation, or the result less resembles a have been previously developed by the work product, the less likely the selecuser, or by other users or by a test tion procedure is to be content valid, publisher.
and the greater the need for other evi(4) Standards for demonstrating con. dence of validity. tent validity. To demonstrate the con- (5) Reliability. The reliability of setent validity of a selection procedure, lection procedures justified on the a user should show that the basis of content validity should be a behavior(s) demonstrated in the selec- matter of concern to the user. Whention procedure are a representative ever it is feasible, appropriate statistisample of the behavior(s) of the job in cal estimates should be made of the requestion or that the selection proce- liability of the selection procedure. dure provides a representative sample (6) Prior training or experience. A of the work product of the job. In the requirement for or evaluation of specase of a selection procedure measur- cific prior training or experience based ing a knowledge, skill, or ability, the on content validity, including a specifiknowledge, skill, or ability being meas cation of level or amount of training ured should be operationally defined or experience, should be justified on In the case of a selection procedure the basis of the relationship between measuring a knowledge, the knowledge the content of the training or experibeing measured should be operational- ence and the content of the job for ly defined as that body of learned in which the training or experience is to formation which is used in and is a be required or evaluated. The critical necessary prerequisite for observable consideration is the resemblance beaspects of work behavior of the job. In tween the specific behaviors, products, the case of skills or abilities, the skill knowledges, skills, or abilities in the or ability being measured should be experience or training and the specific operationally defined in terms of ob- behaviors, products, knowledges, skills, servable aspects of work behavior of or abilities required on the job, wheththe job. For any selection procedure er or not there is close resemblance bemeasuring a knowledge, skill, or abili. tween the experience or training as a ty the user should show that (a) the whole and the job as a whole. selection procedure measures and is a (7) Content validity of training sucrepresentative sample of that knowl. cess. Where a measure of success in a edge, skill, or ability; and (b) that training program is used as a selection knowledge, skill, or ability is used in procedure and the content of a train
ing program is justified on the basis of construct(s) believed to underlie succontent validity, the use should be jus- cessful performance of these critical tified on the relationship between the or important work behaviors in the content of the training program and job or jobs in question. Each construct the content of the job.
should be named and defined, so as to (8) Operational use. A selection pro- distinguish it from other constructs. If cedure which is supported on the basis a group of jobs is being studied the of content validity may be used for a jobs should have in common one or job if it represents a critical work be- more critical or important work behavhavior (i.e., a behavior which is neces- iors at a comparable level of complexsary for performance of the job) or ity. work behaviors which constitute most (3) Relationship to the job. A selecof the important parts of the job.
tion procedure should then be identi(9) Ranking based on content valid fied or developed which measures the ity studies. If a user can show, by a job construct identified in accord with analysis or otherwise, that a higher subparagraph (2) above. The user score on a content valid selection pro- should show by empirical evidence cedure is likely to result in better job that the selection procedure is validly performance, the results may be used related to the construct and that the to rank persons who score above mini- construct is validly related to the permum levels. Where a selection proce- formance of critical or important work dure supported solely or primarily by behavior(s). The relationship between content validity is used to rank job the construct as measured by the secandidates, the selection procedure lection procedure and the related work should measure those aspects of per- behavior(s) should be supported by formance which differentiate among empirical evidence from one or more levels of job performance.
criterion-related studies involving the D. Technical standards for construct job or jobs in question which satisfy validity studies—(1) Appropriateness the provisions of section 14B above. of construct validity studies. Con (4) Use of construct validity study struct validity is a more complex strat. without new criterion-related eviegy than either criterion-related or dence.-(a) Standards for use. Until content validity. Construct validation such time as professional literature is a relatively new and developing pro- provides more guidance on the use of cedure in the employment field, and construct validity in employment situthere is at present a lack of substan- ations, the Federal agencies will tial literature extending the concept accept a claim of construct validity to employment practices. The user without a criterion-related study should be aware that the effort to which satisfies section 14B above only obtain sufficient empirical support for when the selection procedure has been construct validity is both an extensive used elsewhere in a situation in which and arduous effort involving a series a criterion-related study has been conof research studies, which include cri- ducted and the use of a criterion-relatterion related validity studies and ed validity study in this context meets which may include content validity the standards for transportability of studies. Users choosing to justify use criterion-related validity studies as set of a selection procedure by this strat forth above in section 7. However, if a egy should therefore take particular study pertains to a number of jobs care to assure that the validity study having common critical or important meets the standards set forth below. work behaviors at a comparable level
(2) Job analysis for construct valid of complexity, and the evidence satisity studies. There should be a job anal- fies subparagraphs 14B (2) and (3) ysis. This job analysis should show the above for those jobs with criterion-rework behavior(s) required for success- lated validity evidence for those jobs, ful performance of the job, or the the selection procedure may be used groups of jobs being studied, the criti- for all the jobs to which the study percal or important work behavior(s) in tains. If construct validity is to be genthe job or group of jobs being studied, eralized to other jobs or groups of jobs and an identification of the not in the group studied, the Federal