This

for length and type of experience or written tests as a measure of qualifications. One real measure of qualifications is actual performance on the job. obviously would not be practical as a recruiting or testing device. No effective method has been devised to determine the equivalency of experience to college education. Because these methods of evaluating qualifications deal with more or less tangible items, the intangible requirements which may be equally important are overlooked. Because of this the applicants attracted to the Federal service are frequently not the best qualified available. It has been our experience that frequently employees with demonstrated ability may not be utilized to a maximum because the standard would not permit a change without a serious monetary loss to the employee. Many employees in the clerical field are precluded from entering the so-called technical or professional positions because of the standards established, even though they demonstrated marked ability and potential, since the type of experience they have had is not considered qualifying. It would appear that if the qualification standards are the best that can be devised for a guide to outside recruiting there should at least be some flexibility whereby the agency can take advantage of clearly established potential capacity and ability.

The qualification standards presently in effect have chiefly been developed without adequate recognition of the needs and changes in the various agencies and geographic localities in which Federal activities are situated. Further there has been no direct effort made to fully validate the validity of the combination of experience and training or education established as the requirements for various occupational categories. As a direct result, the qualification standards set forth in Civil Service Handbook X-118 are not responsive to our needs unless authority to lower, raise or otherwise amend is available.

One of the largest single employers in the Federal Government had this to say on qualification standards:

Exist

This is one of the more serious problems in the entire present structure. ing standards, as set down in Handbook X-118, have a deceiving appearance of exactness and specificity. The weight of evidence indicates that they are not sufficiently relevant to true ability to perform actual jobs:

1. Evidences of inaccuracy

(a) Length of time and experience requirements are disturbingly uniform for all grade levels and occupations, as though it took a carpenter as long to learn his trade as a plumber, or a textile man as long as a nuclear physicist.

(b) Lowering these standards, as is done when shortages occur or when waivers are approved, does not result in poorer employees. Our experience gives the edge to those employees hired on such waivers as far as performance goes. Different factors, however, entered into these selections. We do not claim that anyone with lesser qualifications would be better-simply that the factors now used do not measure ability and that other factors do.

(c) Quality of experience and individual differences in applicants appear to weigh more heavily than length of experience. A carpenter who has worked for 2 years for a good contractor and who has been well trained in all phases of his work is a better selection for us than one who has worked for ten years without such development. A man who can think creatively on research problems is a better bet for us on research work than one who has spent years applying formulae developed by others to standard situations.

(d) Existing standards tend to limit creditable experience to the same occupation as the job being filled. Actually, skill and ability are much more fluid than this. Attached as Tab F is a description of the experience of Douglas Aircraft in finding tool design ability in hitherto untapped occupations while finding also that lack of form and space relationship preception ruled out many people with the experience the company had formerly specified in their selection standards. (e) Employees who have been detailed to fill in on jobs for which they would not qualify under present standards have turned in outstanding performances. For example: a wage board employee at White Sands has devised controls to keep buildings at the exactly even temperatures needed and can do a WB-17 level job of maintaining them. We cannot get him qualified above WB-7. His is by no means an unusual case.

(f) Applicants who have solved on other jobs the same type and level of problems as ours cannot be employed. A full example of one such case is shown

below.12 He was disqualified because the Commission could approve only the experience he gained after he got his belated degree and that experience only as so many years of time and not as so many research problems just like ours successfully solved. Additional examples of disqualification of able men and excellent performance of people hired as exceptions or detailed to fill in on higherlevel jobs can be supplied.

2. Apparent causes of inaccuracy

A part of the inaccuracy of existing standards appears to stem from the way in which they have been developed:

(a) Those preparing them are remote from actual jobs and actual job content. Accordingly, the standards seem to be applied more to craft and professional fields than to our positions. Yet it is our positions which must be filled.

(b) As mentioned earlier, these standards have been applied to broad classes, some as broad as a whole occupational field. They do not readily permit of adjustment to fit a position which deals with a specialized aspect of a field or with several different fields. A man who has the abilities needed must sometimes compete with others who, while they have "qualifications" do not have those specifically required.

The job

(c) Standardization appears to have been interpreted as uniformity and the Commission seems to have gone to some lengths to assure uniformity. Uniform standards are useful only to the extent actual job content is uniform. that differs gets its true requirements twisted all out of shape by such uniform standards. If true merit means hiring the best man for the job to be done, invalid standards would appear to damage the merit principle.

(d) Present standards are based on opinion. It is claimed that this is the opinion of experts. The procedure assumes that a technical expert can discern just which of several aspects of an employee's background and native ability gave him a recognized level of performance. Our experience shows that the following situations occur frequently:

(1) The standards closely resemble the experts' own background.

(2) The standards eliminate those who do not qualify for a craft or professional organization.

(3) Employees whom the expert considers outstanding do not, on examination, prove to have the qualifications specified.

(4) The expert, in his own hiring, forgets his standards when he finds a man who can answer his questions well. Sometimes he forgets them when he goes seeking a man.

In short, the expert can state the problems an employee must face on a job if he knows that specific job well. He has much more difficulty in generalizing with any degree of validity on what background a man must have. When he tries to generalize about many jobs that he does not know closely, he tends to revert to the traditions of his trade or profession.

whom

12 This is a case history of an applicant for a position as electronics engineer, GS-11, at we were unable to employ because of existing standards. Electronic engineering is a shortage skill with us, with many unfilled vacancies and a situation in which we are losing ground in trying to keep our jobs filled. This case is not an unusual example. It is relatively typical of some 500 cases over the past year and of thousands of cases over the past 5 years.

The applicant had completed 2 years of electrical engineering training at the University of Michigan in 1940. He was called to military service. In the service, he attended radar school at MIT and the Canadian Army radar school at Kingston, Ontario. He did design work on antennas at MIT and was a radar evaluation officer at Eglin Field, Fla., on remote control aircraft. Leaving the service in 1945, the applicant was engineer in charge of construction of a radio broadcasting station in Michigan. In 1949 he took his bachelor's degree in electrical engineering.

Called back into the service in 1950, the applicant worked under our observation at one of our bases on ultra-high-frequency work in electronic fuse development. Before his release from the service, we offered him the GS-11 post at another of our installations.

Our ultra-high-frequency expert at the installation personally interviewed the applicant and discussed a number of problems with him to test his experience and ability in the electronics field and the methods he had sed to solve electronic problems similar to ours. Our expert was well satisfied that the applicant could perform the work of our electronic engineer at the GS-12 level, above the level for which application was made, based on this examination and on the work done on his last job.

The applicant's papers were sent to Denver for rating by the Civil Service Commission. Because the applicant had only 11⁄2 years of experience since obtaining his degree, he was rated as eligible only for GS-9. It did not matter that his war experience was in line with our needs or that the 112 years was experience on problems of the same nature as those of the job to be filled. Two appeals were made, one in person to Denver but the Commission decision stood.

The applicant then applied to the Western Electric Co., which had a contract for research work at one of our installations along lines requiring the same abilities as our jobs. He was immediately appointed at $7,500 per year plus a $60 a week cost-of-living allowance, a salary much above our GS-11 offer. He has had one raise since this appointment. When contacted, the applicant's present supervisor informed us that Mr. is "one of the best men they have had in a long time."

In short, a time limit which is uniform for any and all professions was held to be a more valid index of this man's ability than demonstrated ability in our own service on similar work and demonstrated ability to solve our problems when presented in an examination, borne out later by demonstrated performance on like work for an industrial concern.

(e) Present standards, as far as we know, have never been validated by the Civil Service Commission against actual job success.

3. A suggested method for improving accuracy

It is suggested that the problem of setting standards may be overcome by changing the factors employed. In lieu of the present time and occupation factors, the standards might well be based on the job situations which will face the employee on the actual job to be done. The applicant must be able to demonstrate that he has either solved similar problems or dealt successfully with similar situations in his past experience or be able to pass an examination which confronts him with them.

This "critical incident" or "critical situation" method is the one which, in our experience, the experts tend to use when they make their own actual selections. They pose for the applicant problems that are typical of those he will face. A standardized use of this technique, used by the Wisconsin Office of OPA during World War II, resulted in a force which produced 3 to 1 over regional averages in terms of cases prepared well enough to win court victories and was consistently first in the agency's national statistics on production.

A study of the possibilities of the critical situation technique of standard setting and examining is suggested.

4. Placing responsibility for setting standards

Since standards must relate closely to actual job content if they are to measure true merit, the responsibility for setting them should be placed as close to the actual jobs to be filled as possible. This means that responsibility should rest with the agencies, and that it should be delegated within agencies to the lowest level that trained competence and sound control of abuse will permit. It is felt that this delegation can be made to the station level if adequate criteria and guidance are developed, adequate training is given to local people, and frequent reviews of local actions are made on a post audit basis.

There appear to be three principal methods which may be used in setting qualification standards.

(1) Statements of required experience and education. This is the method utilized in the present system of setting qualification standards. (2) Statements of required knowledges, skills, abilities and personal traits. To some extent, these factors enter into written and performance tests as explained earlier in this report.

(3) Statements of job situations which must be met or must previously have been met successfully. This method, sometimes referred to as the "critical incident" technique, is not utilized at all by the Civil Service Commission, nor is its use authorized. A couple of agencies have, however, undertaken pilot or developmental studies of this method of setting qualification standards.

As a part of the Civil Service Commission's "quality examining program" (described in a succeeding section of this report) which was proposed in July 1950, the Commission contemplated exhaustive research into all known techniques for setting qualification standards. Unfortunately, this program was abandoned at the outset of the Korean emergency.13

Our analysis of the present system of qualification standards in

13 Note agency comments on need for such study:

"If the Civil Service Commission would concentrate on doing a first-rate job for the agencies many of the complaints voiced on qualification standards would disappear. The Civil Service Commission should concentrate on issuing 'standards for standards' which will meet the legislative requirement of uniformity yet, contain enough built-in flexibility to give the agencies some freedom of operation in the field of qualification standards. In this manner, the Civil Service Commission would free itself from the burdensome job of constructing a rigid standard for each job or job category. Free of this time-consuming task the Commission staff would have an opportunity to re-examine the entire area of qualification standards with a view toward developing a system suited to the needs of the service.

"We recommend, therefore, that the Civil Service Commission concentrate its activities to preparing and issuing guidelines to be used by the various agencies in establishing qualification standards that will be responsive to their needs and properly geared to meet situations peculiar to their respective missions and localities. This procedure is, in our opinion, consistent with Civil Service Commission policies favoring decentralization of operating activities to local boards and Civil Service Commission regional offices. Such action would enable the Civil Service Commission to spend more time in providing staff assistance and guidance by freeing it of operating details which can be performed more realistically on a decentralized basis."

light of the goal which the standards are intended to achieve the accurate measurement of job applicants-leads to the following conclusions:

(1) The present method of setting qualification standards is not accurate for all Government jobs in all situations. To the extent of this inaccuracy in each situation, merit principles are not served and the best qualified available people are not being placed in Government jobs.

(2) A part of the inaccuracy of the present system is due to the policy of the Commission in attempting to construct or centrally approve, individual standards which, in the process of central development or approval, lose work level meaning. Basically, realistic qualification standards can be fixed only at the work level. Any change or modification thereafter is at the expense of accuracy. Moreover, this centralized control is time-consuming and costly in terms of delayed production.

Recommendations

1. The Civil Service Commission should, at the earliest practicable date, undertake a research program designed (a) to test the validity of methods currently used in setting qualification standards; (b) to evaluate and explore the use of other existing techniques and methods not currently employed by the Government; and (c) to explore the development of new techniques and methods for measuring job qualifications.

To the extent found feasible and practical, the results of this program of research should be applied to replace or supplement present methods for setting qualification standards.

We do not agree that this program should be put off until the termination of the present emergency. In fact, we consider the expansion caused by the current emergency to be a motivating factor behind this recommendation.

2. The Civil Service Commission should discontinue its practice of constructing and/or approving qualification standards for individual jobs or job categories and concentrate on the establishment of guidelines within which agencies may develop their own qualification standards. The Commission should delegate authority to the local boards of examiners to approve qualification standaras developed at the local level and to amend existing standards. The subcommittee contemplates that the Civil Service Commission will have ample opportunity to check for any abuse of this authority in its periodic audit and inspection of agency personnel programs.

THE USE OF THE INTERVIEW AS A PART OF THE EXAMINING
PROCESS

The Civil Service Commission as stated earlier is reluctant to use the interview as a part of the examining process. In only a few instances may it be used and then only on an "in or out" basis. The applicant is rated eligible or ineligible and no attempt is made to evaluate and rate him based on the interview. The only time the interview may be used as an examining tool is when the situation meets the criteria set forth in civil service regulations:

The standard interview is used only for positions in which the ability to deal effectively with other persons is essential-so essential, in fact, that lack of these qualities proposed to be tested should completely disqualify an applicant.

The Commission elaborates in Handbook X-111:

The interview might be desirable in an examining program for the position of contact representative or administrative officer, while it would contribute very little in the selection of a research technician with no supervisory duties.

Dr. John C. Flanagan,14 however, reporting on his work with the critical incident technique in a paper presented at the Civil Service Assembly in New York stated that a job study for "critical incidents" (incidents critical in terms of success or failure in important aspects of the job) revealed that 24 to 75 percent of the incidents represented motivational and personality factors as differentiated from technical competence.

As mentioned earlier, the junior management assistant examination includes an interview. Actually there are two interviews, a group oral type where the candidates are observed in group discussion and an individual interview. Contact representatives for the Veterans' Administration are also interviewed in the examining process. In any instance where the Civil Service Commission standard interview is used there must be a minimum of two raters.

It is the opinion of the subcommittee that the interview has a place in the formal examining process for a great number of positions in the Federal Government. Present examining procedures examine only a fraction of the whole man. Personality traits and attitudes are extremely important in most jobs. Our examining techniques-the written examination or rating of training and experience-give no weight to job interest, ambition, manner, voice, appearance, and other characteristics and motivational factors which are every bit as important to job success as length of experience and training. In many jobs the above factors mean the difference between success and failure, in most jobs they are the difference between a good employee and a "seat warmer." The Federal Government can ill afford to hire "seat warmers" when there are good applicants available.

The value of the interview is not limited to the evaluation of personality traits and attitudes. It is an excellent instrument for further evaluating the quality of experience and training. Applicants misrepresent their abilities intentionally and, more frequently, they misunderstand the job and misrepresent unintentionally. Furthermore, in applying for Federal employment applicants frequently state their experience in general terms in the hope of gaining consideration for the maximum number of vacant positions. The interview can serve as a means of probing experience claims and evaluating experience in terms of job situations. The applicant may be interrogated on specific job problems and his solutions or reactions observed. Personnel officials in private industry have reported that they consider the personal interview essential in the examining process.is

The interview is, in fact, used in the selection process to complete

14 American Institute of Research, Pittsburgh, Pa.

15 An extreme situation is reported by a large company with an outstanding recruiting program which brings to level recruits to the various divisions of the company. "At that point (after interviews are completed) the test results are brought into the picture and carefully reviewed for each of the candidates interviewed. Occasionally clues will appear in the tests which will initiate a discussion and change opinions previously formulated. However, tests are used simply as clues and, if they cannot be substantiated, they are not relied upon."