Using and abusing the GRE in graduate admissions

I preface this statement with an acknowledgement that my views here are intended to stimulate discussion at Texas A&M–Texarakana and do not apply to my feelings about the use of the GRE at other institutions.

I have been asked to revisit the Graduate Record Exam (GRE) requirement for admission into the ITED program at TAMUT. I have tried to address the issue with as much clarity as possible, and have concluded, after closer inspection, that the policy of using the GRE as part of admissions criteria is not only inappropriate for TAMUT’s ITED program, but is also potentially misleading in making admissions decisions. This statement lays out my rationale and a proposed replacement to the GRE requirement.

There have generally been four purposes of the GRE in the admissions process. Those reasons are:

The industry surrounding improving test scores for admissions committees that will likely never read them.

(1.)  As a minimum cut-off used in order to manage large numbers of applications for programs where the application pool is larger than department or universities’ resources allotted to review applications.

(2.)  As a predictor of success in graduate school in order to locate learners who may need additional support once admitted;

(3.)  As an additional point of observation to compare similar boarder-line admissions candidates;

(4.)  As evidence of sincerity in a candidate’s desire for admittance to graduate study. I will address each point separately.

1. A minimum cutoff to manage large application pools

The ITED program is not presently overwhelmed with applications. To make a statement that this is the purpose for the requirement would be unethical. In an effort to make our admissions as transparent as possible, we should exclude this purpose from our application statements. This purpose is also advised against by the test developers: “A cutoff score based solely on GRE scores should never be used as the sole criterion for denial of admissions.” (from the ETS website.) Using the test in this way is misuse.

2. As a predictor of success

The test does not assess the skills associated with the program, and in doing so, provides no relevant measure of potential success in the program. ITED is a design program, but the test measures other types of knowledge not sufficient in the process of design. Design programs train learners to uncover unforeseen solutions, while the test measures how well a learner can mimic the test creator’s logic. Furthermore, even for more strictly academic programs, the use of the test as a predictor has been widely discredited (This Chronicle of Higher Education article and dozens of others).  Previous studies show that the GRE only explains 6% of the variation of first year scores, and even has negative correlations with graduation rates (Onash 1994). Using the test to predict success in the ITED program hinders our vision of who might actually be the most promising candidates because even high scores will have no relevance to program success.

3. Identifying students who may need support after admittance

The GRE measures analytical writing, verbal (primarily English language manipulations) and high school mathematics. Apart from the skill of constructing coherent and complex arguments explaining rationale, most of the areas measured on the GRE fail to align with problem areas common to the ITED program—difficulties in understanding and applying instructional and learning theories, and the technical aspects of computing and digital design.  Since the GRE does not identify problem areas, another criterion would be more appropriate for this purpose. For example, learners who have difficulty with digital skills could be identified early on via a digital artifact. They could be required to take a TAMUT basic computing course or web design course as a condition of enrollment. Using the GRE weakens our ability to detect these deficiencies in preparation for graduate study in Instructional Technology.

4. Evidencing sincerity in desire for graduate study

Were it the case that all applicants had access to the same financial resources, this might be a plausible use of the GRE. However, having no other purpose, imposing this financial barrier on the population that TAMUT is situated to serve is suspect on a number of levels. The price of the test may be equivalent in dollar value to each learner, but in terms of percentage of resources the cost may vary dramatically from learner to learner. Thus the practice of requiring the GRE may favor more financially well off students, and obfuscate the institution’s vision of the most qualified students. This dynamic works against our mission to provide equitable access to learning, and on these grounds cannot be assumed to discriminate fairly among candidates.

An alternative requirement: A designed artifact

In comparison to other institutions, an equal number use and forgo the GRE as a requirement. Among those that do require the test, we do not know why they use it. I can only conclude that the negative aspects of using the GRE on admission requirements outweigh the positive aspects of requiring the test. An alternative measure might provide the additional level of granularity some programs are searching for in their requirement of completion of the GRE. Thus, an alternative may prove to be a more useful and appropriate means of assessment for admittance.

One possible alternative is requiring a designed artifact. Requiring a designed artifact holds the potential to address the skills we hope to enrich in the ITED program- applied creativity and innovation in creating learning. An artifact of design can clue us into a candidate’s ingenuity, their ability to take information and apply it to real world contexts, and a learner’s tenacity to complete a design task. The practice may avoid the caveats presented with using the GRE as a criterion for admissions. Requiring a designed artifact may weed out learners who have little or no concept that ITED is a program where the creation of materials for learning is the primary objective, thus narrowing the pool to those who have a concept and desire to study in the program for which they are applying. A designed artifact also directly aligns with the learning objectives in the program and in so provides a more appropriate measure of potential success. Unlike the GRE, a designed artifact may expose learners’ limits in relevant skill areas, thus pointing to better means of preparing learners for graduate study in Instructional Technology in the early stages. And finally, requiring a designed artifact is a non-prejudicial practice, discriminating on relevant experience and knowledge as opposed to financial resources.

References:

Onasch, C. (1994). Undergraduate Grade Point Average and Graduate Record Exam Scores as Predictors of Length of Enrollment in Completing a Master of Science Degree. ERIC Document No. 375 739.

Sacks, P. (June 8, 2001) How Admissions Tests Hinder Access to Graduate and Professional Schools. The Chronicle of Higher Education.

Addendum on 4/3/2012

I presented the argument above to the education faculty in March. Subsequently, through consultations with Dr Bill McHenry, an alternative course of action was decided– an interview with supporting rubric for scoring the interview. The tentative draft of the rubric for scoring is as follows:

On April 10, this rubric was tested with a real, live, breathing student. Our ratings came within .5 point of each other. We fumbled with various questions which brought the rubric into a reality within th context of the interview into a reality. Kevin and I met for an hour afterwards to hash out some of the questions we felt produced the most clear cut and score-able answers. Those refined questions are:

1a. Describe something you designed?

1b. Describe how you use technologies you use now—in your teaching or other creative things that you do.  

2a. Explain how you handled a technological change in your work or creative endeavor.

2b. Describe a poor technological choice or poor technological implementation strategy in a learning context. How would you have done it differently?

3a. Where you see yourself in the field of Instructional Technology? How does what you like to do match with the tasks people who hold positions in this field actually do?

3b. If accepted, how will this degree help you meet your professional goals?