The Pilot Curriculum is a long-range experiment in the College of Arts and Sciences which was designed to produce information that would be useful determining the configuration of the next undergraduate curriculum. Developed by the Committee on Undergraduate Education (CUE) during the academic years 1998 to 2000 and introduced with freshman classes starting in 2000, the experiment tests an alternative to the College’s current general education curriculum with a subset of students in each class. By tracking and evaluating the academic programs of those students over their entire undergraduate careers compared with students enrolled in the regular curriculum, the experiment seeks to understand the effects of the two curricula on the educational choices students make and on their academic achievements over the course of their entire undergraduate careers.

For students in the Pilot Curriculum, the College suspended its standard degree requirements. In place of them, it imposed an alternative set of requirements characterized by (1) a more concentrated and more compact set of general education requirements, (2) a corresponding increase in the number of free electives, (3) an emphasis on planning with an academic advisor, and (4) a research experience, normally in the context of the student’s major.

1. Pilot General Requirement. In place of the ten-course General Requirement of the regular curriculum, Pilot students take four courses specially designed to introduce students to interdisciplinary study and to open up a variety of modes of inquiry. A few such courses exist in the regular General Requirement. However, the General Requirement courses taken by the vast majority of College students are designed to provide a comprehensive introduction to a single major, to cover the territory of that field from a certain vista, and to draw students into further study in that field.

2. Free electives. Theoretically, the decrease in required general requirement courses should afford pilot students a corresponding increase in free electives. In practice, however, this increase was offset to some degree for some students by the fact that there are fewer opportunities to double count Pilot general requirement courses toward major and pre-medical requirements. (This may change as departments, under pressure from students, consider whether to count various Pilot courses toward the major.) Also, students were not able to substitute AP credit for Pilot general requirement courses as they can with a number of required courses in the regular general requirement. Consequently, some Pilot students found that they had no more free electives or flexibility in their schedules than they would have had if they had enrolled in the regular curriculum.

3. Academic Plan. In general, arrangements for academic advising for students in the Pilot Curriculum are similar to arrangements for students in the regular curriculum. Like students in the regular curriculum, students are required to meet with their academic advisors at least three times during the freshman year. Unlike students in the regular curriculum, however, Pilot students are expected to prepare a written academic plan in consultation with their advisor. Discussions about the plan are normally concluded in the second semester of the sophomore year, which is also the deadline for all students to declare a major. The advisor’s signature is required, not to indicate approval of the plan but rather to signify that the student has been sufficiently reflective about his or her goals to commit them to writing and discuss them with a more mature scholar and has given thought to observations and suggestions the advisor may have about the plan.

4. The Research Requirement. The purpose of the research requirement is to ensure that all students in the Pilot gain some degree of hands-on experience with the processes of discovering and validating knowledge in at least one field of study. The normal expectation is that this experience takes place within the field that one has studied most thoroughly, namely, the major. Nevertheless, some students have arranged to meet this requirement outside their major.
In summary, the objective of the experiment is to see how Pilot students as compared with students in the regular curriculum make use of this alternative curriculum to shape interesting, intellectually engaged and coherent programs of study.

Experimental Design

It is unusual for a college to implement a new curriculum on an experimental basis for only subset of potential students. It is, perhaps, unique that the College chose to implement this curriculum as a true, randomized experiment. Each May starting in 2000, the College sent brochures to the approximately 1600 matriculating freshmen describing the Pilot and the regular curricula and inviting them to apply for the Pilot Curriculum or to indicate that they would pursue the regular curriculum. The brochure explained that 200 students admitted each year to the Pilot Curriculum would be selected at random from the applicants. Comparison of the applicants versus the non-applicants revealed no significant differences in the standard academic indicators (SAT-V, SAT-M, a “Predictive Index” computed by the Penn Admissions Office to predict students’ first semester GPAs). No bio-demographic differences (sex, ethnicity, etc.) were evident except for a slightly higher proportion of first-year Pilot applicants whose country of origin was outside the United States.

In each year, the goal was to recruit a pool of 400 applicants, from which to randomly select 200 for the Pilot Curriculum and 200 for the primary control group, namely, those who applied for the Pilot Curriculum but who were not selected at random. In some years, as shown in Table 1, the number of applicants was substantially less than 400, leaving a control group somewhat smaller than 200. In all years, the selection method was by simple random sampling (without blocking).

Although the virtues of a randomized experiment will be evident to many, it is worth emphasizing what this methodology accomplishes. If we had merely selected the first 200 applicants in each cohort and compared them with all other students in their cohort, any differences we observed in their educational outcomes could potentially be attributed to differences in such things as attitudes, motivations, and abilities that existed prior to their arrival at Penn. On the other hand, since participants in the Pilot Curriculum were randomly chosen from among all who applied, differences between them and the applicants who were not chosen can be only be attributed to their differing experiences at Penn.

In many of our research projects, we compared three groups of students: Pilot students, students who applied for the Pilot Curriculum but were not selected, and students who did not apply for the Pilot Curriculum. These comparisons are referred to as comparisons by Pilot status.

Table 1. Numbers of Applicants and Non-applicants to Pilot Curriculum

Evaluation Committee

In the spring of 2000, CUE called for the creation of a separate and independent Pilot Curriculum Evaluation Committee consisting of four faculty members and a student. Three faculty candidates for the committee were vetted by the Undergraduate Chairs. The fourth candidate was selected by CUE from among those of its own members whose terms were about to expire and who were intimately acquainted with the formulation and development of the idea for the Pilot Curriculum.

The Evaluation Committee has been accumulating data for this experiment over the past three and a half years not just with the first class of Pilot students but also with each subsequent class. (The last Pilot class will matriculate in the fall of 2004.) Even for the first class, though, the data are incomplete, and the picture we have of what differences this curriculum makes will not be complete until after the first class is graduated at the end of the current academic year.

The Evaluation Committee decided last spring (2003) to prepare this interim report during the fall semester as a “dry run” for the final report that it is expected to submit to the Dean of the College and CUE sometime after the end of this academic year (2003-2004). This report is a preview of the general form of the final report but not necessarily a preview of its content. Its purpose is just to share with the Dean and with CUE the general form the final report is expected to take. It will present the range of questions the final report will address and the kinds of assessment the committee has done to date, as well as our findings thus far.

This report is not intended to make definitive conclusions that will appear in the final report, for three reasons. First, as indicated above, we have not yet gathered and analyzed all the data on the class of 2004. Second, we believe it is essential to replicate and compare our results with data on the class of 2005, but essential data on that class will not become available until late September 2005. This comparison is particularly important because some features of the Pilot experience (e.g., the prevalence of team teaching in Pilot General Requirement Courses) changed over time. Third, part of the purpose of sharing the report at this time is to get feedback from the Dean and from CUE about what they may find missing from the report. What questions should the committee be asking that it has not asked so far? What additional analyses should the committee perform? By sharing the interim report at this time, the Evaluation Committee is open to feedback that could change both the form and the substantive conclusions of the final report.

Over the next several months, the Committee plans to undertake the following additional projects:

• Additional statistical analysis of student records:
  • Choice of major by Pilot status.
  • Courses taken outside of SAS by Pilot status.
  • Number of courses taken in major subject by Pilot status.
• A survey designed to assess the quality of the research experience for Pilot students and other seniors.
• Content analysis of Academic Plans for the Class of 2005.
• Senior survey in the Fall of 2005: to include Science Survey and overall evaluations of the educational experience.
• Identification of regular General Requirement courses that are similar in structure and content to Pilot General Requirement courses.
• Analysis of conventional course evaluation data to determine how course ratings relate to whether the courses are taken to satisfy a requirement.

This interim report is organized in the following way. We begin with a summary report that presents our interim findings in each of several content areas: Pilot General Requirement courses, advising, patterns of course choices, science education, and the research requirement. This is followed by more detailed descriptions of the various research projects we conducted over the past three and one-half years, along with discussions of specific findings. In the summary report, numbers in parentheses refer to the section numbers of the research projects that provided the relevant information.