The Pilot Curriculum is a long-range experiment
in the College of Arts and Sciences which was designed to produce information
that would be useful determining the configuration of the next undergraduate
curriculum. Developed by the Committee on Undergraduate Education (CUE)
during the academic years 1998 to 2000 and introduced with freshman
classes starting in 2000, the experiment tests an alternative to the
College’s current general education curriculum with a subset
of students in each class. By tracking and evaluating the academic
programs of those students over their entire undergraduate careers
compared with students enrolled in the regular curriculum, the experiment
seeks to understand the effects of the two curricula on the educational
choices students make and on their academic achievements over the course
of their entire undergraduate careers.
For students in the Pilot Curriculum, the College suspended its standard
degree requirements. In place of them, it imposed an alternative set
of requirements characterized by (1) a more concentrated and more compact
set of general education requirements, (2) a corresponding increase
in the number of free electives, (3) an emphasis on planning with an
academic advisor, and (4) a research experience, normally in the context
of the student’s major.
1. Pilot General Requirement. In place of the ten-course General Requirement
of the regular curriculum, Pilot students take four courses specially
designed to introduce students to interdisciplinary study and to open
up a variety of modes of inquiry. A few such courses exist in the regular
General Requirement. However, the General Requirement courses taken
by the vast majority of College students are designed to provide a
comprehensive introduction to a single major, to cover the territory
of that field from a certain vista, and to draw students into further
study in that field.
2. Free electives. Theoretically, the decrease in required general
requirement courses should afford pilot students a corresponding increase
in free electives. In practice, however, this increase was offset to
some degree for some students by the fact that there are fewer opportunities
to double count Pilot general requirement courses toward major and
pre-medical requirements. (This may change as departments, under pressure
from students, consider whether to count various Pilot courses toward
the major.) Also, students were not able to substitute AP credit for
Pilot general requirement courses as they can with a number of required
courses in the regular general requirement. Consequently, some Pilot
students found that they had no more free electives or flexibility
in their schedules than they would have had if they had enrolled in
the regular curriculum.
3. Academic Plan. In general, arrangements for academic advising for
students in the Pilot Curriculum are similar to arrangements for students
in the regular curriculum. Like students in the regular curriculum,
students are required to meet with their academic advisors at least
three times during the freshman year. Unlike students in the regular
curriculum, however, Pilot students are expected to prepare a written
academic plan in consultation with their advisor. Discussions about
the plan are normally concluded in the second semester of the sophomore
year, which is also the deadline for all students to declare a major.
The advisor’s signature is required, not to indicate approval
of the plan but rather to signify that the student has been sufficiently
reflective about his or her goals to commit them to writing and discuss
them with a more mature scholar and has given thought to observations
and suggestions the advisor may have about the plan.
4. The Research Requirement. The purpose of the research requirement
is to ensure that all students in the Pilot gain some degree of hands-on
experience with the processes of discovering and validating knowledge
in at least one field of study. The normal expectation is that this
experience takes place within the field that one has studied most thoroughly,
namely, the major. Nevertheless, some students have arranged to meet
this requirement outside their major.
In summary, the objective of the experiment is to see how Pilot students
as compared with students in the regular curriculum make use of this
alternative curriculum to shape interesting, intellectually engaged
and coherent programs of study.
It is unusual for a college to implement a new curriculum on an experimental
basis for only subset of potential students. It is, perhaps, unique
that the College chose to implement this curriculum as a true,
randomized experiment. Each May starting in 2000, the College sent
to the approximately 1600 matriculating freshmen describing the
Pilot and the regular curricula and inviting them to apply for
Curriculum or to indicate that they would pursue the regular curriculum.
The brochure explained that 200 students admitted each year to
the Pilot Curriculum would be selected at random from the applicants.
Comparison of the applicants versus the non-applicants revealed
significant differences in the standard academic indicators (SAT-V,
SAT-M, a “Predictive Index” computed by the Penn Admissions
Office to predict students’ first semester GPAs). No bio-demographic
differences (sex, ethnicity, etc.) were evident except for a slightly
higher proportion of first-year Pilot applicants whose country
of origin was outside the United States.
In each year, the goal was to recruit a pool of 400 applicants, from
which to randomly select 200 for the Pilot Curriculum and 200 for the
primary control group, namely, those who applied for the Pilot Curriculum
but who were not selected at random. In some years, as shown in Table
1, the number of applicants was substantially less than 400, leaving
a control group somewhat smaller than 200. In all years, the selection
method was by simple random sampling (without blocking).
Although the virtues of a randomized experiment will be evident to
many, it is worth emphasizing what this methodology accomplishes. If
we had merely selected the first 200 applicants in each cohort and
compared them with all other students in their cohort, any differences
we observed in their educational outcomes could potentially be attributed
to differences in such things as attitudes, motivations, and abilities
that existed prior to their arrival at Penn. On the other hand, since
participants in the Pilot Curriculum were randomly chosen from among
all who applied, differences between them and the applicants who were
not chosen can be only be attributed to their differing experiences
In many of our research projects, we compared three groups of students:
Pilot students, students who applied for the Pilot Curriculum but were
not selected, and students who did not apply for the Pilot Curriculum.
These comparisons are referred to as comparisons by Pilot status.
Table 1. Numbers of Applicants and Non-applicants to Pilot Curriculum
||Total Responses to Invitation
||Selected for Pilot
|Class of 2004
|Class of 2005
|Class of 2006
|Class of 2007
In the spring of 2000, CUE called for the creation of a separate and
independent Pilot Curriculum Evaluation Committee consisting of four
faculty members and a student. Three faculty candidates for the committee
were vetted by the Undergraduate Chairs. The fourth candidate was selected
by CUE from among those of its own members whose terms were about to
expire and who were intimately acquainted with the formulation and
development of the idea for the Pilot Curriculum.
The Evaluation Committee has been accumulating data for this experiment
over the past three and a half years not just with the first class
of Pilot students but also with each subsequent class. (The last Pilot
class will matriculate in the fall of 2004.) Even for the first class,
though, the data are incomplete, and the picture we have of what differences
this curriculum makes will not be complete until after the first class
is graduated at the end of the current academic year.
The Evaluation Committee decided last spring (2003) to prepare this
interim report during the fall semester as a “dry run” for
the final report that it is expected to submit to the Dean of the College
and CUE sometime after the end of this academic year (2003-2004). This
report is a preview of the general form of the final report but not
necessarily a preview of its content. Its purpose is just to share
with the Dean and with CUE the general form the final report is expected
to take. It will present the range of questions the final report will
address and the kinds of assessment the committee has done to date,
as well as our findings thus far.
This report is not intended to make definitive conclusions that will
appear in the final report, for three reasons. First, as indicated
above, we have not yet gathered and analyzed all the data on the class
of 2004. Second, we believe it is essential to replicate and compare
our results with data on the class of 2005, but essential data on that
class will not become available until late September 2005. This comparison
is particularly important because some features of the Pilot experience
(e.g., the prevalence of team teaching in Pilot General Requirement
Courses) changed over time. Third, part of the purpose of sharing the
report at this time is to get feedback from the Dean and from CUE about
what they may find missing from the report. What questions should the
committee be asking that it has not asked so far? What additional analyses
should the committee perform? By sharing the interim report at this
time, the Evaluation Committee is open to feedback that could change
both the form and the substantive conclusions of the final report.
Over the next several months, the Committee plans to undertake the
following additional projects:
- Additional statistical analysis of student records:
- Choice of major by Pilot status.
- Courses taken outside of SAS by Pilot status.
- Number of courses taken in major subject by Pilot status.
- A survey designed to assess the quality of the research experience
for Pilot students and other seniors.
- Content analysis of Academic Plans for the Class of 2005.
- Senior survey in the Fall of 2005: to include Science Survey and
overall evaluations of the educational experience.
- Identification of regular General Requirement courses that are
similar in structure and content to Pilot General Requirement courses.
- Analysis of conventional course evaluation data to determine how
course ratings relate to whether the courses are taken to satisfy
This interim report is organized in the following way. We begin with
a summary report that presents our interim findings in each of several
content areas: Pilot General Requirement courses, advising, patterns
of course choices, science education, and the research requirement.
This is followed by more detailed descriptions of the various research
projects we conducted over the past three and one-half years, along
with discussions of specific findings. In the summary report, numbers
in parentheses refer to the section numbers of the research projects
that provided the relevant information.