Paul D. Allison, Chair
Jacob Cytryn
Irma Elo
Robert Perelman
D. Kent Peterman
Ingrid Waldron

Comments should be directed to Paul Allison, allison@soc.upenn.edu

Pilot General Requirement Courses

For the General Requirement component of the Pilot Curriculum, students choose from a small set of interdisciplinary courses selected and designed for the purpose of general education. Unlike most courses in the regular General Requirement, Pilot Curriculum General Requirement courses are designed to suggest a variety of areas for further study and modes of inquiry rather than to channel students into the field and disciplinary methods of a single major. Students are required to take fewer of these courses, four instead of the ten general requirement courses students must take in the regular curriculum.

The first four Pilot General Requirement courses were offered in the fall of 2000 and had a total of 10 faculty instructors. These faculty were interviewed in spring 2001 about their experiences in the courses (5). Faculty were generally very positive about the courses. Those who taught in teams were particularly enthusiastic about the quality of the intellectual experience, but mentioned the additional work required to prepare for the classes and coordinate their efforts with other members of the team. Similar views were expressed in a forum of faculty who taught Pilot courses in 2000 or 2001.

Students had less favorable views about the Pilot General Requirement Courses. Focus groups of randomly chosen Pilot students held in 2000 and 2001 expressed both positive and negative opinions about the courses (6). Complaints centered on the workload, which some students found excessive, and the lack of integration in the team-taught courses. Many students were unhappy with the limited choice of courses.

To get a quantitative assessment of student satisfaction, supplementary course evaluations were distributed in all Pilot General Requirement courses (2). For courses taught in 2002, we used both standard and supplemental evaluation forms to compare 15 Pilot general requirement courses with a selected sample of 28 general requirement courses taught in the standard curriculum. Based on the standard evaluation forms, there was little if any difference between courses in the two curricula with regard to students’ overall evaluation of either the course or the instructor. For the supplemental evaluation (for which we had reports from only 12 standard curricula courses), the Pilot courses were generally evaluated less favorably than the standard courses, although none of the differences was statistically significant.

Advising

One goal of the Pilot Curriculum was to improve academic advising during the first two years. Specifically, it was hoped that advisors of Pilot students would have more meaningful conversations about academic goals and planning because the reduced number of general education requirements would reduce the need to discuss how these requirements would be filled and because the requirement for Pilot students to prepare an Academic Plan during the sophomore year would focus thoughtful attention on achieving academic and intellectual goals.

Academic advisors for Pilot students reported that they felt this goal was achieved to some extent, although the advisors often felt unprepared to provide practical recommendations in response to questions about academic interests (7). Correspondingly, both Pilot and non-Pilot seniors complained about poorly informed advisors (4), and Pilot students expressed frustration about the lack of information regarding the research requirement (6). Seniors reported considerable dissatisfaction with advising during their first two years at Penn (4). Ratings of satisfaction with advising did not differ significantly by Pilot status.

The preparation of Academic Plans was perceived as helpful by some students, but considered redundant with planning for the major by others (6). The usefulness of the Academic Plan may have been limited by the tendency of students to focus on courses that had already been taken, with relatively little attention devoted to plans for future selection of elective courses (7, 8).

In summary, our evidence suggests only very limited success in the goal of improved advising for students in the Pilot Curriculum. One pervasive problem is how to provide advisors and students with helpful information on the enormous array of academic options available to students in the College. Advising in the Pilot Curriculum framework might be improved by refocusing the Academic Plans on the future more than the past. In addition, the information which is now available on the research requirement for Pilot students may also contribute to reduced student frustration.

Patterns of Course Choices

One of the major goals of the Pilot Curriculum was to allow students greater freedom to choose courses to pursue their intellectual interests by reducing the number of general education courses required. In the student focus groups, students often said that they had applied for the Pilot Curriculum because it has fewer general education requirements and thus provides greater freedom to choose electives than the regular curriculum (6). Nevertheless, few differences in course choices were found in comparisons between Pilot and non-Pilot students in analyses of data from transcripts and students’ self-reports (1, 4). No significant differences were found by Pilot status in the proportion of students who were earning double majors, dual degrees, sub-matriculation masters, or minors, and no significant differences were found in measures of study abroad, foreign language learning, and several additional learning experiences. One significant difference was that more of the Pilot students reported a “research-oriented independent study or other research experience” but this is unsurprising since the Pilot curriculum requires a research experience. Another significant difference was that more Pilot students reported a “culminating senior experience (comprehensive exam, capstone course, thesis, etc.)”, which probably relates to the research experience requirement, since research experience correlated strongly with participating in a culminating senior experience.

One concern about the Pilot Curriculum has been that non-science majors in the program might take fewer natural science courses, given that they are only required to take one general education course in Category 3, "Earth, Space and Life" and one in Category 2, "Science, Culture and Society". In comparison, the regular curriculum requires three or four courses in science and math. Analyses of student records showed no significant difference between Pilot and non-Pilot students in the average number of science courses taken (whether required or elective), and this lack of difference also held both for students who were science majors and for those who were not science majors. Nevertheless, among non-science majors, Pilot students were significantly more likely than non-Pilot students to have taken only one science course and to have taken no mathematics or statistics courses (1).

Science Education

To evaluate scientific and quantitative literacy, we developed and validated a multiple-choice "Science Survey" which assessed scientific reasoning, understanding of concepts, and knowledge of important facts and terminology in the natural sciences and math/statistics. Data for seniors indicated no significant difference between Pilot and non-Pilot students in either the mean score or the proportion with low scores on this scientific and quantitative literacy test (3). Data for seniors also indicated no significant difference by Pilot status in (a) a scale that measured interest in science or (b) whether the students had a science major (1, 3). Thus, the Pilot Curriculum appeared to have no detectable effect on science learning.

Students’ Assessment of Their Education

There were no significant differences by Pilot status in students’ ratings of their “entire educational experience at Penn” or in their ratings of the contribution of a Penn education to none of ten intellectual abilities assessed. However, Pilot students did give somewhat more favorable ratings of the contribution of their Penn education to the “ability to learn on your own, pursue ideas, and find information you need” (4).

Research Requirement

One of the goals of the Pilot Curriculum is to offer students significant opportunities for individual research, scholarship and/or creative projects, and all Pilot students are expected to engage in research prior to graduation. During their freshman, sophomore and junior years, Pilot students expressed considerable frustration about the lack of information regarding this requirement and what research experience was necessary to fulfill it (6). In addition, several Pilot students who participated in focus groups during the spring semester of their junior year reported that although they had conducted research in a variety of settings, none had engaged in this research for the purpose of fulfilling the Pilot Curriculum research requirement.

Pilot students have now been told that they can fulfill the research requirement in a number of ways – by taking upper level research seminars, independent studies, writing a senior thesis, or conducting lab research or experiments – and that they can usually do so in the context of their major. Given considerable variation in departmental research requirements, which are related to disciplinary considerations and the number of undergraduate majors, the nature of students’ research experiences are likely to vary substantially. A survey of Pilot students conducted in the fall of 2003 reflects this variation. Almost one-half of the respondents reported survey research or secondary data analysis, one-fourth reported laboratory research, and almost one-fourth reported library research as their primary type of research. Also, based on students’ self-reports of their educational experience, more Pilot students reported “a research oriented independent study or other research experience” and/or “a culminating senior experience” than non-Pilot students (4).