The Task Force on Professional Implications of Learning Technologies was established
in Fall 1998. The Task Force's charge was to examine the effects of electronic
teaching and scholarship on the processes of evaluation, especially in tenure
and promotion, and to suggest ways in which departments and the Dean must prepare
in order to be ready to assess the quality and promise of electronic teaching
and scholarship. The Task Force broadened its charge slightly, seeing a responsibility
to examine how the "new technologies" will affect evaluation of teaching, research,
and service in all faculty evaluations, not just at the time of tenure or promotion.
The members of the Task Force were:
Joe Kearney, Computer Science (Chair)
Bob Boynton, Political Science
Brooks Landon, English
and Bob Mutel, Physics and Astronomy
Frederick J. Antczak, Associate Dean for Academic Programs,
served ex officio and JoAnn Castagna staffed the committee.
The committee sought advice from colleagues on and off campus, the AAUP, and
professional organizations.
PREFACE
This is a transitional moment––the number of faculty using new technologies
as an integral part of their teaching, research, and service is rapidly growing
and these new technologies are dramatically changing the scholarly and teaching
enterprises of the University.
After some general comments, we have divided our suggestions into three broad
areas of concern. The Task Force believes it is appropriate and important to incorporate
a consideration of the use of new technologies in the review of all three components
of faculty evaluation: research, teaching, and service.
Back to the Top
GENERAL STATEMENTS
Throughout this report, we will use the term "artifact" in an attempt to include
all the possible products of the new technologies. Just as creative work in many
media have long been recognized as part of the "research" output in the fine arts,
artifacts like databases, simulators, theorem provers, and hypertexts of various
sorts are becoming increasingly important research products in many disciplines.
Some of these artifacts will be accessible solely in forms of the new technologies.
It may be useful for the College to examine possibilities for moving beyond text-related
vocabulary when evaluating research, teaching, and service.
Among individual faculty members, it is likely that differences of subject
matter, research methodology, or personal preference also affect use of the new
technologies. It is completely congruent with the highest standards of intellectual
freedom that these differences exist. The Task Force urges the College to make
clear that the use of specific technologies is not required for the positive evaluation
of research results or teaching. However, it is in the best interests of the College,
the departments, the faculty, and our students that the University invite, encourage,
and celebrate the involvement of faculty with the new technologies.
II. We recommend that when faculty use the new technologies, this use be considered
as a component of many different kinds of evaluation:
- At every stage in the review of junior faculty
- At the time of the tenure review
- At promotion reviews and during Peer Review of Tenured Faculty reviews
- At the time of yearly salary reviews
- As a component of research and other funding reviews
III. Although our suggestions below are divided along traditional lines of
research, teaching, and service, we urge the College, departments, and our colleagues
to recognize that new technologies can contribute to improving the integration
of these three aspects of scholarship. Scholars who begin to use the new technologies
often find their research, teaching, and service activities enriched and made
more seamless than has been previously the case. Evaluations at every level, from
the department to the Provost, should appreciate this integration.
Back to the Top
THE EVALUATION OF RESEARCH
There may be very wide differences among disciplines and departments, now
and in the future, in the ways in which new technologies are adopted and how the
resulting artifacts should be evaluated. At both the level of departmental review
and in the College's tenure and review committees, these differences should be
recognized. In some disciplines, the changes wrought by the adoption of new technologies
by faculty members may be so great as to challenge even the traditional definitions
of "research" in the field. At both the departmental and Collegiate level the
University of Iowa can become a leader in recognizing new research paths.
The Task Force reiterates what we feel is a universal understanding within
academe that "peer review" is the essential foundation for evaluation. We also
acknowledge the equal importance of what, for lack of a more precise word, must
be called "publication." By "publication" we mean every form of distribution of
an artifact, including, but not limited to, printed publication of linear text.
We urge the College, departments, and individual faculty members to be vigilant
in identifying ways in which new technologies may change the venues and forms
of publication used in a discipline, and to recognize such forms, as they become
accepted in the faculty member's field, as valid means of disseminating scholarly
products.
Back to the Top
Publication
In some disciplines there is already a move away from traditional formats for
the presentation of research (for instance the peer-reviewed journal that is paper-printed
and charges for subscriptions; the printed monograph for which an academic press
serves as gatekeeper, the physical-space gallery show, the real-time performance,
and so on) and toward an acceptance of new formats as equally valid forms of publication.
In other disciplines the traditional formats are still the only formats that are
generally accepted as representing "real" paths toward the sharing of research.
We recommend that Collegiate and departmental Promotion and Tenure procedures
become more explicit in identifying criteria for the inclusion of technology-mediated
scholarship in the research record.
Peer Review
The new technologies challenge all review committees to identify the experts
and sages who will be able to provide accurate evaluation of the artifacts of
research expressed in the new technologies. Expertise in the evaluation of traditional
research products may not automatically or even easily transfer to artifacts produced
with the new technologies. It will be important for departments and the College
to be alert to developing fields and to identifying those outside reviewers who
can with credibility comment on artifacts produced and disseminated through the
new media.
In some cases, there may need to be a greater reliance on evaluation at the
institutional level rather than by "outside experts." At other times increased
specialization may instead necessitate greater reliance on experts in the field
to judge both the quality and significance of a scholarly artifact. It will be
important, therefore for the College and departments to consider carefully the
composition of review committees and to ensure that all review committees have
access to the advice of individuals knowledgeable in the new technologies.
Back to the Top
Other Concerns in the Evaluation of Research
Departments are already aware of the need for nuanced evaluation of individual
faculty members, and are already working with probationary faculty and in the
development of faculty portfolios to create clear expectations for the research
plan. Departments will need to be most directly responsible for recognizing differences
in expectations that reflect the differences in technologies used while maintaining
equity between those who do and those who do not use the new technologies. The
Task Force encourages departments to be explicit in discussions with probationary
faculty and in the development of portfolios for tenured faculty. Faculty members
who depend on new technologies may have longer startup periods as they build laboratories,
develop software, and train graduate students who will assist them in their research.
The time to degree for Ph.D. students who use new technologies may also be longer
than it is for students in more traditional areas. We encourage the College to
work with central administration, the Graduate College, and other units to explore
these issues, and to incorporate into the discussion of expectations for tenure,
promotion, and other evaluations consideration of these temporal differences and
possible impediments.
Back to the Top
TEACHING
Many faculty members have found the new technologies important in their teaching.
The traditional principles for the evaluation of teaching will not change, but
the practice of this evaluation may change.
Just as peer review and publication are the essential criteria of the evaluation
of research, the evaluation of teaching has depended on a few clear parameters.
We traditionally evaluate teaching through
- an examination of course materials and course design
- an examination of the faculty member's communication with students
- student and peer evaluation of the faculty member's overall teaching.
The Task Force foresees ways in which each of these core criteria may be changed
by faculty adoption of the new pedagogies. First, we recommend that departments
and the College consider the ways in which the new technologies integrate teaching
and research more tightly than is typical in traditional teaching/research models.
We also recommend consideration of the ways in which the new technologies will
affect course design. Patterns of the use of class time and physical classroom
space may be greatly changed by new technologies. There will undoubtedly need
to be some changes in the ways in which instructor skills are evaluated. We encourage
departments and the College to support innovation in teaching practices and to
develop evaluation procedures that recognize variations from the traditional models.
Back to the Top
Some of the changes that may occur:
- Course materials may consist of a hypertext site, a course bulletin board,
and so on, in addition to or instead of traditional printed materials.
- Traditional methods of evaluating teaching may not capture the experience
of a class taught with the new technologies. Peer evaluation of teaching via classroom
visits will be difficult, or at least very different, when the "classroom" is
in a virtual rather than a physical space.
- Student evaluation may take place throughout a semester rather than through
a summary evaluation at the end of the course. We have already heard from our
colleagues that faculty-student interactions may increase dramatically with the
use of the new technologies, (e.g. through email exchanges and electronic discussion
groups) though these interactions may be cumbersome to track and difficult to
assess.
- Evaluation of courses taught using the new technologies should also recognize
the work involved in the preparation of materials that incorporate the new pedagogies.
Back to the Top
SERVICE
The new technologies bring with them increased opportunities for faculty members
to provide service at every level, from the department to the general public.
The Task Force heard many reports of the ways in which general outreach to the
state and the nation is facilitated by the new technologies. Web sites produced
in the Departments of Computer Science and Mathematics are generating email questions
and many "hits" from around the globe. The Department of Physics and Astronomy
houses the "Iowa Robotic Telescopes Site" that enables students all over the country
to do research. Faculty in the Department of English have participated in the
development of the Walt Whitman Hypertext Archive, another widely-used resource.
We encourage departments, the College, and the University to recognize and reward
those faculty members who act on the opportunities presented to them to interact
directly with members of the public and to share their research and expertise
in these ways. At the same time, we ask departments and the College to be aware
of the ways in which the new technologies increase the likelihood that faculty
members will be asked to perform more and different kinds of service than has
been traditionally true. The College should be vigilant in ensuring that there
are adequate numbers of staff and technical support personnel. We should not ask
or expect faculty members to do as "service" tasks such as software and hardware
installation or computer maintenance for departments; such work is more appropriately
done by paid professionals. Similarly, departments should mentor probationary
faculty carefully to ensure that they do not become burdened with legitimate service
opportunities that arise from their skills in and use of the new technologies.
Back to the Top
