Standard 2
Planning and Evaluation
Team Members:
Jill O’Hagan,
lead, Director of Business & Industry Services
Frank Aleman,
Professor
Brian
Donohue-Lynch Associate Professor
Marion Egan,
Professor
Cheryl Goretti,
Assistant Professor
Dianne Williams,
President
Description
Planning
When the goals for the coming year have been finalized, each College division adopts Divisional Strategies to support the goals, and every individual in each division adopts activities to support the divisional goals. Divisional budget requests identify funds needed to accomplish institutional goals, and final decisions are based, when possible, on the Learners First guiding principle. The projected activities of faculty members are reflected in their contractually stipulated faculty additional responsibilities plans and reports. At mid-year and at year-end, each individual and division provides Goal Attainment Reports on the status of goal achievement at the College, division, and individual levels.
Given the College’s commitment to Learners First and “Total Quality” service, the College planning process includes considerable constituent input. For example, in 1998 when Quinebaug Valley Community College was developing its Facilities Master Plan, the College hosted four information-gathering meetings with College constituencies including business and industry, community organizations, educational entities, and legislative and municipal leaders. As part of the strategy to gain information for planning, College staff members serve widely on community boards and in community organizations. Also, the self-study process at the College also enables College programs to receive input on constituent needs to further program planning. Since 1991, eleven College programs have completed the self-study process and submitted self-study reports. In addition, occupational programs at the College have advisory boards that meet to assist in program planning.
The CCTC system also collects
information regarding constituent needs in order to develop system goals and to
inform local planning efforts. Between 1998 and 1999, for example, the Board of
Trustees and the Chancellor’s office sponsored a series of four regionally
based information-gathering meetings and a market research project
to undertake a comprehensive analysis of current and future educational needs
of key regions and constituencies in the state. Individuals from business,
government, education, and regional communities participated in the meetings.
Summary information from the meetings is available.
Evaluation
Evaluation is an integral component of College planning and operations. Various strategies are used to collect information, determine goal achievement and satisfaction, and respond to external requirements. These strategies involve the broad College community, including external groups and agencies and of course the learners themselves. Evaluation focuses on the College overall, programs and services, processes, outcomes achieved by learners, and College employees. The results of evaluation are used in a continuous improvement cycle.
In 1995, while preparing the Interim Report for NEASC, the College applied for the Connecticut Award for Excellence (CAFE), which is the Connecticut award based on the Malcolm Baldrige Total Quality criteria. The criteria included information and analysis, strategic and operational planning, human resource development and management, educational and business process management, school performance results, student focus or stakeholder satisfaction, and a special emphasis on leadership. The application was submitted as a mechanism to gain external feedback as the College began to prepare for review of its institutional effectiveness in 2001. A team of outside reviewers evaluated the CAFE Application and a site visit was made to the College. A panel of judges granted the College the award at the Nutmeg level. None of the examiners or judges was associated with higher education and all were experienced Total Quality examiners. The College was the first State of Connecticut agency to receive this award and, as of this writing, it is the only public higher education institution to have received it. The business community viewed QVCC’s receipt of the CAFE Award as a significant indicator of institutional effectiveness. The evaluation and examiner’s report provided valuable information to the College for the continuing integration of improvement cycles in all facets of the College.
From 1997 through 1999, under the
direction of the previous CCTC Chancellor, Indices of Success were developed with
specific targets that each of Connecticut’s twelve community colleges reported
to the Chancellor and the Board of Trustees. These included such items as funds
raised, access to and use of technology for learning, and enrollment. The
College documented considerable success in achieving these indices. The Board
no longer uses these indices, having replaced them with a new system strategic
planning process.
The Connecticut Department of Higher
Education, with involvement of the four constituent units of State public
higher education, has developed Department of Higher Education performance
measures. Each college or subgroup of CCTC system colleges has been
benchmarked against other peer institutions nationally and has reported
achievement. The finished report for 2000 is available on the system website
www.commnet.edu/co/academic/reports/
Accountability%20Report.PDF. This replaces the biennial
institutional assessment reports previously required and submitted.
Degree programs and special areas are
evaluated using a process and schedule regulated by the Board of Trustees.
Special areas at QVCC have included the English as a Second Language and
Humanities areas, as well as the Learning Center. Also, the Medical
Assisting program is
accredited by the American Association of Medical Assistants and is
evaluated every six years to maintain that accreditation.
The self-study process for academic
programs has recently been redesigned. All colleges in the system have implemented
the Discipline
Review Process, a systematic and introspective process designed to
ensure a curricular area’s continuous improvement. The new review’s purpose is
to examine a discipline’s effectiveness through both internal and external
perspectives.
Individual programs and services have
developed evaluation procedures with the results used to direct improvement.
Examples include the library user satisfaction surveys and the developmental
student progress study of relationships between developmental and
other students concerning enrollment, retention, graduation, and QPA. Relevant
staff members initiate such evaluations, and the results are communicated at
full staff meetings. College goals and their achievement by division are also
reviewed, evaluated, and communicated at these meetings.
Process improvement teams (PITs) may be
chartered by the QVCC Total Quality Steering Committee or initiated by relevant
staff members based on identified problems or documentation of dissatisfaction.
Examples of issues addressed by PITs are building a better course schedule,
streamlining the disbursement of student financial aid awards, moving mail and
material between the two campuses more efficiently, and improving communication
via telephone. The Total Quality Steering Committee has developed a process to
track and evaluate changes. PITs are cross-functional, and each team generates
a PIT
final
report.
Since the last College self-study in
1991, the faculty has been involved in classroom assessment activities. When
the system developed a new Academic Model and identified eleven
initial components, the College chose to take the lead in developing the
component on outcomes assessment, building on its prior efforts. A manual,
titled A Step-by-Step Guide to Outcomes Assessment
in the Connecticut Community Colleges, was developed, and it has
been made available for use by other colleges in the CCTC system. Also, the
College’s general education objectives were revised into general education
outcomes, entailing a shift in language from articulating what the College will
teach students to do or know to articulating what students will be able to do
upon completion of coursework. Emphasis was then placed on assigning
identifiable outcomes for each course and academic program. The Learning and
Student Development Division of the College identifies at what point in a
program a desired outcome should be assessed. Once assessments clarify whether
outcomes are being achieved, curriculum revisions can be made as needed.
Information
is regularly collected and analyzed in an effort to improve programs and
services. Results of graduate surveys are used to target
programs and services for further study. Scanning sessions have been held with
members of the community to collect information, identify perceptions and
satisfaction, and solicit ways the College could better meet its mission and
the needs of the local community.
New programs and services include a
formal evaluation component. Examples include the evaluation of compressed video courses
between the Danielson and Willimantic campuses and an evaluation of the new student advising
system. In addition, when the organizational architecture changed
from three deans to two, a formal evaluation was carried out by the
College, as well as a questionnaire on reorganization
specifically focused on the Learning and Student Development Division. All of
the above evaluative information is shared with staff on a regular basis.
A new system-wide faculty evaluation process, the Faculty Development and Review Plan, was implemented during the 1999-2000 academic year. The new process is the result of a lengthy effort by a system labor-management committee. In the new process, for the first time student ratings and faculty self-appraisal are involved in the review of full-time faculty. Also, a new evaluation tool for non-teaching professional staff was implemented in 2001. Classified staff continue to be evaluated in accordance with their respective collective bargaining agreements. A new evaluation process for managers was developed by the Board of Trustees and a system committee of managers and implemented in 1999-2000. At QVCC this process applies to the President, the executive assistant, deans, and the director of community and professional learning. Achievement of individual goals derived from divisional and College goals is included in the evaluations of all staff. Finally, it is important to note that virtually every staff member has been involved in the current self-study.
Appraisal
Planning
The
College’s current planning process resulted from its assessment that the staff
at large was not embracing, or not actively involved in, the traditional
five-year planning system and that the ensuing document from that planning
process was not well used.
The
new planning process has proved to be successful for the institution in various
ways. First, efforts to include all staff in developing and working to attain
the goals have been achieved. This is evident by the way goals are being
communicated. Goals are discussed and adopted by the staff and then routinely
focused on at staff meetings. At division meetings, members supply input and
work to
develop the divisional strategies and identifiable measurements. Individuals
then construct their yearly activities to support the College goals. The
planning process has reached all levels of the College.
The planning process is also successful in the way the progress of attaining the goals is reported. Through semi-annual reports on the status of divisional strategies, staff can see tangible results regarding accomplishments made toward meeting the institution’s goals. In addition, each employee is required to assess, through a year-end report, how she or he has fulfilled individual goals and supported the achievement of the institution’s goals. The year-end report is used as part of the employee’s evaluation.
The College was able to measure the success of its planning process when, in 1999, the Community-Technical College system decided on a complete change in its strategic plan. Quinebaug Valley was able to incorporate easily the demands of the system into its planning process, since the College already had a proven process underway. In addition, prior to new contract language requiring that individual faculty additional responsibilities reports directly address the institution’s goals, Quinebaug Valley was the only college in the system already requiring that the reports do so.
Since the College is successful in meeting system-wide strategic goals and incorporating these goals into its plan, it needs further to develop a means to measure how individual and divisional goal attainment relates to the attainment of the College’s goals. In addition, certain operational goals sometimes lie outside the scope of the stated system and College goals. For example, the College’s Administrative Services Division supports the academic area and other areas and establishes operational goals that include such activities as setting up budgets and accounting systems for various programs; however, it is often difficult to relate directly activities of this nature directly to a specific College goal. Another example can be drawn from the Learning and Student Development Division, which establishes departmental goals that express the need for faculty to recruit, hire, and evaluate adjunct instructors each semester: these essential operational goals are nevertheless difficult to align with stated College and system goals. The College must keep working on ways of tracking and measuring important operational goals and incorporating them into the planning process, while making sure that broad statements of goals reflect the essential missions of the institution.
Assessing
the effectiveness of evaluation procedures has been a continuous focus for the
faculty and staff of Quinebaug Valley Community College. A key example is the
way recommendations from the Connecticut Award for Excellence evaluation team
have been integrated into College procedures (as discussed above in the
Description section and evidenced in CAFE recommendation and feedback reports).
Yet more could be done if the College had a full-time institutional researcher.
The current researcher is shared with two other colleges and thus primarily
provides data related to compliance reporting, such as the multitude of
Integrated Postsecondary Education Data System reports and reports for the
Department of Higher Education. Over the years there have been many evaluation
procedures mandated by external agencies, and it is not always clear how they
address the College’s core mission.
When degree
and career programs are evaluated in accordance with the regular self-study
review schedule, knowledgeable individuals from outside the College are invited
to review all evaluative processes and comment on both the results of the
review and how the review was conducted. Also, small groups of instructors meet
to review and evaluate classroom assessment processes and to monitor their
effectiveness. This project has evolved into the College’s contribution to the
system Academic Model project—the Step-by-Step Guide to Outcomes
Assessment--which has been distributed to all colleges in the system to guide
their evaluation of their achievement of program and general education learning
goals. Also, QVCC has revised all general education and program goals into
learner-centered and measurable terms, concomitantly articulating how these
goals will be met and how the achievement of these goals will be measured.
These measures provide continuous feedback on how the College’s goals are being
met and where improvements should be implemented.
In the new
Faculty Development and Review Plan, both the faculty member and the Dean of
Learning and Student Development (the immediate supervisor of all full-time
faculty) have input in the review process. The plan incorporates an
instructional observation with pre- and post-conferences, a self-appraisal,
student ratings, a performance review, and an action plan for further
professional development. It is expected that the new plan will improve the
evaluation process; the College will be working with the CCTC system to develop
a process to monitor and measure that improvement.
The College fosters a proactive climate in which it seeks opportunities for improvement, rather than reacting to problems as they arise. A process of continuous improvement can be seen in QVCC’s three-year effort to evaluate the effectiveness of the implementation of recommendations made by Process Improvement Teams. The ideas of Total Quality management and continuous improvement have become ingrained in the College’s culture. These ideas have led to a greater focus on measuring the success of various endeavors and have led the College to question how best to monitor the measurement. A first monitoring program was developed and used, but upon review it was found to be inadequate. Using the original plan and input from those who worked as monitors, a second plan was developed, implemented, and evaluated. During the 2000-2001 academic year, QVCC implemented a third version of the monitoring program while continuing to collect data that can be used to continue refining the evaluation process.
The lack of a
full-time institutional researcher inhibits the analysis of quantitative
research regarding College needs and goals and often leads to heavy reliance on
qualitative data alone. For example, QVCC’s ability to prepare grant
applications, to complete program reviews and accreditation reviews such as the
NEASC self-study and the Medical Assisting accreditation review, and to track
student progress would all be enhanced by a full-time institutional researcher.
Projection
Documents
Academic Model documentation
Annual committee reports
Biennial institutional assessment reports
Blank Management Evaluation Report
CAFE Application, recommendation, and feedback reports
Department of Higher Education performance measures
Developmental student progress study, English and math
Discipline Review Process
Divisional Strategies
Evaluations of compressed video courses
Evaluations of the new student advising system
Example of a college plan prior to 1996
Facilities Master Plan
Faculty additional responsibilities plans and reports
Faculty Development and Review Plan
Faculty Evaluations
Goals and Goal Attainment Reports at the College, division,
and individual levels, 2000-2001
Graduate surveys
Hoshin Kanri Training
Manual
Indices of Success and related report
Lead Planning Team year-end reports—2000-01, 1999-2000,
1998-99, 1997-98
Library user satisfaction surveys and results
Medical Assisting Program accreditation report
Process Improvement Team Reports
Questionnaire on LSD reorganization
Regionally based information-gathering meeting summaries
Report on Plastics program
Self-study reports, 1991-present (11)
Step-by-Step Guide to Outcomes Assessment in the Connecticut
Community Colleges
Tech Plans, including the Blue Sky Tech Plan
Usage statistics for the Learning Center