Home / SACS Self Study at CVCC / QEP / GOAL 4

Quality Enhancement Plan

GOAL 4

GOAL 4: The College will centralize the online assessment process to include a Director of Research, Assessment, and Planning to oversee the development, collection, analysis, feedback, distribution, and use of results.

As the number of objectives associated with this goal (thirteen) attest, assessment is an integral and essential component of the College’s plan to achieve the highest possible quality of online instruction. All of the subcommittees considered assessment in varying degrees and for differing aspects of online instruction, but it was the Assessment of Online Learning subcommittee which had assessment as its sole charge. This goal represents one of their foremost conclusions.

The Learning Outcomes subcommittee defined assessment as an ongoing process aimed at understanding and improving student learning. The subcommittee agreed that assessment needs to occur at three separate levels to provide the most comprehensive data: instruction, program, and institutional system. By conducting various kinds of assessments at all three of these levels, the institution will have the kinds of data it needs to gauge progress, track changes, and monitor the directions online learning is taking. This particular goal is concerned with assessment at the program and system level, thus the identification of the Director of Institutional Research, Assessment, and Planning as the lead figure who will oversee the activities directed by this goal. (Assessment at the instructional level will be discussed in greater detail below.)

The need for centralizing the process of collecting information about online and traditional courses became very clear through the contacts with other colleges and the survey of the literature. The colleges that provided the subcommittee with information regarding this topic (including Cochise Community College, Florence-Darlington Technical College, Forsyth Technical Community College, Guilford Community College, and Western Piedmont Community College) have developed a centralized system for collecting the data needed to evaluate both online and traditional courses. The numerous articles dealing with the evaluation of the two methods of course delivery also showed the necessity of having a centralized collection process. The many colleges discussed in these articles simply could not have achieved the level of analysis they reached without having some type of centralized system for collecting the information.

CVCC currently has in place all of the components to assess online courses. However, this process is not centralized. Each of the academic divisions has part of assessment; IT also has a part. These disparate pieces need to be tied together, and this objective makes both concrete and immediate the intent of the College to do so.

This objective is another result of the findings of the Assessment of Online Learning Subcommittee. The literature and examination of survey instruments developed by institutions or purchased from testing agencies indicate that a variety of survey designs exist. Institutions such as those in the Kentucky Virtual University System through the Kentucky Technical and Community College System (KCTCS) and CVCC use one long comprehensive survey instrument that covers many areas. (CVCC’s assessment document comprises some eleven pages and incorporates 65 questions.) KCTCS assessed areas such as Introductory Questions, Choosing Courses, and Demographics. On the other hand, Cochise College, University of Phoenix and Guildford Technical College use separate short surveys to assess specific areas such as Online Course Evaluation and Student Satisfaction. (See Appendix C, figure 2.) After studying these instruments, subcommittee members agreed that CVCC should develop separate evaluation instruments instead of continuing to use the one comprehensive document.

In addition to the above criteria, the subcommittee determined from the survey instruments observed that demographic data and space allocated for students’ general comments should be included on at least one survey instrument being used by CVCC. Demographics should provide additional information that will facilitate the measurement and comparison of traditional and online learning outcomes. A section for general comments will encourage students to give additional information that can be used to improve courses and programs.

The criteria included in the instruments are mostly supported in an article published by Michigan Virtual University which identifies seven principles of effective teaching while focusing on evaluating online courses. Members observed these principles in the surveys being used by Brevard Community College, Franklin University, Kentucky Virtual University, University of Phoenix, St. Petersburg College, and the North Carolina Community College System. The seven principles are as follows:

• Good practice encourages student-faculty contact
• Good practice encourages cooperation among students
• Good practice encourages active learning
• Good practice gives prompt feedback
• Good practice emphasizes time on task
• Good practice communicates high expectations
• Good practice respects diverse talents and ways of learning (Graham et al)

This objective reflects further findings of the Assessment of Learning Outcomes subcommittee. CVCC does not currently have any kind of assessment instrument designed to gather this kind of information. Especially influential to the subcommittee was Elements of Quality Online Education by Janet C. Moore. She acknowledges the importance “of a quality framework with practices that show institutional commitment to learning effectiveness, cost effectiveness, access, faculty satisfaction, and student satisfaction” (3). Adding this particular instrument to the rest of the assessment package will allow the institution to systematically gather data which before was available only anecdotally. These data will provide a valuable new perspective into online teaching from the instructor side of the desk.

According to the data the College has been able to collect so far, the overall attrition rate for online courses is higher than that for face-to-face classes. This rate varies, however, from class to class for reasons that so far remain unanalyzed. The institution is aware that rates of attrition and retention are significant measures of student satisfaction that are integral to creating successful learning outcomes. The research further suggests that indicators used to examine attrition/retention rates may include family size, family income and educational background, age, gender, education, placement scores and ethnicity (Moore 44). PeopleSoft, the new student information software system, will allow CVCC to create a database of many of these demographic elements so that that they can be used to generate data that determine rates such as retention and attrition. Data collected from these analyses and other assessment instruments can be analyzed and used to (a) change or modify programs and/or courses, (b) compare learning outcomes among courses and programs, and (c) compare online and traditional completion rates and learning outcomes. Ultimately, a collection and study of these statistics will lead to changes and improvement of learning outcomes at CVCC.

This objective will be alluded to again in the discussion of Goal 7 below. In summary, however, Goal 7 commits the College to provide similar student support services to individuals taking online classes as are provided students in traditional classes. These student support services are to be phased in over the five-year course of the QEP. Objective 1 comprises Academic Records, Admissions, Registration, Exam and Test Proctoring, Tuition and Fee Payment, Tutoring, Technical Support, and Bookstore. This objective calls for the creation of a method to ascertain to what extent this portion of Goal 7 is being achieved.

As has been explained under Objective 4 immediately above, collecting student demographic data can be valuable in helping to determine reasons behind attrition and retention rates. In order to effectively assess differences between online and traditional methods of teaching, the College needs to also analyze the differences among our students. Currently, for example, adult learners comprise 80% of web students (Newcombe). Some of the studies that claim online courses to be as effective (or more effective) as traditional courses fail to take into account differences in the student populations (Hiltz, Zhang, and Turoff 14-15, 18-22). However, researchers maintain that differences between students in online and traditional courses can significantly affect the levels of success. They see demographic data as strongly affecting student performance. (Grayson, MacDonald, and Saindon; Parker and Germino). Another key area highlighted in the research is that students have a variety of different learning styles that can affect their performance. Some of the studies examined reveal that since online courses frequently employ instructional methods that differ from those used in traditional courses, determining the learning styles of the different students could be a useful indicator as to which type of learning environment would work best for the individual student. (Grayson, MacDonald, and Saindon; Johnson et al). According to some research, for example, although the classroom behaviorist model may be used for online learning, the constructivist model (the instructor acts as facilitator) is better suited to the online environment (Hazari; Driscoll; Seels; Ryan et al; Polichar & Bagwell, 2000).

Demographic data can yield much, but as the timeline for this objective makes clear, the process of collecting and analyzing the data will be an ongoing process. The institution will need to determine which specific demographic data items are most valuable and in what contexts. The multi-year timeline for this objective will provide the necessary time for gathering data, establishing baselines for the baseline database, and then analyzing what information the data reveal.

Much research has been conducted on what works pedagogically in the traditional classroom and work is currently being conducted to gather this same information for online courses. This objective represents CVCC’s effort to do the same for our particular courses and students. Research examined by the Assessment of Online Learning subcommittee found that Brevard Community College in Florida, for example, has been compiling data that compares the grades and retention rates of students in the baseline online and traditional classes to data compiled from students taking online courses to examine attrition and retention rates. The surveys that will be developed at CVCC will allow the College to gather similar information plus additional data. These surveys will allow CVCC to help determine what kinds of students take and succeed in an online course and compare their characteristics to those of students in traditional classes.

Measuring student satisfaction in this fashion will provide another perspective into what works--and how--in traditional and online courses. Examining data from classes taught by the same instructor will permit the survey to limit the number of variables assessed and focus on those elements which delineate online from traditional classes. At the same time, these surveys should ideally identify how the two delivery methods are similar. These data may also suggest that strategies or methodologies employed in one delivery mode can be beneficial in the other. For example, anecdotal evidence gathered at CVCC seems to suggest that students appreciate and are growing to expect that traditional classroom classes have an online component. Students most often mention the helpfulness of lecture notes and study guides that some teachers in the traditional classroom make available on a web site.

When CVCC first began offering online classes back in fall 1998, most instructors found that offering an on-campus orientation was beneficial both to them and their students. Such a meeting allowed instructors to associate a face with the names on their class roster; more important, this meeting allowed the instructors to go over the mechanics of their online class. Instructors could show students how to access the course on the Internet, where to find course assignments, and how to submit them. In those days too, different online formats proliferated. CVCC had courses in FrontPage, WebCT, and Blackboard.

Many CVCC students were not sophisticated computer users, and these orientation sessions gave them the knowledge they needed and permitted the instructor to disseminate essential course material efficiently. However, with each passing term students gained computer knowledge, and some instructors found orientations less necessary. Now that the VCCS has purchased and installed system-wide Blackboard servers, all online courses share a common platform, and as students become familiar with Blackboard, this may further lessen the need for orientation sessions for online courses. This objective will allow the institution to gather data to determine the best orientation practices and implement these practices throughout the online curriculum.

This objective seems self-explanatory. It commits the College to continue to examine and refine the various assessment instruments and policies associated with the Quality Enhancement Plan to insure their maximum effectiveness.

This objective is analogous to Objective 5 immediately above. As discussed under that objective, Goal 7 (see below) commits the College to offer online the full complement of student services currently available on campus. Various student services are to be phased in over the life of the QEP, and this particular Objective is designed to assess the online efficacy of the following student service functions: Academic Advisement, Financial Aid, Learning Resources, Disability Services, and Placement Testing.

This objective represents yet another facet of the assessment process that will provide much of the information that guides this QEP. By looking at student performance data—grades certainly, but also other aspects that might include competencies or skills, test scores, written work, et cetera—the institution will be in a position to determine how the two course delivery methods compare in terms of learning outcomes. The question has been raised, especially from those strong adherents of online instruction, whether online education might prove to be a more effective way of teaching than the traditional classroom. The data gathered as a result of this objective will speak to that particular question, at least as far as CVCC students are concerned, and should offer interesting insights into the nature of how (and what) students learn. The best outcome for such investigations and analyses of the resulting data might be to provide instructors with the strategies that produce the most effective learning; as a result, students who opt for either mode of instructional delivery can achieve the maximum benefit each has to offer.

The results from the analyses of the data gathered to meet this objective can be combined with additional research already available to help instructors design new courses and modify existing courses to make them more effective. Accordingly, their characteristics should be considered when creating, editing, or reviewing web courses (Darkenwald & Merriam; Weiss, Knowlton, & Speck).

This objective is a corollary to Objective 5 immediately above. The previous objective will look at attrition and retention rates; this objective, however, will gather data that specifically address the reasons students withdraw. While some of the reasons will doubtless be common to both online and traditional classes, it will be interesting to see if certain reasons are more common to one or the other. If this turns out to be the case, the institution will be able to offer more effective guidance to students who are planning their curricula, and instructors will be better able to assess their students’ progress and counsel them if they fall into any of the at-risk categories.