About Online TCEs
To view the Teacher Course Evaluations you must login with your linkblue id.
Overview
For five years until Fall 2014, the Office of Institutional Research has administered online course evaluations for academic units who request them. Just over 1,900 sections were evaluated using online forms in fall 2012. The online mode constitutes 43% of all course evaluations handled by the Institutional Research (IR) staff. The growth in online ratings has placed some strain on the efficient coordination of the teacher-course evaluation process. The IR Office has used online survey software—first SNAP and later Qualtrics—to evaluate courses and professors. In terms of analysis and reporting, IR’s current online course evaluation process involves: 1) importing the Qualtrics dataset into SAS; 2) running the SAS program to analyze students’ responses and generate the separate types of reports; 3) distributing the reports to college contacts; and 4) having the college contacts send the reports to instructors and administrators. Thus, the cobbled-together reporting process now takes four steps. This is an extremely inefficient process compared with today’s commercial course evaluation software. However, most evaluation software possess the functionality to cut the number of steps in half by automatically analyzing students’ evaluations and distributing a series of reports in PDF files to instructors and administrators immediately after grades have been turned in.
Finding a Solution
In January 2013, a committee was formed with the charge of recommending a consistent university‐wide technical solution for students’ evaluation of courses and teaching based on best practices in the field related to instrument design, administration, data collection, report format and use of results.
Furthermore, the previous task force’s findings and recommendations from 2010 were included and considered in the evaluation of this technical solution via incorporating previous research and team members from the effort.
Committee Members:
|
Anna Bosch, College Arts & Sciences |
Mary Watts, College of Engineering Tara Rose, Director of Assessment Connie Baird, Director of Distance Learning Doyle Friskney, Chief Technology Officer Jason Conley, Information Technology |
Evaluation
Three vendors (eXplorance, CoursEval, and Scantron) and their respective software solutions were identified for further study by the group based upon the
previous group’s findings, current use at UK, peer references and technical requirements.
CourseEval is TCE software used currently by the Colleges of Pharmacy, Dentistry, and Nursing so they were strongly considered due to the existing online footprint. Scantron has for many years printed the bubble sheets used in the traditional paper-based course evaluations at UK so their Class Climate
software was evaluated due to the ease of a phased-in approach to online evaluations. Finally, eXplorance’s Blue was considered due to reports of their advanced technical approach, as well as their strong endorsement from the University of Louisville.
The committee attended various webinars and question & answer sessions during which the companies discussed:
- Distinguishing features of their software
- Procedures involved in administering the evaluations
- Features of the online rating form
- Visualization options for reporting the results
- Process of sending the findings to instructors and administrators
- High-level functional and technical requirements as outlined, previously
Next, the committee called at least two references given to them by each of the software vendors. As anyone would expect, each reference gave positive feedback on the product from each vendor. However, the group solicited every lesson learned to see how each implementation might compare to an online implementation at UK. In the case of eXplorance, several committee members traveled to the University of Louisville to discuss their assessment of the company’s software.
Recommendation
The committee’s unanimous choice of software was eXplorance - Blue. Overwhelmingly, eXplorance - Blue was the winner on every evaluation point with
a few exceptions. The decision was based on several factors: the crisp, attractive online rating form and the ease of administering the evaluations, and the visual appeal of the graphs and other elements of the customizable reports. In addition, it should be mentioned that Scantron lost the support of some committee members due to its plain, visually bland online form. CoursEval did not appear to be a viable option because it did not offer a way to conduct paper-based evaluations that could be easily integrated into the current reporting process. Furthermore, eXplorance – Blue technically was the superior product with
Web Service integration available with UK’s SAP SLCM and a proven track record with Blackboard’s LMS systems. These advantages delivered a higher quality product than Scantron’s Class Climate, which was integrated via legacy APIs unique to Class Climate and had no track record with either SAP SLCM or Blackboard LMS.