Skip to content

Compare and Evaluate Online Courses

June 5, 2013

Beth Transue

EdTech522

Evaluation of Two Online Courses

This activity was eye-opening regarding online courses at my campus. We are migrating from Sakai to Canvas as our learning management system starting this summer through the Fall term.  Therefore this was a very timely assignment for me to learn, evaluate and compare the new learning management system.

I was surprised by how easy the new system can be to navigate. The left menu is very clear and it can be customized to the course. I thought that Course A had better navigation because there were more menu items through which a student could access materials such as syllabus, assignments, discussions and quizzes. Course B only allowed the student to access materials through the Modules menu option. This provided a structured framework, but limited direct access to specific assignments for later review.

Neither faculty posted online office hours. The software allows direct conferencing which is very easy to use within the system. Therefore it is regrettable that the faculty do not make better use of this feature for synchronous meetings or assistance with students. When I start working directly with courses which utilize Canvas next academic year, I will ask to post librarian office hours and allow students to contact me synchronously for research assistance through the conference feature.

The assignments seemed appropriate for the courses and consisted of expected discussions, papers and quizzes/exams. I did not see as many external resources as I would have hoped, although I could only see week 1-2 for Course A. Course A did provide contact information for the liaison librarian, although neither course provided direct library links which would make them more easily accessible to students. When working with my liaison courses, I will request contact information and library links.

Rubric for Evaluating Online Courses

Course A: CIS180 – Introduction to CIS

Element 0 1 2 3 Score Comments
Design Framework No framework or navigation Limited framework and navigation on home page Apparent framework. Limited navigation throughout course Apparent framework. Ease of navigation throughout course 3 Intuitive navigation. Professor also explains navigation in detail.
Modules No learning modules Modules are inconsistent. Materials not relevant to learning objectives. No introduction or summary. Limited consistency. Resources current and relevant. Only introduction or summary included. Modules are consistent. Resources are current and relevant. Information presented with introduction and conclusion. 2 Modules exist but are not explicit. They are simply divided as weekly assignments. The connection and progression could be more explicit.
Appearance Poor color choices. Busy icons. Inconsistent style Minimal color scheme. Minimal consistency Acceptable color scheme. Acceptable uniformity Appealing appearance and color. Uniform style 3 Very easy to look at and read.
Assignment Navigation Difficult to find Found with minimal searching Easily found Easily found. Site map available 2 Easily found as a list under the assignment menu button. But acronyms in assignment names are confusing.
Variety No variety. Minimal or unclear instructions At least two types of assignments. Confusing instructions At least two types. Limited instructions At least three types. Clear and concise instructions 3 Discussions, essays, papers, summaries to news stories. Instructions are very clear
Learning Resources No learning resources Minimal learning resources Not easy to find. Inactive links Comprehensive. Credible resources. Accessible 1 Only learning resources provided are textbook and teacher online lecture. I can only see the first two weeks of the course so this may change further on.
Learning Community No learning community At least two discussions At least three discussions Regular discussions and virtual classroom 3 At least one discussion every week. Students must post before they can read other posts thereby ensuring original content before replies.
Use of technology tools One one or two tools Three tools Four to five tools More than five tools 2 Four tools visible in the first two weeks. Email, asynchronous discussion, Gradebook, Quizzes. More tools may be provided later in the semester.
Access to Faculty No office hours Posted office hours Posted office hours. Face to face and virtual. Times vary Posted office hours. Face to face and virtual. Times vary. Increase frequency prior to due dates 0 No office hours listed. Professor does encourage email or texts at any time and states there will be a response within 24 hours.
Outcomes NA Outcomes not visible to me as an observer and student.
Discussions No new information presented Faculty reinforces student ideas Faculty adds limited new information Faculty adds to the body of knowledge NA This is difficult to judge. I can only see introduction discussions. More content-focused discussions are hidden until student gives a response therefore I cannot see them to determine faculty response to posts.
Links No links Minimal links. Irrelevant. Appropriate links. Relevant Credible and relevant links add to the learning experience 1 The only  external link I saw in the first two weeks of material was to easybib in response to introduction question. Links might be present for future modules.
Objectives Not identified Identified but not measurable Measurable objectives. One engages the learner in analysis, synthesis or evaluation Measurable and appropriate. More than one engages learner in analysis, synthesis or evaluation 3 Clear measurable objectives. Many objectives require analysis and synthesis such as explaining similarities and differences among computer science fields.
Assignments, Readings, Projects Not related to objectives Some related to objectives Purpose identified and related to objectives Related to objectives. Appropriate and manageable 3 Projects visible in the first two weeks were explicitly related to objectives and were appropriate.
Multimedia Not used Minimal use Limited use Used throughout. Reflects progression of content 2 Lectures given using webcam and presentation screencaptures.
Knowledge Limited expertise Inconsistent expertise Expertise evident Expertise evident. Provides expertise in interaction with students 2 Faculty has extensive expertise in computer science instruction. Unable to view discussions with students therefore not able to assign a “3”
Links to Librarian/ Library No link to library or librarian Mention of librarian Link to library Link to library. Link to librarian contact information. Highlight library resources or services 1 Provided librarian contact information. No link to library

Rubric for Evaluating Online Courses – based on and adapted from Ternus, et. al. (2007) rubric

Course B: THEA240 – Survey of Dramatic Literature and Criticism

Element 0 1 2 3 Score Comments
Design Framework No framework or navigation Limited framework and navigation on home page Apparent framework. Limited navigation throughout course Apparent framework. Ease of navigation throughout course 2 Professor explains initial navigation and course structure. However there is a lot of scrolling on home page to see it.
Modules No learning modules Modules are inconsistent. Materials not relevant to learning objectives. No introduction or summary. Limited consistency. Resources current and relevant. Only introduction or summary included. Modules are consistent. Resources are current and relevant. Information presented with introduction and conclusion. 3 Modules are clearly arranged and progression through course is evident.
Appearance Poor color choices. Busy icons. Inconsistent style Minimal color scheme. Minimal consistency Acceptable color scheme. Acceptable uniformity Appealing appearance and color. Uniform style 3 Clear, concise and appealing
Assignment Navigation Difficult to find Found with minimal searching Easily found Easily found. Site map available 1 There is no assignment button which would make finding a list of assignment easy. Instead student must open modules to see assignments within the module.
Variety No variety. Minimal or unclear instructions At least two types of assignments. Confusing instructions At least two types. Limited instructions At least three types. Clear and concise instructions 3 Discussion, quiz, research paper, exam. Instructions seem clear based on course content.
Learning Resources No learning resources Minimal learning resources Not easy to find. Inactive links Comprehensive. Credible resources. Accessible 3 Many different resources provided in appendix section such as glossary flashcards and supplemental reading materials.
Learning Community No learning community At least two discussions At least three discussions Regular discussions and virtual classroom 3 1-3 discussions weekly.
Use of technology tools One one or two tools Three tools Four to five tools More than five tools 1 I could only see three tools listed: asynchronous discussion, Gradebook, Quizzes
Access to Faculty No office hours Posted office hours Posted office hours. Face to face and virtual. Times vary Posted office hours. Face to face and virtual. Times vary. Increase frequency prior to due dates 0 No office hours posted. Students are invited to email professor. No timeframe on response given. Professor is not available during one of the eight weeks of class.
Outcomes NA Outcomes not visible to me as an observer and student.
Discussions No new information presented Faculty reinforces student ideas Faculty adds limited new information Faculty adds to the body of knowledge 3 Faculty response includes links to additional resources and new content for students to consider.
Links No links Minimal links. Irrelevant. Appropriate links. Relevant Credible and relevant links add to the learning experience 3 Appropriate and credible links. Often links which students may not otherwise know.
Objectives Not identified Identified but not measurable Measurable objectives. One engages the learner in analysis, synthesis or evaluation Measurable and appropriate. More than one engages learner in analysis, synthesis or evaluation 1 Course objectives listed but often not measurable. Such as “discover” systems of analysis
Assignments, Readings, Projects Not related to objectives Some related to objectives Purpose identified and related to objectives Related to objectives. Appropriate and manageable 1 Assignments are measurable and seem to be somewhat related to objectives
Multimedia Not used Minimal use Limited use Used throughout. Reflects progression of content 2 Powerpoint and pdf. No webcam communication or other multimedia.
Knowledge Limited expertise Inconsistent expertise Expertise evident Expertise evident. Provides expertise in interaction with students 3 Faculty is expert in this field. Provides expertise knowledge in discussion with students
Links to Librarian/ Library No link to library or librarian Mention of librarian Link to library Link to library. Link to librarian contact information. Highlight library resources or services 0 No mention of librarian or library. This is unfortunate given the research paper and extensive library resources required.

Ternus, M., Palmer, K., & Faulk, D. (2007). Benchmarking quality in online teaching and learning: a rubric for course construction and evaluation. Journal of Effective Teaching, 7(2), 51–67.

Advertisements
Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: