Assessing Online Course Quality

If you’re looking for answers, this is the wrong place. I’ve got a lot of questions though.

We are trying to increase the number of online courses we offer. I have my own opinion about what a good online course looks/acts like but I’ve been looking for something from people who have more credibility and more experience. So, let’s take a look at one of the major players.

iNacol

The National Standards of Quality for Online Courses

This document is fairly large- detailed in odd ways, vague in others. I get the feeling after breaking it down that it’s something a consumer/purchaser would use to evaluate a course as opposed to a creator evaluating their own course to improve it.

The rating scale for each element:

  • 0 Absent—component is missing
  • 1 Unsatisfactory—needs significant improvement
  • 2 Somewhat satisfactory—needs targeted improvements
  • 3 Satisfactory—discretionary improvement needed
  • 4 Very satisfactory—no improvement needed
  1. Content – 14 items in this section

    I get frustrated that

    The course content and assignments are of sufficient rigor, depth, and breadth to teach the standards being addressed.

    is given the same weight as

    Issues associated with the use of copyrighted materials are addressed.

    Giving these factors equal weight in the rubric makes no sense to me. There are many similar examples. I have no confidence that the end numeric score would equate to quality.

    I also find the descriptors to be fairly limited if the rubric is to become a decent guide for teachers creating courses.

  2. Instructional Design – 16 items in this section

    I realize rubrics have limits but phrases like the one below don’t seem to give the reader enough focus or clarity to really do anything.

    The course provides opportunities for appropriate student interaction with the content
    to foster mastery and application of the material.

    I understand the need to be general but statements like these leave me with more questions rather than less. Maybe that’s a good thing but I don’t think it’ll help the majority of teachers I know make better online courses.

  3. Student Assessment – 7 items
    I wonder about the relative weight of this section.

    Assessment strategies and tools make the student continuously aware of his/her progress in class and mastery of the content beyond letter grades.

    Lots of questions here.

  4. Technology – 13 items

    The course accommodates multiple school calendars; e.g., block, 4X4 and traditional schedules.

    I have no idea what that has to do with technology or why that’s really a key component of an online course. I can see why that would matter if you were making courses as a vendor but it’s still course design in my mind.

    The course makes maximum use of the capabilities of the online medium and makes resources available by alternative means; e.g., video, CDs and pod casts.

    I don’t think you can do both of these things. It’s like the backup laptop lesson. If you are really using laptops to do something unique and powerful, you’ll have a hard time providing a paper equivalent.

  5. Course Evaluation and Management – 7 items
    This section is particularly consumer focused but I like this idea although I’ve never seen it done.

    The results of peer review and student evaluations of courses are available.

  6. 21st Century Skills – 1 item
    Essentially, it says go look at an additional rubric which references P21. That makes this thing even larger and more unwieldy and still it’s not providing the kind of guidance I’d hoped for. It may be that I’m looking at the wrong tool or the wrong provider.

I’m feeling a lot like I felt when we made the TIPC. I looked at the things that P21 and ISTE offered and felt that neither helped teachers see what quality meant in the classroom. The audience always seemed to be some other group. The size of the documents were too large while also remaining too vague. So we made our own version. I’m hoping we don’t have to do that again with online courses.

If you know of quality rubrics for this kind of thing, I’d like to see them.

Comments on this post

  1. Michelle Bourgeois said on November 30, 2010 at 1:10 am

    Tom,
    I’ve been searching recently for the same. I’m a bit more happy with the following two as they align with the NSDC (which now goes by the odd name Learning Forward, but that’s another story…)

    http://publications.sreb.org/2004/04T04-Standards_Online_Prof_Dev.pdf

    http://www.learnnc.org/lp/pages/2747 – this has been the best fit for me as it also goes into some thoughts about instruction including social presence and facilitation.

    I’d be interested on your take on these two.

    • Tom said on December 2, 2010 at 7:50 am

      I like this much better. It gives me something to build on. I’ll post what we end up using fairly soon.

Leave a Reply

Trackbacks and Pingbacks on this post

No trackbacks.

TrackBack URL