A Brief Rant About Misleading Research and Unaccounted for Variables

This article came across my feed today:

Five studies find online courses are not working well at community colleges

And of course, I immediately clicked and scanned to see what exactly the studies had been looking at in terms of instructional design. After all – we know that it is important to plan and effectively design online courses.

I wondered if this article was looking at synchronous talking head classes with some blended content offered online? Maybe they were looking at xMOOC style video, read and test classes? I’m always excited to find one more article that debunks the effectiveness of those classes.

As I scanned the article, I realized the researchers hadn’t taken into account the mode of the class (or at least this wasn’t discussed in the article). All classes – synchronous or asynchonous – were addressed in the same way regardless of the design model. There wasn’t even mention of whether the class was created by the teacher, or by a centralized unit in the Community College. Not a peep about the background of the teachers offering these classes. In short, the article addressed every online class as if it were created equal.

But the statement was decisive and unqualified.  Online courses aren’t working well at community colleges. Five studies say so.

As I read down seeking for any acknowledgement of the complexity of the numerous models of online teaching and learning I did see this statement:

Online students might see a brief video clip of the professor each week, mentioning upcoming topics, but rarely is an entire lecture videotaped.



If a student proposed to do research on the effectiveness of online courses in community college in one of my own classes, I would have pushed the student proposing that research to dig more deeply into online teaching and learning and the methodologies that influence it. I would have asked the student why we should do a study that characterizes a medium for interaction as “effective” or “ineffective” and to drive my point home, I would have pointed out the numerous studies that examined whether technology in and of itself was “effective” in the classroom. I would have pointed out to this student that any tool – chalk, paper, a pencil, technology – CAN be effective if wielded by a teacher who is using it with purpose. And then I would have raised the issue of whether it is ethical to create a study such as this, ignoring so many important variables, when it could be consumed (and believed) by many, many people. I would have pointed out that as researchers, we have a responsibility to be as thorough and transparent in our design as possible.

The article does at least acknowledge that instructional design exists through this comment:

To be sure, the design and production of online community college courses are decentralized and primitive. Professors largely make their own, from soup to nuts, and most don’t have technical or video training.

Not “most” online community college courses, or even “community colleges in general”.  But apparently the design and production of ALL online community college courses are decentralized and primitive.

Of course there is the case of the Mississippi Virtual Community College which has worked to centralize, vet and produce high quality classes for Mississippi Community College students for nearly 20 years. To be very honest – I don’t know how well MVCC has kept up with its lofty goals – I left Mississippi 15 years ago. But if I know the ed tech people in Mississippi (and I do) every effort has been made to ensure the design of these classes are of high quality.

On the positive side, it is possible that this article could spur debate about the design of courses which could enhance online offerings for Community College students. On the negative side, it’s more likely this article could spur administrators to purchase the next great lecture capture tool for Blackboard in an attempt to get those full lectures – all three hours of them – online! On the extremely negative side, here’s more fodder for those who want to say online teaching and learning is just inferior.

I hope that all of us in the ed tech and online teaching and learning community will keep underscoring the need for research that is useful to improving online teaching and learning. I would hope that at this point in the development of online pedagogy, the need to effectively and intentionally design online classes isn’t one of those concepts.

And to community colleges – I would suggest not investing in that lecture capture software, or in video technicians – but instead, starting campus conversations about Quality Matters and peer review for online quality instruction.


3 responses to A Brief Rant About Misleading Research and Unaccounted for Variables

  1. Dr. Virgil Fredenberg says:

    I read the article as well. Why do people with no experience teaching online think they know how to do research into it? It is important that online teachers receive education on how to teach in this medium. You don’t just get a computer and begin teaching online. Online teaching is different and difficult.

    I will say that many professors who are “teaching” online are likely not very good at it and are definitely not good online instructors, as the article states, “To be sure, the design and production of online community college courses are decentralized and primitive. Professors largely make their own, from soup to nuts, and most don’t have technical or video training.”

    This article does lump all online courses into one big pot, “At first blush, it would seem that online courses, where students can log in at their convenience and complete assignments at their own pace, might be an ideal solution for community college students, many of whom are older and juggling jobs and parenting.” There are online courses like that out there. But there are many other types of online courses out there.

    Some online courses are taught simply to get as many students into them as possible. These can be very poorly designed and taught. Other online courses are taught so that the students can have access without having to travel great distances to a classroom. Many of these courses are synchronous and make use of advantages technology brings to a course. There are other forms of online courses as well. And there are blended courses and combinations of these types.

    This article does concede that, “Rigorous studies at four-year colleges have tended to find no difference in student performance between traditional face-to-face lecture courses and online courses.” Let me point out that “Rigorous studies” find no difference. Perhaps this is an admission that the premise of this article is based on studies that are less than rigorous. Even this article is not well researched. Even the example of an online course provided is not the course, it is the instructions on how the course is set up, the way the course is managed, and how to navigate around the website. It is not an example of a class session in the course.

    Let me close with the question I posed earlier, Why do people with no experience teaching online think they know how to do research into it?

  2. akedtech says:

    For me – it isn’t that they even have to have experience in it – but I do wish they would at least do a literature review of the past thirty years or so so that they could understand the driving questions behind it.

    Saying “online teaching is ineffective” is like saying cars are “ineffective”. If all you have known is a horse you might find a car very ineffective. It won’t eat. When you pull the steering wheel it doesn’t stop. It can’t go through streams, or accross deep ditches.

    So when you are talking about the effectiveness of cars over horses, it’s the same principle as looking at the effectiveness of online teaching over f2f in any given situation. You can’t expect online teaching to look the same as f2f – any more than you can expect a car to do the things that a horse will do.

    I guess I am still ranting a little. My serious concern is the number of administrators who will read the headline, and use this “finding” (I use the term loosely) as ammunition for more pedagogically bad online teaching initiatives.


    • akedtech says:

      Your point about trying to get as many students as possible into a course without the proper support in place is a really good one, too! Yet another unaccounted for and unmentioned variable – who signed up and why – and what were their perceptions of online learning?


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s