For the last three Thursday mornings, I’ve sat through presentations by a combination of developers and sales teams from three different learning management systems (or LMSs), since USF’s IT department wanted to look at the institution’s choices and asked faculty and students from around the university to help organize an evaluation of different LMSs. ((The group of faculty and students have no power to make decisions here, but my impression is that IT staff are taking the input extremely seriously.)) We currently have Blackboard 9.0, and the options being considered are to upgrade to Blackboard 9.1, use Blackboard’s newly-acquired Moodlerooms hosting service for Moodle, or switch to Instructure Canvas.
Before I lay out a few first impressions for each option, a broader observation: I was struck by some data IT staff gave out in some early organizational meetings, showing that for the majority of listed features in Blackboard 9.0, very few courses use the features (and that is true in many cases for face-to-face, blended, and online classes). Because of this, an additional question that came to mind at several points in the last few weeks has been, “Okay, that looks like an interesting added control no one has at USF right now. But turning that on looks like it requires some effort, and I wonder how many would really use it.”
Many of the features I saw strike me as the LMS equivalent of kale chips. What are kale chips, you ask? So did I when I saw a package in a health-food store earlier this year, so I bought it as an experiment–kale is incredibly healthy, but it tends to require a lot of preparation time and energy. So why not give it a whirl? I have decided I do not like kale chips. At least in that first bag, kale chips are what you get when you boil the essence of kale bitterness into a triangular bit of fiber that has been baked or fried to have the mouthfeel almost precisely of tortilla-chip-textured cardboard. It is an amazing product from one perspective, but it’s not to my taste, or rather, not worth the effort in time or money.
If you think about using a particular feature of an LMS or other online system, you have to learn how to use it and what to use it for. There’s always some investment, and some things look really cool if you don’t remember that for each of the 20 assignments in a course, you would spend a few minutes tweaking the settings for that cool feature (let alone the other cool features), on top of everything else you do. So just because an LMS will let you set up special discussion-board subscription options for the students who declared “piercings/body mods” in their list of personal interests, that doesn’t mean any faculty member is going to use that option. That sort of thing feels like kale chips–somebody may love it, but not too many somebodys.
More importantly, USF shouldn’t choose an LMS based on kale-chip features. It doesn’t matter how granular you can set up some features if no one is going to use them. So in thinking about the options, I’ve decided it’s the big picture that counts, what will happen in most courses and for most students and faculty. Please keep in mind that the following observations are personal, not the result of conversation with anyone else in the group IT has used to help plan the review process.
Blackboard 9.1
Blackboard 9.1 improves moderately on 9.0 — it adds the ability to create grading rubrics, it removes some clicks from how you hide assignment grades from students (important for hand-graded assignments as you submit grades to the system), and it allows the creation of template course organizations so faculty and teaching assistants can just “get in it and drive.” It has some ability to draw in media from Youtube and Flickr (and probably a few other sets), though the only items that are easily embedded (and not by default, either) are the items you can search for publicly. (Otherwise, you embed media through the same HTML hack that’s available in 9.0.) In other words, it’s an incremental improvement of the old, clunky Blackboard.
The main advantage to sticking with Blackboard is the cost of transition. I am worried less about the typical faculty member who has created content in Blackboard (which can migrate to other systems fairly easily, for most stuff) than the departments and colleges that have invested considerable time in choosing proprietary “cartridges” that work well in Blackboard but probably would not work well (or at all) in either Moodlerooms or Canvas. That’s a transition issue. At least at first impression, there is no overwhelming reason to stick with Blackboard other than the transition problems that could result. But it is important to acknowledge that those transition issues are significant for some departments and colleges.
Moodlerooms
Moodlerooms is a service that combines the open-source Moodle platform with added features the company (now Blackboard unit) calls Joule. At least from my first impression, Moodlerooms and Moodle are what Blackboard could have been but will probably never be–the feature sets are broader and deeper than in Blackboard, and it feels about 20% easier to move around and just get things done. But having said that, the basic look and feel of Moodle/Moodlerooms is very similar to Blackboard — courses are organized into little boxes that you can call folders/weeks/modules/whatever, and in the boxes you can add different things students do, whether read, view, complete, or interact with. Moodlerooms had the course organization templates long before Blackboard, and there are more of them to help you get in and drive the course as a teacher.
The big value added by Moodlerooms in the past year is a unique feature called Personalized Learning Designer. This is something like a combination of adaptive/responsive material release and targeted emails, but on steroids. Based on events in the system, a faculty member could open up material, or alert or email students, including an in-system “mail-merge” setup allowing a student to receive something like the following:
On entering a course: “Hello, Sherman! Welcome to XIV 9384, Introduction to Complexification Studies. I am so glad you are taking the course, and here is what I recommend you look at first…”
On poor performance in a quiz: “Dear Sherman, you scored under 70% on Quiz 4. I have opened up a special review for concepts from the quiz, Quiz 4 Review Materials, and I would encourage you to review that before clicking on the next link, Quiz 4 Retake….”
The Personalized Learning Designer is an awful name, but it’s a very powerful tool, and it is the major advantage I see for moving to Moodlerooms, even beyond the active user community for Moodle.
Canvas
Instructure Canvas is somewhat different in look and feel from either Blackboard or Moodle. While there are structures to build inside a course, there is far less effort required than the multiple-click experience of either Blackboard or Moodle. For example, in an area of a course designed for discussion (discussion boards in Blackboard, forums in Moodle), in either of the systems you need to click to add an entry, type in the entry box, and then scan through all of the varied options while hunting for the submit button. In Canvas, it’s much closer to the experience of adding a status update to Facebook or a Tweet in twitter: click inside an empty box, type, click submit. Or click on an icon to record yourself talking with a webcam or (for audio only) a microphone. Unlike in BB or Moodle, links to Youtube or the equivalent become live embedded video rather than clicks to links. The reduction in friction in general makes Canvas a qualitatively different experience.
There are a number of other places where that reduction in friction is evident — Canvas has a side-by-side display of student work along with places to mark a rubric or other grading feedback — and while any one of them may be mimicked by the other systems, it appears to be baked into the Canvas user interaction much moreso than with Blackboard or Moodle. Blackboard and Moodle shows every single one of their kale chips. Canvas may have kale chips, but they’re generally hidden. In general, it looks like where there is evidence available about perceptions of faculty and students at other universities, the response is very similar to my first one: everything just works easily, and in its first few years the company has been very responsive to user feedback.
Snapshot impression… the choice depends on priorities and a shrewd guess about the future
At least right now (Thursday evening, April 19th), I can see some clear distinctions between the three platforms. We could stay with Blackboard if the transition issues dominate the decision-making. If the most important quality of a course management platform is not to disrupt existing published material, then sticking with Blackboard would make sense.
We could move to Moodlerooms if the existence of the algorithm-driven interaction (Personalized Learning Designer) is the most important feature. I can see how this could help a number of key functions at a university, and if the complexities and remaining “user friction” of Moodle is less important than the potential uses of that algorithmic capacity, then we would move to Moodlerooms.
We could move to Canvas if the modern interface and removal of friction is the most important feature. I can see how some of the behind-the-scenes mechanics could be as important as Moodlerooms’ algorithm functions, but it’s the “wow, it can be this easy?” factor that would make Canvas a very different experience. If removing most of the friction of using online tools is critical, we would move to Canvas.
It is important to keep in mind that current functionality is not what we’re working with: part of the decision depends on some shrewd guesses about the future. Will Blackboard’s new owners follow through on their commitment to Moodlerooms as it exists now and might in the future? Will the rudimentary rubric tools in Blackboard become as sophisticated as they exist in Canvas? Will either Blackboard or Moodlerooms make embedding media as simple as it is in Canvas? Will Canvas develop an algorithmic tool like Moodlerooms’? To some extent, the decision depends on the platforms as they currently exist, but not everything. I just hope they don’t depend on kale chips.
Comments most welcome
If you teach or take classes at USF, please feel free to follow the link to the LMS review page, view any of the presentation sessions, and comment below.
Note: The University of Central Florida is also evaluating LMS options.
What a great blog post – thank you for sharing this information. It’s been 8 years since we went through a process like this at my school (Univ. of Oklahoma) and we’ve been with D2L all that time; D2L was looking good 8 years ago, but it looks exactly the same now as it did 8 years ago, which is not so good. They are rolling out a huge version change that I imagine is going to be pretty disruptive, but unfortunately we did not get out in front of that to use it as an opportunity to review our other options. We are going to put ourselves through the upgrade to v.10 and the inertia clock will start ticking all over again. I wonder if we will ever have to re-evaluated.
Meanwhile, just from personal use, I am very impressed by Instructure Canvas. I’m the kind of online instructor who prefers to have everything open and I love the you can create open course materials with Instructure, just opting to have that be published on the open Internet with real addressable webpages, no guest log-in, nothing – the real and true Internet. I know that’s not something that most faculty will probably ever want, but it’s always been a dealbreaker for me. I keep zero content in our Desire2Learn cms because it is just not worth it to me to develop the content if I cannot share it with a wider audience. So, if I had to lobby for something at my school it would be Instructure but, more importantly, I would be lobbying for Google Apps (we don’t even have that) so that we could have website publishing, blogging and document sharing options. Right now, uh, we don’t.
Anyway, thanks for this post and I am very curious which way your school will go! I found out about your post via Bob Calder at Google+. Are you at Google+ also? I will look for you there!
You have shared a great example of the increasingly complex decision in choosing an LMS, or any software solution for that matter. What makes your situation problematic is in trying to compare traditional in-house run applications (apples) with those same applications run as off-premise hosted applications (oranges) and of course the additional dimension of commercially supported applications versus open source applications and the commercial variants of OSS.
Most IT investment decision models and RFP templates do not account for each of these dimensions. This forces organizations to treat the options as all the sames which is just bananas since they have different value models (risk, rewards, costs, & benefits). You can find some additional insights at http://blog.thehigheredcio.com/2011/11/10/rfp-template-open-source-cloud-computing/ [Sherman’s note: This is a URL of a commercial consulting site, which generally kicks comments into spam, but this is a substantive comment, so I pulled it out of spam to make it visible.]
I don’t think there’s a way to avoid thinking about apples and oranges. Your approach is interesting but assumes that a university can accurately predict things like net present value and internal rates of return. The obvious payback for an LMS is generally qualitative, or at least the reasons why a university might see benefits to a particular LMS have different levels of quantifiability. Given human nature, an attempt to conduct a rigorous ROI analysis is likely to lead people to analyze the most easily quantified items rather than the ones that should dominate a decision.
One other factor is how for-pay course content packages from textbook publishers integrate into the system you choose. Will the math homework and its grades be within the “L”MS or separate?
Thank you so much for this review – I’ve been using Moodle for years, and didn’t know about the Personalized Learning Designer (you’re right – what a stupid name) since I have never used joule. I find Moodle much easier than Bb because the kale chips are opt-in rather than opt-out. So many things are enabled in Bb that one wouldn’t use, and students get confused seeing all of them in the “tools”. And embedding media isn’t too bad in Moodle – I’ve had problems in a few areas, but not the forum where I need it to work.
The disadvantage to Canvas seems to me to be that you can’t change the names of the course buttons. What if I don’t have my class organized as they suggest? I wouldn’t use Bb back when it first came out until they had course menu buttons I could change, so in that sense Canvas feels like going backward.
Dear Sherman,
Thank you for your insightful and detailed personal analysis of the process, products and related issues. Considering a new LMS is a very difficult choice. Your blogpost is helpful in providing insight into the process at USF thus far and opening the discussion more broadly as the decision approaches.
My great concern is faculty learning a new version or new product. I learn technology easily, but I know it is an uphill battle for many people.
Therefore somehow we have to evaluate the cost-benefit ratio of multiple variables among multiple preferences and skill sets including, but not limited to, upgrading or switching LMS with learning curves, tech support, contracts, benefits and more…
Thank you for your contribution on behalf of the faculty and students!
Dr Kathy King
Thanks for chiming in!
(a few days later…) I’m trying to figure out how to think more clearly about the issues you raised in terms of comfort with an LMS. Any transition has to consider not only those who have invested time in learning the current system but also those who might pick up the new system as a result of change and those who are going to be hired in the future. It’s too early in the morning for me to sort through this, though… I need another cup of coffee!
Did anyone tell you that simply having the “ability to create grading rubrics” doesn’t mean you have the ability to use them as part of the grading process?
We appear to have Bb9.1, and that is what I learned during a training program on some of the new features. Maybe using it will come later, in 9.2?
You can use rubrics inline for grading discussion boards, blogs, journals, wikis and test questions. If that didn’t come through in your training, drop me an email and I would be happy to walk you through it. ramsey dot chambers @ blackboard dot com.
Great post, excellent thoughts, and a really good comparison. I wanted to chime in on Rubrics in Canvas. They are just cool. This is my second semester with Canvas, and I must say this is the first time I have been willing to use Rubrics, only because Canvas has made them really easy to use while grading. Even their iPad Speedgrader app does the rubric thing really well. I am hooked.
Really enjoyed this post. Our college recently completed an LMS migration, and it’s nice to know we’re not the only school who wrestled with decisions like this.
We were on WebCT CE. Our user base had outgrown the infrastructure, new technology was so far out ahead of the platform, and the lifecycle of WebCT was pretty much at an end. So, an upgrade was defiantly in order.
There were a few “must haves” on our shopping list:
1. As of Spring 2012, we’re the fastest growing community college in Texas, so scalability is a must have! The technology has to scale up as quickly as our program grows.
2. Over 60% of our total enrollments are online, so air-tight data security, disaster prevention/recovery, and “as close to 100% uptime” SLA are must-haves. The platform simply cannot go down.
3. Our teachers are not web designers or computer experts, and we have a ton of non-traditional students, so overall ease of use for teachers, students, and admins is a must.
4. One instructional support guy against hundreds of online teachers and courses means that built-in migration tools that work are a definite must.
Our final decision was between Bb9, Moodle, and Instructure Canvas. The original direction was to “upgrade” from WebCT CE to Bb9, but we heard a lot of scary stories about migration and the demos were not impressive. We actually found Bb9 a more difficult user experience than WebCT; it was definitely an “iceberg ahead” moment for us.
Our Moodle findings were pretty lackluster too.
Neither Moodle nor Bb9 were native cloud platforms, the migration tools were kind of sketchy, and ease of use just didn’t meet our requirements. Then, we met Instructure Canvas.
Honestly, we went into our first Canvas demo without a lot of hope, but by the end of the first demo, we knew Canvas was the platform for us. Just to make sure, though, we ran demos with faculty and students and had a sixteen week pilot with thirteen teachers/six hundred students.
So many great stories to tell! The boosted productivity and excitement of our teachers were really encouraging; they would tell us how much time the SpeedGrader in Canvas was saving them. The student experience was pretty great too: Only three (of around six hundred) students came to the pilot orientation, but hardly any (if any) help desk calls from students. Crazy.
So, what really rocked our socks and sealed the deal?
1. Canvas is a native cloud app. It’s scalable, load balanced, and physically redundant.
2. Functionality of the grade book, calendar, migration tools, and the rich content editor — these aren’t afterthoughts or extensions, they’re core features that our teachers can’t live without now.
3. SPEEDGRADER!!!!
4. Mobility. Canvas works great on mobile devices and there are iOS apps (including SpeedGrader for iPad) that we don’t have to pay extra for.
5. Support and product development. We know the support team at Instructure; they provide immediate support, especially for critical matters. And the development cycle is every two weeks! Improvements (most ideas coming from the Canvas community) roll out every two weeks!
…Okay, sorry to be so long winded. We are one of Instructures first customers, and although we’ve experienced some growing pains, we’re still really, really happy with Canvas. Not only has the platform itself improved our online program, but the excitement of embracing a new platform that gives our teachers and students everything they need to express themselves in learning has brought fresh energy to the program and the campus.
Hope our story is helpful. Wishing you all the best!
Jason Gilbert
Coordinator of Instructional Technology
Panola College
Our institution is currently in the middle of a gradual migration from WebCT/CE6 to Canvas, and so far the response from students and faculty alike to the new LMS has been overwhelmingly positive. We’re especially impressed with how easy it is to integrate third-party applications such as Khan Academy and Quizlet. Instructure has been receptive to our feedback and nimble in implementing changes. We expect to have our migration substantially completed by the end of 2012.”