Master’s Thesis on “OS software evaluation model” focused on Course Management Systems

http://www.karinvandenberg.nl/Thesis.pdf

This thesis by Karin van den Berg was part of her Master’s program at Tilburg University in the Netherlands. It does a nice job of boiling down many of the previous attempts at developing criteria for evaluating open source projects (see her references for the extensive list) and then uses as a case study the field of course management systems to try out the evaluative framework. Somewhat unsurprisingly to me, she hones in on Moodle and Atutor as being the top two contenders when these additional open source considerations are factored into the equation; it is nice though to have some substance to back up ones’ instincts.

This piece was quite close to my heart for a few reasons, and thanks to Stuart Yeates posting on the Educause community blogs for finding it. It’s meaningful to me first off because I’ve been looking at these CMS thingies for far too long now through the Edutools project. It was meaningful to me because she cites Edutools numerous times throughout the thesis, and it is nice to get some academic ‘props.’

And finally it is meaningful because for the last year I have been going around giving a presentation that I think basically says a lot of the same thing, though I frame it slightly differently. The jist of that presentation is that making good open source choices is all about picking projects that are a suitable fit with the capabilities and maturity of your own organization, and trying to educate people on what some of the qualities of OS projects are that they can base those judgements on. The presentation was actually a summary of a funding proposal for an “open source suitability decision making tool,” a proposal that didn’t get funded. And I am really glad it didn’t get funded. Not because such a tool wouldn’t be useful. But instead because, as one of the reviewers astutely pointed out, and as this thesis backs up, the judgement of an open source project’s “maturity” is too multivariate (and evolves too quickly) for it to be serviced well by both such a small one-time grant (we were really not asking for much) but also from such a centralized research model as we had proposed. Still, it is really vindicating for me to see basically the same set of criteria brought out in full in this thesis, as it makes me fill slightly less half-cocked (so, what, two-thirds cock’d?) – SWL

ELGG vs. Moodle – defusing a false dichotomy

http://www.unisa.edu.au/odlaaconference/
PPDF2s/13%20odlaa%20-%20Anderson.pdf

I’ve had a few people come up to me at conferences recently and ask me to compare ELGG and Moodle, and choose between them as if they were somehow mutually exclusive. Indeed, even within the Moodle community itself there seems to be a bit of dismissiveness about what ELGG does, and the notion that with just a couple of twists of code Moddle can easily replicate its functionality.

Well maybe, but this is what excited me so much about the paper linked to above by Terry Anderson and the work he describes taking place at Athabasca University. I had the pleasure of seeing Terry present on this recently and wish I could link to those powerpoints as I think the illustrate the point I’m trying to make better than the article does, but what is exciting for me is that Terry and Athabasca are putting together a large, production environment in which Moodle and ELGG will seemingly co-exist quite nicely, thank you very much, and take care of different problems. Hopefully I am not going to mangle this too much, but as I understood it, Moodle was being positioned to handle conventional ‘course management’ problems like the delivery of content, assessments, discussions. In Athabasca’s case (and I’d argue in all of our cases, but that’s another post) they also have to deal with a continuous uptake model, where instead of cohort-based programs they also have very much self-paced programs with differing start times. Thus they are using ELGG as one of the ways to build community “between” the space of courses, community that is formed not because of one’s membership in a pre-ordained group or cohort but out of your interests. Sounds to me like a job for social software!

Can Moodle support similar ad-hoc community formation across course (and even institutional) boundaries? Maybe, and it sounds like we will find out fairly soon through upcoming releases. And bully for them if they can. But what I love about ELGG is that it is built from the groud up around the user and their connections as they key focus, rather than on ‘courses’ or ‘content’ (I’m not trying to levy a criticism at Moodle here as I like it very much as well). Far from being only a ‘blogging’ tool or a ‘eportfolio’ tool, what excites me about ELGG is that it is becoming a social networking ‘framework’ (o.k. you can dispute that term as much as you like) that while it has initially focused on tools to create blog posts and share files, isn’t interested in restricting you to only its blogging tool (and why would it? RSS anyone?) and is looking at a whole set of other interesting apps (Calendaring? Synchronous tools?) that are also of intrinsic value but become even more useful if people can use them with other semantically related users.

Should elearning providers be looking to one single tool to provide all of these aspects and more? Maybe. Right now though, the best bet seems like trying to get the best solution possible through a set of provisional measures. Personally, I’m more interested in making these and others co-exist, and seeing if we can get the integration between them to be more than lame-ass ‘pointing to their URLS’ or simple single sign-on; if instead we see if we can get shared identity happening across a number of these services in a way that takes identity mean more than your username and password. – SWL

ECAR Students and IT Study: CMS Chapter

http://www.educause.edu/ir/library/pdf/ers0506/
rs/ers05065.pdf

It would be easy to make too much of this chapter from a recent ECAR report on Students and Information Technology, 2005: Convenience, Connection, Control, and Learning, as it seems to paint an overall positive picture of student experiences with CMS. In the highlights it states that of students who report having used a CMS in one of their courses, 75% reported a positive or very positive experience.

A deeper look reveals, however, that the respondents were reflecting overwhemingly on an experience of using a CMS in a blended or mostly f2f situation, and that much of what was viewed as positive was focused on CMS’ ability to facilitate somewhat administrative tasks, like accessing syllabi, course notes and grades. – SWL

More on the new behemoth – Timing, Open Source and Interoperability

So I’ve had a bit of time to digest the <a href="http://www.blackboard.com/webct/"big announcement yesterday and process what I heard on the analyst call, as well as see some of the feedback from around the edtech blogosphere. Here’s some more thoughts:

Timing and Rationale
First off, let’s put aside any euphemisms about this being a “merger” of equals. This was an acquisition by the largest player in the market (Blackboard, with $90 million in licenses last year) of its next biggest rival (WebCT with approximately $30 million last year in licenses). The offered price allows WebCT’s venture backers to recoup their investment (approx $120 million) with a decent return, far better than they were going to see anytime soon from the 5% operating margin that WebCT was turning at the time of the acquisition. Blackboard has been turning a profit plus is cash rich after it’s IPO, and with that and its line of credit, was in position to take over WebCT.

And the timing does make sense, in retrospect, in relation to WebCT’s recent release of CE6. Very few customers of their existing CE4 product have had time yet to do the upgrade. Sales for Vista have I think not been as good as hoped. So WebCT CE4 customers, expect some vigorous sales calls from Blackboard folks in your near future. BB hopes to convert many of these customers to its platform, and for those it doesn’t, it can still profit from sales of CE6 licenses without having to invest much by way of development in the near future. It will be surprising if there is anything much more than a few bug fix releases of WebCT CE6 before the new behemoth platform emerges.

Open Source
When challenged with the threat of “open source” both Blackboard (and WebCT in the past) have made incredibly vaporous statements in the past about how they are “open systems” if not open source, in an attempt to stave off interest from their customers in open source options. No doubt they will continue to do so, and Bryan Alexander’s right, we are likely to see more FUD coming from that camp. But on the analyst call yesterday they did come off as relatively nonchalant about the open source threat, and unfortunately not all of that is posturing.

Undoubtably, there are existing and future customers who will choose to go with the likes of Moodle and Atutor. Maybe if some of the bigger schools would weigh the value of ease of use and pedagogical flexibility more heavily than IT concerns we’d see even more adopt these two and for them to get the respect they deserve (I know, I know; look I’m not levying this claim, I’m just reporting what the common perception is, rightly or wrongly).

But for now at least, the promise that has been held out for those ‘bigger’ institutions has been lately from Sakai, the ‘enterprise’ open source CMS. And in my eyes this is the real tragedy here. In theory this presents the perfect opportunity for Sakai to shine and come into its own, to convert a ton of both BB and WebCT customers. Certainly, David Wiley seems to lament it not having showed up on a recent RFP in his state. Well true enough, I can understand David’s frustration at it not even showing up in the competition. But in reality it would have had to be an RFP so heavily weighted to business concerns at the expense of current functionality for Sakai to have stood a chance. You may hate the vision of online learning that CMS represent, but if you are trying to compare apples to apples, I just can’t see how the current release of Sakai measures up to these loathed commercial competitors.

But wait, what’s that you say – “but it’s open source, it will grow and and bloom as more people adopt and develop it.” Well, maybe. Hopefully. Last I heard, the soft money didn’t have too much longer to go and there look to be only a couple of instances actually in production. And maybe someone from inside that project can comment to the rest of us how many developers who are not funded through that soft money, and who are outside the “core schools,” are actually contributing code back into the core project right now. ‘Open Source’ at its best means more than just ‘source code availability.’ I will leave it at that lest I spark a religious debate. I do truly wish that project well, as monoculture in elearning is not a good thing. But I speak from personal experience in saying that adopting software solely or primarily because it is open source, and not weighing heavily enough its fit to functional requirements and one’s own capabilities for rectifying that lack of fit, is a fatal mistake in software acquisition and development..

Interoperability and Open ‘Standards’

This is the piece that really gets my goat. IMS has been around since 1997. The Content Packaging, QTI and Enterprise specs (the ones I take to be of primary concern when it comes to CMS portability) are now all at least 5 years old, if not more. And yet all of us, yes US, the adopters, implementers and purchasers of CMS, have given the commercial CMS a pass on these. (SCORM is a different story here, but unfortunately it’s never really applied that well for the higher ed sector, nor for the CMS that service it).

It’s not totally our fault – yes, the specs were a moving target for a long time. Yes, compliance testing likely means a load of liability insurance that no one can afford. Yes, there are good reasons to accept that the specs need to be able to be extended to accommodate things they couldn’t do in their original incarnations.

But we’ve accepted all of these excuses and what do we have? Instead of content interoperability and portability between systems, we basically have vendor lock in, the very thing the freaking specs were supposed to help avoid! And that just got a world worse too with this consolidation. David Davies has it exactly right when he says that “BigCo vendors were cautious about embracing interoperability too vigourously.” But who the heck thought they were ever going to adopt these voluntarily? Say what you like about the military and SCORM, but there are a whole lot of LMS that spent money to conform to that specification, and we know they do because there is a test you can run to confirm this, and there are procurement folks who insist they prove it before they are awarded a contract.

</end of rant>

Wow, I must have been bottling a lot of stuff up over the last few months of not posting to the blog. At least that’s the excuse I’ll give to anyone I pissed off with this write up! Anyways, that’s my limited view of things. Good luck! – SWL

More on WebCT aquisition by Blackboard

On the analysts call the first piece of work that was identified to bring the two products closer together was unifying an API for the Blackboard and WebCT.

So, what does that mean for things like the IMS Tools Interoperability Profile? Well, seems to me like the new unified API becomes a de facto ‘standard’ that will be even harder to displace for any of the more open approaches to integrating 3rd party tools into CMS. – SWL

HOLY $#@! – Blackboard and WebCT to merge

http://investor.blackboard.com/
phoenix.zhtml?c=177018&p=irol-newsArticle
&ID=767025&highlight=

Well, there goes any claim to being an insider! Just found out about this a few minutes ago and had no idea it was in the works. I believe this is true as I am sitting on the analysts conference call right now. You kind of knew something like this had to happen, but still it’s profoundly shocking now that it has, especially as WebCT is just pushing WebCT Campus Edition 6 out the door in the last few months, which meant it was no longer supporting 2 code bases. Somewhat unsurprisingly, it was referred to as an ‘acquisition’ a number of times, which seems really like what it is. – SWL

New O’Reilly Publication – Using Moodle

http://www.oreilly.com/catalog/moodle/

As if more proof was needed that Moodle has “crossed the chasm,” along comes this new publication from O’Reilly written by Jason Cole. The majority of the content you could likely glean yourself from Moodle’s various online communities, help docs and demo courses, but if for instance you have an administration that remains skeptical about the widespread nature of Moodle adoption, maybe this might help convince them. – SWL

Moodle Forum – Blogs, Forums and the nature of discussion

http://moodle.org/mod/forum/discuss.php?d=27338

One of the things I love about the Moodle community is that, far more than almost any of the other open source CMS, they seem to have really rich discussions about the pedagogical uses of the tools they are building, not just their functionality utility or technical challenges (AND, you can even view them as a Guest if you are adverse to new accounts). And this particular one is no different – starting with a post from Moodle’s founder, Martin Dougiamas, this thread (55 posts long in 2 weeks!) discusses some of the ins and outs of blogs versus discussion forums, and starts to tackle the issue in light of the secured environment that CMS like Moodle provide, and that in many contexts (read K-12) likely cannot be dispensed with. The Blog feature itself is promised in an upcoming (1.6) release, but a demo can already be seen. – SWL

Presentations available from 2005 Alt-i-Lab sessions

http://www.imsglobal.org/altilab/

June has been a busy month in the post-secondary elearning world; along with the release of Sakai 2.0, another major milestone happened this month at the Alt-i-lab sessions in Sheffield, England. The page above links to many of the presentations and demonstrations that took place there, possibly most notable of which was the first practical demonstration of the Tool Interoperability across multiple CMS. A summary of the demonstration by Chris Vento is available, which seems to be cause for cautious optimism; unfortunately, the only ‘independant’ report I’ve been able to find (not having attended myself) is this one from the Learning Technology Standards Observatory. One can only hope that Wilbert Kraan and the folks at CETIS will come to the rescue with another of their lucid and helpful write-ups to explain what this really all means.

But I would be remiss in not pointing to some other sessions of note; for me the one that jumped off the page as I read further was the working session on “A Common Cartridge for Robust Content Delivery.” This group basically proposed to tackle the problem of content interoperability once more in light of the current situation:

“It’s five years later. The major elearning providers have implemented IMS specifications; many customers mandate compliance with some form of them. However, software vendors and suppliers, consumers, and maintainers of content have not worked together to create a detailed de facto understanding of what implementation means. So while elearning firms market ‘compliance’ with IMS specifications, and some have been certified as compliant with a specific version of the specifications; the lack of practical interoperability has left us in a place not sufficiently different than where we were prior to the IMS specification effort began.”

It’s nice to see the problem being owned up to (no real news to folks in the trenchs who have become increasingly dismayed as the variety of implementations of IMS Content Packaging failed to bring them the content portability and freedom from vendor lock-in they had hoped for). Too early yet to say if the proposed idea of “Content Cartridges” can have any better effect, but the idea of compliance testing and publisher involvement in the standards both seem improvements. – SWL

Sakai 2.0 Review posted on Edutools site

http://www.edutools.info/course/productinfo/detail.jsp?id=262

I don’t normally post notifications of every new review we do on the Edutools site, but in the case of Sakai there has been a lot of interest from this community and so I thought it might be warranted. As always, we endeavor to provide descriptive, non-evaluative reviews of the software within a framework that allows you to compare it with other known quantities and for you to make the judgements yourself. For instance you can view a side-by-side comparison of Sakai 2.0 with Blackboard 6 and WebCT CE 4 and Vista here, or look at it in comparison to some other open source CMS (Moodle, Atutor and .LRN) here. – SWL