A Guide to LMS Evaluation Criteria

Choosing a Learning Management System (LMS) is one of the most important decisions you have to make if you offer educational programming of any kind. And it’s not an easy choice. This guide will outline some of the reasons why and help make the decision-making process easier for you.

We also have a free template to break this decision down into smaller steps, walking you through the process whether it’s the first time you’ve done it or you’re replacing a previous system. We’ll help ensure you get the best solution you can for your organization and the people who learn from you.

Let’s get started with an informative and helpful journey through the finer points of selecting from the many LMS solutions available.

What’s in an LMS?

A learning management system (LMS) is a software application that allows teachers and students to access course materials from anywhere at any time. The main purpose of using an LMS is to provide a virtual learning environment where learners can interact with each other and their instructors, access course content, and track progress toward their educational goals.

LMSs have become more important than ever in today’s educational landscape. They allow educators to create courses, manage assignments, track student performance, and communicate with their students, and they allow them to do it all remotely. With the changes in the field of education brought on by the pandemic, the remote learning functionality of an LMS is more important than ever before.

The market for LMSs is highly fragmented, and that means that there are more options than can be easily accounted for. Because so much relies on these tools, it’s important to be sure you’re getting the right LMS for your educational programming, your budget, and your learners.

Choosing an LMS

Now that you’re familiar with the details of what an LMS is and why you need one, let’s talk about what to look for when you’re considering the options. This is a simple but effective process.

Determine Your Required Features

Before you can begin looking closely at different LMS options on the market, you need to decide what you want. Otherwise you can get overwhelmed by the volume of choices. 

It’s important to define which features you absolutely must have in an LMS system. If you’re switching from an older system, this may be easier to determine than if you are choosing an LMS for the first time. 

Consider why your organization chose its previous LMS. Are the needs the same or have they changed? Having a detailed picture of your organizational needs can help narrow down the options as you begin evaluating the various LMS options.

Nonprofits and associations have different needs than other kinds of institutions that use LMSs like universities and colleges. The audience that takes your courses may look very different from the traditional university classroom, or you may focus more on certifications and licenses than other kinds of educational programming. 

It’s important to keep the nature of your educational programming in mind not only as you’re defining your organizational needs, but also as you’re researching different options. Messaging that’s intended for one kind of LMS customer may not be as relevant to others. 

If you’re choosing an LMS for the first time, or if you’ve done it before and want to make sure you’re considering the decision as fully as possible, this list of criteria can help you make sure you’re finding what you need. 

We’ll start with the criteria that are likely to be the most important to you, like cost, and move toward those which are important to consider but may be simpler, such as the quality of the vendor’s public profile. 

Most importantly, we’ll take this potentially overwhelming decision and break it down into more manageable steps so that you’ll be set up for success.

LMS Evaluation Criteria

It’s no secret that breaking a big problem down into smaller issues makes it more manageable. Software products are complex, and with many options available, it’s easy to get overwhelmed.

We’ll walk through a list of major criteria when considering an LMS, talk about why each criterion is important, and help you turn this big decision into a series of easier considerations.

Cost

It’s hard to overstate the importance of cost when considering an LMS. Even if an option is nearly perfect in every other category, if the cost is too high it’s most likely not going to be an option. It’s important to get a clear sense of what your organization’s budget is for an LMS before you start investigating your options.

Discounts

But there’s more to consider than just your budget. Are there discounts you can find for the LMS you want? Memberships with associations or other organizations can sometimes earn you significant subscription markdowns on some options. Vendors may do promotions or sales you can take advantage of as well.

Pricing Models

Cloud-based LMS solutions can use different pricing models, and it’s important to calculate the effective cost if you’re considering options using different models. You may simply pay a licensing fee, or you may pay a more variable rate by number of registered or active users, or even the time you spend using the LMS. Being able to accurately compare across these pricing differences depends on having a clear understanding of how you’ll be using the system.

Soft Costs

Think beyond the obvious, upfront costs of purchasing or subscribing to an LMS. Setup, ease of use, and other ongoing costs can influence the long term costs of an LMS. If a system comes with minimal or no support and requires large amounts of labor within your organization to maintain, that’s part of the cost of the system, and one you’ll probably want to avoid.

Do the Research

Cost, like it or not, is never simple. It is, however, a critical consideration to make and make fully when considering an LMS for your organization. Be sure to thoroughly investigate both the short-term and long-term costs of any system you’re seriously considering switching to.

Technical Requirements and Features

Technical requirements and features are almost as important as cost when considering your options. Everything you need your LMS to do, the number of people that can use it, and the systems that can run it are all part of this criterion. 

Is the LMS mobile-friendly? Does it integrate easily with other software systems you rely on? For instance, TopClass LMS by WBT Systems integrates directly with AMS software iMIS. Not all LMSs do this out of the box without custom LMS integrations.

Does it have all the assessment options you need? This is where thorough analysis of what you need out of an LMS and a complete list of requirements will go a long way. It’s a good idea to eliminate any options you’re considering that can’t meet all your requirements for this criterion. 

You’ll also need to consider what kind of hosting you want for your LMS. Usually this choice will be between a cloud-based option or an on-premises option. There are reasons for both. Cloud-based systems are easier and more cost-effective to scale, for example, while on-premises options can give you a greater deal of customization and control at the expense of being more resource-intensive to maintain.

UX and UI

You want people to want to take your courses, and that means you want people to want to use your LMS. User experience (UX) is an important criterion then, because it refers to how people interact with your LMS and how they feel about using it. Frustrated users spend less time learning and more time wrestling with the LMS.

Frustrated learners also don’t learn as much as those who enjoy using the system. A user interface (UI) that promotes good UX is usually less noticeable and obtrusive: it doesn’t present unintended obstacles to engaging with course materials, discussions, and other aspects of your educational offerings. Your students know where to find things and how to do what they want to do. It just works. 

Good UX means lower stress levels, which are linked to higher performance on tests and other course requirements. If you want your students, employees, or anyone else learning from you to succeed, you owe it to them to provide an LMS platform that gives them the opportunity. A good UI is the key component of that platform. 

How can you test out the UI of an LMS you might want to use? Request a product demo. There are a few ways vendors may offer the chance to try their product before you buy it. For instance, Brightspace by D2L offers free demos of their software for customers interested in the LMS. 

You may be able to sign up for a live demonstration in which a representative will show you the features of the LMS, explain how it works, answer your questions, and more. Another possibility is that a vendor may offer a trial version of the software you can use for a limited time to get a feel for how well the software fits your needs. 

A vendor should offer some way for you to get to know the UI and features of their LMS before committing to a subscription or purchase. It’s not only a crucial component of your decision-making process, it shows that the vendor believes in and stands by their product.

Implementation

Implementing your LMS is a process that begins largely after you’ve selected a new LMS, but that doesn’t mean it shouldn’t factor into your decision-making process. 

One of the main things you can think about before you’ve chosen an LMS is whether or not the vendor can assist in implementing that particular option. A vendor should be able to provide a project-plan template, a document that walks you through the basic tasks associated with implementing the LMS. 

Remember the project-plan template is not exhaustive; it won’t contain anything about the many tasks specific to your organization that will need to happen during implementation, but the vendor may be able to advise you on these tasks separately. 

Implementation within your organization is a team effort, and will require the involvement of project managers, IT staff, educators, and anyone else who would use the system. Together you’ll be able to make a plan to implement the LMS over a certain period of time that will depend upon the kind of LMS you choose and the needs of your organization.

On-premises systems generally take longer to implement than cloud-based ones, so you will need to be sure that you have sufficient time to implement the LMS you choose before you make a decision. 

There is much more to the process of implementation that comes after you’ve made the call. Data migration is a large part of it. Be sure to discuss what needs to happen with the rest of your team and be prepared to start the implementation process soon after selecting an LMS.

Technical Support and Training

In a perfect world, everything would work exactly as it supposed to, without unforeseen issues. Part of purchasing a car is researching common issues the car may have, and doing the same for an LMS is advisable. It’s a complex platform, and like any complex platform there is room for errors and bugs. So do your research on any system you’re looking at to see if there are common issues that users have with it. 

In the process you will likely come across an important source of technical support, the user community. Many LMS and other software vendors moderate these communities and provide technical support as needed. It can be a wealth of information before you make a purchase as well as a continuing source long after. 

An LMS vendor should also offer more traditional forms of technical support such as phone, email, and live chat support. If something goes wrong, it can be a threat to your educational programming, and you need to be able to reach a support technician to find a solution quickly. 

Because an LMS is a complex and potentially customizable tool, technical support is not all a vendor should offer. The vendor should offer training to teach you and your team how to use the LMS effectively. This training can come in more than one form. 

A knowledge base is an essential resource that gives you 24/7  access to knowledge articles, instructional videos, demos, and other content to help you work through issues and learn more about the LMS. The vendor should keep this knowledge base up to date. 

The vendor may offer other training opportunities as part of the purchase process. It’s a good idea to take full advantage of any offering like this. 

If an LMS you’re interested in does not have a robust training and support structure available, look at a different LMS! Being stranded with a seemingly unsolvable problem down the road is not conducive to learning or teaching, and it’s a sign that the vendor doesn’t really stand by their products or care about their customers.

Security and Compliance

Data security and regulatory compliance are important when it comes to online learning. Because your LMS hosts a large amount of learner data, there are responsibilities you, our organization, and your LMS provider have to safeguard that data. 

When choosing an LMS, be sure that there is sufficient information from the vendor on their approach to data security and compliance. They should be able to offer specific information on regulations such as GDPR (General Data Protection Regulation), WCAG 2.0 (Web Control Accessibility Guidelines), ISO certification, and other regulatory standards like HIPAA and FERPA.

Although you have a legal and ethical obligation to protect the information hosted on your LMS, looking into the security and compliance information available from an LMS vendor is also a good way to get a feel for the credibility of the vendor. If this information is absent, it’s likely that the vendor is not as trustworthy as other LMS providers on the market.

Vendor Profile and Reputation

LMS vendors are closely associated with the platform they sell, as it is often their core offering. Researching vendors to get a better sense of what kinds of clients like their product, how satisfied they are with it, and what common issues they may have is helpful in determining which LMS is right for you. 

Software review aggregators like G2 are a good place to start looking for this information. You may also find reviews on social media or video sharing platforms like YouTube. If you are a member of any professional associations, you may be able to reach out to them for additional insights about a vendor.

Summary and Conclusions

Although there are many steps to choosing an LMS, if you follow them the choice is usually not that hard. At this point, you should be able to eliminate many choices that do not meet your needs in one or more of these criteria. As you go through this process, the picture of which LMS is right for you will get sharper until only the clear choice remains.

Unlike some other kinds of software, the LMS market has many viable and respected products without a clear leader in all categories. Although this requires a more rigorous decision-making process than for other software purchases, it also means there’s more competition between LMS providers, and that means there’s greater incentive for providers to create a truly outstanding product and offer it at a competitive price. 

But although the market conditions are likely to produce a lot of high-quality LMS products, it’s still important to put in the work to make the best choice for your organization. Your educational programming is highly unique from that of other LMS customers, and so will your priorities. 

An LMS is the meeting ground between you and the people who learn from you: so much of what happens between you and your audience is mediated by your LMS, maybe even everything that happens if you work exclusively in online learning. Finding the best LMS option means delivering the best educational experience you possibly can.