Learning organizations and the IT organizations that support them are facing the difficult challenge of selecting and implementing the right learning management system (LMS) for their environment. This challenge is becoming more complex with new LMS vendo
by Site Staff
January 2, 2004
Selecting the appropriate LMS and/or learning content management system (LCMS) for an organization starts with identifying the learning strategy and requirements. Purchasing a learning management environment is a major investment, so it is important to clearly define and prioritize requirements in order to find the right LMS that will meet core requirements initially and grow to meet subsequent requirements in the future. Once a purchase is made, it is nearly impossible to replace it without significant additional costs, so it is crucial to understand the full scope of what an LMS should do for an organization.
There are more than 70 major LMS providers, and certainly not all of them follow the same business model. One provider may offer an installed solution, while another will only host content as an application service provider (ASP), and others will offer both. Some LMS solutions are great for small to medium-sized companies while insufficient for large enterprises, while others focus on enterprise solutions that are too overwhelming for smaller companies, and still others try to balance their product by offering scalable solutions.
I have also seen many postings in forums asking for LMS recommendations, but selecting an LMS simply based on the success it had within another company is ill-advised. There are many factors that may make a particular solution work in one environment and not in others. Consider the corporate culture, organizational structure, industry, types of learning programs, number of learning programs and other factors that need to match in order to consider such a recommendation.
It can also be a big mistake to select an LMS solely because of a pre-existing relationship with that vendor or that vendor’s partner. The common thinking is that it will save a lot of money, when in fact it could cost even more if the LMS cannot meet core requirements that are crucial in meeting business objectives. It is best to include such an LMS in an evaluation, but it is equally important to evaluate the LMS based on its capability to meet requirements.
There is no shortcut in selecting the right LMS for an organization, but what is considered in the selection process? In selecting an LMS and/or LCMS provider for enterprise-wide or department-wide training management, the following areas of focus are recommended for functional requirements:
- Skill Assessment: Skill assessment revolves around learners assessing their competency gaps, matching those gaps to a prescribed curriculum to address those gaps and developing a learning plan that is attainable and executable to close those gaps that were identified.
- Content Access: This area focuses on how learners access the content. This involves the medium (e.g., classroom, CD-ROM, online, etc.) in which the content is delivered, the method (e.g., instructor-led, self-paced, blended) in which the content is delivered, the languages in which the content is delivered and to whom the content is being delivered (e.g., employees, customers, partners, etc.). In addition, the access of the content from a learning content management system (LCMS) is also a component of this area.
- Enrollment and Tracking: Enrollment and tracking considers entries in the course catalog that learners can access, enrollment in event-driven classes and launching of online courses, collecting funds through an e-commerce process if applicable, tracking activities and completions, and generating reports on the information being tracked.
- Learning Evaluation: The learning and evaluation area involves the creation of survey instruments and test assessments to collect data associated with evaluating the effectiveness and efficiency of learning programs, as well as tools to analyze the data collected. Evaluation has become major factor for many companies in recent years, so the ability for an LMS to deliver on measuring learning is rapidly becoming a key factor in vendor selection.
- Marketing and Communications: If you build it, will they come? Not if they don’t know about it. The marketing and communications area is concerned with advertising through the Web, e-mail, personal newsletters, personal home pages, etc., as well as communicating status or announcements that learners need to be aware of. This area also considers communications between learners and instructors, learners and administrators and even learners and learners to form learning communities.
- Content Development: Content development encompasses authoring, maintaining and storing the learning content. This is where the issues of authoring-tool compatibility, version control and re-usable learning objects are considered.
It is also very important to consider business relationship areas when developing requirements, such as:
- Financial Stability: The financial stability of a vendor is of key importance in predicting their future in their industry. The learning management industry is a very competitive market that has seen some mergers and acquisitions as well as vendors going out of business. Consider the rating of the LMS vendor from financial reports and, if possible, reports from research firms.
- Account Management: Customer service is a key area to consider in a vendor, and talking with references can be critical in understanding how responsive a vendor actually is. A prepared questionnaire is an appropriate approach to equally evaluate key areas of a vendor from multiple resources.
- Support: When things go wrong, support is needed. Having a strong support structure will make a lot of difference when faced with irate customers. Consider a clearly stated service-level agreement that explicitly outlines roles, responsibilities and response expectations.
- Pricing: Pricing is obviously important, but I listed this last because the other factors I mentioned earlier could produce costly problems if they are not considered wisely. When considering pricing, also consider the quality of the product and what it might cost if another vendor was selected that didn’t conform to quality expectations. Some features may be traded in for a lower pricing structure, but be prepared to consider what is actually gained or lost. In the final analysis, the pricing is always negotiable, so it is fair to consider what payment would be expected for the functionality you expect to be used.
The LMS Selection Process
Consider the following seven-step process. It may not work for every situation, but consider it as a guideline.
1. Determine the Learning Strategy
As a learning organization, there should already be a clear learning strategy in place. If not, now is the time to develop one. In developing a strategy, consider the target audience—their learning preferences, their locations, the resources that are available to them to attend learning programs, etc. Corporate goals and objectives should also be defined and the strategy aligned to them. Also take into account budget constraints, potential realized benefits and return on investment.
This is not a trivial task, and it can be a whole separate initiative in itself, so bear in mind the time invested in this step. A learning strategy should reflect how learning programs are delivered to the people who need them to accomplish business goals.
2. Document Requirements
Specific requirements should be defined in each of the areas mentioned previously. One of the key factors in finding the right LMS for an organization is matching an LMS to requirements, not the other way around.
It is also important to prioritize requirements in a range from core (high) to low. High-priority or core requirements are absolutely necessary for the LMS to meet within the initial implementation or launch of the system. Any core requirement that cannot be met should dismiss the LMS from consideration. Medium-priority requirements are essential to be met in the initial or subsequent phases of implementation. This means that the LMS may not be able to meet the requirement for the initial phase, but a new scheduled release appears to meet the requirement or there is a commitment from the LMS vendor to meet the requirement in the near future. Finally, low-priority requirements are “nice to have” and can be delayed indefinitely, but also run the risk of being promoted to medium priority, so these priorities still have a bearing on how willing or open an LMS vendor is to considering them.
Another consideration is where gaps in the requirements for a particular LMS can be filled with customization or extension of the LMS and how well the LMS adapts to such customizations or extensions. Customization refers to changeable parameters within the confines of the out-of-box design of the LMS, while extensions refer to the ability to integrate or interface additional functionality not originally included in the LMS design. Some organizations struggle with the “build or buy” question when it comes to LMSs, so it is important to note the options that may be open to organizations willing to apply additional resources to meet specific needs.
Also, consider hosted versus installed systems. Hosted systems are maintained by the LMS provider, which acts as an application service provider (ASP). The LMS provider typically grants access to users of the LMS and provides support for the system should problems arise. Modifications or customizations beyond what the application supports in configuration screens may need to be done by the ASP and can be restrictive. Installed solutions, on the other hand, are systems that are installed within a company’s network. The support of the hardware and applications would most likely fall on the IT organization, but there would be more control over customizations and extensions. Compatibility to standards such as SCORM (Shareable Content Object Reference Model) and AICC (Aviation Industry CBT Committee) should also be considered.
3. Research LMS Companies
In order to make the most appropriate decision, it will be necessary to research profiles of each potential LMS and/or LCMS vendor. Information is usually available on their Web sites. Additionally, research and comparison reports may also be available from research firms.
The focus should be on key areas surrounding the core or highest-priority requirements. LMS companies will usually work with a client to meet lower-priority requirements through partnerships, customization or future releases. From these reports, a manageable list of companies that requests for proposals can be sent to should emerge.
4. Prepare the Request for Proposal (RFP)
The RFP should be prepared based upon the requirements. In the RFP, it is not necessary to indicate priorities of requirements, nor list them in any specific order, so that each requirement is responded to equally. Each requirement should be as specific as possible so that the LMS vendor can respond directly to the requirement rather than provide a general response.
Scenarios should also be included in the RFP. Scenarios describe very specific situations that the LMS/LCMS needs to accommodate. This will give a clear indication as to how the LMS vendor can meet specific situations.
A proposed project plan for implementation based on the requirements should also be requested. The project plan must include timelines relative between the start and the end of the project. This will provide an estimate as to how long the vendor perceives implementation will take, ownership for each task and the details of the tasks themselves. If the LMS vendor has had enough experience in implementation, it should already have a template of a project plan that could easily be applied in a proposal.
Finally, provide a short response time for the RFP. This will give an indication as to how hard a company will work for the business and can be a strong indicator as to how they will perform in a business relationship. It should not be the sole indicator, however; there are other opportunities to establish this type of estimate.
5. Review the Proposals
The review team should have sufficient time to review the proposals and establish a rating system that all can agree upon. Each rating should also include comments for both positive and negative impressions. In this manner, quantitative measures of the ratings are not only considered, but also subjective impressions of each criterion.
Once again the focus will need to be on the core or highest-priority requirements. These are usually the requirements that must be present in order to consider the system. If even one of the core requirements cannot be immediately met by the LMS, that LMS should be eliminated from the list. For this reason, only core requirements that truly represent imperative functionality should be incorporated.
The result of the review should lead to a short list of vendors.
6. Schedule Meetings and Demos
After the proposal review is complete, meetings and demos should be scheduled so that the vendors can answer specific questions and demonstrate their claims on the proposal. They should also be required to demonstrate the scenarios provided. This is crucial in determining how compatible or flexible their environment is. It is also important to make clear what part of the functionality is included out-of-box with minor configuration changes and what part requires customization beyond the quoted price.
If any of the review team needs to attend virtually, it would be good opportunity to utilize a vendor’s distance-learning solution. This will provide the capability to experience part of the environment as the learning audience would.
Be sure to question any part of the functionality or implementation that is not clearly understood. It is important that the account representative is able to explain functionality clearly and without ambiguity. Additionally, the flexibility of the project plan should be explored. An organization should not be required to adhere to processes that conflict with internal processes.
7. Make the Selection
Finally, a selection can be made after carefully reviewing and internally discussing the impressions made by each vendor during each meeting. This is a serious and long-term investment, so it is important to have complete cooperation among the members of review team. It is also important to build in contingency plans in case certain features that are expected in the initial implementation are not done in time, or other unexpected delays or problems arise.
This is just a high-level view of an approach to thoroughly examining LMS/LCMS vendors and is flexible to shorten the cycle, although at the cost of quality. Consider that LMS solutions can be a huge investment and if implemented too quickly, can lead to enormous costs later in additional effort to meet requirements.
Potential results from employing the approaches discussed in this article are presented for demonstrative purposes only. Actual results may vary from application to application and are not guaranteed.
Pat Alvarado is an independent consultant specializing in corporate learning and learning technology. Pat can be reached at pat.alvarado@elearning-engineering.com.