Evaluation Instruments

Level One Evaluation

At the completion of the each of the modules, participants will be asked to complete a level one evaluation.  All level one evaluations will be computer based.

Likert Scale Questions

Please indicate the extent to which you agree/disagree with the following:
[scale from 1-5 provided; 1 = strongly agree, 3 = neutral, 5 = strongly disagree]

1.       The purpose of the training module was clearly stated.

2.       The content was presented clearly.

3.       The training module was engaging.

4.       The content of the module was applicable to me.

5.       The module was easy to navigate.

6.       The module was delivered without any technical issues.

Open-Ended Questions

7.       What was most useful in this training?

8.       What was least useful, and why?

9.       Additional comments.

Level Two Evaluation

At the conclusion of each module, participants will be required to complete a level two evaluation which will be administered through the company LMS.  The quiz for each module most be successfully completed (scoring at least an 85%) to display mastery and receive credit for completing the course.

Navigation & Functions

Participants will be evaluated on their answers to the following questions:

1.        List the primary functions used in Lotus Notes.

Answer: The primary uses of Lotus Notes at Company XYZ is for email, calendar, information storage in the form of databases, and electronic forms also within databases.

2.        Describe the structure of the database system.

Answer: Databases are grouped according to the functional group who is the “owner” of said database (e.g., Travel and Expense database is grouped with all other Accounts Payable databases).

Searching in Databases

Participants will be given the following scenario and a simulation screen in which they must successfully complete the steps based on the given scenario:

You recently went on a business trip for Company XYZ and need to submit an expense report to apply for reimbursement.  This is completed through the Travel & Expense Database of which you have never accessed or used.  Using the simulation Lotus Notes site on the screen, demonstrate how you would go about searching for and accessing this database in order to successfully submit your travel expenses.  Clicks are tracked, so be cautious to clicking erroneously as that may result in a failing score.

Level Three Evaluation

After 60 days of completing the Lotus Notes modules, the participants will be provided with an electronic survey, similar to the one provided in the performance gap analysis, to measure if the training met their needs, the transfer of knowledge to their job tasks, and if the learning objectives were met.

In addition to this survey, a report will be gathered to analyze the IT Help Desk requests submitted for Lotus Notes assistance.  Cross checking will take place to see if associates who have completed the course still required IT assistance in the material covered through the training.  If it is found that these individuals did still need help, an interview will be held with them to determine what they needed help with, and then determine if any changes need to be made to the material and/or delivery of the content.

Level Four Evaluation

The performance gap analysis conducted for this project indicated the following current and desired states:

Current State - Of the 576 respondents, 13% reported understanding how to search for databases in Lotus Notes.

Desired State - 100% of associates should be able to easily and quickly search for required databases in order to efficiently perform their job duties.

Delta - Based on the sample of respondents, the percentage increase in database searching capabilities should be 87%.

In the first year of this training being implemented, data based on the level three evaluation will be collected quarterly to determine if Company XYZ is approaching its desired state of performance.  At the end of the first year, the data will be analyzed to determine if further action needs to be taken to close the performance gap or if the established path meets the business needs.