π Lecture 21 β Transfer of Training (continued)
π Overview: This lecture continues the discussion on transfer of training, focusing on the key factors that support the application of learned capabilities on the job. It explains how manager support, peer support, opportunity to use learned capabilities, and technological support each play a critical role in ensuring that training content is effectively transferred and maintained in the workplace.
ποΈ Topics Covered
This lecture covers four major factors that support transfer of training: manager support, including the use of action plans and different levels of managerial involvement; peer support, through support networks, newsletters, and mentors; opportunity to use learned capabilities, defined by breadth, activity level, and task type; and technological support, specifically through electronic performance support systems (EPSS). Each factor is explained with practical strategies and measurement methods.
π Lecture Summary
Factors that support transfer of training
Manager Support
Manager support refers to the degree to which traineesβ managers (1) emphasize the importance of attending training programs and (2) stress the application of training content to the job. Managers can communicate expectations to trainees as well as provide the encouragement and resources needed to apply training on the job. One company asked trainees and their bosses to prepare and send memos to each other describing what the other person should βstart to do,β βcontinue to do,β βdo less,β or βstop doingβ to improve learning transfer.
Managers can provide different levels of support for training activities. The greater the level of support, the more likely that transfer of training will occur. Managers should be actively involved in the design and delivery of training programs. The basic level of support is acceptance, allowing trainees to attend training. The greatest level of support is to participate in training as an instructor (teaching in the program). Managers who serve as instructors are more likely to provide many of the lower-level support functions, such as reinforcing use of newly learned capabilities, discussing progress with trainees, and providing opportunities to practice. To maximize transfer of training, trainers need to achieve the highest level of support possible.
Managers can also facilitate transfer through reinforcement (use of action plans). An action plan is a written document that includes the steps that the trainee and manager will take to ensure that training transfers to the job. The action plan includes (1) a goal identifying what training content will be used and how it will be used (project, problem), (2) strategies for reaching the goal (including what the trainee will do differently, resources needed, and type of support from managers and peers), (3) strategies for receiving feedback, and (4) expected results. The action plan also provides a progress check schedule of specific dates and times when the manager and trainee agree to meet to discuss progress.
The action planning process should start by identifying the goal and the strategies for reaching the goal. Once those are determined, strategies for obtaining feedback and identifying what goal accomplishment will look like are completed. To complete their action plans, trainees may need additional technical support, such as access to experts or reference materials. Trainers or project managers can help trainees get the resources they need to complete their action plans through either face-to-face or electronic meetings.
π‘ Why this matters: The level of manager support directly determines whether training is used on the job; without active managerial involvement, even the best-designed training may fail to transfer.
There are several ways to gain managersβ support for training. First, managers need to be briefed on the purpose of the program and its relationship to business objectives and strategy. Managers should be given the schedule of topics and a checklist of what they should do after the training. Second, trainees should be encouraged to bring work problems and situations to the training session, which can be used as practice exercises or put into action plans. Third, information regarding the benefits of the course collected from past participants should be shared with managers. Fourth, trainers can assign trainees to complete action plans with their managers. Fifth, if possible, use managers as trainersβtrain the managers first, then give them responsibility to train their subordinates.
π Definition β Manager Support: The degree to which traineesβ managers emphasize the importance of attending training programs and stress the application of training content to the job. π Levels of Manager Support: Basic (acceptance/allow attendance) β Highest (participate as instructor) π Definition β Action Plan: A written document that includes the steps that the trainee and manager will take to ensure that training transfers to the job. π Example: A progress check schedule sets specific dates and times when manager and trainee meet to discuss progress in using learned capabilities on the job.
Peer Support
Transfer of training can also be enhanced by a support network among the trainees. A support network is a group of two or more trainees who agree to meet and discuss their progress in using learned capabilities on the job. This may involve face-to-face meetings or communications via e-mail. Trainees may share successful experiences in using training content on the job. They might also discuss how they obtained resources needed to use training content or how they coped with a work environment that interfered with use of training content.
Trainers might also use a newsletter to show how trainees are dealing with transfer of training issues. Distributed to all trainees, the newsletter might feature interviews with trainees who have been successful in using new skills. Trainers may also provide trainees with a mentorβa more experienced employee who previously attended the same training program, or a peer. The mentor can provide advice and support related to transfer of training issues (e.g., how to find opportunities to use the learned capabilities).
π Definition β Support Network: A group of two or more trainees who agree to meet and discuss their progress in using learned capabilities on the job.
Opportunity to Use Learned Capabilities
Opportunity to use learned capabilities (opportunity to perform) refers to the extent to which the trainee is provided with or actively seeks experiences that allow for application of the newly learned knowledge, skill, and behaviors from the training program. Opportunity to perform is influenced by both the work environment and trainee motivation. One way trainees have the opportunity to use learned capabilities is through assigned work experiences (e.g., problems, tasks) that require their use. The traineesβ manager usually plays a key role in determining work assignments. Opportunity to perform is also influenced by the degree to which trainees take personal responsibility to actively seek out assignments that allow them to use newly acquired capabilities.
Opportunity to perform is determined by three dimensions: breadth, activity level, and task type. Breadth includes the number of trained tasks performed on the job. Activity level is the number of times or the frequency with which trained tasks are performed on the job. Task type refers to the difficulty or criticality of the trained tasks that are actually performed on the job. Trainees who are given opportunities to use training content on the job are more likely to maintain learned capabilities than trainees given few opportunities.
Opportunity to perform can be measured by asking former trainees to indicate (1) whether they perform a task, (2) how many times they perform the task, and (3) the extent to which they perform difficult and challenging tasks. Individuals who report low levels of opportunity to perform may be prime candidates for refresher coursesβcourses designed to let trainees practice and review training content. Refresher courses are necessary because these persons have likely experienced a decay in learned capabilities because they have not had opportunities to perform. Low levels of opportunity to perform may also indicate that the work environment is interfering with using new skills (e.g., manager may not support training activities). Finally, low levels of opportunity to perform may indicate that training content is not important for the employeeβs job.
π Definition β Opportunity to Use Learned Capabilities (Opportunity to Perform): The extent to which the trainee is provided with or actively seeks experiences that allow for application of the newly learned knowledge, skill, and behaviors from the training program. π Three dimensions: Breadth (number of trained tasks performed), Activity level (frequency of performing trained tasks), Task type (difficulty or criticality of trained tasks performed) π Definition β Refresher Course: A course designed to let trainees practice and review training content, used when trainees have experienced a decay in learned capabilities due to lack of opportunity to perform. π Example: A trainee who has not used a new software skill for three months is assigned a refresher course to rebuild their proficiency.
Technological Support
Electronic performance support systems (EPSSs) are computer applications that can provide, as requested, skills training, information access, and expert advice. An EPSS may be used to enhance transfer of training by providing trainees with an electronic information source that they can refer to on an as-needed basis while they attempt to apply learned capabilities on the job.
Trainers can also monitor traineesβ use of EPSS, which provides the trainer with valuable information about the transfer of training problems that trainees are encountering. These problems might relate to the training design (e.g., lack of understanding of process or procedure) or work environment (e.g., trainees either not having or not being able to find resources or equipment needed to complete an assignment).
π Definition β Electronic Performance Support Systems (EPSSs): Computer applications that can provide, as requested, skills training, information access, and expert advice. π‘ Why this matters: EPSS offers just-in-time support for trainees, and monitoring its use helps trainers identify specific transfer problems related to either training design or the work environment.
β Key Takeaways
Transfer of training is critically dependent on four work environment factors. First, manager support ranges from basic acceptance to active participation as an instructor, and can be operationalized through action plans that include goals, strategies, feedback methods, and progress check schedules. Second, peer support through support networks, newsletters, and mentors helps trainees share successful experiences and cope with workplace barriers. Third, opportunity to use learned capabilities is measured by breadth, activity level, and task type; low opportunity may require refresher courses or indicate work environment interference. Fourth, technological support through EPSS provides on-demand information and allows trainers to monitor transfer problems. Gaining manager support requires briefing them on training purpose, involving them in action plans, and potentially training them first to serve as instructors.
π§ Quick Revision Questions
- What are the five specific ways to gain manager support for training transfer?
- List the four components that must be included in an action plan for training transfer.
- What three dimensions define "opportunity to perform" trained capabilities on the job?
- How can an electronic performance support system (EPSS) help both trainees and trainers?
- What is a support network, and give two ways trainers can facilitate peer support besides forming groups?
π Lecture 22 β Training Evaluation
π Overview: This lecture examines the critical process of evaluating training programs to determine their effectiveness and value. It explains why evaluation is essential for justifying training investments, describes the two main types of evaluation (formative and summative), and outlines the outcomes and criteria used to measure training success. Understanding evaluation helps organizations ensure training delivers measurable benefits aligned with business strategy.
ποΈ Topics Covered
The lecture begins by defining training effectiveness and training evaluation, then explains the reasons for evaluating training, including the link between training and organizational outcomes. It distinguishes between formative evaluation (during program design) and summative evaluation (after training completion). The overview of the evaluation process is presented as a systematic cycle from needs assessment to results application. Finally, the lecture covers outcomes used in evaluation, introducing Kirkpatrick's four-level framework and a more comprehensive six-category model including reaction, learning, behavior, affective, results, and ROI outcomes.
π Lecture Summary
Training Effectiveness and Evaluation
Training effectiveness refers to the benefits that the company and the trainees receive from training. Benefits for trainees may include learning new skills or behavior, while benefits for the company may include increased sales and more satisfied customers. A training evaluation measures specific outcomes or criteria to determine the benefits of the program. Training outcomes or criteria refer to measures that the trainer and the company use to evaluate training programs. Training evaluation refers to the process of collecting the outcomes needed to determine whether training is effective. The evaluation design refers to the collection of informationβincluding what, when, how, and from whomβthat will be used to determine the effectiveness of the training program.
π‘ Why this matters: Without evaluation, organizations cannot know whether training investments are producing desired results, making it impossible to justify continued funding or improve program quality.
Reasons for Evaluating Training
Companies are investing millions of dollars in training programs to help gain a competitive advantage. Research shows that companies that conduct training are likely to have more positive human resource outcomes and greater performance outcomes. The influence of training is largest for organizational performance outcomes and human resource outcomes and weakest for financial outcomes. Training is more strongly related to organizational outcomes when it is matched with the organization's business strategy and capital intensity.
Training evaluation provides a way to understand the investments that training produces and provides information needed to improve training. It involves both formative evaluation and summative evaluation.
Formative Evaluation
Formative evaluation refers to the evaluation of training that takes place during program design and development. It helps ensure that (1) the training program is well organized and runs smoothly and (2) trainees learn and are satisfied with the program. Formative evaluation provides information about how to make the program better; it usually involves collecting qualitative data about the program, including opinions, beliefs, and feelings.
π Definition β Pilot testing: The process of previewing the training program with potential trainees and managers or with other customers (persons who are paying for the development of the program).
π Example: A group of potential trainees and their managers may be asked to preview or pilot test a Web-based training program. As they complete the program, they provide opinions on whether graphics, videos, or music contributed to or interfered with learning. They evaluate ease of navigation, quality of exercises, and feedback quality. This information is used to improve the program before full rollout.
Summative Evaluation
Summative evaluation refers to an evaluation conducted to determine the extent to which trainees have changed as a result of participating in the training program. It determines whether trainees have acquired knowledge, skills, attitudes, behavior, or other outcomes identified in the training objectives. Summative evaluation may also include measuring the monetary benefits (also known as return on investment) that the company receives from the program. It usually involves collecting quantitative (numerical) data through tests, ratings of behavior, or objective measures of performance such as volume of sales, accidents, or patents.
Seven reasons to evaluate training:
- To identify the programβs strengths and weaknesses
- To assess whether content, organization, and administration contribute to learning and transfer
- To identify which trainees benefit most or least from the program
- To assist in marketing programs through participant feedback
- To determine the financial benefits and costs of the program
- To compare costs and benefits of training versus non-training investments
- To compare costs and benefits of different training programs
Overview of the Evaluation Process
The evaluation process should begin with determining training needs. Needs assessment helps identify what knowledge, skills, behavior, or other learned capabilities are needed and where training is expected to have an impact. The next step is to identify specific, measurable training objectives to guide the program. Besides considering learning and program objectives, it is also important to consider the expectations of stakeholders (trainees, managers, and trainers).
Based on the learning objectives and analysis of transfer of training, outcome measures are designed to assess the extent to which learning and transfer have occurred. The next step is to determine an evaluation strategy, considering factors such as expertise, how quickly information is needed, change potential, and organizational culture. Planning and executing the evaluation involves previewing the program (formative evaluation) as well as collecting training outcomes according to the evaluation design. The results of the evaluation are used to modify, market, or gain additional support for the program.
Outcomes Used in the Evaluation of Training Programs
One of the original frameworks for identifying and categorizing training outcomes was developed by Kirkpatrick. Table 1 shows Kirkpatrick's four-level framework:
| Level | Outcome |
|---|---|
| 1 | Reactions |
| 2 | Learning |
| 3 | Behavior |
| 4 | Results |
The hierarchical nature of Kirkpatrick's framework suggests that higher level outcomes should not be measured unless positive changes occur in lower level outcomes. However, the framework has been criticized because: (1) research has not found that each level is caused by the level that precedes it; (2) the approach does not take into account the purpose of the evaluation; (3) it suggests outcomes should be collected in an orderly manner, but realistically learning measures need to be collected near the end of training.
As a result, training outcomes have been classified into six categories:
| Level | Outcome Category | Description | Measurement |
|---|---|---|---|
| 1 | Reaction | Trainee satisfaction and perceptions | Surveys, questionnaires |
| 2 | Learning/Cognitive | Knowledge acquisition | Tests, assessments |
| 3 | Behavior/Skill-based | Skill demonstration on the job | Observations, ratings |
| 4 | Affective | Attitudes and motivation | Attitude surveys |
| 5 | Results | Business impact (productivity, satisfaction) | Organizational data |
| 6 | Return on Investment | Monetary benefits vs. costs | Cost-benefit analysis |
Both level 1 and level 2 outcomes (reactions and learning) are collected at the completion of training, before trainees return to the job. Level 3 outcomes (behavior/skills) can also be collected at completion of training to determine skill level. To determine whether transfer of training has occurred, level 3, level 4, and/or level 5 outcomes can be collected after trainees return to work.
π Definition β Return on Investment: The monetary benefits the company receives from the training program compared to its costs.
π Formula: ROI = (Benefits - Costs) / Costs Γ 100 β The percentage return on the training investment
β Key Takeaways
Training evaluation is essential for justifying training investments and improving program quality. There are two main types: formative evaluation conducted during program design to improve the program, and summative evaluation conducted after training to measure outcomes. Kirkpatrick's four-level framework (Reactions, Learning, Behavior, Results) is foundational but has limitations, leading to a more comprehensive six-category model that adds Affective outcomes and Return on Investment. The evaluation process must begin with needs assessment and learning objectives, and outcomes should be selected based on training goals and business strategy. For exam purposes, remember that evaluation design must consider what, when, how, and from whom data will be collected to confidently attribute changes to training.
π§ Quick Revision Questions
- What is the difference between formative evaluation and summative evaluation, and when is each type used?
- List and explain the six categories of training outcomes used in modern training evaluation.
- What are the three main criticisms of Kirkpatrick's four-level evaluation framework?
- Why is it important to consider stakeholders' expectations when designing training evaluation?
- At what point in the training process should reaction and learning outcomes be collected, and when should behavior/skills and results outcomes be collected?