VUB Kirkpatrick Case Study

Implementing the training effectiveness measurement program in the VUB bank

Bank photo

THE BUSINESS CHALLENGE

Employing over 4000 employees, the VUB bank, had faced a long-term effort to decrease costs and prove effectiveness of the invested financial means. The HR training and development department could not disregard this trend either. Since DEVELOR has cooperated with the bank in the past to improve the level of HR specialists and shift their skills to the role of internal consultants and business partners, they approached us with a task to establish a methodology for measuring the effectiveness of the invested training resources.

In 2011 the top management has set the priority to create and implement a system for measuring the effectiveness of the invested training resources.

Mobiltel has become a supplier with convergences strategy. Key elements in it are strong customer centric approach along with innovation and technology edge over the competitors.

HR DEPARTMENT PRIORITIES

The HR department consequently defined its own priorities:

  • To gain control over the budget effectiveness invested into external development; in the beginning especially to the hard skills trainings that represent 31% of the overall volume
  • To create a methodology for measuring effectiveness of the invested training resources to be able to present the Board with a concrete number
  • To reinforce the take-over of the responsibility for employees’ development by managers, who have the greatest impact not only when defining the training needs, but mainly when supporting application of the newly acquired knowledge and skills on the job that should bring the expected business results

KIRKPATRICK IS THE ANSWER

In 2009 DEVELOR has started to cooperate with Kirkpatrick Partners as an exclusive distributor of their Business Partnership model. Our consultants have successfully completed the Kirkpatrick Four Levels TM program certification. Based on this and their know-how they have trained the HR department of the VUB bank to utilize the Kirkpatrick model internally.

Michal Bača
Director of Training and development department

“I SEE THE BENEFIT OF MEASURING THE TRAINING EFFECTIVENESS ON TWO LEVELS:

1) FIRSTLY IT IS THE PRACTICAL LEVEL, WHEN WE, THANKS TO DEFINING AND EVALUATING EXPECTATIONS FROM A PARTICULAR TRAINING, PRACTICALLY INCREASE THE LEVEL OF UTILIZATION OF THE INVESTED TRAINING RESOURCES – IT HELPS US TO PRIORITIZE THE TRAINING REQUESTS.

2) WE CAN ARGUE BETTER WHEN PROMOTING DEVELOPMENT ACTIVITIES AND WE CAN REFER TO THE CONNECTION BETWEEN TRAININGS AND BUSINESS RESULTS OF THE BANK, BE IT IN SUPPORTIVE OR BUSINESS AREAS. SURELY IT IS MORE WORK FOR BOTH THE TRAINING & DEVELOPMENT DEPARTMENT AND THE MANAGERS, BUT NOTHING IS FOR FREE. THE SET SYSTEM IS NOT PERFECT, THERE ARE STILL THINGS TO BE IMPROVED AND SIMPLIFIED, TO MAKE IT EASIER FOR MANAGERS.”

The Solution

If you want to learn more about the Develor Solution contact us: info@develor.com

resultcasestudy_narrow

Project Preparation And Measurement

ROLE OF THE BOARD IN THE PROJECT

The fact that the Board perceived creating and implementing of such system as a great priority has really helped the success of the project. It provided the project with sufficient importance and created commitment for the managers. Without it we could face the risk that the managers would deem the project-related activities a useless bureaucracy and would not consider the requests as mandatory.

FORMING THE ATTITUDE OF MANAGERS

Another prerequisite for successful functioning of the system was to organize workshops for the managers. The goal was to create a positive attitude towards work with the system and to understand their role as managers, whose task is not only to properly identify the development needs of their people, but mainly to appropriately formulate the training expectations; what should employees apply from a particular training on the job and how the success will be evaluated.

METHODOLOGY – DEFINING RETURN OF EXPECTATIONS

The methodology itself was quite simple. The managers should set, in cooperation with their subordinates, development objectives and mark them with priority 1 or 2. Such prioritization should serve as a basis for increasing responsibility for the invested financial means, since trainings with priority 1 were being approved preferentially and only if the budget allowed it, the trainings with priority 2 were authorized.

The form “Training effectiveness evaluation – ROE – Return of Expectations” had been introduced during workshops, to be used by managers when ordering trainings from their training specialists. Simple notes navigated the managers what had to be filled in and how.

Three areas have been defined, to be evaluated in the following order from the most important:

  • Results (to which results or business objectives should the training contribute)
  • Activities (what are the specific activities of employees proving that they have implemented the training know-how)
  • Know-how (what is the specific know-how the employee should acquire and for what purpose)

Thus managers were supposed to define at least one expectation on each level (results, activities, know-how).

The greatest importance was gradually put on the RESULTS level, then ACTIVITIES and the smallest on the KNOW-HOW level. The importance had to be defined at least on the RESULTS level or on the ACTIVITY level. This scaling system has helped managers to become aware of the importance of evaluating individual levels from the viewpoint of the company’s business objectives. Thus it supported the idea of dealing with the company’s business strategy as well as authorization of financial means promoting the strategy in the most direct manner.

METHOD TO MEASURE EFFECTIVENESS

ATTITUDE FORMING WORKSHOPS

Total participation at workshops: 168 managers; maximum 10 participants per group; 22 workshops in duration of 3 hours, lead by HR director and training & development specialists.

The effectiveness of the ROE introduction workhops was measured by the feedback questionnaires.

We checked on the level of understanding and acceptance of the new system (identification with it).

Scale 1 to 5 (5 being the best)

  • I understand the ROE methodology and can apply it: 4,7 (out of which 0% for 1 & 2)
  • I deem the ROE methodology beneficial to the bank: 3,5 (out of which 10% for 1 & 2)
  • The workshop was held professionally and was understandable: 4,8 (out of which 0% for 1 & 2)

TRAININGS

The planned training evaluation date: not later than 4 to 6 months after the training

The form contains:

name and personal number of the employee, his/her direct manager, name of the training, date of the training, evaluation date.

When creating the form the greatest effort was put into making it as simple as possible.

Since the company has its own functional Learning Management system – the ROE form was added as an on-line application.

Marta Jedináková
Training & Development Specialist

“I PERCEIVE USING ‘KIRKPATRICK’ AS A SUCCESSFULL HANDLING OF A PART OF ‘PANDORA’S BOX’…

…WHICH SOME PEOPLE CONSIDER TO BE THE TRAINING EFFECTIVENESS MEASUREMENT. IT WAS NOT EASY, IT IS NOT EASY, IT WILL NOT BE EASY – BUT WE HAVE THE BASIS AND MOST IMPORTANTLY WE KNOW IT IS FEASIBLE.

DEVELOR HELPED US WITH CLARIFYING ‘KIRKPATRICK’ ITSELF AND WITH FORMING AN ATTITUDE TO MEASURING EFFECTIVENESS – HOW FAR WE CAN GO AND THAT EVERYTHING IS MEASURABLE, AT LEAST ON THE LEVEL OF ACTIVITIES.

WHEN I LOOK BACK, I PERSONALLY ASK MYSELF: IT IS NOT NUCLEAR PHYSICS, WHY THE OTHERS DON’T DO IT… THE QUALITY OF TRAINING SERVICES WOULD IMPROVE (BOTH INTERNAL AND EXTERNAL), SINCE ULTIMATELY ACCEPTING RESPONSIBILITY FOR OUR OWN WORK IS REQUIRED.”

The share of individual training types within the whole development system, where the bank managed to measure the invested training resources by means of the Kirkpatrick method (i.e. paid trainings) for 2012.

  • Managerial skills 37% – the company has measured this type of trainings by action plans issued after every training, whose fulfillment had been subsequently evaluated. As part of the employee satisfaction surveys it also measures employee engagement that is directly linked to the quality of managers and their skills, i.e. the company measures results (e.g. employee engagement increase by 4%) – to measure the progress it uses the 360° managerial feedback
  • Professional skills 31% – ROE forms, effectiveness measured up to the results level
  • Sales skills 28% – also handled by action plans issued after every training. Fulfillment of sales plans is monitored as well.
  • Legislation 4% – not evaluated by the Kirkpatrick method, since the courses are mandatory
  • Language skills 1% – the company invests in these courses just minimally (only Slovak language courses for ex-patriates)

PROJECT EVALUATION

Fulfilment of expectations by evaluation dates:

The total return rate of the evaluations measured up to date is 87.7%

Thanks to the system the HR department could verify that in 95% of the cases employees reasonably and necessarily needed the trainings in order to perform their work and they could prove these claims by probative evidence.

PROJECT EVALUATION

PROGRAM IMPLEMENTATION BENEFITS

  1. Continuous consultations provided by training & development specialists, making themselves more visible in the eyes of managers, perceiving specialists as important business partners
  2. Training & development specialists in a more direct contact with strategic objectives of managers
  3. Continuous system training for new managers by the HR department
  4. New managers appreciate copying the training history of their people
  5. Continuous information exchange among managers (when an employee is transferred from one department to another, ROE goes along with him/her
  6. Continuous system improvement via phone interviews between HR and employees (verifying how their managers brought ROE into their attention, how and whether the expectations and consequent evaluation was defined)

In conclusion we can state that at present we are not aware of another company of a comparable size in Slovakia, which would have such an accomplished training effectiveness measurement system and whose managers would feel to be such an important part of the system.