While it is commonly recognized that training is a critical component of operations, most companies are unable to evaluate their training beyond the first two Kirkpatrick levels, simply because they lack the data collection tools to make higher level evaluations.
CallSource, which has been tracking and evaluating industry statistics for nearly 15 years, recently achieved level four training with Results, the company’s own integrated marketing analysis and sales training system that incorporates Donald Kirkpatrick’s four levels of training evaluation: Reaction, Learning, Behavior and Results.
Those four levels, described by Kirkpatrick half a century ago, still comprise what is considered the foremost training evaluation model.
Even with the advent of the Internet and other technological innovations, not much about the model has changed.
CallSource released its Results system last June at the National Apartment Association’s education conference and exposition in Orlando, and, several months ago, after significant beta testing, clients began using the system. “Today, it is in about 150 different management companies and more than a thousand communities,” said Elliot Leiboff, CallSource’s chief strategy officer.
Happy coincidence
It was only by coincidence that the CallSource platform and Kirkpatrick’s term for the highest level of training evaluation share the name “Results.”
“At the time we introduced CallSource Results, we hadn’t even heard of Kirkpatrick and we didn’t understand the added significance of the name we chose. We picked it because our goal is to give our customers results, not just reports. We had become frustrated by the fact that every monthly report for every client was pretty much the same as the reports from prior months. It drove us crazy to see what was happening without being able to impact our clients’ performance and help them achieve their business goals,” said Leiboff.
“So we took a look at our platform and recognized we are uniquely able to evaluate the effectiveness of leasing training on the third and fourth levels. When we introduced our Results system, we said, ‘It’s not about reports, it’s about results.’ Once we integrated our learning management and content modules, we were also able to say ‘It’s not about training, it’s about results,'” he said.
Results is a confluence of business tools the company has fine-tuned over the years. It tracks, evaluates and continuously improves marketing and leasing performance by measuring both the quantity and quality of marketing leads and by reviewing actual interactions between the staff and the customer. The final piece is the customized learning management system that also interfaces with property management, CRM, and payroll software, accessing their data to bridge the gap between training and performance, giving true level four feedback.
The platform starts with an evaluation of marketing effectiveness, lead traffic and appointment-setting ratios, determining how many leads are needed from marketing and how much the leasing team will have to improve at appointment setting to reach the goal.
CallSource first looks to see if its customers are getting enough traffic to fill vacancies, then how many of those leads are really leasing opportunities and how many are something else (e.g., vendor calls, maintenance calls, personal calls, misdials).
“It’s amazing to see how many non-prospect calls are generated each month. All of those used to get lumped together, so we did not see an accurate picture of what our ads were bringing and we were receiving a much inflated number,” said Jessica Simmermon, director of marketing for the Breeden Company Once the number of true leasing opportunities has been determined, CallSource assigns a value to each lead and derives an accurate cost per lead. Clients can use this information to negotiate advertising rates and to support future marketing decisions.
“We can also give a fair evaluation of the call handler. Before, marketing might have said to sales, ‘We gave you 100 calls, why are there only three leases?’ Without listening to all the calls, they didn’t know that there really were only 50 true leasing opportunities among them. Now we can begin to accurately assess whether we have issues with the quality of the leads, or issues with the quality of the call handling. And by filtering out the non-leads, we can, for the first time, establish an accurate performance baseline for each leasing agent and then track his or her improvement after training,” said Leiboff.
Crucial clarity
Call monitoring also gives visibility to a number of issues that exist or could arise during the leasing call.
“In reviewing one of the telephone TPA (telephone performance analysis) reports last week, we saw that our teams have come light years in the introductory information and in setting appointments, but still struggle with handing out the price and not referring people to our Web sites to view floor plans, which is helpful to people calling from out of town. Now we know we need to take that training further.
We also found that some of our people had been taught to handle questions about pets one way and others had been trained another way.
We were able to clarify our policy to ensure that everyone handles it the right way.
“We actually had a potential fair housing issue pointed out by a CallSource screener when listening to a call. They immediately contacted us, bringing it to our attention within an hour of the call, and we were able to take immediate action,” said Alethea M. Parker, VP of property operations for Artcraft Management Inc., which owns and manages a portfolio of 2,200 apartments in 19 communities in Virginia and South Carolina.
Two steps beyond
Level one evaluation usually mean surveys collected after a training session, rating student reaction to the trainer and the content. But sheets with smiley faces do not measure the effectiveness or the value of the training.
The most common level two evaluations consist of pre- and post- training tests to measure learning. But were the right lessons learned? Does the training positively impact performance on the job?
“Many of our employees have taken training classes, but that hasn’t always ensured they were using the information,” said Parker about her company’s previous employee training that was conducted during a once- a-year training day at a conference center, or through rare and random trainings when employees themselves requested education tuition assistance.
By switching to CallTrack, LeadScore, Telephone Performance Analysis, Our University, and Training on Demand by CallSource, Artcraft managers are able to run reports and see which employees are taking advantage of the training courses and use that information to evaluate learning and influence promotions, she said.
“It has been eye opening to us to realize that it’s not about the venue in which you deliver the training. An e-learning environment is not inferior to a classroom. In all cases, what matters is how well the course is designed and delivered. The real point is that you derive a benefit from the training that helps you to achieve your business goals,” said Leiboff.
CallSource also conducts pre- and post-tests for all its courses, but, with the baseline for employees’ performance established, advances to a level three evaluation of whether there is a measurable difference in behavior and performance after the training — which can be quickly seen by reviewing the scores in the TPA.
“We’re looking for specific behavior. We either see it or we don’t.
And if we don’t see it, or if performance is still less than optimal, then we have to provide performance support. We have to go back to the specific skill gaps we have detected and help reinforce prior learning, or deliver additional learning needed by that specific student. It’s a very individual process,” said Leiboff.
“With CallSource monitoring all our calls and giving us a report card on how we do, our numbers have gone from 35, as the highest performance score company wide, to hitting 85 out of 100, for several employees last month,” said Parker.
CallSource repeats the entire process over and over, reviewing clients’ employees each month — analyzing, evaluating, ranking and showing the results in Our University, a client-customized, e-learning software system that establishes an individual Web site for each employee.
Our University serves as an internal resume for the student and a repository for resume-like information, so that when the next job opportunity arises within the company there is an easy and obvious way for management to evaluate which employee is ready to take the next step.
“If they are subscribing to the Telephone Performance Analysis, the level of performance for each student becomes very clear. Then, by interfacing with the property management software, or the CRM program, we can extract such information as how many leases are being written.
Within the Telephone Performance Analysis we can hear how many appointments are being set. So, with property management systems that allow us to interface, we have a glimpse of level four return-on- training investment. In all instances, we definitely have a full view of level three behavior.
“Last year we announced a new level of synergy between marketing, sales, and management. This year, in addition, we have achieved a new level of training that no one else has and that the industry has never seen before,” said Leiboff.
New product flow
With Results in place, CallSource continues to apply the platform in innovative ways, aware that new products and services are the lifeblood of business. “For us that means providing resources to help our clients succeed that may not be available anywhere else,” said Leiboff.
On target for release over the next few months are a number of new features and services, including a train-the-trainer program and an expanded product line for the maintenance side of the multihousing industry.
CallSource is introducing routing and interactive voice response services to ensure maintenance calls get to the right people at the right time, based on the urgency of the maintenance demand. Recording, call monitoring and telephone performance analysis will be part of the program. Another exciting new addition will be outbound call tracking that will close the loop on unfinished maintenance business. Outbound calls will be tracked and reported the same way as inbound calls.
“Typically some calls to maintenance end up in voice mail. Now we can see and know if there was anything done about it, or if leads were vaporized into the ether,” said Leiboff.
And, as a follow-up, CallSource is creating automated surveys that can determine if clients’ maintenance teams are keeping their customers happy.
Another new product is training for the trainers themselves, through Our University. CallSource is dedicating a learning management system to trainers, whether they have that formal designation, or are just managing others.
“A big part of every good manager’s responsibility is developing their people. But a lot of people in the apartment industry earned their stripes coming up through the ranks, learning by doing, and may not have had very much formal training on the academic side of the training world. So we are bringing some of those skills sets into the working world–things that maybe they didn’t learn in school, but that now are very important in their new positions and as they move up through the ranks in the multifamily industry,” said Leiboff.
Next for introduction is iREVIEW, a brand new product aimed at clients who like the concept of TPA, but prefer to listen to and review their own calls. Leiboff describes it as a do-it-yourself version of the tool CallSource uses for TPA analysis; a Web tool that can be accessed by a trainer or someone else on the client’s staff designated to listen to calls, grade them and indicate if survey questions need to be created.
“If a company has a unique way of measuring the performance of their leasing agents, they can reflect that in the survey they design so their people can look for the behavior when they review,” he said.
iREVIEW will be released shortly after the National Apartment Association Education Conference & Exposition in Las Vegas in June.
CallSource also is cross-pollinating its vertical markets, bringing clients from other industries into the multifamily space. “A large hearing aid manufacturer and an air conditioning giant that have been eyeing the apartment industry will be attending future NAA events, at our urging. We have convinced them that they should pay more serious attention to multifamily, because it really is quite a fine market to be operating in,” said Leiboff.
