Vendor Evaluations – Understand an Outsourcing Vendor’s Operations
Outsourcing vendors’ sales executives make well more than $200k per year. Needless to say, if you have made the right choice to run a RFP despite the vendors’ sales team’s best efforts (sales teams never want objectivity, they want to sell based on their relationship with your executives), you can pretty much ignore much of the vendors’ written responses. Behind the scenes, a well-paid group of junior salespeople are busy copy and pasting responses given to other companies into your proposal format. Of course, a well-written RFP prevents much of this, and we’ll soon be publishing more on that topic, but needless to say, it’s fairly impressive how much sales jargon, three letter acronyms, and cryptic diagrams a vendor can jam into a proposal.
This is why site visits are essential. You have to see the actual operations management team that will process your work. The trick, however, is to have absolute control over the agenda, otherwise the vendors’ sales team will copy and paste their same lame material into a PowerPoint.
Here’s the agenda we suggest you follow, assuming you begin at 8am:
8:00-8:15am – Introductions and review the agenda (to be sure nothing has changed)
8:15-8:45am - Vendor corporate review, to ensure your team understands the vendor’s business and corporate structure.
8:45-10am – Vendor solution presentation, where the vendor walks through the solution they propose to provide you. They present the solution at the entry point of transaction processing to the end of the process.
10:00-10:15 - Break
10:15-10:45am - Interview the operations management team, the specific team members that will manage the flow and reporting of your transaction processing or call center. The key areas to focus on is the management team’s skills. Do they know what reports and metrics to watch? How often do they watch the metrics? What do they do if something goes wrong? Ask for specific examples of how they handle peak volumes, inventory backlogs and system outages. Understand how they build staffing schedules and ensure sufficient quantities of trained agents are available.
10:45-11:15am - Interview the recruiting team, the specific members that handle advertising, marketing, and interviews. Where do they source talent from and how does marketing and the vendor’s brand recognition compare to other vendors you talk to? Do they use any techniques or automated skill testing? Get a sense for the type of labor they employ: % of full-time, % of college graduates, location of most of the agents’ homes. What role do operations leaders play in interviewing? Do agents have the opportunity to see what day-to-day life is like? How do they source managers and specialty skills (e.g., workforce management)
11:15am-12pm - Interview the quality team. Do they understand statistical sampling? How do they track quality errors? How do they identify root causes? How do they ensure identified problems are permanently resolved? In many cases, you should be look for evidence of Six Sigma processes…and not just the mention of them. How many black belts do they have? Green belts? Ask them to see case studies of actual examples of six sigma projects in process or completed. Do the people giving the explanations understand what they are discussing?
12-12:30pm - Break and lunch.
12:30-1pm - Meet with the training staff. Seek to understand the content and length of the proposed training program. What components do the vendors add to the content (e.g., accent neutralization, industry introductions)? Are there sufficient numbers of training rooms? What level of accountability will training teams take toward developing and maintaining training materials? Ask about supervisory, management, and specialty role training programs.
1pm-1:30pm – Question and answer time.
Note that we excluded technology from the agenda. If you have an IT team member traveling with you, which you should, you should break them off during the middle of the presentations and have them meet with the technology team. They should seek to understand infrastructure, telecommunications, applications, and support. What is the vendor’s 2 or 3 year technology strategy plan? How often will they update desktops and other infrastructure elements?
Lastly, do a site walk through and understand how the vendor secures your area and how they support their agents.
What do you think of our sample agenda? Let us know!
Related posts:
I think you have treated the site tour too lightly. I always work with my clients in advance of traveling to determine a list of things to ask and validate during the actual tour. I often find that what we are told in the conference room cannot be validated on the operations floor. This is the only way to check.
Rosemary, I completely agree with deep diving into vendor operations! When I evaluate a vendor’s quality team, I begin with a generic capabilities presentation, which must be delivered by the actual site’s lead quality resource (not the sales person and not the vendor’s “global EVP of quality” – I want to meet the person who actually leads the team). Then we move into site’s quality area to see the work in action. In a call center environment, Clients should be encouraged to side-jack into a quality auditor’s phone and listen to a call being reviewed and watch the auditor score/grade the call. The client should also ask the auditor what they think about the call and how it could be improved – with the intent to understand the analytical capability of the auditor and the thought process. Do they discuss how changes are made to training? How new hire recruiting has been adapted? How the client is involved? How Six Sigma DMAIC or other quality methods are employed? Do they discuss the length of the call and how the call could be handled quicker? Is there a discussion around tone, cultural differences, and language? In a data processing center, Clients should watch the auditor and ask the same question. In an IT outsourcing shop, you want to see the quality team’s scripts and, if possible, their manual test execution (although the problem in the IT world has more to do with requirements understanding). You want to see specific examples of the testing, how six sigma processes have reduced error and how changes have been made to the requirements gathering and high level design phases. In all cases, you want to look at the use of tools and applications. Is the vendor using home-grown MS Excel? Or are there the applications that automate the quality process?
Then it’s time to see the training function – and you repeat the process through each functional area – begin in a conference room and migrate into the functional area to see the work performed. Like you, I am skeptical of copy and paste sales pitches made in a fancy conference room. I prefer to linger in the work areas where I’m free to ask (and encourage clients) to ask questions of everyone they see. I vividly remember watching a data entry senior agent pound through 2nd pass error correction without the use of OCR technology, despite a very persuasive presentation made by the global IT VP discussing the use of OCR. I asked the agent why there was no use of OCR, and he said, “It’s not accurate enough for the client’s quality standards.” At that point I knew that the vendor’s IT team lacked the the skills to manage a high quality OCR application, and that overall quality would be difficult to control in a manually audited world.