Creating a customer satisfaction survey

One way to ensure your law department is delivering superior value to the business is to use a survey. In this article, we cover some basics to get your started.

There are a number of reasons why you would want to send out a client satisfaction survey for your legal department. First, it helps you gather helpful data and comments about how the legal department is performing and how it is perceived throughout the organization. Second, you can use the survey as a tool to market and promote legal by reminding people that the department is there, that it provides valuable services to the company, and that you are directly seeking their input into improving the quality and value of the services it provides.

This article walks you through the process of creating, distributing, and analyzing the results of a client satisfaction survey. Though the discussion deals with in-house counsel sending a survey to their internal business clients, the “why” and “how” apply equally to the relationship between outside and in-house counsel. And outside attorneys can easily use this post to create a satisfaction survey to distribute to their in-house counsel clients.

1. What is the purpose of your survey? The first thing you need to do is spend some time thinking about what it is you want to know or capture with your survey. Typically, you will want the survey to give you insight into some measurable metrics showing how efficient or effective the department is, such as timeliness, quality of advice, accessibility of the lawyers, etc. Likewise, you will probably want to get some general feedback on the department as a whole or about individual performers. In addition to your own thoughts, gather a cross section of your team and get their input on the survey as well. Spending time up front thinking about what you want to accomplish gives you an excellent road map to prepare the survey questions.
2. What tools can I use to create my survey? The easiest and best way to go is to use an online survey tool. This will allow you to create a professional-looking survey, let your clients complete the survey online, and provide you with the ability to tabulate and “slice and dice” the results in a number of interesting ways. Check first to see if your company has its own internal survey generator. If it does, use that tool as it will give you the widest variety of options/analytics (and you’ll probably have access to help and support in creating and managing your survey). If not, there are a number of tools available online that are either free versions (which usually provide all the tools you need) or paid (which give you more questions and more powerful analytics/flexibility). Whether you use the free or paid versions will depend on the purpose of your survey, the number of questions, and the level of information and detail you need. Three easy-to-use survey tools for you to consider are Survey Monkey, Zoomerang, and Survey Planet. If you have the budget, you can also hire a professional consulting company to create a survey for you (e.g., Altman Weil). But, unless you believe you have some really complex issues to sort out, I am confident you can create a valuable and useful survey on your own using the tools and ideas set out here.
3. Creating the questions. Once you have selected your survey tool, it’s time to prepare the questions. To start, don’t make the survey too long or complicated. If you do that, people will not complete it. The maximum number of questions should be 20 or, preferably, less. You will also want to mix up the “type” of questions you ask, i.e., yes/no, rate on a scale, multiple choice, open-ended, etc. Mixing up the types of questions keeps the survey interesting. Most of the online tools give you numerous templates you can use to design your questions. Your survey should not take more than 15-20 minutes to complete. Additionally, before finalizing your questions, seek the input of your team. Let them propose questions or give feedback on your questions. Likewise, and more importantly, identify a group of clients and get their feedback on the types of questions they would they like to be asked as well as on some of the questions you propose to use. This will help you draft questions that clients will want to answer. You should think about the order of the questions so your survey has a logical flow, and do your best to not build bias into the questions (another reason to have several folks review your questions). Instead of listing a bunch of potential questions for you to consider, I am providing links to sample surveys that you can use as guideposts in creating your own questions (or copy verbatim if you wish). Click here to view a survey I recently created using Survey Monkey. I used very similar questions in the surveys we circulated when I was general counsel at Sabre and Travelocity. If you want more ideas, type “law department client satisfaction survey” in your search engine and you will get other examples.
4. Confidentiality of responses. As you prepare your survey, you need to create a short introduction. You will see an example in the sample survey I created. Let the user know why you are seeking their feedback, why it is important that they participate, how long the survey will take to complete (e.g., “15 minutes or so”), and, most importantly, let them know that their answers and feedback will be kept confidential and their identity will not be revealed or tied to any of the responses. Confidentiality is important so as to obtain maximum participation and to generate more honest responses. Still, one of the frustrating problems with an anonymous survey is that you have no ability to follow up with someone if they have particularly good or important feedback or ideas for the department. One way around this is to add a place near the end of the survey to allow users to give their name and email address/phone number if they are willing to disclose that information. You will see such a question at the end of my sample survey.
5. Who and how often do you survey? One critical question to answer before launching your survey is “Who gets it?” This is important because you certainly want the survey to go to people who are frequent users of the department, but you do not want the survey to go only to “fans” of legal, as the results will be skewed and not useful. The first few surveys I sent out as general counsel were limited to vice presidents and above, along with some additional select employees. While this gave us a pretty useful sample, we realized that a large group of people who used the legal department regularly were left out. We then increased the survey distribution to include two levels down from vice president. We got better results by doing this, but it still seemed wrong to leave out anyone who wanted to take the survey. As a result, we began sending the survey to every employee in the company, around 10,000 people. We did not really add a significant amount of information by doing this (as many employees had very little, if any, interaction with the legal department), but I know the employee base felt better about the department because we included them and asked for their opinion – even if they did not complete the survey. That goodwill alone was valuable. Depending on your situation, my advice would be to just make sure you have a wide enough sample of people who will likely give both positive and negative feedback to the department – making for a much more useful survey.
   
   Another question is how often should you send out a survey (and when should it go out to your clients). Any more than once a year is too much in my opinion, and you will see response levels suffer from “survey fatigue” if you seek input more frequently than that (as employees get bombarded with company surveys all the time). Our surveys tended to go out about every 14 months or so. I found that a little bit longer than one year between surveys got the best overall response rates. As to when to send out the survey, be sure to avoid the summer (vacations) and the major holidays (e.g., in the U.S. avoid November and December as well as March – spring break). In my experience the best months are September or October because then you can use the results to plan for next year. Also, coordinate with other departments to make sure your survey does not clash with or overlap another major survey going out to employees. Keep the survey open for 30 days post-launch, and send out several email reminders during the time that the survey is open, noting that you greatly appreciate the 15 minutes people take to complete it. You can also promote the survey on your company’s internal website and the legal department website, in the company newsletter, at town hall meetings, on TV monitors around the facility, etc. Ask your team to encourage their internal clients and colleagues to complete the survey.
6. Weighing the responses. Not all responses are created equal. In my experience, the opinions and thoughts of frequent users of the department and those of the senior management tend to carry more weight. The former are more important because their frequent use means the feedback is generally more specific and useful. The latter are more important because they control the purse strings and their perceptions of the legal department as an important and valuable part of the company (vs. a black-hole cost center) are critical as to whether the legal department thrives and has a seat at the table or is constantly fighting for funding and a voice. One way to account for this is to weigh the responses from those who are frequent users or who are part of senior management a bit higher than other responses. For example, give a 1.25 weight to the former vs. a 1.00 weight to the latter. To do this, you will need to ensure that you have users identify themselves (generically) as “frequent” users of Legal – however you want to define that – and/or as senior management (e.g., VP and above, or SVP and above). You will see an example of this in the first page or two of my sample survey.
7. Create a “word cloud.” You’ve probably seen a number of word clouds on a day-to-day basis. These are charts that show different words at different sizes, shapes and colors. Using the available data set, the words that appear most frequently are “larger” and “brighter” than the words used infrequently. In my sample survey you will see a question that asks the user to list three words that they think best describe the legal department. I used this same question in my actual surveys, created a data base of the words provided by the users, and then used a word cloud generator to create a word cloud. I included the word cloud chart in the survey results I shared with my team and with my CEO. It is a great way to visually capture a snapshot in time of what your clients think of the department. You can see a list of word cloud generators online. I used Wordle to generate my word cloud.
8. Analyzing the results. Once your survey closes, it’s time to analyze the results. A few things to understand about surveys. First, don’t be disappointed if you don’t get a “90%” response rate. That would be amazing. You should more realistically expect anywhere from 5%-10% of the people to respond (more if the survey pool is small). Second, you will tend to get more responses from people who are “unhappy” with legal. This is just human nature. Don’t be put off by this. Accept the criticism as just something you and the team need to work on to fix. The fact that you asked for feedback has probably made some positive in-roads with these unhappy clients. There will always be some people that, no matter what you do or how unreasonable their complaints, you will never satisfy – just a cold hard fact of in-house life. Hopefully, none of those people are part of senior management. Depending on the capabilities of your survey tool, you should be able to slice and dice the data in a number of different ways, e.g., by region, by line of business, by type of legal matter, etc. This will give you a lot of insight into where things are going smoothly or where you need to put in some work. If you included the ability for people to write open-ended comments, there is a lot of great information you can extract. Look for patterns involving certain types of matters or certain groups or members of your team (good or bad). All of these are areas for improvement or enhancement. Prepare yourself for the odd curveball, e.g., someone complaining on the survey about things that are not part of Legal’s purview, such as procurement or system security. The simple fact is, not everyone has a true understanding of what fits under Legal and what falls outside of Legal. Your word cloud will give you additional insights as well.
9. Sharing the results. After you have been through the results yourself, it’s time to share them. Most importantly, you will want to share them with your team. I would first share the results with my department leaders and, after they had a chance to digest everything, make the results available to everyone else. We would then go over the results in detail as a team at an offsite or during one of our monthly staff meetings. Be sensitive about sharing the results with the full team if there are any extremely negative or unfair comments about any individual person set out in the comment sections. If this happened, I would edit out names of those people from any version I shared broadly (though either I or their manager would show them the comments and discuss the matter). After your team, you want to share the results with your CEO and senior management. When you do this, your cover email should list the top three positives and the top three “things to improve” gleaned from the results. Be sure to note that you have plans in place or are making plans to address any particular shortcomings. Finally, share some of the key results with the company at large. You solicited feedback from a large group of people; let them see how the department fared (and read some general comments from you thanking them for their feedback and that you are putting into place action plans to work on issues identified through the survey). You can post this on the legal department webpage or work with corporate communications for the right way to proceed.
10. Create an action plan. Lastly, you want to be sure to use the data and the comments to make changes. Develop a formal action plan to deal with the most important areas. You do not need to do this alone. Form a team with your department heads (and potentially others) and ask them to help develop plans for the department, themselves, and their teams. You should also solicit ideas generally from the entire department so they feel vested in the process. Ask everyone on your team (and don’t forget yourself) to make the relevant sections of the action plan part of their development plans for next year. If users identified themselves for follow up, be sure to follow up with them and get their thoughts and ideas, and show them that their feedback is important and valued. Finally, share your action plan with the CEO (especially during any mid-year or full-year review) and senior management to let them know you are moving forward with fixing any problems. Consider asking for a short meeting with your senior management peers to individually go over the key survey results and get their feedback on the action plan and/or the department in general. Even if they don’t take you up on the invitation, they will greatly appreciate being asked for their input.