Conducting Surveys: The Good, the Bad, and the Ugly

By: Guest | June 6, 2012 | 

Today’s guest post is written by Brian Meeks.

I know almost as much about PR as I do about women. I won’t go into the math, but suffice it to say that it is a whole integer well south of zero.

I do, however, know a thing or two about data.

There is frequent discussion of measurement, which is what I did for GEICO (Where a 15 minute call could save you 15% on your auto insurance), when I was in their marketing department.

It makes sense a PR professional would like to quantify how their campaign has brought love and understanding to their client. That is how you keep them writing those checks, right?

What is the best way to find out what people think about your “Bob’s House of Bacon” YouTube campaign? Well, you could ask them. Surveys are always good…well…unless you do it in such a way that it makes the customer mad and they decide to write a scathing blog post for a wildly popular PR blog.

ESPN Sucks,” said the disgruntled blahger who grew up watching not one episode of Sports Center, but three of them. (For those who don’t know, back in the day, they did a one-hour long show and ran it three times in a row. Your average inebriated male, after having been threatened with a restraining order from the brunette waitress, would seek solace in the achievement of guys who get tons of babes. Da da da..da da da).

The blahger – who shall remain nameless (unless you want to buy his books, then you should probably look up Brian D. Meeks on Amazon), but to everyone else, he is just some random guy – took a survey today.

The problem is the box asking if he would like to take the survey came up almost immediately after he clicked on the “All Scores” tab. The mystery blagher had never been to that tab before.

These are his thoughts:

“If you are going to ask me my opinion about your website, you’ll need to give me a freaking moment to form one. I realize I was probably chosen at random, so you probably didn’t know I’d never been to the site, but you could have randomly selected people who had been hanging out for a few minutes. Yes, really you could have, it is call c o m p u t e r  p r o g r a m m i n g. The problem is, the guys you hired from India, can only design what you request. They are brilliant and if you had taken a moment to consider how much it might make me angry, to get your survey before I knew anything about the questions you were asking, they could have done it. Now, I’m left with the opinion that you are stupid. I don’t just mean low I.Q. stupid, but your mother was a raging alcoholic during her pregnancy, stupid. I’m sure she is very disappointed in you. “

The point is, this blagher is mad. He is going to tell this story over and over again. ESPN failed to get any meaningful data from him and now has a small measure of negative PR. And this is from a rabid ESPN fan.

So, be careful when you ask for feedback. That is my point. Try to imagine EXACTLY who will be getting the survey and that they will be someone who is able to respond.

Also, and this is unrelated, spend some time learning about survey design. Don’t ask a question with six possible degrees of answers, it should always be an odd number, so there is a “neutral” response. ESPN got that part right.

Also, please try to hide your own bias.

Q1: Are you happy with the current President, the really handsome and charming Barack Obama, or would you have preferred to see Hillary Clinton (wife [still] of the impeached and cheating Bill Clinton), in her XXL pant suits, as President?

The public isn’t as stupid as everyone thinks. Trying to game your survey will only result in someone catching it and calling on their Snark-O-Potamous to take you down a few pegs.

Don’t be that PR guy/gal.

Brian Meeks has delusions of novelist, which he feeds by authoring Henry Wood Detective Agency. (Available on his blahg). When the economy went south, he turned to social media and does this to feed and clothe himself.  In his free time, he does… well… social media… and publishes the Extremely Average. He can be reached on Twitter or by carrier pigeon at the house with the big tree out front.

  • quick test, will delete

  • Garbage in Garbage out. Often surveys from the RNC and DNC have trick questions. Same with marketing surveys. They always corral you into the answers they want…vs your real answers.
    I also blogged awhile back that people lie in surveys. I do. All the time. For various reasons. I run a billion dollar multinational with thousands of employees. I will always slam walmart and sams club when asked if I have shopped there (yes even though I never do) and say they are filthy (but haven’t been to one in years). YouGov always asks if I have heard good or bad things in the last two weeks about brands. I just picks ones I like and lie and say yes good things. Ones I don’t I lie and say yes bad things. As a Democrat I always say the economy is looking up.
    So between bad survey questions written by people who want…NEED…certain responses…and messed up responders…like me…..

  • I’ve been thinking a lot about surveys of late. They typically aren’t very effective because they get a result designed by the “researchers.” Without open-ended questions, you can’t really get true results, but people aren’t willing to take the time to complete those. Heck, I won’t even complete a multiple choice survey. Plus, anytime a survey is published, @HowieSPM makes me look at all of it and then dig 10 layers down to find what we’re missing. It’s exhausting, really.

  • rdopping

    Seriously funny post Mr. Weeks, sir. You certainly can’t please everyone but the idea that you get lambasted the moment you arrive is common. I do a lot of research on ye old web and more and more sites are asking for my opinion. They rarely get it.

    What is painfully clear is that there are very few programmers out there, er, check that, people in positions of influence that really get it. Just goes to show you that the internet is not as well seasoned as my grandmother’s roasted chicken.

  • YES! I am not a data person like you, as you well know, but I know just enough to be dangerous… which is why I try really, really hard to surround myself with “true” data people when it comes to doing anything related to numbers. They can be phenomenal, and tell great stories, but they absolutely have to be used wisely.
    Brian, I think it’s time resurrect that guest post you’d written for me a while back, on measurement… don’t you?

    •  @Shonali Maybe, but I’m not sure it was very good.  I may try to give it a read and see what I think.

      •  @ExtremelyAvg It was very good. It was also quite harsh… and that’s why I didn’t run it, because I didn’t think it represented the person I’m pretty sure you are. So I would love it if you gave it another read/try.

  • I meant to respond to to this yesterday. @Shonali is right about numbers, you do need to use them wisely and all too often they aren’t.
    My guess is that it happens because it is useful to be able to spout out stats. It makes you sound educated and important, at least that is the theory.
    The reality is sometimes we start sharing numbers without understanding why they are significant and that lack of understanding can really hurt us.

    •  @TheJackB “It makes you sound educated and important, at least that is the theory.”

  • patmrhoads

     @ExtremelyAvg Not only did I feel like you made a good point, but it was funny and fun to read. Thanks for brightening my day while educating me!

    •  @patmrhoads I’m pleased you got a chortle or two from the piece. That is always my goal.

  • nikkihilton247

    Conducting survey always good to know your product or services details, if you do personally. If you give outsource you might be get fake survey data.

  • nikkihilton247

    Conducting survey always good to know your product or services details, if you do personally. If you give outsource you might be get fake survey data. 

  • Pingback: Digital Times and the Innovation Report by @makeaner Spin Sucks()