One of the beauties of working in qualitative research is the variety. One week you’re listening to people’s thoughts on a new, fruity breakfast cereal. The next, you’re asking them what kind of an animal David Cameron would be.

And over the last few years, I have experienced some extremes. After years as an agency account planner — using qualitative research mostly in creative development, I grew up and joined COI. Not in its research unit, but working alongside it, using the research commissioned both by COI and other public sector bodies.

As you can imagine, it’s a bit of a culture shock going from an ad agency to becoming a civil servant. So much so, in fact, that I felt inspired to write down these observations. They are, I should add, personal views untroubled by any kind of approval from the organisations involved. And I have generalised shamelessly, but bear with me.

Chalk? Cheese? The similarities and differences between ad agencies and the public sector are intriguing — but not necessarily in the ways you would expect.

Both are big users of qualitative research. And there’s quite a bit of overlap — COI commissions heaps of advertising research, often working alongside the ad agencies developing government campaigns.

But talking about research for an ad agency still conjures up images of glamorous people in flashy offices talking passionately about big ideas. While public sector research makes you think of middle aged, grey suited civil servants trying to get their heads around impenetrable instruments of policy.

At least that’s what I thought.

The corridors of power?

Government is scarily labyrinthine. Making a research project happen can be complex and daunting — you need buy-in from lots of stakeholders, each with their own agenda, before you even begin.

Then projects sometimes take years to materialise and suddenly disappear because of a new face arriving or a shift in policy. Many projects mooted simply don’t happen. They just get lost along the way. This is hardly unique to the public sector, but working in this environment, I learned the full meaning of the expression “kicked into the long grass”.

And, working with government, the kind of communications research I was used to — using normal qualitative methods — can sometimes feel like the poor relation compared with social research and academic studies that can span years and involve much larger samples. This can be an obstacle to having qual findings taken seriously.

But this is to paint a misleading picture of the public sector. The environment may be more staid than advertising and there are certainly lots of stakeholders to accommodate. And there’s politics. And setbacks. Plus career civil servants are a very different breed to advertising folk.

And yet, and yet… There are certain characteristics of public sector qual that make it uniquely attractive, in spite of all of this.

First — the subject matter. It is just plain interesting to talk to people about public policy issues. How do they feel about the health service, education, sexual health, drugs, obesity, anti-social behaviour, climate change?

Second, it really matters. Now — don’t get me wrong. I was never motivated to work in the public sector for altruistic reasons, though there are many admirable people who are. But there is a certain unique buzz associated with doing work that might make a difference.

Then there’s the topicality. It’s strangely intoxicating to be working directly on the subject matter that’s today’s big news story on the front page of the Times or the Sun.

Finally, they get to use — or create — fascinating, leading edge methods. For example, research among hard-to-reach samples (could be anything from sex workers or drug addicts to captains of industry). It’s a great challenge for any qualitative researcher. It creates opportunities to be really inventive. Forget off-the-shelf solutions like eight group discussions.

So, using qual in the public sector has a rather different mix of swings and roundabouts to those I expected to find.

Sexy advertising?

And the same is true of the kind of work ad agencies commission.

True, advertising is inherently interesting. And everyone loves talking about ads. Don’t they?

And glamorous? Attending car groups in a viewing facility recently, I was mildly surprised to find the fridge full of champagne. Sadly, we were all too scared of the budget controller to touch it.

And of course it’s brilliant working with creative people — even though you’re often just a little bit afraid of them, too.

But anyone who has worked on advertising research knows that it can be a political minefield. To say there are different agendas is stating the obvious. Agencies’ profitability is largely determined by whether campaigns get through first time, without having to go back to the drawing board. So objectivity is difficult.

And there’s real passion invested in those ideas being exposed to the slings and arrows of outrageous focus group respondents. It’s hardly surprising agency people get tetchy about creative development research. We’re not just being arsey for the sake of it. Well not usually.

Above all else, when commissioning qual on advertising, I have always had one nagging doubt. Does anyone (and by anyone I mean people outside our immediate team) really care? Ads for most things are just such a trivial event in people’s lives (as indeed are the things themselves). Sometimes it’s hard not to feel a bit of a fraud, asking them to have a long, in-depth conversation about something that matters so little.

Some years ago, I recall watching a group talking about a gentle washing product for delicate clothes. It had been painstakingly recruited — we selected creative respondents who had completed homework diaries and brought along their most valued and delicate clothes to talk about. The moderation was excellent.

All was going well when, after maybe 40 minutes, one of the most likeable and articulate respondents suddenly broke the spell. She sat bolt upright and said “Look, I’m not sure how to say this; I know it’s been an interesting evening and everything … but this new product, the things we’re talking about .... I’m sorry but I just don’t care. Truly I don’t.”

This is surely one of the biggest challenges facing any advertising research — how to deal with the sheer unimportance of the material to the people we need to engage with.

And then again, this is an unreasonably harsh judgement on advertising research. Advertising enjoys an ambivalent relationship with research of all kinds because it is in many ways a baffling mix of science and art. And, ironically, that ambivalence — the “logic meets magic” quality is exactly what makes it so fascinating.

Maybe the fact that creative development research can be fraught is really just to say that creating truly memorable communications is pretty motivating and people do get emotional about it.

Chalk and cheese

So, qualitative research in an ad agency and in the public sector can indeed be very different.

I always sensed in the public sector that research was something remarkably akin to a genuine search for truth. Maybe that’s why the research professionals can be sticklers for correct methodologies. It really matters. “Quick and dirty” would never be an option, even though timescales can be every bit as demanding as the private sector.

In an ad agency, research is an obstacle creative work has to overcome in order to be ‘bought’. It’s also an honest endeavour to improve and optimise, but there’s a significant feeling of being a means to an end. Ask any creative team.

Is any of this particularly surprising? Well some of it surprised me. I expected ad research to be sexy — and it is — but it also turns out respondents don’t care much about the outcome. I expected policy comms research to be dull and yet it is strangely exciting.

Obviously there are a thousand differences between working in an ad agency and working in the civil service. The way they use qualitative research is one of the more subtle contrasts. It’s a reflection of two very different cultures.

Truth — or something close to it — actually matters in public policy, so the search for truth and understanding drives the process. And now that the public sector is so driven by the twin mantras of delivery and measurement, rigour in qualitative research is a big deal. But what’s “truth” in advertising? Insight maybe or originality would be more meaningful. Agency folk are often seemingly on a mission, but I would hesitate to describe it as a quest for truth.

Sublime? Ridiculous? Well yes, all of these, but I’m still not sure which is which.

In the end, I’m glad I’ve had the opportunity to experience both environments, and to be a user of qualitative research in two such contrasting cultures. I eventually went back to work in an advertising agency. Somewhere, I reckon there’s still a fridge full of champagne with my name on it.