The "bad apples" debate

It’s amazing how history repeats itself. The March issue of Research magazine featured a cover story on what it termed ‘Bad Apples’, the fake and/or serial respondents who crop up in the course of qualitative work.

A search through AQR’s archives will reveal that this is an old chestnut, but one that has never been satisfactorily resolved. That’s why, following the Research article, the topic was put up for discussion on our website.

Members were asked to consider four central themes. First, is a central database thought to be the answer, combined with identity checks? If so, who should bear the cost of its creation and implementation? Secondly, is the root cause linked to the speed with which research and field agencies are asked to recruit candidate? And if this is the case, is the situation ever going to get any better? Thirdly, how much is the case of the ever-more-niche respondent exacerbating the issue, and finally, is there a dearth of discussion between clients and agencies in relation to recruitment briefs and the recruitment process?

One month on and, at the time of writing, there have been 25 responses to date with another seven letters on the topic featured in the April issue of Research. And if you still haven’t added your comment to the AQR website, it’s not too late.

The majority thought fraudulent respondents were the biggest problem, but adopted a more relaxed attitude towards those who come under the ‘repeat’ heading (see site for further details). The points raised included:

So where does the debate go from here?

At the April AQR Committee meeting the issue was thrashed over once more and it was agreed that it did need resolution.

The Committee decided to formulate a set of ‘best practice’ guidelines, endorsed and hopefully written by both clients and researchers, which will offer advice on how to deal with respondents — and recruiters — who aren’t perhaps participating as they should.

This will focus on problems that can occur with the original selection of respondents, maybe recommending random spot checks, to those that occur at the groups in question, even if this boils down to the researchers initiating post-mortems with recruiters the following day or shortly thereafter.

“We don’t live in a perfect world,” said AQR chair Fiona Jack. “Sometimes the client’s specifications are difficult to adhere to. There is always an element of time pressure, but we do need to try and create a shared understanding of limitations, manage people’s expectations better, and for everybody to make it their business to raise the bar.”