Share this
Podcast: Karine Pepin on finding value in survey incompletes
by Infotools on 09 Aug 2022
As demand for survey participants exceeds supply, Karine Pepin of CV2 says we may need to change our views toward partial survey completes and find the value in this data.
We recently welcomed Karine Pepin, Vice President at 2CV, to discuss the concept of incomplete or partial data. She sets the stage by talking about how rapidly the world of insights is changing due to things like shorter attention spans overall and the massive impact of technology, with ResTech market research solutions transforming the industry across the board from data collection, to survey platforms and data analysis tools. We shouldn’t underestimate the profound impact this has had on so many aspects of our work, including how we should be treating incomplete data in our dataset.
She says that there is a “traffic jam” in the survey response collection process. The demand for surveys has increased while the willingness to take surveys has decreased. In addition, there have been more fraud and quality issues which means that many of our completes are tossed out in the back-end. There is a shortage of good participants, and we continue to experience persistent problems with sample quality.
Still, survey conversion is still the ultimate metric. We cannot afford good panelists who have qualified to not finish. Every panelist counts – literally and figuratively. So we have to optimize for the new reality to not only encourage respondent engagement, but also consider the value in the incomplete responses. To boost engagement at the front end, Karine says our first line of defense is to provide a better survey experience so people stay. This can include practices like shorter surveys that are device agnostic, which should be matter of course. We need to make questionnaires more engaging overall.
She says that in an ideal world, we want to create an experience where the participant looks forward to seeing the next question, like a book. There is a lot of conversation about how to make the “questions” better or to gamify questions but she encourages thinking about “storification.” This concept is more about improving the structure and the flow of the survey at a macro-level. Some elements of that include providing an enticing landing page, improving the user interface, using illustrations, icons, emojis, using a more conversational tone, and offering a sense of progression.
But despite our best efforts, people will still inevitably drop out. This is where the idea of keeping partial completes comes in. Karine mentioned “Every participant counts and it kills me to see people drop-out especially when they are almost done. I always ask myself ‘what could have I done better’?” Depending on the sample size, there will usually be hundreds of participants who dropped out at various points in the study.
When you keep partial completes, you can actually help reduce the traffic jam, speed up fieldwork and value the participant’s voice even if they didn’t finish. And the quality of the data is often high, because the respondents went through the same entry quality checks as those who completed. While the demographics of people who dropout are not always the same, they are typically people who are less engaged with the category and could provide valuable nuances in the findings. For example, in a consumer electronics study, it could be older females dropping out. On a gaming study for a casual game designed for women, it could be male players dropping out. This means that we certainly always skew the data to more frequent / involved / higher spender consumers by only keeping completes.
She says that, of course, she still prefers a complete dataset. And the value of keeping partial completes varies from study to study. On a high incidence study, it would be best to simply wait until you hit your quotas with completed interviews. If the incidence is low and replacing these individuals is difficult, then it might be a study where keeping partials makes sense. Keeping partial completes can create inconsistency in the data, so it is something to be aware of when examining the overall results.
Karine acknowledges that even 10 years ago, we would not have considered keeping any incomplete data to feed into our research results or insights. We didn’t need to because we could get plenty of respondents, and they would be more likely to finish a long survey. Today, she says we are facing new challenges. We have to be creative in how we approach these challenges and evolve with the participants and the technology.
Share this
- October 2024 (1)
- September 2024 (4)
- August 2024 (6)
- July 2024 (7)
- June 2024 (4)
- May 2024 (7)
- April 2024 (6)
- March 2024 (3)
- February 2024 (8)
- January 2024 (3)
- December 2023 (6)
- November 2023 (5)
- October 2023 (3)
- September 2023 (8)
- August 2023 (4)
- July 2023 (6)
- June 2023 (6)
- May 2023 (3)
- April 2023 (6)
- March 2023 (6)
- February 2023 (4)
- January 2023 (2)
- December 2022 (2)
- November 2022 (8)
- October 2022 (6)
- September 2022 (6)
- August 2022 (7)
- July 2022 (5)
- June 2022 (6)
- May 2022 (5)
- April 2022 (4)
- March 2022 (8)
- February 2022 (7)
- January 2022 (1)
- December 2021 (2)
- November 2021 (2)
- July 2021 (4)
- June 2021 (2)
- May 2021 (4)
- April 2021 (2)
- March 2021 (5)
- February 2021 (3)
- January 2021 (3)
- December 2020 (1)
- November 2020 (5)
- October 2020 (2)
- September 2020 (5)
- August 2020 (4)
- July 2020 (4)
- June 2020 (1)
- May 2020 (3)
- April 2020 (6)
- March 2020 (3)
- February 2020 (4)
- January 2020 (2)
- December 2019 (4)
- November 2019 (4)
- October 2019 (3)
- September 2019 (2)
- August 2019 (4)
- July 2019 (5)
- June 2019 (2)
- May 2019 (4)
- April 2019 (4)
- March 2019 (2)
- February 2019 (4)
- January 2019 (3)
- December 2018 (5)
- November 2018 (2)
- October 2018 (1)
- September 2018 (3)
- August 2018 (5)
- June 2018 (4)
- May 2018 (4)
- April 2018 (3)
- December 2017 (1)
- November 2017 (2)
- October 2017 (1)
- September 2017 (3)
- August 2017 (2)
- June 2017 (2)
- February 2017 (2)
- January 2017 (2)
- December 2016 (2)
- September 2016 (1)
No Comments Yet
Let us know what you think