Last year, Gongos research did a study comparing character counts from mobile vs. online surveys that surprised us. Contrary to what we would have expected based on our experience, they found that the average character counts for open-ended questions were similar between online and mobile surveys taken on smart phones, with the latter actually yielding slightly longer responses. Our hunch was these findings were explained not by the fact that people are eloquent in their mobile survey responses, but rather, that we are accustomed to seeing much longer online survey responses than are typically derived from a panel.
To test our hypothesis, we conducted a quantitative comparison of 1,178 member-generated, open-ended survey responses gathered via both online and mobile surveys, analyzing character counts from each. In parallel, we qualitatively assessed the thoughtfulness of responses, and compared themes between each of the two methods of data collection. Our findings suggest that, while online surveys produce lengthier member-generated text, members’ responses to open-ended questions posed via mobile survey tend to be more valuable to an in-the-moment, in-context learning objective. Findings from our study have informed our own best practices for using mobile techniques within the online community setting – both when to employ mobile for research, and how to pose questions most effectively.
Click here to download the paper, and read on for the top 3 things to keep in mind when employing mobile surveys with your online community:
Take right approach
Sure, mobile is hot right now (and is, undoubtedly, here to stay), but be sure to use the right tool for the task at hand. If your research objective is to understand how consumers feel about their phones, for instance, this can most likely be accomplished via an online survey. Conversely, if you’re looking to gather situational, top-of-mind feedback from members (e.g. sitting in a movie theater, entering a grocery store), a mobile survey would be the way to go.
Get to the point
Question text should be succinct. After all, if a participant is taking a mobile survey in-store, we want them to feel like they’re primarily shopping, and secondarily taking a survey.
Less can = more
Our online surveys typically yield long, reflective responses from our members, so the amount of text gathered via mobile survey can look a little scant in comparison. However, though not yielding the same level of detail and thought, these short, in-the-moment responses are often just as (if not more!) valuable in revealing the top of mind, and often more emotional response.