Colleen Carlin pointed me toward an interesting post on the Pew Web site. It compares results from landlline and cell phone samples from three polls on the presidential election. The polls were conducted in late June, late July/early August, and mid-September. The key findings:
- In all but the first poll in late June, the landline samples show a tied race.
- In all three polls, the cell phone samples show Obama leading by around 20 points.
- When you combine the two samples the impact is minimal--a two point increase for Obama in June and a one point increase for the two other polls.
These results suggest to me that the story we have been hearing for some while about the impact of wireless substitution is holding. That story has been that unless you are especially interested in the younger demographic the impact of not including cell phones in samples is largely corrected with traditional demographic weighting. Given the substantial cost often associated with calling cell phone samples we should continue to ask ourselves whether it is worth it for the sake of one and two point differences in estimates.
However, there is a caveat and it relates to subject matter. We probably should not infer too much from political polling. Other domains such as customer satisfaction or new product research may be very different. As always, more research is needed.