Scientific Testing: The Most Reliable Way to Test a Network

The FCC is seeking to more closely regulate a key tactic in mobile carrier marketing—their performance and speed claims.

The commission already does this for fixed broadband and has proposed to use crowd data to set the upper limit for carrier marketing claims.

But here’s the problem: There are significant differences between crowd and scientific testing.

Crowd testing is easier to conduct but tough to draw out any useful conclusions, while scientific testing takes significant resources to conduct but provides easy-to-understand and useful results based on a methodical process that is accurate and enables apples-to-apples comparisons. As a result, the FCC, in taking a shortcut with crowd testing, will not present the full or fair picture of the performance and speed of mobile providers.

Although the differences between crowd and scientific testing could just be chalked up merely to competition, with both sides advocating their approach, a major government agency has decided to throw its lot in with a crowd tester. Such an approach will provide a limited view of the mobile consumer experience and won’t provide an accurate reading of the service providers’ strengths and weaknesses.

In this report, we provide an overview of both scientific and crowd testing and provide a number of observations on the right policy direction.

Download the Report Now