Speed Testing

[peachy]

I just read the note in the "Informational" about this and was also chatting to Mike over email but I still don't really understand your speed tests.

> However, after we stopped testing but continued transferring the file, the 50Mbps connection was very slow, very sluggish, trying to pull up a web site on a very fast remote site that we often use was not fast, it was very slow to load the page, taking nearly 30 seconds.
I don't understand the point here, a web site is much more complicated to measure that a file transfer. Where was the time taken? Initial connection (slow webserver), pulling down the elements of the page (webserver or connection), rendering the page?

I'm getting about 2-300Mbps on my wired speedtest, it's still useful in that i will spot a relative degradation but i'm pretty sure if we used iperf or qperf to test between our connections we'd almost always get 1000Mbps.

I'm assuming (and reading between the lines in your article) that you are concerned ISPs might have traffic shaping in place that would e.g. throttle a connection. So if we want to test a 1000Mbps line we'd ideally xfer a 1TB file or something, but then of course I'd see a degradation in performance over the duration of this test, so you have to xfer a smaller file and calculate up?

My guess is the speedtest uses a very small file and so on my big connection it's measuring the time taken to establish the connection too which is why it nets out at about a quarter actually bandwidth?

[Mike]

Great question.

The main point of the article is that there is no way to get anything conclusive. It's just another test, another stat that needs to be taken into account with more information.

Speed testing is definitely useful but is a moment in time test using shared bandwidth where the test can go across networks we have no control over and have no idea how they are set up. More importantly, most of these tests seem to terminate on CDNs which are optimized to cache data mainly for streaming and other media.

That's why we used a big file so that the transfer could settle and we could (should) get a fairly sustained speed going. Even when using iperf which is a more real world test, it is inconclusive because the test travels over networks we don't control.

The article is mainly questions like why did the transfer drop down to kilobytes when it's a 50Mpbs pipe that is barely being used? I don't think we are implying anything about the provider but pointing out that we simply don't know since we don't control their network. We can make some assumptions by testing different kinds of data to different locations but in the end, it is inconclusive.

Yes, a web site could be slower but 30 seconds is way up there in terms of time to render pages. The point there was simply to see if there was something going on with the file transfer specifically or the overall bandwidth. Meaning, maybe the sustained speed caught the attention of an application manager which automatically tried to keep the transfer at a certain speed. No idea.

One of the more interesting things during the testing was that we checked the hops to the destination and could see that the provider actually had something on the edge of the data center we were testing to. Meaning, our location, the provider, level3, back to the provider then into the DC. It seemed odd that we could never get anything close to 50Mbps.

I'm getting about 2-300Mbps on my wired speedtest, it's still useful in that i will spot a relative degradation but i'm pretty sure if we used

iperf or qperf to test between our connections we'd almost always get 1000Mbps.

When you say test between your connections, do you mean with the same provider or would those tests go across multiple network owners to get from source to destination. Testing between servers in a data center or even to another data center managed by the same org usually always shows the correct throughput but that's because in these cases, the network owner is under obligation to make those speeds can handle all of the traffic required. Cable companies don't work this way, it's a best effort service that has 'acceptable' ranges to cover all their needs. I cannot speak with authority since I've never been part of a cable company but I've spent years fighting with them to get what we pay for and to get them to fix problems for us and customers when we offered ISP and MSP services.

We did a lot of this kind of testing and while speeds can remain around what ever they tested at, most of the time, it's up and down as expected.

My guess is the speedtest uses a very small file and so on my big connection it's measuring the time taken to establish the connection

too which is why it nets out at about a quarter actually bandwidth?

In fact, OTM tests against fast.com which has an interest in making sure that consumers are getting the bandwidth they are supposed to be getting. If you are seeing what you think is a set up time as a delay, it's possible since it has to spawn a process, hit at least three servers, calculate the results, etc. The browser test is instant if you go to fast.com.

However, hardware agents that we sell do something different, similar to what you said. They run a shorter, smaller test to get a quick average.

Unlike the usual speed testing that people are doing, we aren't really interested in fully saturating everyone's connections over and over again.

The main purpose of our speed test is not about finding out what your maximum speed is but if you have usable bandwidth for your requirements.

Our speed testing tools are still in their infancy as we try to find the best way to give the most useful information without fully saturating and using up all kinds of data since many are on data plans. The speed testing will end up being to and from our network only at some point when we can find the best way to get useful results. We've also tested limiting the speed to fast.com so as not to be using up users bandwidth/data or wonderful fast.com service.

BTW, there are a lot of interesting blogs/articles about fast.com and why they built the test and made it download only. it's quite interesting.

I hope this helps clarify the bit of testing we did that day :).