Cloud Service Latency: What you should know

Performance testing of cloud services is of great interest to me and many other software engineers. Therefore, I created an application anyone can use to do their own point-to-point Linxter messaging performance testing. It’s an open source Visual Basic console application. In addition to reviewing our posted results, you can create your own testing scenarios and customize the console application as needed to better understand your own cloud service latency issues.

Here are a couple of important points regarding performance testing:

• Point-to-point and point-to-multipoint are different kinds of tests. In many systems, the time it takes to send 1 message from 1 to 1 (another) endpoint is the same as it takes to send 1 message from 1 to 100,000 endpoints.
• Comparing how many messages a cloud messaging system as a whole can process per second is a completely different kind of test than point-to-point messaging.
• There are many factors that can affect the results. When running your tests, be sure to run your tests multiple times on the same machine, and be aware of what else is running on that test machine. How busy your local area network is plays an important role too.
• Machine specs have a significant impact on results, as you will see by reviewing our posted performance results. In fact, my old laptop at its best in the recorded test run (I’m located in Cooper City, Florida - USA) was only sending 14.646 messages per second. And my colleague, Felipe in Florianopolis, Santa Catarina – Brazil, on a newer machine was sending 18.017 messages per second. The specs of both machines can be found in the results spreadsheet.
• A server running in a commercial data center will almost always outperform a laptop or desktop running on a shared DSL (home or office) network. An old blade server of ours (running in a commercial data center in Boston, MA) was sending 46.262 messages per second.
• The lowest latency can be achieved by using a Linxter private cloud package running in the same data center as your systems. The test results posted all used a Platform-as-a-Service package.
• Some systems turn off security and reliability features in order to show optimal results. As designed, Linxter’s various levels of security and reliability cannot be turned off.

You can download the Visual Basic solution file here, along with our test results and a setup document.