Is the "compare across clouds" part coming soon? The ffmpeg report for example only had AWS shapes (I was hoping to see a nice head-to-head of AWS, GCE and maybe SoftLayer).
On a more direct note, it seems like your reports should also filter based on input size / memory requirement. The t2.micro (and GCE's f1-micro) is good value if you can fit in its memory and you only need to process for a short burst, but if either of those things doesn't hold you're probably not going to be happy.
Thanks for the feedback. Multi-loud is currently implemented for AWS and a few Rackspace instances. The ffmpeg report should be fixed now - a small bug on our part in the publicly visible reports.
We are working with several cloud providers as well as HPC system providers to get them integrated, including SoftLayer. If you could connect us with someone at GCE to get this rolling that would be of course appreciated.
Regarding your comments on instances these are very valid, and the filters you suggest would be indeed very useful. Presently, if an instance runs out of memory for a particular benchmark it returns a failed code and will be excluded from the Profiler results. On the t2.* instances absolutely agreed that they are good for bursty loads, but once the credits are depleted then the performance drops off the cliff. For this there are two potential solutions: First, a long running workload will expose this drop off, i.e something that runs for more than ~30 minutes on a t2.micro. Another way, would be for us the benchmark a credit depleted instances to show the performance delta - we do have that on the schedule so that people can understand the performance implications.
Point taken. Thank you very much for the feedback. We thought this may be a bad idea. Currently the Profiler supports instances from AWS and Rackspace. We are working with additional providers to get them on board soon.
This is an excellent idea. It might not have a lot of features that you'd expect right now, e.g. being able to compare DO and AWS side by side, but it can give you a comprehensive picture of what you're in for with different cloud providers.
8 comments
[ 3.4 ms ] story [ 24.9 ms ] threadOn a more direct note, it seems like your reports should also filter based on input size / memory requirement. The t2.micro (and GCE's f1-micro) is good value if you can fit in its memory and you only need to process for a short burst, but if either of those things doesn't hold you're probably not going to be happy.
Disclaimer: I work on Compute Engine.
We are working with several cloud providers as well as HPC system providers to get them integrated, including SoftLayer. If you could connect us with someone at GCE to get this rolling that would be of course appreciated.
Regarding your comments on instances these are very valid, and the filters you suggest would be indeed very useful. Presently, if an instance runs out of memory for a particular benchmark it returns a failed code and will be excluded from the Profiler results. On the t2.* instances absolutely agreed that they are good for bursty loads, but once the credits are depleted then the performance drops off the cliff. For this there are two potential solutions: First, a long running workload will expose this drop off, i.e something that runs for more than ~30 minutes on a t2.micro. Another way, would be for us the benchmark a credit depleted instances to show the performance delta - we do have that on the schedule so that people can understand the performance implications.
Can only view one "report" before being forced to register an account that needlessly requires my personal information? No chance. You're dead to me.