How to save up to 80% in cloud HPC costs?

To test the availability of preemptible instances, a Techila Distributed Computing Engine computing system of 1000 n1-highcpu-2 Linux Techila Worker instances was deployed and the time required to start the instances was measured and logged. Each instance had 2 CPU cores, meaning the system consisted of a total of 2000 CPU cores. The time required to start the Techila Worker instances is illustrated in the image below.

The orange line represents the amount of Techila Workers that are ready for computations. In the deployment test, 99% of the capacity was online in under 3 minutes. The remaining 1% of the instances were terminated by preemption events during start-up, which increased the total start-up time as new instances had to be started.

To test the effect of preemptible instance terminations on the amount of computing power available, a deployment consisting of 50 n1-highcpu-2 Linux Techila Worker instances was kept running for a time of 24 hours. Each instance had 2 CPU cores, meaning the system consisted of a total of 100 CPU cores. These tests were repeated several times on different dates. During these tests, the number of preemption events varied greatly, ranging from 1 to 401 events. More general information about preemption selection is provided by Google and can be found here.

Each time an instance is preempted, the instance will be terminated. However, because TDCE uses a managed instance group to run the instances, any terminated instances will be automatically replaced with new instances. The self-configuration features in TDCE will also mean that the new instances will be automatically configured to process the previous Techila Workers' workload. This minimizes the impact of the termination to the user.

The graph below illustrates the preemption events observed during one of our tests and contains two key metrics:

The orange line shows the cumulative amount of preemption events during the 24-hour test. There was a total of 168 events, meaning a total of 168 instance terminations took place during the test. Each time an instance was terminated, a replacement instance was automatically started and reconfigured by Techila Distributed Computing Engine.

The blue line shows the amount of available Techila Workers. Each time a preemption event takes place, there is a corresponding short downtime period for a Techila Worker. Despite there being a total of 168 termination events during the test, the average amount of Techila Workers online is close to 100% all the time, never dropping below 90%.

As preemptible instances use the same underlying cloud hardware as normal instances, the main cause for performance differences will be the preemption terminations, which can happen at any time. Compute Engine always terminates preemptible instances after they run for 24 hours, assuming the instance has not been terminated in another preemption event prior to that.

To measure the effect of these preemption terminations on computational workloads, an identical run was repeated using a set of normal instances and preemptible instances. The workload consisted of 36,000 MATLAB Jobs, which were processed using 50 x n1-highcpu-2 Techila Worker Linux instances. Each Job took 5 minutes to complete.

The duration of each run was just over 30 hours. In short, the run executed on the preemptible instances was only 8 minutes 17 seconds slower (0,45%) than the run executed on the normal instances, which was a pleasant surprise for the test team.

The total execution times of the runs are shown below:

The image below shows how the performance of preemptible instances (orange line) compares with normal instances (blue line). As can be seen, the orange line is overlapping the blue line during the majority of the Project duration. This means that both preemptible and normal instances are processing workloads at the same rate. This is possible because Techila Distributed Computing Engine keeps the amount of computing power on the desired level despite preemption terminations and reconfigures any replacement instances, so they automatically rejoin the computations.

There was a total of 57 preemption termination events during the test, 37 of which occurred at the 24-hour mark. The reason why only 37 instances of a total of 50 were terminated at the 24-hour mark, is that all other instances had been preempted by an unexpected preemption event (which resets the timer). Each time a preemption termination occurred, a replacement instance was automatically provisioned and configured by Techila Distributed Computing Engine.

The most visible effect of these preemption terminations can be seen around the 24-hour mark, where 37 instances were terminated. Even though the interrupted Jobs were automatically restarted after replacement instances had been provisioned, the performance impact of these interruptions resulted in the orange line (preemptible capacity) trailing the blue line (normal capacity) in the amount of completed Jobs for the remainder of the Project.

There was also one unexpected instance termination when using normal instances. This instance was automatically replaced and reconfigured to process the MATLAB workload, but still delayed the Project completion by approximately 5 minutes.

To compensate for the unexpected preemptible terminations, preemptible instances are priced competitively, allowing you to save up to 80% on instance infrastructure cost.

The table below shows how the cost of the computations can be calculated for normal and preemptible instances. These calculations are for the computations that were discussed in the Performance Chapter.

In this particular use case, we were able to save over 75% on the cost by using preemptible instances.