Understanding vCore CPU utilization in MuleSoft CloudHub

MuleSoft CloudHub platform offers multiple worker sizes to deploy APIs. The worker sizes range from slice of a vCore to full sized vCores. It is often sufficient to deploy APIs to slice of a vCore like 0.1 or 0.2 rather than to a one full vCore, unless the API is CPU intensive or the deployed application consists of many complex APIs packaged as a bundle.

Most of us know that MuleSoft workers are AWS EC2 instances behind the scenes, but what we may not know is how CPU is utilized in a slice of vCore and what could be the possible implications of using a slice of vCore.

The below table shows the various worker sizes and their corresponding EC2 instance types.

Burstable Performance Instances:

Instances providing slicing of vCores are called burstable performance instances. In these instances, full CPU utilization is achieved when an instance reaches baseline utilization percentage defined for the instance type.

What happens when an instance hits baseline utilization percentage?

Burstable performance instances accrue certain number of credits per hour when the load on the instance is below the defined baseline percentage for that instance type. Credits are accrued until the maximum limit for the instance type is reached. 

When the CPU utilization for an instance bursts beyond the baseline percentage, then the instance would start using the accrued credits. When the instance runs out of accrued credits, then the application slows down as the processes starve for CPU.

Any of the following scenarios will use one CPU credit.

• One vCPU at 100% utilization for one minute
• One vCPU at 50% utilization for two minutes
• Two vCPUs at 25% utilization for two minutes

The following table shows baseline utilization percentage and CPU credits by instance type.

Instance type	CPU credits earned per hour	Maximum earned credits that can be accrued*	vCPUs	Baseline utilization per vCPU
T2
t2.nano	3	72	1	5%
t2.micro	6	144	1	10%
t2.small	12	288	1	20%

* The number of credits that can be accrued is equivalent to the number of credits that can be earned in a 24-hour period.

Life span of accrued credits:

When a T2 instance is launched, it gets 30 launch credits per vCPU. For example, t2.micro or t2.small has 1 vCPU, so these instances are launched with 30 launch credits whereas t2.medium has 2 vCPUs and get launch credits of 60. Launch credits do not count towards the CPU credit balance limit. 

For T2 instances, the CPU credit balance does not persist between instance stops and starts. If you stop a T2 instance, the instance loses all its accrued credits.

Monitoring CPU usage:

You can monitor CPU usage for an API through the built-in dashboards available in Anypoint Monitoring section of the Anypoint platform. The overview section for an API provides valuable metrics like CPU% utilization, heap usage, thread count etc...

The following is a screenshot of the CPU% utilization metric for an API.

CPU Usage alert:

Alerts can be configured in Runtime manager to get notified when the CPU usage is beyond a certain threshold for a consistent period of time. The threshold can be determined based on the instance type(worker size) used.