November 8, 2026 · Temple Compute
Research Computing Cost Control: Budgets That Stick
Research computing budgets fail in a predictable way. A postdoc launches a pipeline on Friday, a misconfigured stage asks for forty GPU nodes, and the cloud console sends a polite email on Monday. By then the grant line is spent. The bioinformaticians already know which tool they want. This post is for the person who has to sign off on it: research IT, the HPC manager, the finance partner.
How do you control research computing costs?
You control research computing costs by enforcing limits before compute is provisioned, not by reading reports afterwards. On a shared Slurm cluster that means accounts, QOS and fairshare. For cloud spend it means budgets scoped to the people and teams that spend money, a cost estimate checked at launch, a watchdog on running jobs, and rules on which hardware can be requested at all. Alerts alone only document the overrun.
Here is how the common layers compare:
| Control layer | What it limits | When it acts | Stops spend? |
|---|---|---|---|
| Cloud provider budgets | Account or project total | After spend or forecast crosses a threshold | No, alerts only unless you script it |
| Slurm accounts, QOS, fairshare | Share of an on-prem cluster | At scheduling time | Limits priority and size, not money |
| FinOps dashboards | Nothing, they report | After the bill | No |
| Horus, by Temple Compute | Cloud spend per user, organization and workflow | Before launch and during the run | Yes, denies or cancels cloud runs |
Each layer has a job. Slurm fairshare keeps a shared cluster fair. Provider budgets catch account-level drift. Neither one knows that a single pipeline run is about to cost three times what the project has left. That gap is where research overspend lives.
Showback vs chargeback: which model should research IT use?
Showback tells each group what it used and what it would have cost. Chargeback turns that number into an internal invoice against a grant or cost centre. Showback is the right first step for almost every institution: it surfaces usage without starting a fight about rates. Chargeback follows once the numbers are trusted and groups agree the rates are fair.
Both models need the same foundation: every unit of compute tagged to a person, a project and an organization at the moment it runs. Retrofitting attribution from scheduler logs and cloud billing exports is where most showback projects stall. If the orchestration layer already knows who launched which workflow under which organization, attribution is a query, not a quarterly reconciliation exercise. For cloud runs, Horus records the cost of every machine against the organization that launched it. For on-prem Slurm, the scheduler's accounts remain the system of record, and Horus adds per-task CPU, memory and wall-time measurements that make showback numbers concrete.
A worked chargeback rate model
The rate model does not need to be clever. It needs to be explainable to a PI in one table. The figures below are illustrative. Replace them with your own hardware cost, depreciation period and utilization.
| Resource | Formula | Illustrative inputs | Rate |
|---|---|---|---|
| CPU core-hour (on-prem) | (node cost ÷ years of life + yearly power and support) ÷ (cores × 8,760 h × utilization) | €12,000 node, 5 years, €1,500/yr, 64 cores, 70% | ≈ €0.010 |
| GPU-hour (on-prem) | Same formula per GPU | €40,000 node, 5 years, €4,000/yr, 4 GPUs, 60% | ≈ €0.57 |
| Cloud CPU or GPU hour | Provider on-demand price × (1 + overhead) | Instance list price, 10% overhead | Pass-through |
| Storage TB-month | (array cost ÷ months of life + support) ÷ usable TB | Your array | Your rate |
Two rules keep the model honest. Charge for requested resources, not used ones, because an idle core reserved for a job is unavailable to anyone else. And publish the rates where researchers can see them, so the cheapest option is also the obvious one.
Why do cloud budget alerts fail for research workloads?
Cloud budget alerts fail for research workloads because they act at the wrong scope and the wrong time. A provider budget watches an entire account, while research spend happens one pipeline run at a time. The alert fires after the money is committed, and stopping resources requires custom automation that someone in IT must build, test and keep working through every provider API change.
Research pipelines make this worse. A single run can fan out to hundreds of tasks, each provisioning its own machine. One wrong resource request, multiplied across a sample sheet, turns a €40 run into a €4,000 run in an afternoon. By the time an account-level forecast reacts, the nodes have been running for hours. The control has to sit where the run is created.
What Horus enforces, and where
Horus puts cloud cost control in the orchestration layer, the one place that knows who is launching what, on which hardware, under which budget. Budgets gate cloud spend; runs on your own cluster are governed by Slurm's own accounts and fairshare. Budgets are configured for your organization with Temple Compute and are visible to organization managers. These controls are enforced by the platform, not by convention:
- Budgets at three scopes. A monthly budget per user, a collective budget per organization, and a budget per workflow. A workflow budget denies only that workflow, so one expensive pipeline cannot starve the rest.
- Denied before launch. Horus estimates a run's cost from its task resources, and from the actuals of earlier runs once they exist. A run whose projected cost would cross a budget is refused before a single machine starts. Spot requests are gated at the on-demand rate, so a spot price drop never sneaks a run past its budget.
- Watched while running. A watchdog accrues the cost of every open cloud node. On a breach the run is either flagged or cancelled, depending on the mode you choose.
- Per-member token caps inside a shared organization.
- Hardware rules. An instance-type allowlist refuses any cloud machine type not on the list, and cloud targets not managed by Horus are refused for everyone.
- Concurrency limits on both runs and tasks.
Cloud machines are created when a run reaches a stage set to run in the cloud and destroyed when the run ends. Nothing idles overnight because someone forgot to tear it down.
Who can do what: roles and accountability
Spend control is also access control. Horus separates builders, who create and change workflows, runners, who launch them, and viewers, who read results. A member can run a workflow without being able to install new components. Launching requires the runner role, and a member removed from a project can no longer launch in it.
Every manual change to a token balance records who made it and why, so "who raised this group's allowance?" has an answer. SSO, a full audit trail and organization reporting are part of the Enterprise plan. For IT, that adds up to a short list of people who can spend money, a log of every exception, and budgets that the platform enforces rather than asks people to respect.
Use the hardware you already paid for first
The cheapest core-hour is the one on a cluster you already own. Horus connects to Slurm over plain SSH with each researcher's own account: no agent, no daemon, no shared service account to review. Jobs land in the partitions and accounts your users already have, so existing fairshare and QOS policies keep working unchanged.
Each workflow stage declares where it runs. A stage pinned to the cluster never leaves it. Only the stages you allow to run in the cloud ever reach it, and only within the budgets above. Sensitive data can stay on-premise while a GPU-heavy step runs on a rented machine, or the reverse. That decision belongs to IT and is written into the workflow, not left to whoever happens to launch it.
Right-sizing is the budget lever nobody pulls
The fastest way to cut research compute cost is to stop paying for resources jobs never use. On nf-core/rnaseq with 12 human samples on an AWS Slurm cluster, a right-sized Horus run finished 35% faster and 32% cheaper than Nextflow with nf-core's default resources. The default resource labels asked for about 2.2 times the CPU-hours the jobs actually used.
That gain came from Horus measuring every task, plus a scripted step that turned those measurements into new sizes. The same reasoning applies to every group on your cluster. For the hands-on version, see right-sized Slurm jobs. For GPU-heavy teams, the same budget controls apply to GPU-heavy LLM fine-tuning.
IT evaluation checklist
Use this list in any procurement review of a research orchestration platform, ours included.
| Question | Why it matters | Horus answer |
|---|---|---|
| Can a cloud run be stopped before it overspends? | Alerts after the fact do not protect a grant | Yes, projected cloud cost is checked at launch |
| At what scope are budgets set? | Spend happens per person and per team | Cloud budgets per user (monthly), organization and workflow |
| What happens when a running cloud job breaches? | Long runs drift past estimates | Warn or cancel, per configured mode |
| How is on-prem Slurm usage governed? | Shared clusters need fair access | Your Slurm accounts, QOS and fairshare, plus Horus per-task usage data |
| Can we restrict hardware types? | One wrong instance type dominates a bill | Instance-type allowlist |
| Does it install anything on our cluster? | Every agent is a security review | No, plain SSH with user accounts |
| Who can launch, edit or only view? | Spend authority should be explicit | Builder, runner and viewer roles |
| Are manual exceptions logged? | Auditors ask who approved what | Actor and reason recorded |
| Does idle cloud capacity linger? | Forgotten nodes are pure waste | Machines destroyed when the run ends |
| SSO and audit trail? | Identity and compliance | Enterprise plan |
If you are comparing orchestration options more broadly, our overview of orchestration platforms with cost controls covers the rest of the field. For how Horus itself works, start at the Horus engine page.
FAQ
What is the difference between showback and chargeback in research computing?
Showback reports each group's compute usage and its cost without billing anyone. Chargeback turns that usage into an internal invoice against a grant, department or cost centre. Most research IT teams start with showback to build trust in the numbers, then move to chargeback once groups agree the rates are fair.
Do AWS, Azure or Google Cloud budgets stop spending when the limit is reached?
Not by default. Provider budgets send alerts when actual or forecast spend crosses a threshold. Stopping resources requires extra automation that you build and maintain yourself, and it acts on accounts or projects, not on individual pipeline runs.
How does Horus stop a run from going over budget?
Budgets in Horus gate cloud spend. Before a run launches, Horus estimates its cost from the task resources and, once they exist, the actuals of earlier runs. If the projected cost would cross the user, organization or workflow budget, the launch is denied. While the run executes, a watchdog accrues the cost of open cloud nodes and either warns or cancels the run on a breach, depending on the configured mode.
Does Horus need an agent or service account on our Slurm cluster?
No. Horus talks to Slurm over plain SSH with the user's own account, the same way a person submits jobs. Nothing is installed on the cluster, there is no daemon, and no shared service account to audit.
Can we limit which cloud instance types researchers can launch?
Yes. An instance-type allowlist denies any cloud machine type that is not explicitly listed, and cloud targets not managed by Horus are refused for everyone. Concurrency limits on runs and tasks cap how much can run at once.