High-Compute AI Agents: A Governed Architecture for Long-Running Work

Many agent designs assume a tool call will finish quickly. The agent sends a request, waits for a response and continues. That model becomes fragile when the requested work involves a long GPU job, a large scientific dataset or a pipeline that can run for hours.

On October 16, 2026, Google Cloud published a reference architecture for high-compute biotech agents. Its examples cover protein folding, compound screening and structure-activity analysis. The engineering pattern is useful beyond biotech. It applies wherever an agent plans work that must run in a separate compute system, including simulation, media processing, risk modelling and large-scale data analysis.

The important shift is architectural. The agent should coordinate the work without carrying the full workload inside its own context or runtime.

Separate reasoning, execution and governance

Google's design has three layers:

1. A cognitive orchestrator interprets the goal, breaks it into steps and decides which specialist or service should handle each step.
2. An execution layer runs the expensive or specialized computation on infrastructure suited to the task.
3. A governance plane controls identity, approved destinations, policies, traffic and audit evidence.

This separation keeps each part honest. The language model can plan and summarize without pretending to be a numerical engine. The compute service can process large inputs without sending every intermediate value through an LLM. Security teams can apply controls at the boundary between them.

Google's implementation uses Gemini Enterprise Agent Runtime for orchestration and containerized scientific models on Google Kubernetes Engine for execution. The exact products will vary by organization. The boundary matters more than the product names: reasoning produces a well-defined job request, and execution returns a durable result plus evidence about how it was produced.

Treat heavy work as a job, not a tool response

A short API call can return one response. A long-running compute task needs a lifecycle. It may queue, start, pause, retry, fail or finish after the user has left the original session.

For production systems, we recommend making that lifecycle explicit in a job manifest. This is our design recommendation based on the reference architecture; Google Cloud does not require this format. A useful manifest records:

• a unique job ID and idempotency key
• the requested operation and versioned input references
• the execution environment, model or pipeline version
• the initiating user and agent identity
• cost, region and data-classification constraints
• approval state and approver, when required
• created, queued, started and completed timestamps
• output references, checksums and execution logs

The orchestrator can submit the manifest, return the job ID and check status without holding an open request for the whole run. If a retry occurs, the idempotency key helps prevent an expensive duplicate job.

This also makes incident review possible. A team can answer which version ran, who authorized it, what inputs it used and where the outputs were stored.

Pass large data by reference

Google's architecture writes large structural files and trajectory logs to Cloud Storage, then passes signed object references and compact summaries between agents. That avoids filling the model context with data the model does not need to read.

The same rule is useful for video, design files, financial scenarios and large analytical outputs. Keep the binary or high-volume state in controlled storage. Give the agent only the reference and the metadata required for the next decision.

A reference should still be treated as sensitive. It needs:

• a short validity period
• access limited to the intended workload identity
• a content checksum or immutable object version
• encryption and regional placement that match the data policy
• an audit record of creation and access

A URL is not a security boundary by itself. The storage policy and identity attached to it do the real work.

Match concurrency to the execution system

Agents often generate parallel tool calls because parallelism can reduce perceived latency. High-compute backends may serialize requests, share a small GPU pool or impose their own queue. More client-side concurrency can then increase timeouts and contention.

Google recommends aligning agent concurrency with the execution topology. In practice, the orchestrator needs capacity signals from the job system: queue depth, available workers, estimated start time and per-tenant limits. Those signals should influence whether the agent starts work now, batches it or asks the user to approve a longer wait.

Cold starts introduce another trade-off. Google's reference design suggests warm nodes and preloaded model weights for latency-sensitive scientific workloads. A warm pool reduces startup time but carries an idle cost. Teams should choose it from measured demand. For low-frequency work, a visible queue and a realistic completion estimate may be the better experience.

Stream milestones instead of silence

An agent should not disappear while a thirty-minute job runs. The execution layer can publish meaningful state changes such as accepted, queued, running, validation complete, failed and completed.

Milestones need to reflect actual system events. A conversational message that says "still working" without an underlying state change is not useful evidence. Each update should be tied to the job ID, timestamp and execution stage.

Good progress reporting also changes how failures are handled. If preprocessing succeeded but simulation failed, the operator can see what is safe to reuse. A resumable pipeline can restart from a verified checkpoint and avoid paying for every earlier stage again.

Keep waiting out of the model context

Long jobs can consume tokens even when the agent is doing no useful reasoning. An October 14 OpenAI customer story describes how Third Moment Research found agents repeatedly polling a backtest that took several hours. Those checks filled the context while the agents waited. The company reports that a caching and memory-packing change reduced that particular backtest from roughly three hours to three to five minutes.

That result is a customer-reported example, not a general benchmark. The underlying failure mode applies more broadly: a model loop is an expensive place to wait for an external system.

Move waiting into the job system. A queue or workflow engine can track state without asking the model to reason on every check. Resume the agent when the job emits a meaningful event, reaches a deadline or needs a decision. If the backend cannot send events, use a thin supervisor for rate-limited polling and keep raw status responses out of the working context.

Measure waiting separately from reasoning. Useful operational fields include polling count, time spent in each state, tokens consumed while no state changed and the reason the agent resumed. These records show whether latency comes from compute, queuing or orchestration overhead.

Put retry decisions outside the worker

The same OpenAI story says Third Moment places its working agent inside an outer harness. When work stops early, the outer harness examines the trajectory and chooses whether to retry, terminate or request human review.

This pattern can prevent a stalled worker from retrying itself indefinitely. The supervisor needs an explicit policy with named failure classes, budgets and escalation rules. Define:

• which failure classes are safe to retry
• the maximum attempts, elapsed time and cost
• whether a retry resumes from a checkpoint or starts again
• which state changes require a person to review the evidence
• which credentials and tools the supervisor may use

Using another model call does not make the supervisor independent. For costly or sensitive work, enforce budgets and stop conditions in the job system where the worker cannot rewrite them. The model can classify what happened and prepare evidence, while deterministic controls enforce the final limit.

Use asymmetric approvals

The October 16 architecture describes "asymmetric gating": automate reversible operations and require explicit approval for consequential ones. That is a practical way to avoid two bad extremes, unbounded autonomy and an approval prompt for every small step.

An approval matrix for a high-compute workflow might look like this:

| Action | Default handling | Reason | |---|---|---| | Read approved metadata | Automatic | Reversible and low cost | | Validate or transform an input | Automatic with logging | Output can be discarded | | Start a large GPU job | Policy check or human approval | Material cost and capacity impact | | Release a result to another system | Human or policy gate | Changes the distribution boundary | | Trigger a physical or regulated action | Named domain approver | Difficult to reverse and potentially high impact |

Risk is contextual. A small job on public data may be safe to run automatically. The same operation on proprietary or regulated data may need a different region, private endpoint and explicit approval.

The approval record should become part of the job manifest. Keeping it only in a chat transcript makes later reconciliation harder.

Put security controls at the boundary

Google's supporting services show what a governance plane can cover:

• Agent Identity assigns a distinct cryptographic identity to an agent. Permissions can be granted to that agent principal instead of sharing a broad service account credential.
• Agent Registry catalogs approved agents, tools, MCP servers and endpoints.
• Agent Gateway enforces connectivity and policy checks across user-to-agent, agent-to-tool and agent-to-agent traffic. Its default posture blocks connections unless an IAM policy grants access.
• Model Armor can inspect prompts and tool responses for risks such as prompt injection and sensitive-data leakage.

These controls are related, but they are not interchangeable. Identity answers who made the request. The registry describes what may be reached. The gateway decides whether the interaction is allowed. Content inspection looks for unsafe material crossing the boundary.

There is also a lifecycle detail worth planning: Google's Agent Identity documentation says deleting an agent does not automatically remove IAM bindings granted to its principal. Decommissioning therefore needs an explicit access-cleanup step.

Evaluate two systems, not one

A high-compute agent combines orchestration quality with domain-execution quality. One score can hide which part failed.

Evaluate the orchestration layer on whether it chose the correct tool, formed a valid job, respected approvals, handled retries and reported state accurately. Evaluate the execution layer with domain-specific checks for numerical, scientific or business correctness.

Then test the handoff between them:

• Did the manifest preserve the user's intent?
• Were inputs immutable and correctly scoped?
• Did a retry create one job or two?
• Could the agent summarize results without overstating confidence?
• Did every consequential action have an identity, policy decision and evidence trail?

This resembles a spec-driven agent workflow: the contract between components becomes as important as the prompt.

A practical pilot sequence

Start with one workflow where heavy compute is genuinely useful and the outcome can be reviewed before it affects production.

1. Define the job manifest and state machine before connecting an agent.
2. Run the compute pipeline manually and measure queue time, startup time, runtime and failure modes.
3. Give the agent access to a narrow submission interface, not the underlying cluster controls.
4. Store large inputs and outputs by reference with scoped access.
5. Add milestones, cancellation and idempotent retries.
6. Classify actions by reversibility, cost and data sensitivity, then attach approval rules.
7. Test orchestration and domain results separately before testing the full workflow.
8. Review the audit trail with security, platform and domain owners.

The useful question is whether the organization can understand, control and reproduce a job after the original conversation has ended.

Bayseian helps enterprise teams design and evaluate AI systems with these production boundaries in place. A focused pilot can test the handoff, policy model and evidence trail before broader automation is considered.

Sources

• Google Cloud, "Architecting autonomy: A design pattern for high-compute, governed agents in biotech", October 16, 2026.
• Google Cloud, Agent Runtime documentation, updated October 15, 2026.
• Google Cloud, Agent Identity overview, accessed October 18, 2026.
• Google Cloud, Agent Gateway overview, accessed October 18, 2026.
• Google Cloud, Agent Registry, updated October 17, 2026.
• OpenAI, "How Third Moment Research runs 10,000 agents for 36+ hours", October 14, 2026. Customer-reported operating details are identified as such in the article.