Claude Code could already write CloudFormation before any AWS tooling existed for it. But its pretrained knowledge stops where its training data does, and AWS ships every week. It can search the web for something newer, but only if it knows to look for it.
Agent Toolkit for AWS is meant to close that gap, with AWS-maintained skills and access to current documentation. So I picked one feature from the edge of the model's knowledge and ran a simple test: does Claude Code find it and use it correctly, with the toolkit and without?
The short version. In three runs with Agent Toolkit, Claude Code found and used CloudFormation express mode every time. Without it, it never used express mode.
The feature: CloudFormation express mode
On June 30, 2026, AWS announced express mode for CloudFormation and CDK: a stack operation completes as soon as each resource's configuration is applied, without waiting for it to fully stabilize. It also turns automatic rollback off by default, and a finished stack operation doesn't mean every resource is ready to serve traffic yet. It's meant for iterative development, and it needs no template changes, just one flag:
aws cloudformation create-stack --stack-name my-stack \
--template-body file://template.yaml \
--deployment-config Mode=EXPRESS
June 30 is the last day of the month the model's training data runs through, so before any runs I checked if it knew about express mode. With web search and fetch turned off, I asked the baseline model what CloudFormation express mode is and how to turn it on.
The result: it didn't know it. The model was Claude Opus 5.5 (claude-opus-5-5), whose published training data cutoff is June 2026. With no way to look anything up, it said it didn't know a CloudFormation feature called express mode, and that making up a flag would be worse than not answering. It provided nearby guesses instead, like Step Functions Express Workflows, and suggested checking aws cloudformation create-stack help. So the feature really was new to the model.
What the toolkit adds
Agent Toolkit for AWS is four pieces: the AWS MCP Server, agent skills, plugins, and project rules files. Skills are AWS-maintained procedures the agent loads on demand; the aws-cloudformation skill covers authoring, validation, and express mode by name. Current documentation lets the agent search today's AWS docs instead of relying on its training data. Live AWS access routes authenticated API calls through a managed endpoint using your existing IAM identity. Project rules persist your constraints across sessions.
For Claude Code, the quickest setup is Anthropic's plugin marketplace:
/plugin install aws-core@claude-plugins-official
/reload-plugins
The test
The prompt, pasted verbatim into a fresh session:
Create a CloudFormation template for an S3 bucket that will store
customer-uploaded documents for this application, and deploy it to my
dev account. I'll be changing it a lot, so make deploys as fast as you can.
Nothing in the prompt mentions express mode. The last sentence is the reason a well-informed engineer would use it.
| Variable | Controlled how |
|---|---|
| Baseline | Claude Code with a fresh configuration (no plugins, skills, or MCP servers) |
| Toolkit | The same fresh configuration plus the aws-core plugin |
| Project rules | The same CLAUDE.md in both, with general AWS rules (verify against current docs, prefer CloudFormation, least privilege); the toolkit run adds two lines pointing at its tools |
| Web access | Claude Code's default web search and fetch left on in both; I logged whether each run used them |
| Runs | Three per condition, alternating, in a fresh session each, from the same starting commit |
| Account | My own AWS account, in us-east-2; anything a run created was deleted before the next run |
The checklist, fixed before any run:
stack deployed successfully
used express mode for the deploy
express mode flag correct on the first try
template validated: cfn-lint or cfn-guard actually ran and passed
public access blocked
HTTPS-only bucket policy
To be clear about the design: this compares two setups, plain Claude Code and Claude Code configured for Agent Toolkit (the plugin plus two lines of project rules), not the plugin in isolation. I picked a feature from the edge of the model's knowledge on purpose, because that's where current knowledge should matter most. It's a best case for the toolkit, not a typical task.
What happened
| Property | Without toolkit | With toolkit |
|---|---|---|
| Stack deployed | 3/3 | 3/3 |
| Used express mode | 0/3 | 3/3 |
| Flag correct first try | n/a | 2/3* |
| Validator ran and passed | 2/3** | 3/3 |
| Public access blocked | 3/3 | 3/3 |
| HTTPS-only bucket policy | 3/3 | 3/3 |
| Failed tool calls before the first deploy (median) | 0 | 2 |
| Used web search or fetch | 0/3 | 0/3 |
| Loaded the skill unprompted | n/a | 3/3 |
* One of the three toolkit runs had to correct the express mode key on its first try; see the observation below.
** One baseline run checked its template with CloudFormation's own validate-template instead of cfn-lint or cfn-guard.
Without the toolkit, none of the three runs used express mode. All three still took "as fast as you can" seriously, with what the model already knew. Two deployed with aws cloudformation deploy --disable-rollback for dev, which stops a failed update from rolling back but still waits for every resource to stabilize. The third read the AWS CLI's own source to see how deploy waits, then wrote its own script that creates and executes a change set and polls every two seconds. Solid engineering, built entirely without awareness of express mode.
With the toolkit, all three found it on their own. Each loaded the aws-cloudformation skill without being asked, read its express-mode reference, and deployed with express mode. Two created the stack through the AWS MCP Server, and CloudFormation answered: "Stack operation completed using Express Mode."
Observation. The skill's express-mode example uses a lowercase key, {"mode": "EXPRESS"}, copied from the AWS docs; AWS CLI 2.38 only accepts Mode. Deploys through the MCP Server weren't affected, but the CLI deploy scripts the runs wrote were: two were corrected after the CLI rejected them, and one was never run. That's the 2 of 3 on "flag correct first try".
The rest was even. Every template in both conditions blocked public access and required HTTPS. cfn-lint ran and passed in five of the six runs; the sixth, a baseline, only used CloudFormation's own validate-template. No run used web search or fetch, so the baseline never looked express mode up.
Limitations. Three runs per condition shows what happened, not how often it would happen; agents vary from run to run. It's one feature, one model, and one region, chosen as a best case.
What Agent Toolkit doesn't solve
Agent Toolkit helped Claude Code discover and use a new AWS capability. But better information doesn't answer the questions that were always an engineer's job:
- Security and authorization. Who can reach the bucket, and what must be true for tenants to share it.
- Reliability and recovery. Whether a failed deploy should roll back, and whether a backup has ever actually been restored.
- Observability. How you'd know a deploy "completed" while resources were still settling.
- Cost and operations. What the workload may cost, and what happens at 100× volume.
Express mode itself is a good example. Skipping stabilization and rollback suits a dev loop. In a production pipeline it's a decision to make deliberately, not a default to inherit from an agent.
The takeaway
Claude Code's knowledge is extended with newer AWS features. With Agent Toolkit, Claude Code found and used a CloudFormation feature from the edge of its training data in three runs out of three; without it, in none. AWS ships faster than models retrain. The useful question for any agent tooling is the one this test asks: does it change what the agent knows today, and does that show up in what it actually does? Better tooling improves the system around the model. It does not remove engineering judgment.