Improving agent effectiveness with CAFE(S) and DX
Putting CAFE(S) into practice with DX
Kali Watkins
Product Marketing
We recently collaborated with authors from GitHub, Capital One, and the University of Victoria to publish the CAFE(S) framework for evaluating the quality of context provided to AI agents. The research starts from the idea that model capability is only part of what makes an agent effective. Even highly capable agents are limited by the quality of the context they receive. CAFE(S) defines five properties of high-quality context: Clarity, Actionability, Fidelity, Efficiency, and Security.
At DX, this research is also informing how we’re building our products for measuring and improving agent effectiveness. Here’s what we’re building to make the framework viable.
Build the foundations for effective agents
Alongside the CAFE(S) research, we’re introducing an AI Readiness scorecard that helps teams evaluate whether their software environment is set up for effective agent use.
Before teams can assess context quality with CAFE(S), they need the right foundations in place. The AI Readiness scorecard looks at those prerequisites, including whether context exists, is accessible, and is organized so agents can find what they need. It also evaluates broader engineering standards and guardrails that help agents work effectively within the software environment.
For example, teams can check whether repositories have owners and linked documentation, whether README and agent instruction files exist and are current, and whether build scripts, test scripts, and other guardrails are in place to help agents safely write and ship code.
The scorecard gives platform teams a way to put these practices into place across their organization, with standards for the context, resources, and guardrails agents need to work effectively.
The AI Readiness scorecard is available in beta for DX AI Enablement customers. Reach out to your DX Representative to set it up in your account.
Understand what’s limiting agent effectiveness
Agent Experience, powered by AI Code Insights, helps teams understand where developers encounter friction when working with agents. It evaluates individual agent sessions across requirements, scope, and steering, helping teams identify recurring patterns and drill into individual sessions to understand what happened and why.
For example, Agent Experience might show that requirements are a recurring source of friction. Looking at the underlying sessions could reveal that agents repeatedly have to reconstruct missing conventions or context before they can complete their work.
The DX research team is actively testing how Agent Experience can map to the CAFE(S) dimensions, validating the relationship between context quality and observed agent effectiveness before bringing that measurement into the product.
Coming soon, the Agent Effectiveness report will build on Agent Experience to give engineering and platform leaders a broader view of how teams work with coding agents across different types of work, models, tools, Skills, and MCPs.
Agent Experience is available to customers using AI Code Insights. Reach out to your DX Representative to set it up in your account.
Continuously improve how developers and agents work together
By understanding where agents struggle in real work, teams can identify opportunities for changes to their environment, apply those improvements consistently across the organization, and see whether they actually make agents more effective over time.
CAFE(S) gives teams a framework for understanding what high-quality context looks like. DX is bringing that research into practice, helping teams create better conditions for agents and developers to work effectively together.
Current DX customers can reach out to their CSM to get started with AI Code Insights and the AI Readiness Scorecard. If you’re new to DX, request a demo to see how DX can help your organization measure and improve agent effectiveness.