Changelog

Three quality-of-life improvements to the Playground: You can now see provider costs per million tokens directly in the model selection dropdown. You can run evaluations directly from the Playground without navigating to the evaluation menu. And you can collapse test cases to navigate large test sets more easily.

You can now track multi-turn conversations with chat sessions. All traces with the same session ID are automatically grouped together, letting you analyze complete conversations instead of individual requests.

The new session browser shows key metrics like total cost, latency, and token usage per conversation. Open any session to see all traces with their parent-child relationships. This makes debugging chatbots and AI assistants much easier. Add session tracking with one line of code using either our Python SDK or OpenTelemetry.

Minor improvements:

• Added time filtering to the analytics dashboard. You can now view metrics for the last 6 hours, 24 hours, 7 days, or 30 days.
• Added the ability to batch delete multiple traces at once. Select traces using checkboxes and delete them in a single operation.

The new JSON Multi-Field Match evaluator validates multiple fields between JSON objects. Configure any number of field paths using dot notation, JSON Path, or JSON Pointer formats. Each field gets its own score (0 or 1), and an aggregate score shows the percentage of matching fields. This evaluator is ideal for entity extraction tasks like validating extracted names, emails, and addresses. The UI automatically detects fields from your test data for quick setup. This replaces the old JSON Field Match evaluator, which only supported single fields.

The Playground now supports PDF attachments for chat applications. You can attach PDFs by uploading files, providing URLs, or using file IDs from provider APIs. This works with vision-capable models and extends to evaluations and observability. You can now build and test document processing applications like invoice analysis or contract review.

AI coding agents like Cursor, Claude Code, and VS Code Copilot can now access Agenta documentation directly through the Agenta MCP server. Connect your IDE to get instant answers about Agenta features, APIs, and code examples without leaving your editor. The server supports multiple clients and requires no authentication.

You can now create projects within an organization. This lets you divide your work between different AI products. Each project scopes its prompts, traces, and evaluations. Create a new project or navigate between projects directly from the sidebar.

You can now use provider built-in tools in the Playground. Add web search, code execution, file search, and Bash scripting tools directly to your prompts. Supported providers include OpenAI, Anthropic, and Gemini. Tools are saved with your prompt configuration and automatically used when you invoke prompts through the LLM gateway.

You can now configure reasoning effort for models that support this parameter, such as OpenAI's o1 series and Google's Gemini 2.5 Pro. The reasoning effort setting is part of your prompt template, making it available when you fetch prompts via the SDK or invoke them through Agenta as an LLM gateway.

You can now use Jinja2 templates in your prompts. Jinja2 is available in both the Playground and in prompt management.

Learn more in our blog post or check the documentation.

We're open sourcing the core of Agenta under the MIT license. All functional features are now available to the community. This includes the evaluation system, prompt playground and management, observability, and all core workflows.

Development moves back to the public repository. We're building in public again. Only enterprise collaboration features like SSO and audit logs remain under a separate license.

Get started with the self-hosting guide. View the code and contribute on GitHub. Read why we made this decision at agenta.ai/blog/commercial-open-source-is-hard-our-journey.