The new Response API from OpenAI combines the simplicity of chat completions with the tool usage capabilities of the Assistant API. This allows developers to build their own agents or use the API for their applications.

The Response API provides access to several default tools:

1. Web Search: This is the same web search capability as in ChatGPT, exposing a powerful search engine to developers. It outperforms other models on information retrieval benchmarks.

2. File Search: This enables Retrieval Augmented Generation (RAG) or "chat with your documents" functionality, similar to the Assistant API but with improvements.

3. Computer Use: This exposes a computer-using agent that can monitor a screen, determine actions, take them, and generate screenshots - all through the API. While the performance is not yet production-ready, it showcases the potential for building innovative applications.

The pricing for these new tools is as follows:

• Web Search: $30 per 1,000 queries for GPT-4, $25 per 1,000 queries for GPT-4 Mini
• File Search: $2.5 per 1,000 queries, plus storage costs of $0.10 per GB per day
• Computer Use: $3 per 1 million input tokens, $12 per 1 million output tokens

Overall, the new Response API provides developers with powerful tools to build advanced agents and applications, combining the simplicity of chat completions with the capabilities of external tools.

OpenAI has introduced three new default tools as part of their Response API: Web Search, File Search, and Computer Use.

Web Search:

• The Web Search tool allows developers to leverage OpenAI's specialized web search model, which outperforms other models on information retrieval tasks.
• It provides a simple interface to perform web searches directly through the API, without the need for an external search service.
• The web search capability is priced at $30 per 1,000 queries for the GPT-4 model and $25 per 1,000 queries for the GPT-4 Mini model.

File Search:

• The File Search tool enables a Retrieval Augmented Generation (RAG) pipeline, allowing agents to interact with a set of documents.
• It handles document chunking, embedding, and ranking, providing a turnkey solution for building RAG-based applications.
• The File Search API is priced at $2.5 per 1,000 queries, plus storage costs of $0.10 per GB per day (with the first GB free).

Computer Use:

• The Computer Use tool allows agents to capture and execute mouse and keyboard actions, enabling the automation of computer-based tasks.
• The performance of the current Computer Use agent is reported to be 38% on the OS-World benchmark, indicating that it is not yet highly reliable for automatic tasks.
• The pricing for the Computer Use tool is $3 per 1 million input tokens and $12 per 1 million output tokens.

These new default tools provide developers with powerful capabilities to build intelligent agents and applications that can interact with the web, documents, and even computer systems. While the pricing may be relatively expensive, the capabilities offered by these tools are unique and can enable the creation of innovative solutions.

The Agents SDK is a minimalistic implementation of an agentic framework, providing a lightweight yet powerful solution for building multi-agent workflows. It is built on top of the Swamp framework, which was released last year, and OpenAI has made several improvements to it.

The Agents SDK defines an agent as an LLM that has a given set of instructions and access to built-in tools. This framework enables intelligent handoffs, allowing for the seamless transfer of control between agents. For example, you can have a triage agent that orchestrates the workflow and smaller sub-agents that perform specific tasks.

The framework also includes configurable safety checks, known as guardrails, for input and output validation. This is crucial for preventing prompt injection or other undesirable effects. Additionally, the Agents SDK provides comprehensive tracing and observability features, allowing you to visualize agent execution traces to debug and optimize performance.

The implementation of the Agents SDK is straightforward. You can import the AgentsRunner and various tools, such as a support agent, a shopping assistant, and a triage agent. Each agent has access to a specific set of tools, making the selection and operation of agents more specialized and efficient.

The Agents SDK's simplicity and flexibility make it an attractive choice for developers looking to build multi-agent workflows. By leveraging the power of LLMs and providing a well-designed framework, the Agents SDK empowers developers to create innovative and robust applications.

The pricing details for the new tools announced by OpenAI are as follows:

Responses API:

• Standard token rates apply for just chatting with the API endpoint.
• Web search is priced at $30 per 1,000 queries for GPT-4 or GPT-4 mini.
• This is relatively expensive compared to Google's free grounding with Google search.

File Search API:

• Priced at $2.5 per 1,000 queries.
• Additional storage cost of 10 cents per GB per day, with the first GB free.
• This is relatively expensive compared to some vector store solutions, but OpenAI handles the custom RAG pipeline.

Computer Usage:

• Priced at $3 per 1 million input tokens.
• Output tokens are priced at $12 per 1 million output tokens.
• This is a relatively expensive option, but there are no other solutions with these capabilities at the moment.

Overall, the pricing for these new tools is on the higher side, but they provide powerful capabilities that may be worth the investment for certain use cases. Developers should carefully evaluate their requirements and the trade-offs between cost and functionality when deciding which tools to use.

The announcement of OpenAI's new tools and capabilities is an exciting development for developers. The Response API, which combines chat completion with tool usage, provides a powerful way to build agents and integrate them into various applications. The inclusion of web search, file search, and computer usage tools within the API offers developers a comprehensive set of capabilities to work with.

The introduction of the Agents SDK is particularly noteworthy, as it provides a lightweight yet powerful framework for building multi-agent workflows. The ability to orchestrate agent handoffs, implement configurable safety checks, and leverage the built-in tracing and observability features makes the Agents SDK a valuable tool for developers.

While the pricing for some of these new tools may be relatively expensive, the capabilities they offer are unique and can enable the creation of innovative applications. Developers should carefully evaluate their use cases and the potential benefits of these tools to determine if they are a good fit for their projects.

Overall, the announcements from OpenAI demonstrate their commitment to providing developers with the tools and resources they need to build advanced AI-powered applications. The combination of new APIs, tools, and the Agents SDK opens up exciting possibilities for developers to explore and push the boundaries of what's possible with AI.