Everything you need to know about OpenAI Assistants API and GPTs

Some major announcements were made at the OpenAI Dev Day event in San Francisco, including the introduction of the OpenAI Assistants API, which allows developers to build their own AI assistants within their own applications. These assistants can leverage models, tools, and knowledge to respond to user queries. The API currently supports three types of tools: code interpreter, retrieval, and function calling. In this article, we'll go deeper on these features and potential use cases for the OpenAI Assistants API.

What's the flow of the OpenAI Assistants API?

To understand how the Assistant API works, let's take a look at the typical integration flow:

1. Create an assistant: Developers can create an assistant in the API by defining custom instructions and selecting a model. They can also enable tools like code interpreter, retrieval, and function calling to enhance the assistant's capabilities.
2. Create a thread: When a user starts a conversation, a thread is created to capture the interaction.
3. Add messages to the thread: As the user asks questions or provides input, developers can add messages to the thread to simulate a conversation in order to collect more relevant inputs.
4. Run the assistant on the thread: To trigger responses from the assistant, developers can run the assistant on the thread. This automatically calls the relevant tools and knowledge and generates a response.

The OpenAI Assistants API provides a powerful platform for developers to create AI assistants that can engage in dynamic conversations with users.

‍

What are the limitations of the OpenAI Assistants API?

While the Open AI Assistants API offers a wide range of capabilities, it does have some limitations. Here are a few important points to note:

1. File uploads: The API allows a maximum of 20 file uploads per assistant, with each file limited to 512 megabytes. The organization level has a maximum limit of 100 GB.
2. Function calling: Function calls have a maximum wait time of 10 minutes for execution.
3. Streaming output: The API does not support streaming output for image generation.
4. Image analysis: Image analysis is not supported directly in the API. Developers need to call the Vision API separately for image-related tasks.
5. Retrieval capabilities: The API does not extend retrieval capabilities to XLS or CSV files.

Despite these limitations, the OpenAI Assistants API offers a powerful set of features that can be leveraged to create innovative and interactive AI assistants.

‍

What are some examples of OpenAI Assistants API in action?

The OpenAI Assistants API opens up a world of possibilities for content creators, companies, and individuals who want to provide interactive and engaging experiences. Here are a few examples that were shared:

1. Conversational AI assistants: Developers can create AI assistants that can have conversations with users. These assistants can be customized to provide information, answer questions, or even entertain users. For example, a chatbot could be built to answer questions about a specific topic, such as global warming, by using retrieval data and custom functions.
2. AI chatbots for content creators: Content creators can leverage the OpenAI Assistance API to create AI chatbots that can interact with their audience. For instance, a blogger could use the API to build a chatbot that answers questions about their blog posts or provides additional information based on the content.
3. Customized GPTs: One of the most exciting features of the OpenAI Assistance API is the ability to create customized versions of the chat GPT model. These custom GPTs can be tailored for specific purposes by providing instructions, expanded knowledge, and actions. They can be more helpful and work better in various contexts, giving users better control over the AI assistant.

What the difference between ChatGPT and custom GPTs?

One of the most game-changing aspects of the OpenAI Assistants API is the introduction of custom GPTs. These are tailored versions of the ChatGPT model that can be created for specific purposes. CustomGPTs combine instructions, expanded knowledge, and actions to provide more helpful and context-specific responses than what ChatGPT is able to provide.

The big differences are:

1. Instructions: ChatGPT can only follow the prompts that you type in. But a custom GPT comes with baked in prompts for how it should respond. So the results are more accurate and consistent.
2. Knowledge: ChatGPT can only answer based on what it’s model already knows. But a custom GPT can be given custom information like text, websites, or images and deliver it’s output from there.
3. Actions: ChatGPT gives you words and code. But a custom GPT can take actions on your behalf. For example, theres a GPT that create a fully functioning website based on a simple text description.

With custom GPTs, anyone can create their own AI assistant with natural language. You don't have to know how to code. Which means anyone and everyone will be introducing these in the coming weeks and months and it's going to get NOISY. There are already hundreds of custom GPTs circulating on X and Reddit.

What are some examples of a custom GPT?

Several examples of custom GPTs were showcased during the OpenAI DevDay event. Here are a few notable ones:

1. Startup Mentor: Sam Altman built a GPT called "Startup Mentor" that uses his previous speeches from his time leading Y Combinator to provide advice to startup founders in a chatbot format. By uploading the transcripts of his speeches and configuring the capabilities of the GPT, Sam was able to create a personalized assistant that can give founders advice automatically. Others like John Cutler have already created their own versions of what I'm calling "advice bots" based on prior writing or content. And it takes about 3 minutes to set up.
2. Trend Analyzer: Professor Ethan Mollik demonstrated a custom GPT called "Trend Analyzer" that looks up the latest trends for a specific product category on the web and creates prototype images based on those trends. This tool can help designers and product developers stay updated with the latest trends and incorporate them into their presentations quickly.
3. GIF PT: Nick Dobos created a custom GPT called "GIF PT" that automatically turns static images into GIFs. By leveraging the code interpreter and DALL·E image generation capabilities, Nick was able to package his workflow into a GPT that simplifies the process of creating GIFs.

These examples highlight the potential of custom GPTs. The ability to package complex workflows and prompts into a single GPT makes it easier for users to leverage AI capabilities without the need for extensive coding knowledge or prompt engineering.

Are there risks to custom GPTs?

There are tons of concerns and open questions that have yet to play out. Here are a few of the top questions that OpenAI will need to figure out.

1. Creator monetization: No plans on how people will cash in on custom GPTs. But without a clear monetization model, there's not a big incentive for creators and developers to continue building out the "App Store for AI".
2. Misinformation and Disinformation: The ability to create custom GPT models could be used to spread misinformation and disinformation more easily. Malicious actors could use these models to generate fake news, social media posts, and other forms of content designed to deceive or mislead people.
3. Bias and Discrimination: Custom GPT models could reflect the biases of their creators, leading to the generation of biased or discriminatory content. This could perpetuate harmful stereotypes and unfairly disadvantage marginalized groups.
4. Privacy and Security: Custom GPT models may require access to personal data in order to train and operate. This raises privacy concerns about the collection, use, and storage of sensitive information. Additionally, these models could be used to create phishing attacks, malware, and other security threats.
5. Lack of Oversight: OpenAI has stated that it will review custom GPT models before they are made available on the marketplace, but it is unclear how rigorous this review process will be. There is also a risk that models with harmful or malicious content could slip through the cracks.
6. Ease of Access: The GPT marketplace could make it easier for people to create and distribute custom GPT models, including those with harmful or malicious intent. This could make it more difficult for OpenAI and other organizations to identify and remove these models.

OpenAI has stated that it is committed to mitigating the risks associated with the GPT marketplace. The company has implemented a number of safety measures, such as requiring developers to verify their identity and providing users with the ability to report harmful or malicious content. However, it is important to be aware of these risks and to use custom GPT models with caution.

‍