Should you use OpenAI Assistant API with Bubble?

Headshot of Matt the Planet No Code Bubble Coach

Need 1 to 1 help?

Your no-code consultant & Bubble tutor.

Let's discuss whether to use the OpenAI Assistant API with applications. Exploring the key differences between the assistance API and chat completion, and the challenges and benefits of incorporating AI into your Bubble app.

Introduction to OpenAI Assistance API

I've received quite a lot of questions about the OpenAI assistance API, so I thought I'd record a quick video just to talk about what exactly it is and why it's different from chat completion and whether you should use it in your Bubble app or not. So here I am on the OpenAI documentation and yeah, I'm just going to compare and contrast the two different ways of getting AI into your Bubble app. We've done plenty of videos on the technicalities of it, but this is more about choosing the right method to get AI into your Bubble app. So with the assistance API, first thing to notice is that it is in beta, and if you've been watching the AI landscape, as I'm sure you have, you'll realize that things change really rapidly. So it is always a risk if you build something that is production ready and you're using beta software applications in it, because OpenAI could shut it down, they could change it.

Understanding Text Generation Chat Completion API

We just don't know what's going to happen. Second is just to realize how exactly they work. And so actually I'm going to go in and start us with the text generation chat completion API, because this is where the bulk of the AI and Bubble videos that I've recorded come from. It is using the chat completion API, and you send through an API request and you record all, you record, you submit through to OpenAI all of the messages that have been part of that conversation. And that's how the AI can be aware and contextually aware of the history of the conversation.

Challenges with Chat Completion API

So there are some downsides here, particularly in the early days when you were limited to how much data, how many tokens you could submit in your input. That's no longer such a big issue because we're into the tens of thousands of tokens. You can submit half a book, a whole book now in your prompt. So that's no longer an issue. But it does mean that it is your role as the app creator in Bubble to keep track of the conversation, because your conversation won't be able to have that historical awareness if you aren't able to submit the previous steps.

Advantages of Chat Completion API

Now this is by far the easiest way to add in chat completion AI text generated content into your Bubble app. Because when you say you submit the system prompt and you submit user, that's when your Bubble app waits for a response from the AI, and the AI will respond with this message here, and you can save that into a thread or a series of chat messages. Again, we've got loads of videos covering how to actually implement this, but it is that your Bubble app waits and then it gets a response. Like it's, it gets notified when the AI has completed writing the assistant part of the conversation. Now this is different to how the assistance API works, because the assistance API, you have to create an assistant.

Creating Assistants with Assistance API

Now the advantage here is that you can have your users create different assistants so you can almost, it's like putting your system prompt elsewhere and kind of wrapping it in the Persona of an assistant. And the benefits of that are that you can draw on that assistant any way that you like in the app once you've created it and apply that Persona to a conversation. The other advantage with the assistance API is that you can upload files and that can basically inform the knowledge base upon which the API draws upon. But we create an assistant, then we create a thread. So this is the main difference.

Tracking Conversations with Threads

It is no longer your role as the app creator or the Bubble app maker to keep track of the conversation. Threads are the way they kept track. The message history is held is maintained on the OpenAI server. So you basically add a message to the thread. So notice we're not sending a list of messages, we're just sending the most recent message.

Executing the Run Endpoint

And yeah, OpenAI is keeping track of what's going on. And then that doesn't do anything. You can literally have your user ask a question and then nothing happens. Because then you need to run. So you need to use the run endpoint to basically bring it all together.

Limitations of the Assistance API

So you run the thread with any assistant and then that, that's then when the AI adds the assistant message as the latest message to the thread. Now the main difference, and the main reason that I'm not using the assistance API to build AI apps of Bubble is because nothing happens at that point. You run, then nothing happens on your Bubble app. On the OpenAI servers they write the assistance prompt and then nothing happens. There's no notification back to your Bubble app.

Challenges with Continuous Checking in Bubble

And this is perhaps a weakness of Bubble, which is that you need to be continuously checking the thread to see if a new message has appeared. And this is quite tricky in Bubble because there isn't a great way, especially if you're trying to really conserve those workload units, there isn't a great way of checking in. You want to be checking every second because that matters when your user is waiting for a response from the AI. And if you are checking every second, then you're running a
every second to check that a new message has been created. And I've seen a number of YouTube Bubble educators do a few different approaches on this.

Non-Optimal Solutions for Real-Time Checking

And simply put, I don't think that there is an optimal way to do it. They all have downsides. So you can say check every minute, no, every second. You could check every second that a new message has been written into the thread, but then you can add in only when statements. So you could say only check if you put the chat into a state that the user has submitted the message and is waiting for a response.

Cost Impact on Workload Units

But then you're actually adding to the workload units that it's going to cost to run the operation because you're making the check the thread for new messages workflow more complicated and therefore it's going to cost more workloads units. So yeah, that's it. That's basically my assessment of the pros and cons. Like I say, it's not going to be an API that I'm using mainly because it's in beta. Hopefully they'll add in webhooks.

Future Improvements and Current Recommendations

That would give it the ability that when you submit a new message to a thread, you can say notify my Bubble app of a response here. And that would be perfect. But it's in beta. They say they're developing new features. The AI landscape is constantly changing.

Final Thoughts

So right now, if you wanted to add AI into your Bubble app, I would recommend using chat completion. Now the two main differences are that you can't upload files for chat completion to inform it of knowledge, or at least it's not quite as easy to do as with the assistant. But if you wanted to give a dynamic Persona like the assistant in chat completion, you could just make your system prompt dynamic. So instead of, you know, you could have you are a helpful assistant for one user and you could have you are a pirate for another user, and that's how you can inform the Persona of the chat. Anyway, I'd love to hear your thoughts and your opinions, your experience.

And if I've missed anything, if there is a really creative way of using the assistance API in Bubble that doesn't cost a ton of unnecessary workload units, please leave a comment down below and let me know.