BrowserBear is a fantastic web scraping tool with a really easy but versatile API. And in this Bubble tutorial video, I'm going to show you how you can add web scraping into your Bubble app. And in particular, we're going to focus on how to take a screenshot. And we'll be taking a screenshot of the New York Times homepage and showing how we can get that data back into Bubble.

Discover our Bubble tutorials

But before I launch into that, if you're learning Bubble, well, you probably are because you're watching this video. And we've got hundreds of Bubble tutorial videos just like this one, many of which can only be accessed by our members on our website planetnocode.com.

Using Browserbear

But let's dive into BrowserBear. And so I've signed up in BrowserBear and in fact, I've modified one of their example tasks. And so I'm saying, let's take a screenshot of the New York Times. And so if I go in and examine the steps here, I've got Go To, the homepage of the New York Times. And then I simply say, take a screenshot. And I do that by adding in a step. And just to give you an idea, this is really how versatile BrowserBear is because there's all of these steps that you can add into a task. And then all you need to do, and I'll be demonstrating that in just a few seconds, all you need to do is run this task from your Bubble app. And it will run the task and return the data to your Bubble app. And yeah, previous video I've done on this demonstrating that AI features really cool way to set up quick web scraping without having to deconstruct the page. Anyway, we're focusing on screenshots in this tutorial. So I'm going to go back. Go back. And right, how do I run this task in Bubble?

Bubble.io API Connector & Browserbear

So I'm in my Bubble API connector and I've set up BrowserBear. I've added in all of the essential fields up here, including authorization, bear and my API key, my content type. If you're unsure where I've got this data, I've taken out of the BrowserBear API documentation. I'm on runs, create run, and then I've taken out the essentials here. And all I really need to send through to run the task is to send through a webhook because that's where I want to receive the data when BrowserBear has completed its operation. And so if I go into a separate tab in Bubble, I've set up an API workflow as a back end workflow called BrowserBear. And if I click detect data, I get this URL. And so going back into my API connector, I'm adding the parameter of webhook URL and I'm pasting my endpoint for my Bubble API back end workflow into there because that's where I'm going to receive the data when it's taken the screenshot. Just change that to action. I'll be demoing all of this within the API connector. Do check out our other videos such as our OpenAI tutorials where you'll see about how we can save data from an API call. But this is just the technicality of using the BrowserBear API. So the other thing I need to update is my task ID because this is a different task from a previous video. Where do I find my task ID? Well, I can just click up here and click copy and then go back to Bubble. Paste in my task ID and I can reinitialize the call.

Testing

Okay, I get this instant response back. Basically, because the process of taking a screenshot can take a little bit of time, that's why I have to use a webhook. I don't get the data right the way back in the return values for my API call. I can go into BubbleBear and I can go into logs. Here it is. So it's still running and I'm going to go back here because I'm waiting for the data to come in. Because by putting it into the request data mode or initialize, Bubble is ready to learn the structure of the content as it returns. So let's just wait here a few more seconds. Our screenshot should arrive. Okay, so we've actually got an error. Now I should have tested this beforehand. BrowserBear is not able to connect to the New York Times.

Wrap up

Let's see, it's recommending I change the setting. Let's try that one. And I'll run the API again. Go into my logs. This is my most recent call. Let's see what happens here. Okay, it's now saying finished. So I'm going to go back here and this is the data that's been sent in. I'm looking for... here we go, the ID of the screenshot. And so I'm going to just approve it. I'm going to take this, copy it, open in a new tab. And here is the New York Times. Now what it's done obviously is it's scrolled to get the full page and I've got this terms or other websites might have a cookie notice. So I'm going to stop this video here and I'm going to have a go at using some of the additional tools in BrowserBear to try and get rid of that cookie notice or terms notice. And I'm going to record a follow-up video to this right away. You