Web scraping with Page2API Premium Proxy

In this Bubble.io tutorial video we demonstrate how to use the Page2API API to web scrape content from a website by pretending the visitor is from a different country using Page2API's Premium Proxy.

Still stuck?

Schedule a free 1 to 1 Bubble coaching Zoom call.
Learn more
Bubble coaching learn no-code

Welcome to part two of a mini series showing how to get a user's country code in Bubble, save it to the database that's in the previous video. And now we're going to use the country code to take advantage of the Premium Proxy with Page2API.

Advantages of using Page2API Premium Proxy

This will allow us to scrape data from a website. Take, for example, Amazon. No matter which Amazon domain, like if it's Amazon.com or Amazon.co.uk, Amazon detects the location of the user as they visit the page and will adjust the information on it according to the location that they're judging the users from based on their IP address.

But what we want to do is ensure that, say, a user is in Germany and they present us with an Amazon URL that they get back the data from the product page that Amazon presents to German users.

So let's get into it. We've got a simple page here that saves the country code to the Current User. And we need to make adjustments to our API connector with Page2API. Now, if this all looks alien to you, check out an earlier video where I go through every step that's needed in creating this API call for Page2API, because right now I'm going to be just referring to the documentation.

So in order to make an API call with the Premium Poxy faking that the visitor, the web scraper is from another country, I need to add this line into my JSON here, making sure that all the JSON syntax is correct, like a comment afterwards. And then I change this into a dynamic value and in here is where if I untick private, it lets me put it into a workflow. And so I could put like 'de' in here and I don't know what's going to happen bbc.co.uk if you visit and the BBC website thinks that you're in Germany, let's have a look. Oh, okay. There we go.

Premium Proxy is only available for paid accounts. Now, I'm creating this video because I literally just got off a call with one of our Planet No Code members and they were facing this exact conundrum and they were trying to scrape data from Amazon. But what I just show you is how to set this up because I know that all of this is correct. I'll show you how to set this up for the workflow. So if I go back onto my page here and then let's put in an input.

Okay, this is going to be my URL. And then instead of this saving the country code, which I demonstrated in the previous video, I'm going to use it to scrape the data.

Okay, so go into plugin and scrape and for some reason, it's not showing up the country code. Oh, private. There we go. I just had to re-add it behind the scenes simply because I'm not going to upgrade to a paid account right now.

Anyway, what you're seeing on screen now is going to work. There we go, country code. And so now I go Current User country code. Okay? And when I run that workflow by the click of the button, it's going to dynamically fill in the URL and the country code.

And Page2API is going to go to the URL I provide under the aegis of coming from the country code that I provide and then it's going to return the values that I've requested that it scrapes.

Troubleshooting

So there's one thing to highlight where you're likely to get this wrong, get frustrated. In fact, two things is you need a paid account in order to do the premium proxy, I hadn't checked that out ahead of time. And then also make sure that your country codes are coming back in this format, which is lower case.

I tried uppercase with the client sorry, with our Planet No Code member, because that's the way that the plugin returns to country code. But it needs to be lowercase because that is what Page2API is expecting.

 

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
camera-videouserscodelinkmenu