JackCrow 4 Posted January 18 Report Share Posted January 18 We have been running a google search bot without isue for months. Now all of a sudden, we get catchas everytime. Doesn't matter the proxie. Or even without a proxy. Any ideas? Quote Link to post Share on other sites
SourceUltra 10 Posted January 28 Report Share Posted January 28 Google recently made an update that affected many search data providers (giant ones even) ability to scrape google. Some believe that it is a move to save company resources as they are spending so much on their AI Gemini, and bots likely account for a HUGE amount of traffic expenditure for them. Some search data providers are still able to collect data, so it seems to be possible with the right tools. It's likely google has a significantly larger fingerprint database and can more easily detect web bots behaving a certain way or using selenium or other popular browser automation engines. I don't scrape google, so I'm not sure if ubot has a built-in solution that works. I also don't have exbrowser but I believe it is selenium based and probably will be detected --but maybe someone will chime in and let us know if it works. I know multiloginapp is a powerful anti-fingerprinting browser engine that last I checked has an API that you can work with in ubot. This is probably a more robust solution. The other solutions I can think of involve using Ubot's 'Run Process' command to launch external solutions/scripts/tools and passing that data back into Ubot with the 'On Output' command, or simply reading the data from a file. With this way you can at least continue using the parts of your bot that parse and manage the data you need and save some time from having to build it all in an entirely new tool. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.