juno 1 Posted November 29, 2018 Report Share Posted November 29, 2018 I've been scraping a site for probably 2 years now and I just noticed today the URL changed to https. Is there something they might be doing to keep my bot from navigating to that url? When I call it the built in browser stays on the default Ubot page. It refuses to load the html. This is happening in uncompiled mode. I'm running the latest version 6.04 Developer Edition Quote Link to post Share on other sites
MasteRofBotS 10 Posted November 29, 2018 Report Share Posted November 29, 2018 Did you try to use different browsers? Ubot 6 has them built in so give it a try (in Options you can change that) and see if that helps. Quote Link to post Share on other sites
juno 1 Posted November 29, 2018 Author Report Share Posted November 29, 2018 I did try that. The options were Chrome21 and Chrome67 I believe but nothing happened other than the address bar twitching side to side every time I tried rerunning that part of the script or hitting the refresh icon. Quote Link to post Share on other sites
Code Docta (Nick C.) 638 Posted November 30, 2018 Report Share Posted November 30, 2018 It is hard to say without the site. In general for SEO and security purposes site owners have chosen HTTPS over HTTP. Google won't rank a site without a SSL certificate. All it is is encrypting data that you send/receive to/from the site. https://www.instantssl.com/ssl-certificate-products/https.html Regards,CD Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.