UBotDev 276 Posted August 28, 2010 Report Share Posted August 28, 2010 TinyURL.com created a api, so we dont scrape their main page. The API can be found here and its works so, thatyou replace "YOURSITE". http://tinyurl.com/api-create.php?url="YOURSITE"The problem is that when I visit that site with UBOT (Navigate), it tries to download api-create.php file, so it is not possible to scrape that site. Any idea how to acomplish this? Quote Link to post Share on other sites
MiriamMB 63 Posted August 28, 2010 Report Share Posted August 28, 2010 TinyURL.com created a api, so we dont scrape their main page. The API can be found here and its works so, thatyou replace "YOURSITE". http://tinyurl.com/api-create.php?url="YOURSITE"The problem is that when I visit that site with UBOT (Navigate), it tries to download api-create.php file, so it is not possible to scrape that site. Any idea how to acomplish this? hmmm..have you tried just going to the website and putting in your urls one by one (from a list) and scraping the result into a list? Otherwise, with the method you are using, you would have to save them all in little files one by one..which seems tedious Quote Link to post Share on other sites
UBotDev 276 Posted August 29, 2010 Author Report Share Posted August 29, 2010 hmmm..have you tried just going to the website and putting in your urls one by one (from a list) and scraping the result into a list? Otherwise, with the method you are using, you would have to save them all in little files one by one..which seems tediousThats strange, because if you visit api link in explorer, the page doesnt download the file, instead it displays it as the only text on site.I have tried by scraping tinyurl.com, but I have problems bi choosing the link, it always scrapes <A href="http://tinyurl.com/">Home</A> instead of right url. :/ Quote Link to post Share on other sites
MiriamMB 63 Posted August 29, 2010 Report Share Posted August 29, 2010 Thats strange, because if you visit api link in explorer, the page doesnt download the file, instead it displays it as the only text on site.I have tried by scraping tinyurl.com, but I have problems bi choosing the link, it always scrapes <A href="http://tinyurl.com/">Home</A> instead of right url. :/ Have you tried choosing the link that says "Open in new window"? That seems like it would scrape your tinyurl link for you. When you add it to a list, just set Duplicates to "NO" so that if it scrapes it twice, you only have one copy of the link. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.