flexfanatic 0 Posted June 7, 2015 Report Share Posted June 7, 2015 I'm currently working on a clickbank marketplace scraper and I have all of the information like hoplinks, description, etc stored in a database. I'm now trying to obtain the actually website url by navigating to the page and capturing the url but I'm having some issues with figuring out how to obtain this. Looking at the page source there are a few tags that tell me what the full url is but is there a way to capture this from the address bar and save it back to the database table? Quote Link to post Share on other sites
Code Docta (Nick C.) 638 Posted June 7, 2015 Report Share Posted June 7, 2015 $urlfunction gets the current url in the browser Quote Link to post Share on other sites
flexfanatic 0 Posted June 7, 2015 Author Report Share Posted June 7, 2015 $urlfunction gets the current url in the browser I love when its that simple. Works like a charm. Thanks Code Docta Quote Link to post Share on other sites
HelloInsomnia 1103 Posted June 8, 2015 Report Share Posted June 8, 2015 Just a heads up, you can get the entire marketplace via their feed that updates every day! Here is the page that talks a bit about it: https://support.clickbank.com/entries/22824126-Marketplace-Feed Or the direct link to the feed: https://accounts.clickbank.com/feeds/marketplace_feed_v2.xml.zip Quote Link to post Share on other sites
flexfanatic 0 Posted June 10, 2015 Author Report Share Posted June 10, 2015 Just a heads up, you can get the entire marketplace via their feed that updates every day! Here is the page that talks a bit about it: https://support.clickbank.com/entries/22824126-Marketplace-Feed Or the direct link to the feed: https://accounts.clickbank.com/feeds/marketplace_feed_v2.xml.zip Correct, I downloaded the marketplace and then imported it into a MySQL database. What I'm trying to accomplish is getting the actual website URL from the hoplink url. I'm able to get the results I want but now I'm running into a issue with updating the column (last line where I'm performing an update and I don't get a error but no update either). plugin command("DatabaseCommands.dll", "connect to database", "mysql:server=\'localhost\';uid=\'myusername\'; pwd=\'password\'; database=\'cboffers\'; port=\'3306\'; pooling=false") { plugin command("DatabaseCommands.dll", "query with results", "select hoplinkurl from hoplinks where url IS NULL limit 1", &hoplinksurl) clear cookies navigate("{$table cell(&hoplinksurl,0,0)}.","Wait") clear list(%results) add item to list(%results,$url,"Don\'t Delete","Global") plugin command("DatabaseCommands.dll", "query", "UPDATE hoplinks SET url=\'%results\' WHERE hoplinkurl=\'&hoplinkurl\'") Quote Link to post Share on other sites
flexfanatic 0 Posted June 10, 2015 Author Report Share Posted June 10, 2015 plugin command("DatabaseCommands.dll", "connect to database", "mysql:server=\'localhost\';uid=\'myusername\'; pwd=\'password\'; database=\'cboffers\'; port=\'3306\'; pooling=false") { plugin command("DatabaseCommands.dll", "query with results", "select hoplinkurl from hoplinks where url IS NULL limit 1", &hoplinksurl) clear cookies navigate("{$table cell(&hoplinksurl,0,0)}.","Wait") clear list(%results) add item to list(%results,$url,"Don\'t Delete","Global") plugin command("DatabaseCommands.dll", "query", "UPDATE hoplinks SET url=\'{%results}\' WHERE hoplinkurl=\'{$table cell(&hoplinksurl,0,0)}\'")} Quote Link to post Share on other sites
HelloInsomnia 1103 Posted June 10, 2015 Report Share Posted June 10, 2015 It looks like you are using the list %restults and the table &hoplinkurl when I think you should be using the list item and table cell you want, that may be the issue. I'm just looking at your code here and so it's kind of hard to say but that may be it off the top of my head. Quote Link to post Share on other sites
flexfanatic 0 Posted June 10, 2015 Author Report Share Posted June 10, 2015 It looks like you are using the list %restults and the table &hoplinkurl when I think you should be using the list item and table cell you want, that may be the issue. I'm just looking at your code here and so it's kind of hard to say but that may be it off the top of my head. Correct, I wrote the last line from the code view and forgot and messed up on the syntax. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.