qualityeye 1 Posted July 27, 2017 Report Share Posted July 27, 2017 I'm not sure what I'm missing but when the bot I'm working on saves to CSV the items go into all different rows. What I'm looking to do is have it in 3 columns. So title would be in column 1, description in column 2, picture in column 3. Then each row would be what was scraped off of the page. I have the scraping working, I just can't figure out how to make everything in a neat output. I also played around with tables but I got the same result. I probably messed it up since I never used the tables option before and deleted it. I've attached a picture of what my setup is now. What am I doing wrong here? I appreciate any help I can get. Thanks. 1 Quote Link to post Share on other sites
qualityeye 1 Posted July 27, 2017 Author Report Share Posted July 27, 2017 I added tables. The output still doesn't look good. I'm not sure what I'm doing wrong here. Quote Link to post Share on other sites
HelloInsomnia 1103 Posted July 27, 2017 Report Share Posted July 27, 2017 When you want to scrape several items per ad (or whatever it may be) what you want to do it scrape each item individually and insert it into a table cell. I've seen people try to scrape lists of things and think they can just combine them at the end but you will often find this leads to problems because sometimes ads may not have a description or a picture or perhaps it has more than one and it messes up the offset when you combine them at the end and so it gives you invalid data. So what you want to do is to scrape each little item individually and insert it into a table cell, here is an example of what I'm talking about: navigate("https://houston.craigslist.org/search/rva","Wait") clear list(%adContainers) add list to list(%adContainers,$scrape attribute(<class="result-row">,"outerhtml"),"Don\'t Delete","Global") set(#row,0,"Global") clear table(&ads) loop($list total(%adContainers)) { set(#ad,$next list item(%adContainers),"Global") set(#title,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #ad, "//a[@class=\'result-title hdrlnk\']", "innertext", ""),"Global") set table cell(&ads,#row,0,#title) set(#price,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #ad, "//span[@class=\'result-price\']", "innertext", ""),"Global") set table cell(&ads,#row,1,#price) set(#location,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #ad, "//span[@class=\'result-hood\']", "innertext", ""),"Global") set table cell(&ads,#row,2,#location) increment(#row) } 1 Quote Link to post Share on other sites
Dallashox 2 Posted July 28, 2017 Report Share Posted July 28, 2017 Legend. Been wondering that question as well but have been trying stuff in Selenium. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.