a-harvey 0 Posted February 23, 2018 Report Share Posted February 23, 2018 Hi, I have been playing about with a clients ebay listings trying to get the following but seem to not be doing to well. https://www.ebay.com/sch/m.html?item=350996870504&_ssn=archivesonline&_ipg=400&rt=nc I have set their account above at 400 deep listings as they will not have more than that running at any one time. I am trying to do the following: scrape each listing:1) Title2) Listing url3) The main image or images if multiple once each is clicked into. Having issues, when scraping the above page, I can get the urls, but the title using the class pulls in every title on the page and not as seperate in a list so I can match the url to them, also how would one get all the images from the popup. Reason, I am looking to try and use the data to then auto build a catalogue for them via database merge in indesign that will then become a digital version with clickable links to the ebay listings. Any thoughts on how best to tackle this. Many thanksAndrew Quote Link to post Share on other sites
LoWrIdErTJ - BotGuru 904 Posted February 23, 2018 Report Share Posted February 23, 2018 use offsets so your not pulling all.you can tell it offset position 0 for the first one in the page. 1 Quote Link to post Share on other sites
a-harvey 0 Posted February 23, 2018 Author Report Share Posted February 23, 2018 ok will try with offset, always thought we shouldn't use them but maybe wrong Quote Link to post Share on other sites
HelloInsomnia 1103 Posted February 23, 2018 Report Share Posted February 23, 2018 Try this, you will need this free plugin: https://www.bot-factory.com/free-ubotstudio-xpath-plugin/ clear list(%containers) add list to list(%containers,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", $document text, "//ul[@id=\'ListViewInner\']/li", "innerhtml", ""),"Delete","Global") clear table(&data) set(#row,0,"Global") set list position(%containers,0) loop($list total(%containers)) { set(#container,$next list item(%containers),"Global") set table cell(&data,#row,0,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//h3[@class=\'lvtitle\']", "innertext", "")) set table cell(&data,#row,1,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//h3[@class=\'lvtitle\']/a", "href", "")) set(#img,$nothing,"Global") set(#img,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//img[@class=\'img\']", "imgurl", ""),"Global") if($comparison(#img,"= Equals",$nothing)) { then { set(#img,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//img[@class=\'img\']", "src", ""),"Global") } else { } } set table cell(&data,#row,2,#img) increment(#row) } 2 Quote Link to post Share on other sites
a-harvey 0 Posted February 23, 2018 Author Report Share Posted February 23, 2018 Ok so finally got the plugin and added it and the code above with the clients ebay listing url at the top,It runs but throws an errorThe given kew was not present in the dictionary. Source: untitled bot -> Untitled Script -> run command ? don't know the issue, my Ubot is fairly limited to scrapping Auction Houses urls for their auctions. Quote Link to post Share on other sites
a-harvey 0 Posted February 23, 2018 Author Report Share Posted February 23, 2018 (edited) I think its almost there which is really great Ok so if I ignore the error it does seem to pull all the data into table &data. I presume I just need to save &data to csv and I will get what I need.Where or how do I download the image files ? I can see its pulling the urls Edited February 23, 2018 by a-harvey Quote Link to post Share on other sites
a-harvey 0 Posted April 25, 2018 Author Report Share Posted April 25, 2018 Hi HelloInsomnia, or someone. I have used the code as above, when I scrape the data, in ubot it shows 3 colums (debugger) but when it save it to the csv it puts everything into one ?????? Any reason its doing that ? also if you run it, I need to then move to the next page, the issue is the > arrow always stays so if you use the click function it will move through but then locks in a cycle ? Sorry any help would be greatly appreciated. Andrewnavigate("https://www.ebay.com/sch/m.html?item=350996870504&_ssn=archivesonline&_ipg=200&rt=nc","Wait")wait(4)clear list(%containers)add list to list(%containers,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", $document text, "//ul[@id=\'ListViewInner\']/li", "innerhtml", ""),"Delete","Global")clear table(&data)set(#row,0,"Global")set list position(%containers,0)loop($list total(%containers)) { set(#container,$next list item(%containers),"Global") set table cell(&data,#row,0,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//h3[@class=\'lvtitle\']", "innertext", "")) set table cell(&data,#row,1,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//h3[@class=\'lvtitle\']/a", "href", "")) set(#img,$nothing,"Global") set(#img,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//img[@class=\'img\']", "imgurl", ""),"Global") if($comparison(#img,"= Equals",$nothing)) { then { set(#img,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//img[@class=\'img\']", "src", ""),"Global") } else { } } set table cell(&data,#row,2,#img) increment(#row)}save to file("C:\\Users\\Andrew\\Desktop\\archivesebay.csv",&data) Quote Link to post Share on other sites
a-harvey 0 Posted April 25, 2018 Author Report Share Posted April 25, 2018 (edited) This is what it pulls, all into row 0 in the csvOnly on this its not showing the url that's pulled first, then a , and the image url that it is showing.so item ulr and image url are on same line. ABNC Archives c.1860-70s Jose Galves, Peru Banknotes Intaglio PROOF Vignette Unc ,https://www.ebay.com/itm/ABNC-Archives-c-1860-70s-Jose-Galves-Peru-Banknotes-Intaglio-PROOF-Vignette-Unc/391986008221?hash=item5b442fd09d:g:KFkAAOSwn9VajbbX,https://i.ebayimg.com/thumbs/images/g/KFkAAOSwn9VajbbX/s-l225.jpg Franklin Banknote Co. Proof Intaglio Vignette Michigan State Arms c.1880 XF+ ,https://www.ebay.com/itm/Franklin-Banknote-Co-Proof-Intaglio-Vignette-Michigan-State-Arms-c-1880-XF/352283628916?hash=item5205bd9574:g:jjcAAOSw5M9ah2lr,https://i.ebayimg.com/thumbs/images/g/jjcAAOSw5M9ah2lr/s-l225.jpg Norway Norges Bank 10+50 Kroner 1975-1978 P-36 P-37 (3 pcs, Unc) ,https://www.ebay.com/itm/Norway-Norges-Bank-10-50-Kroner-1975-1978-P-36-P-37-3-pcs-Unc/391982872887?hash=item5b43fff937:g:FKcAAOSw3wVah2ox,https://i.ebayimg.com/thumbs/images/g/FKcAAOSw3wVah2ox/s-l225.jpg NY. International Banknote Co SPECIMEN Stock Certificate ABNC ,https://www.ebay.com/itm/NY-International-Banknote-Co-SPECIMEN-Stock-Certificate-ABNC/391985596914?hash=item5b442989f2:g:UjMAAOSwPtpag7Fv,https://i.ebayimg.com/thumbs/images/g/UjMAAOSwPtpag7Fv/s-l225.jpg DuraNote Polymer Ad Test Note 1980-90s Oil Rig White & Silver Maple leaf Gem UNC ,https://www.ebay.com/itm/DuraNote-Polymer-Ad-Test-Note-1980-90s-Oil-Rig-White-Silver-Maple-leaf-Gem-UNC/352283628083?hash=item5205bd9233:g:JvkAAOSwBmlah2j0,https://i.ebayimg.com/thumbs/images/g/JvkAAOSwBmlah2j0/s-l225.jpg Edited April 25, 2018 by a-harvey Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.