Scrape A Clients Ebay Listings

[a-harvey 0]

Hi,

 

I have been playing about with a clients ebay listings trying to get the following but seem to not be doing to well.

 

https://www.ebay.com/sch/m.html?item=350996870504&_ssn=archivesonline&_ipg=400&rt=nc

 

I have set their account above at 400 deep listings as they will not have more than that running at any one time.

 

I am trying to do the following:

 

scrape each listing:

1) Title

2) Listing url

3) The main image or images if multiple once each is clicked into.

 

Having issues, when scraping the above page, I can get the urls, but the title using the class pulls in every title on the page and not as seperate in a list so I can match the url to them, also how would one get all the images from the popup.

 

Reason, I am looking to try and use the data to then auto build a catalogue for them via database merge in indesign  that will then become a digital version with clickable links to the ebay listings.

 

Any thoughts on how best to tackle this.

 

Many thanks

Andrew

[LoWrIdErTJ - BotGuru 904]

use offsets so your not pulling all.

you can tell it offset position 0 for the first one in the page.

[a-harvey 0]

ok will try with offset, always thought we shouldn't use them but maybe wrong

[HelloInsomnia 1103]

Try this, you will need this free plugin: https://www.bot-factory.com/free-ubotstudio-xpath-plugin/

clear list(%containers)
add list to list(%containers,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", $document text, "//ul[@id=\'ListViewInner\']/li", "innerhtml", ""),"Delete","Global")
clear table(&data)
set(#row,0,"Global")
set list position(%containers,0)
loop($list total(%containers)) {
    set(#container,$next list item(%containers),"Global")
    set table cell(&data,#row,0,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//h3[@class=\'lvtitle\']", "innertext", ""))
    set table cell(&data,#row,1,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//h3[@class=\'lvtitle\']/a", "href", ""))
    set(#img,$nothing,"Global")
    set(#img,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//img[@class=\'img\']", "imgurl", ""),"Global")
    if($comparison(#img,"= Equals",$nothing)) {
        then {
            set(#img,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//img[@class=\'img\']", "src", ""),"Global")
        }
        else {
        }
    }
    set table cell(&data,#row,2,#img)
    increment(#row)
}

[a-harvey 0]

Ok so finally got the plugin and added it and the code above with the clients ebay listing url at the top,

It runs but throws an error

The given kew was not present in the dictionary.

 

Source: untitled bot -> Untitled Script -> run command

 

? don't know the issue, my Ubot is fairly limited to scrapping Auction Houses urls for their auctions.

[a-harvey 0]

I think its almost there which is really great

 

Ok so if I ignore the error it does seem to pull all the data into table &data.

 

I presume I just need to save &data to csv and I will get what I need.

Where or how do I download the image files ? I can see its pulling the urls

Edited February 23, 2018 by a-harvey

[a-harvey 0]

Hi HelloInsomnia, or someone.

 

I have used the code as above, when I scrape the data, in ubot it shows 3 colums (debugger) but when it save it to the csv it puts everything into one ??????

 

Any reason its doing that ? also if you run it, I need to then  move to the next page, the issue is the > arrow always stays so if you use the click function it will move through but then locks in a cycle ?

 

Sorry any help would be greatly appreciated.

 

Andrew

navigate("https://www.ebay.com/sch/m.html?item=350996870504&_ssn=archivesonline&_ipg=200&rt=nc","Wait")
wait(4)
clear list(%containers)
add list to list(%containers,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", $document text, "//ul[@id=\'ListViewInner\']/li", "innerhtml", ""),"Delete","Global")
clear table(&data)
set(#row,0,"Global")
set list position(%containers,0)
loop($list total(%containers)) {
    set(#container,$next list item(%containers),"Global")
    set table cell(&data,#row,0,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//h3[@class=\'lvtitle\']", "innertext", ""))
    set table cell(&data,#row,1,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//h3[@class=\'lvtitle\']/a", "href", ""))
    set(#img,$nothing,"Global")
    set(#img,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//img[@class=\'img\']", "imgurl", ""),"Global")
    if($comparison(#img,"= Equals",$nothing)) {
        then {
            set(#img,$plugin function("XpathPlugin.dll", "$Generic Xpath Parser", #container, "//img[@class=\'img\']", "src", ""),"Global")
        }
        else {
        }
    }
    set table cell(&data,#row,2,#img)
    increment(#row)
}
save to file("C:\\Users\\Andrew\\Desktop\\archivesebay.csv",&data)
 

[a-harvey 0]

This is what it pulls, all into row 0 in the csv

Only on this its not showing the url that's pulled first, then a , and the image url that it is showing.

so item ulr and image url are on same line.

 

 

ABNC Archives c.1860-70s Jose Galves, Peru Banknotes Intaglio PROOF Vignette Unc        
,https://www.ebay.com/itm/ABNC-Archives-c-1860-70s-Jose-Galves-Peru-Banknotes-Intaglio-PROOF-Vignette-Unc/391986008221?hash=item5b442fd09d:g:KFkAAOSwn9VajbbX,https://i.ebayimg.com/thumbs/images/g/KFkAAOSwn9VajbbX/s-l225.jpg       
Franklin Banknote Co. Proof Intaglio Vignette Michigan State Arms c.1880 XF+        
,https://www.ebay.com/itm/Franklin-Banknote-Co-Proof-Intaglio-Vignette-Michigan-State-Arms-c-1880-XF/352283628916?hash=item5205bd9574:g:jjcAAOSw5M9ah2lr,https://i.ebayimg.com/thumbs/images/g/jjcAAOSw5M9ah2lr/s-l225.jpg       
Norway Norges Bank 10+50 Kroner 1975-1978 P-36 P-37 (3 pcs, Unc)        
,https://www.ebay.com/itm/Norway-Norges-Bank-10-50-Kroner-1975-1978-P-36-P-37-3-pcs-Unc/391982872887?hash=item5b43fff937:g:FKcAAOSw3wVah2ox,https://i.ebayimg.com/thumbs/images/g/FKcAAOSw3wVah2ox/s-l225.jpg       
NY. International Banknote Co SPECIMEN Stock Certificate ABNC        
,https://www.ebay.com/itm/NY-International-Banknote-Co-SPECIMEN-Stock-Certificate-ABNC/391985596914?hash=item5b442989f2:g:UjMAAOSwPtpag7Fv,https://i.ebayimg.com/thumbs/images/g/UjMAAOSwPtpag7Fv/s-l225.jpg       
DuraNote Polymer Ad Test Note 1980-90s Oil Rig White & Silver Maple leaf Gem UNC        
,https://www.ebay.com/itm/DuraNote-Polymer-Ad-Test-Note-1980-90s-Oil-Rig-White-Silver-Maple-leaf-Gem-UNC/352283628083?hash=item5205bd9233:g:JvkAAOSwBmlah2j0,https://i.ebayimg.com/thumbs/images/g/JvkAAOSwBmlah2j0/s-l225.jpg       
 

Edited April 25, 2018 by a-harvey