Scraping Prices then setting up some sort of alert to notify price changes.

[positivity13 4]

Hi Guys,

 

I am creating a bot. And one of the jobs it does is scrape the price of each listing and put it in a list.

 

What I would like to do is set up some kind of alert that the price has dropped say the next day for that listing. 

 

How would the best way to do this be, as this has got me stumped? I have the dev version

 

 

Thanks in advance

 

 

 

 

[positivity13 4]

Just a bit more info.

 

Currently it sticks the unique URL price and image into individual lists and then into a table, which is saved as a Csv file on the desktop.

 

Is there maybe a way I can get ubot to create a excel pivot table to highlight the price hange or maybe something actually in ubot?

 

Please help

[positivity13 4]

Right I was thinking if I combined the URL which will be the unique identifier plus the price in one line making a string per line. Then I will save this on the computer. Then when the bot is run it will check against the list first searching the first bit of the string (URL) to match up against the new list and then comparing the last part the prices. Then if there is a price change it can add this to a list

 

What do you think?

[HelloInsomnia 1103]

If you use a table you can store the last days price and then just say if its lower than that day then maybe add some css or formatting like bold it add a yellow background those kinds of things. And if it is higher then maybe another color or font color. But the best way to track it would be through tables and not lists.

 

One thing you can do is save off a table everyday that is for that days prices, then the next day it opens table1 and adds in current prices for that day on top of the prices from yesterday then saves that as table 2 and overwrites table 1 with the current days prices - I hope that makes sense.

[positivity13 4]

Hiya that does make sense, but the orders of the URLs will change and some will be deleted and some new ones will be added. So the table could be different each time. So how would you overcome this?

 

Thanks

[HelloInsomnia 1103]

You can loop through the table rows and only add in the price if the url is the same or the title or another unique identifier. 

[positivity13 4]

Yeh I sort of get it, better down some coffee and give this ago, thanks insomnia

[LoWrIdErTJ - BotGuru 904]

you can actually save unique url and data to a csv, then next day have it loop that csv day, and make a new column with price scraped, then compare the 2 prices and alert or email as needed.

 

 

have it keep updating that save new price cell on second run and each run after and compare.

 

 

i made a google products bot like this before for a client that was comparing there product prices and wanted to know the difference high or low with and without shipping on products of theirs in comparison to 5 top competitors in google products.

[positivity13 4]

Thanks tj

 

You don't have the code lying around do you :-). Lol

Edited August 15, 2013 by positivity13