DjProg 3 Posted September 28, 2010 Report Share Posted September 28, 2010 Hello guys, I don't succeed to do what I think must be really simple. On some websites, I found dropdowns with an HTML code like this : <center><select name="id_cat" class="list_cats"> <option value=0>Choose a Category</option> <option value=0>--------------------------</option> <option label="Category 1 " value=11>Category 1</option> <option label="Category 2 " value=5>Category 2</option> </select> </center> I "just" want to scrape :"Category 1 " value=11>Category 1"Category 2 " value=5>Category 2 I tried this : CHOOSE BY ATTRIBUTE-----InnerHtml-----<OPTION label=*</OPTION>-----Wildcards Which is supposed to get the multiple categories as far as i know ADD TO LIST-----%scraped-categories-----$scrape chosen attribute----------innerHTML Which is supposed to put all the scraped categories in a list And it just don't work at all... Could you please help me, i'm kind of desperate and it MUST be simple i'm sure. Cheers, Quote Link to post Share on other sites
Pete 121 Posted September 28, 2010 Report Share Posted September 28, 2010 I would think you need more wild cards<option label="Category * " value=*>*</option> Quote Link to post Share on other sites
UBotBuddy 331 Posted September 28, 2010 Report Share Posted September 28, 2010 First. I have done what you are trying to do. But it was a lot of coding on my side that made it happen AND my solution will not work on your site. Here is where you are going to get tripped up. the " Now here is a potential solution Choose by attribute the innerhtml let the following be your Search String also select Wildcards* Then do an Add to List and for your content use the $scrape chose attribute using innerhtml That will get you each of these into your list Category 1 Now setup a loop to go through each item in the list and what you want to do is perform a $replace search for "" and replace with Null/nothing perform another $replace search for "" and replace hat with a Null/nothing You will have to have enough "Category 1", "Category 2", "Category 3" until "Category x" is met. Remember, I am doing this from memory. Not knowing your website to see the actual dropdown limits me to some degree. But this is the essence in how I achieved my scrape. Quote Link to post Share on other sites
DjProg 3 Posted September 28, 2010 Author Report Share Posted September 28, 2010 Hi thanks for your help ! I tried but it still doesn't want to collect the categories (i'm just speaking of unformated categories, formating is another story...) You can have a look at the bot attached, in case it inspires you category-scrapping.ubot I also tried to do a pagescrape but it doesn't work either... (I thought it would work with the following : pagescrape LEFT : <option label= RIGHT : </option> but it doesn't scrape anything...) Cheers, Quote Link to post Share on other sites
UBotBuddy 331 Posted September 28, 2010 Report Share Posted September 28, 2010 Hmmmm The only way that I can see to do it is to write some Javascript to extract the text between ">" and "" It can be done just not by me. It would take me a few months. LOL I was able to scrape the contents of the dropdown and your code can too. Just change your Search String in your "choose by attribute" node to *>* When you examine the output file you will see all of the options. Now, had the website coder written the For instance, search for "" and replace it with "". Sorry that my answer does not get you closer to a solution. Quote Link to post Share on other sites
IRobot 43 Posted September 28, 2010 Report Share Posted September 28, 2010 $page scrape does work. See: category-scrapping.ubot 1 Quote Link to post Share on other sites
DjProg 3 Posted September 28, 2010 Author Report Share Posted September 28, 2010 I was able to scrape the contents of the dropdown and your code can too. Just change your Search String in your "choose by attribute" node to *>*</OPTION> It's weird, I thought they way i had written my choose by attribute was correct. I'll try yours. Thanks ! Quote Link to post Share on other sites
DjProg 3 Posted September 28, 2010 Author Report Share Posted September 28, 2010 $page scrape does work. See: category-scrapping.ubot Thank you VERY VERY much. I don't know why but the following didn't work : http://screencast.com/t/MWNlOTVhYzkt Quote Link to post Share on other sites
IRobot 43 Posted September 28, 2010 Report Share Posted September 28, 2010 Thank you VERY VERY much.No problem. I don't know why but the following didn't work :http://screencast.com/t/MWNlOTVhYzktIt didn't work because the left and right 'option' $page scrape items are lower case, whereas in the working bot I attached, they are upper case. Quote Link to post Share on other sites
UBotBuddy 331 Posted September 28, 2010 Report Share Posted September 28, 2010 Interesting! It did not work for me. Our code is identical. Mine does not work and your does. Hmmmmm Quote Link to post Share on other sites
UBotBuddy 331 Posted September 28, 2010 Report Share Posted September 28, 2010 Found it. I had been rehashing an open test bot. Had I just Refreshed it the code would have worked. Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.