golfer57 0 Posted February 8, 2017 Report Share Posted February 8, 2017 I am trying to scrape the search engine pages and once I have the page scraped, I want to extract all the bolded keywords and put them in a list.I've been trying to make this happen and I just can't seem to put my finger on the right code to get this to work. If you could help me out with this would be greatly appreciated.So again I want to scrape the search engines for a specific keyword. Once I have the page scraped with all the listings this includes the title, URL, and a description that is in the search engines I want to then extract the bolded keywords from that scraped page. I then want to put those bolded keywords into a list. Once the first pages done I'd like to move on to the second then the third getting all the bolded keywords into a list. So I only want to do three pages.Any help in this matter is greatly appreciated thanks Quote Link to post Share on other sites
golfer57 0 Posted February 9, 2017 Author Report Share Posted February 9, 2017 i am wondering how to get the bolded phrases or words highlighted. i tried to use xpath and expressions and still i could not get the bolded words highlights. they can be in the title, url, or description. i am at a loss, trying to watch youtube, but with very little success. i was told to post here but nothing is happening either. not sure what to do or where to get the help, Quote Link to post Share on other sites
pash 504 Posted February 9, 2017 Report Share Posted February 9, 2017 post your codeorhttps://www.google.co.th/search?q=scrape+search+engine+ubot+site:network.ubotstudio.com Quote Link to post Share on other sites
abbas786 78 Posted February 9, 2017 Report Share Posted February 9, 2017 Here is an example code of search and scraping bold text . Am using xpath using aymen http plugin, you can use Dan's free xpath plugin . clear list(%BoldWordsList) set(#keyword,"how to make money online","Global") navigate("https://www.google.com.pk/search?q={#keyword}","Wait") wait for browser event("Everything Loaded","") set(#data,$document text,"Global") set(#boldwords,$plugin function("HTTP post.dll", "$xpath parser", #data, "//em", "InnerText", "HTML"),"Global") add list to list(%BoldWordsList,$list from text(#boldwords,$new line),"Delete","Global") Hope rest of the navigation and move to other pages you can do yourself Quote Link to post Share on other sites
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.