Jump to content
UBot Underground

What's Wrong With My Regex To Scrape Forum?


Recommended Posts

Hi!

 

On this forum, https://www.flashback.org/t1045448, I'm trying to scrape this part (all quotes) in all posts using Regex 

 

https://imgur.com/shWb5LJ

 

Been trying this but doesn't seem to work..

<div class="post-bbcode-quote-wrapper">\n*.+\n*.+\n*.+\n*.(.+)\n*.+\n*.*\n*.*

 Any help would be appreciated :)

Thanks



 

Link to post
Share on other sites

 

 

navigate("https://www.flashback.org/t1045448","Wait")

set(#quote,$scrape attribute(<class="alt2 post-bbcode-quote">,"innertext"),"Global")

alert(#quote)

 

or regex from page source

 

navigate("https://www.flashback.org/t1045448","Wait")

set(#pagehtml,$document text,"Global")

set(#quote,$plugin function("File Management.dll", "$Find Regex First", #pagehtml, "(?<=\\<div\\ class\\=\\\"post\\-bbcode\\-quote\\-wrapper\\\"\\>)(.|\\n)*?(?=\\<br)"),"Global")

set(#quote,$strip tags(#quote),"Global")

alert(#quote)

Link to post
Share on other sites

Take a look at Frank's regex example

http://network.ubotstudio.com/forum/index.php?/topic/7162-using-regex-to-catch-text-between-sections/?/topic/7162-using-regex-to-catch-text-between-sections/

navigate("https://www.flashback.org/t1045448", "Wait")
comment("Replace the line breaks before using Frank\'s regex example.")
set(#DocNoLineBreaks, $replace($document text, $new line, $nothing), "Global")
clear list(%quoted)
add list to list(%quoted, $find regular expression(#DocNoLineBreaks, "(?<=<div class=\"post-clamped-text\">).*?(?=</div>)"), "Delete", "Global")

  • Like 1
Link to post
Share on other sites

or

 

 

to find xpath, use chrome extention: "truepath"

 

to test xpath on chrome, use chrome extention: "xpath helper"

 

Thanks! Thats a handy little plugin. I never even thought to look for a plugin to help with xpath expressions.

Link to post
Share on other sites

Thanks! Thats a handy little plugin. I never even thought to look for a plugin to help with xpath expressions.

Hi, a few more chrome plugin to share with you

 

cookiebro - load and save cookies to chrome (* if you know how to upload cookie file without using open file dialog in exbrowser, please share)

https://chrome.google.com/webstore/detail/cookiebro/lpmockibcakojclnfmhchibmdpmollgn?hl=en

 

chropath - more detail xpath (right click > inspect > chropath)

https://chrome.google.com/webstore/detail/chropath/ljngjbnaijcbncmcnjfhigebomdlkcjo?hl=en

 

coordinates - find x,y on chrome

https://chrome.google.com/webstore/detail/coordinates/bpflbjmbfccblbhlcmlgkajdpoiepmkd

 

postman inspector - it show the http request, but i dont know how to use it

https://chrome.google.com/webstore/detail/postman-interceptor/aicmkgpgakddgnaphhhpliifpcfhicfo

 

truepath (right click > relative xpath)

https://chrome.google.com/webstore/detail/truepath/mgjhkhhbkkldiihlajcnlfchfcmhipmn

 

xpath helper

https://chrome.google.com/webstore/detail/xpath-helper/hgimnogjllphhhkhlmebbmlgjoejdpjl?hl=en

 

xpath helper wizard

https://chrome.google.com/webstore/detail/xpath-helper-wizard/jadhpggafkbmpdpmpgigopmodldgfcki

 

xpath generator

https://chrome.google.com/webstore/detail/xpath-generator/dphfifdfpfabhbkghlmnkkdghbmocfeb

 

to download the chrome extension file to use in exb

https://chrome-extension-downloader.com/

Link to post
Share on other sites

You can just use Chrome dev. tools

 

https://www.youtube.com/watch?v=du2Jnm-TzJc

 

You can copy the JavaScript path in the same manner and use the run jabascript command with it. Like if you need to click a checkbox just add .click() to the js path.

 

The video above shows how to get and verify XPath.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...