Is there a way to speed up table building when scraping emails ?

[webpro 31]

Right now it has been more than 5 minutes and it still scraping 1.5k gmails (creating tables once i'm logged into an account)

(edited it took 9 minutes,it's done now).

 

Is there a way to work this out faster ?

 

i'M in a "Tim Horton" restaurant for a couple of hours (ain't home) connection is 78mbps

 

Thanks

[VaultBoss 310]

You can set the number of emails available to retrieve directly in the Gmail options. If you retrieve less, it will be faster.

I setup mine to allow 1,000 emails when testing, but for working bots, I would rather limit that down to 100 and run the bot more frequently.

[webpro 31]

I see
So might create some sub folders and send certain messages into certain folders to have less.

While i have you, do you have a trick to grab links on plain html emails ?
Right now i got some mails which are showing as plain text into UB when using LOAD HTML command along with $table cell.
So there's no way for me right now to grab the link nor to create a click command.
Works fine when it's html messages by the way.

Thanks

[VaultBoss 310]

You can't create subfolders in Gmail ... eventually you could add labels for sorting.  But that wouldn't help anyway.

What you can do though, is to create separate Gmail accounts for separate purposes - and play with their advanced options accordingly (set the dld max to 200 emails, filter certain emails out directly in Gmail, make sure none go to spam, etc... whatever you want to)

Once you have your different Gmail accounts structured for the various purposes, you could still play with them ALL in a single location, by forwarding them ALL to a central (hub) Gmail that you want to monitor personally... and filter them there using a combo of filters+label for having separate categories, just as before.

 

The advantage is that now, your bots would connect to separate Gmail accounts that would only hold 'certain' topic emails and thus, much less Inbox clutter - which would make your email bots much faster.

[VaultBoss 310]

You need to employ REGEX to be able to extract all the URLs from the text version of the emails and then NAVIGATE to those links, instead of clicking on them.

[webpro 31]

Thanks

I guess i will no choice but to start learning/coding regex

I sould be able to code a bot with it in 2014 lol.....

 

Seriously, it's about debugging a lot as i can see. You need to test with editpad lite with a sort of a cheat list where you got all the regex codes in there and then you start swearing as you can't make it hahahahahahaha

 

I will surely try yours !!! Thanks again

[VaultBoss 310]

Use this to play with and test your REGEX:

 

http://regexhero.net/tester/

 

Good luck!