Replies: 1 comment
-
|
Anyone has any update on this? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I've setup a crawler which works fine using spatie crawler on loads of different websites but on one website it returns a 403 which appears to be cloudflare blocking it. If I crawl this website through browsershot it crawls fine and gets past the cloudflare page, meaning it certainly isn't IP related. What could be causing this I'm not sure if it's something to do with the spatie crawler not having JS enabled but i've added the flag as shown in the image below. I noticed in the documentation you can add a setBrowserShot but not sure if this is something I need and what arguments I pass into if I do.
Beta Was this translation helpful? Give feedback.
All reactions