Python mechanize HTTP Error 403 request disallowed by robots.txt

When running following python script i get error “HTTP Error 403 request disallowed by robots.txt”

To fix this, find

Add below

set_handle_robots is used to disable robots.txt checking. Second line will set a User-agent, so remote won’t won’t block you as a robot.

See Python

Leave a Reply

Your email address will not be published. Required fields are marked *