Don't Allow Website automated Redirect Scrapy

 

I am really sorry if the question is incomplete or unhelpful. I am slightly new at this.

I just started using scrapy for scraping data. The issue that I am facing at the moment is: when i try to scrape data, a website redirect's me to a particular page.

I intend to stop the website from redirecting to that particular page.

To be honest i am not sure if it's even possible or not.

What I have tried so far is:

  • Following the redirect and from there using a href link provided to go to the desired page.The issue in this case is that site follow's a loop and redirects me back to the same page.

  • REDIRECT_ENABLED = False in setting's. The issue in this case is that the scrapper exit's and the script stop's.

    • Any help will be really appreciated.

    Thank you

    链接地址: http://www.djcxy.com/p/89120.html

    上一篇: 我如何改变我的<hr>标签的厚度

    下一篇: 不要让网站自动重定向Scrapy