Baidubot blocked on Udemy
Hey, everyone. I was going through the robots.txt file of the Udemy website, and I saw that the webmaster disallowed the Baidubot from crawling any of the website's pages. They've even included a no-index tag as well.
Is there a reason why they'd do this? Is the website blocked in China? Just curious.
[link] [comments]
Digitalmarketing
0 Comments