How to find pages on a website with a noindex tag?

Hi folks, I have an oddball question for you! How can I get a list of pages on a website with a no-index tag? i.e. How can I find what information a company is hiding on their website?

As some background, I'm a healthcare researcher in the UK and working on a project about doctors conflicts of interest in private healthcare. For example, if a doctor is referring you for an MRI scan, you might want to know that they own shares or equity in the company running the scanner. In fact, the Competition and Markets Authority require private healthcare providers to publish this information on their website.

The company I'm looking at does publish this information, but all of the pages where their doctor's equity stakes are published have a NoIndex NoFollow tag. So this company is following the letter of the law, but by making these notices invisible to search engines dodging the spirit of it.

Now I'm wondering, what else is hidden by a noindex tag? But it's not like I can exactly google for 'Noindex' to find out what else is there. How might you approach this issue? Are there any search engines which don't respect NoIndex tags I could use? Any third party tools that can crawl a website in a similar way? Any thoughts?

submitted by /u/Ahzek117
[link] [comments]

Digitalmarketing

Digital marketing agency in darbhanga

Digital marketing agency in Darbhanga digital marketing agency Local seo service in Darbhanga Best digital marketing agency in patna

0 Comments