URL keeps on changing between product page and landing page for the ranking keyword

Hi guys, In the past month I have noticed that the URL that my ranking keywords have is constantly changing twice a week or once a week. This is causing my ranking to drop significantly.

I am aware that this is due to keyword cannibalisation. In my scenario the two pages are 'Product Page' and my 'New Landing Page' that I created for SEO and has good internal linking as well as content. Whereas the product page only has a small paragraph about the product description.

I am aware that I can 'noindex', 304 redirect or robo.txt the product page but I read a google's blog which suggests that I shouldn't.

Please let me know what you guys think is the right way to deal with this is. Thanks