Any consequences if I change the publication date of an article

Hi guys, I was wondering if Google would penalize my article if I changed its publication date on my next update? Here's my situation: my blog shows the article from newest to oldest based on the publication date. I tried to order it based on the update date, but then it messes up everything each time I add a tiny modification to an article (like adding an internal link). So my alternative would be to switch back to an order based on the publication date and to manually change it for an article that receives a big update so that it can appear first in the list. I'm just worried that Google won't consider it as an "older" content anymore, and gives it less weight in its ranking calculation. Am I worrying too much?