Can Optimizing for Speed negatively affect Organic Traffic?

I recently paid someone to speed up my website. It was atrocious, and I felt it was impacting my organic traffic growth.

He performed services 5 days ago and ever since I have had a steady decline of organic traffic by about 10% every single day so that now I'm done nearly 25%-30% of my organic traffic over a week ago.

Could whatever he have done caused this, and how would I fix it?