Every time you pull the trigger without protection, a bit of your hearing is gone, it's a cumulative effect and permanent.
Telling yourself anything else is just living in denial, which depending on how often you shoot, may become a very quiet place in the coming years.