I would not judge the wear of my suspension on the behaviour over potholes alone.
How is the suspension behaving when not surfing potholes?

If your suspensions are what came with the car from the factory, you have a good excuse to upgrade.