We just had this done by professional services, and I'm so glad we did! We launched with the default Lithium avatars, which just don't really fit our brand, but we didn't take the time to create custom ones before launch.
I'd definitely recommend to newly-launching communities that you do take the time to create custom avatars and delete all of the Lithium default ones, so that you don't have to spend $ on this later.
But, if, like us you were a little lazy / time-crunched, the PS engagement to make this change isn't a big deal and isn't that expensive. Also, it sounds like, in a pinch, you can simply delete the old avatars and upload new ones WITH THE SAME FILENAME - see https://community.lithium.com/t5/Developer-Discussion/Changing-custom-avatar/m-p/430986/highlight/true#M13457 (I haven't tried this myself).
Here's my blog post about our new avatar launch, which happened today (about 6 months after our community went live): https://community.meraki.com/t5/Community-Announcements/Community-update-Our-new-avatars-are-out-of-this-world/ba-p/11570 - may be useful to someone.
Cheers!