Go to “Settings” → “Sound & Haptics”.
You can toggle “Sound effects” on and off, and pick a different sound for the plus and minus buttons under “Sound Effect Type” (tap ▶ to preview).
“Haptic feedback” (the vibration when you count) can be toggled in the same place.
* Sound effects do not play while your iPhone is in silent mode.