Yes, you can turn it off.
Select Audio Recordings while reviewing your sleep data, and choose Disable Recording to turn the function off.
Also, please be assured that audio recordings are never sent to us. They are saved to your device, and are automatically deleted either after 24 hours have passed since your last sleep tracking session or when you begin your next sleep tracking session.
Note: Even with audio recordings disabled, Pokémon Sleep requires permission to access your microphone in order to track your sleep.