I agree honesty and openness is always the best way and if someone complains you have to remain professional and deal with their issue (esp if you have caused it). There is no way out of this one. You may (or may not) get the chance to choose some time in the future whether you want to work with them again.
One thing that may help and reduce the negative 'feel' when this happens is to be clear with them what you will do if they have any issues and working more collaboratively with them (if that is possible) so that you can prevent these instances before they happen.