You have to go in and manually click the Leave Feedback button to leave feedback. If someone leaves you feedback and you click the button that says something like reply to feedback or something like that it will just put a note under the feedback they left you, but not a new one.

Does that make sense?