If a requirement is discovered in the middle of a project or even later, then the cost of implementation of this requirement can be very high.
This can be true or can be false. It depends of the type of change you are referring to. I'm assuming a big change, or something lacking in an important way, or something that deviates in some way from what you have built up to that point. If this is the case, then yes, the change can be expensive. If it's just a requirement like others similar, then implementation will probably not be that high.
The way you phrased that first sentence and the rest of the question implies you are using a predictive approach, or a traditional approach in which you gather requirements at the beginning and then you build a plan to have then implemented. Only in that way can you talk about "the middle of the project".
If you are building the product in an agile way for example, you wouldn't know you are in the middle of the project because you are focusing on delivering value, not on timelines that tell you where you are in the project. However, this does not mean that using Agile or Scrum will guarantee that a requirement late in development will be cheap to implement. If you are building a white airplane and at some point you discover you need a submarine, you can't just chop off its wings, paint it black, and call it done. This kind of change will mess up your implementation no matter if you are using a traditional approach or an Agile one.
However, what Agile does, is that it allows you to figure out sooner that you need a submarine instead of an airplane, because you are incrementally and iteratively building stuff that you can collect feedback on and learn of what's needed, instead of assuming that you know everything from the beginning and you can specify it in requirements that won't change or are not lacking in some way. As already mentioned in the other answers, the Agile approach is different and reduces the chances of discovering something that can really be very expensive to implement at some point down the road.