Your statement is on point. If the project charter gives authority to the project manager, then the project charter should be handed down to the project manager by the project sponsor. But it's not wrong either if the project manager is the one to write this document.
The key word here is "gives". If the project manager is the one to write the project charter, this does not all of a sudden mean that the project manager "takes" the authority. The project sponsor still needs to give the sign-off. They are ultimately responsible for what's written in there. No sign-off, no authority, no matter what the project charter says.
The project charter is often written by the PM and not the sponsor because it's a skill project managers have and the sponsor (who might be very busy or a really Big Kahuna who can't be bothered with writing such documents) just makes use of that skill and delegates the work to the project manager. The PM works with the sponsor and stakeholders to successfully deliver the project and since the charter is part of the initiation phase of a project, it's something that the PM can handle or help with (and it's often a good thing since the project manager gets a better understanding of the project goals right from the beginning, while things are taking shape, and can contribute their experience).
The PM drafts the project charter, gets feedback from the sponsor, makes changes, comes back for more feedback, rinse and repeat, and when things are to the sponsor's liking, they give their sign-off. If the sponsor just signs in blind faith, well... then they get what they deserve.