Yes, a project manager is responsible for completing a project on time. However, "on time" is a date that will often move during the life of the project, and it is the agreed date that the PM should be measured against - not necessarily the original date. The initial date may be agreed by all concerned, however requirements may change, issues are likely to emerge, and the expectations for the project may be modified (e.g. "we have delayed the launch of the new product so we don't need the new system until a later date" - or whatever).
So perhaps you should think of the true responsibility of the PM as delivering the project to the satisfaction of the key stakeholders, which will be delivering against a combination of date, cost, scope, and quality, where these are agreed initially and then modified (re-negotiated) during the life of the project. A successful PM will avoid giving the stakeholders unpleasant surprises by keeping them fully informed of risks and issues, and the impact of them, and agreeing the approach to dealing with each of these - which may be to move timelines, reduce scope, add resources (financial or otherwise), or accept certain risks and the associated consequences (e.g. reduced testing, shorter parallel running, less handover, or whatever). Any of these changes should be subject to formal change management - not just at the whim of the PM, though!
In terms of performance reviews, etc, I would take account of the PM's ability to deliver to the agreed date, and / or minimising the number of changes to the date (or other outcomes), and keep the stakeholders fully informed and comfortable. In my experience, stakeholders don't mind change, as long as they understand the reason for it and as long as the change is fully explained, is justifiable, and isn't just because of poor planning or inept project management.