You need to first understand to what extent the story count is helpful and we do this with standard deviation. If we look at the average story size across a release and the standard deviation is low, you will find that the trends created between story size-based velocity or count based velocity are identical. If the standard deviation is high, they will trend differently. If you're not a big math fan, there is an easy visual way to do this. On the same graph (2 different y-axis scales) plot both of the velocities. Do the lines look the same? If so, you can use either. If not, stick with story size, not count.
There is a fairly large "noestimates" movement that finds a lot of success in right-sizing all stories so they are approximately the same size and then they only look at count. They don't have to be exactly the same. To give a concrete (if hypothetical) rule, a team using point with an average velocity of 40 would probably want most stories around a 3 with a small number of them at a 1 or 5 before they could drop the story points. (This is a deeply flawed example, don't actually use it to implement, just trying to give something concrete.
Now, we should also talk about that last bit about being in line for a present delivery date. From you comments, it sounds like scope and time are fixed, which is almost always problematic. Regardless of what metric you are using, any less-than-desirable picture the forecast paints must be used to manage the backlog, not the team, or you are going to get a lot of dysfunctional behavior and probably lose visibility into your progress.