The reason for using story points is that it allows us to work out the capacity of the team.
The question you have to ask yourself is what is the best way for the team you describe to work out its capacity.
If you force boundaries between disciplines then really you have several capacities and not just one.
For example, say the team consisted of 5 developers. 2 developers can only work on stories relating to the database. 1 developer can only work on the front end and 2 other developers can only work on the web services.
In this situation the team has a capacity for database work, a capacity for front end development and a capacity for web services. That is even before we start talking about testing. Is there just one tester who can work on each area? Do the developers do the testing?
Estimating in Scrum works best when we assume the team members can work on any story. They may be better at working on some kinds of stories than on others, but they can still work on all types of stories. If a developer does not know how to do something, they can get training or pair with a developer who does know how to do it. That way knowledge is shared across the team.
I would recommend you adopt this approach to knowledge sharing and have every team member estimate on every story. You will find that developers estimate high on stories they are unfamiliar with and low on stories that they understand well. That is fine, just get the team to come to a concensus estimate that incorporates every team members input. The way velocity is calculated as a rolling average tends to compensate for this kind of thing over time.
If the team does not want to become cross-functional than you are better off estimating at the task level rather than at the story level. You may well find that estimating in ideal hours is more useful than story points in this approach.