We are working on a technology of which the team is not sure how would it go. I would like to estimate and timebox this story. However, the scrum team has to have goal or a shippable product at the end of this story.
Two things jump out at me.
First, the end result of a Spike is not a shippable product. Spikes are used to learn, and do research. The end result is an answer to a question or finding some information or gaining knowledge in a given area. That doesn't mean that there's not an output associated with a Spike, but it's almost certainly not a shippable product. It also doesn't mean that an iteration that contains a Spike can't also result in a shippable increment that results from other work done.
Second, you shouldn't estimate Spikes. I consider a Spike to be a well-bounded form of backlog refinement activities. One of the end result should be better defined work items in the team's backlog. If you're using Scrum, it is suggested that the team would spend about 10% of its capacity on refinement activities, but this is not a maximum - it can be much more, if necessary. Other frameworks may offer different guidance. The work done to a Spike would fall into the backlog refinement activities, which would be capped at the team.
So: don't estimate Spikes, but do consider the total team effort to perform refinement activities when planning your iteration.
Spikes are typically time-boxed, so it is usually easy to work out the impact they will have on a sprint.
For example, you might time-box a spike on a new technology to be one developer for one day.
The idea with a spike is to do just enough investigation so that the team is then able to estimate the work.
If at the end of the spike the team still does not feel confident enough to estimate then you could potentially do a further spike or alternatively try and break the work down.
Not every story in a sprint needs to result in a (potentially) shippable product. But as a sprint typically contains multiple stories, it should still be possible to have a (potentially) shippable product at the end of the sprint without having one after each story is completed.
The objective of a spike is to gain knowledge that can be used later when working on stories in that area. As such a spike doesn't directly contribute to a shippable product, but gaining knowledge in a particular area can even be a valid sprint goal.
As for estimating a spike, the team will most likely push back if you ask them how much effort it will be to do the investigation, especially as it will most likely be uncharted territory for the team. But what you can do as an estimation is establish how much time you all are prepared to invest in the investigation. That should give you some insight in the team capacity that goes into the research and how much capacity is left for the other work you want to plan.
A "spike" is ... and is intended to be, ... "a calculated 'shot in the dark.'"
To my way of thinking, the essential purpose of this concept is to formally recognize that "sometimes, we do not know," and cannot(!) yet know, "what to do next." Therefore, we simply try to time-box some amount of time into that investigation. (And: "I specifically recommend that you should expand that 'box' as much and as often as necessary," because, in the end, only the results matter. Continue to expand the box until you're sure the decision is "right.")
Because – "sometimes we know 'the right way to proceed,' and sometimes we don't." However, the cost($$) of "rushing or short-changing that decision," should it result in a course of action that "hindsight tells us ..." can be profound. Can indeed be the subject of books like "The Mythical Man-Month."
Importantly: you should be fully-prepared to revise your project plan and schedule, "however 'completely,'" based on the as-yet unforseen outcome of each so-called 'spike.' Because, the objective of a "spike" is specifically the acquisition of new knowledge.