It might even be a stretch to say that business value is the most important factor. Overall, prioritization often comes down to a ROI, but a lot of factors can feed into that, such as:
Types of Value
There is revenue, but there are many other types of value. Security adds value for many companies, as does other hard-to-quantify items like brand recognition, customer satisfaction, and reputation. Then you have other forms of monetary value, like cost savings.
Types of Cost
Even the cost side of ROI can be complicated. Is it a once-and-done cost? Is there a recurring cost, like software licenses or support contracts? Unknowns and risks also represent potential costs.
This is a bit of overlap with costs, but it may be impractical to prioritize in a certain order because of how something is build. Now, I'd say that 90% of the time I hear this, it's not true - it's just easier to build it one way, which means that the PO should really justify why the extra cost is worth it to prioritize a different way or they can lose the trust of the team.
Like it or not, we're humans and there are politics in any organization. They might not be mean and complex, but they are there. Maybe a department has been neglected for a while and creating a feature for them will go a long way in turning them from a detractor into a supporter for your department or team.
Work in Progress
It would be nice if nothing spilled over between sprints, but sometimes it does. There is an assumption that anything spilling over must be at the top of the priority list, but I'd encourage an active conversation about this. It doesn't have to be, but it introduces a huge risk to leave half-done work sitting around. In fact, usually when a team decides not to continue spill-over work, they instead take on the work of backing it out, which means whatever replaces it better be really important.
So, what do we do with all of this? There are a lot of prioritization models out there. Some incredibly complex with lots of spreadsheets and math, others incredibly simple where everything is reduced down to one point estimate for "value" and one for "cost". Personally, I like to list out the major categories, give them each a multiplier to signify how they rank against each other, and then do relative estimation for each feature on each criteria. That gives you something to work with at a feature level and then when you get down to stories or PBI's, I find you can use your instincts and that combined with what you had at a feature levels keeps you in a good prioritization, but that's just me.