This problem isn't unique to Scrum. It's quite common across the Agile methods, which call for frequent delivery of valuable, working software to customers and users. I've come across two ways to handle this situation, both of which have worked out well.

The first way would be to define the infrastructure that is needed as part of both A and B. Then, estimate both A and B as if the infrastructure would be created and tested as part of both. The second approach is to split out the infrastructure into a dependent item that must be completed before either A or B is completed, ensuring that the sizing for A and B is based on the assumption that the infrastructure is in place.

I would avoid the proposed solution of estimating A and B in a way that requires one to be done before the other. This adds an unnecessary and potentially unclear constraint to the way that the team must work. If the wrong one is started first, it will be significantly underestimated.

I have to agree with Jan Doggen's point that defining each Product Backlog Item in terms of customer value isn't the best way to approach it. Loosening the restriction to delivery of value to some stakeholder, which could include the development team, is a better way to think about it. I also like to think of each Product Backlog Item as an independently designable, implementable, testable, and deliverable entity. You can design the infrastructure, stand it up, perform testing on it, and put it into production. This gives you an opportunity to account for infrastructure configuration, performance testing and configuration tuning, security testing. Having the infrastructure already in place before deploying the software that uses it often eases the pressure of deployment. Considering all of this, there is value in ensuring that your infrastructure choices are correct to meet non-functional requirements prior to delivering software that uses it.

The choice between a technical dependency in your Product Backlog and including the work in both Product Backlog Items depends more on your context than anything else. Things like the number of teams supporting the product, technical savvy of the product manager, the tooling used to track your Product Backlog, the time delay between identifying this work and starting this work, and the pressure on the whole team all would affect my decision one way or the other.

I don't think that concerns about the heaviness of the Product Backlog should weigh into the decision at all. After all, you aren't making a firm schedule and committing to it. You're forecasting the amount of work. If you're estimating, all of your estimates have noise. If you happen to have overestimated the amount of work, you can always use that time effectively, by either taking some additional time for Product Backlog Refinement, skill-building, paying down technical debt, or starting some other work early.

This problem isn't unique to Scrum. It's quite common across the Agile methods, which call for frequent delivery of valuable, working software to customers and users. I've come across two ways to handle this situation, both of which have worked out well.

The first way would be to define the infrastructure that is needed as part of both A and B. Then, estimate both A and B as if the infrastructure would be created and tested as part of both. The second approach is to split out the infrastructure into a dependent item that must be completed before either A or B is completed, ensuring that the sizing for A and B is based on the assumption that the infrastructure is in place.

I would avoid the proposed solution of estimating A and B in a way that requires one to be done before the other. This adds an unnecessary and potentially unclear constraint to the way that the team must work. If the wrong one is started first, it will be significantly underestimated.

I have to agree with Jan Doggen's point that defining each Product Backlog Item in terms of direct customer value isn't always the best way to approach it. Some work - such as work to design, build, and validate shared infrastructure - doesn't have direct value to a stakeholder, but removing this shared dependency opens up opportunities for the development team to deliver any number of work that is directly valuable to stakeholders while maintaining the integrity of the system. 

I like to think of each Product Backlog Item as an independently designable, implementable, testable, and deliverable entity. You can design the infrastructure, stand it up, perform testing on it, and put it into production. This gives you an opportunity to account for infrastructure configuration, performance testing and configuration tuning, security testing. Having the infrastructure already in place before deploying the software that uses it often eases the pressure of deployment. Considering all of this, there is value in ensuring that your infrastructure choices are correct to meet non-functional requirements prior to delivering software that uses it.

The choice between a technical dependency in your Product Backlog and including the work in both Product Backlog Items depends more on your context than anything else. Things like the number of teams supporting the product, technical savvy of the product manager, the tooling used to track your Product Backlog, the time delay between identifying this work and starting this work, and the pressure on the whole team all would affect my decision one way or the other.

I don't think that concerns about the heaviness of the Product Backlog should weigh into the decision at all. After all, you aren't making a firm schedule and committing to it. You're forecasting the amount of work. If you're estimating, all of your estimates have noise. If you happen to have overestimated the amount of work, you can always use that time effectively, by either taking some additional time for Product Backlog Refinement, skill-building, paying down technical debt, or starting some other work early.