TL;DR
You're trying to build a Gantt chart based on a process that worked with a previous team, but clearly isn't viable with the current team. There are probably many reasons for this, and I'll enumerate some below, but I think the best approach for you would be to take a step back and assess the best process for planning with your current team.
One of the best ways to do that is to present your planning problem to the team, and ask them to help you develop the plan. While you are responsible for the plan itself, leveraging the people on your team to gather data and develop a functional process is often a useful approach.
Analysis and Recommendations
This question isn't tagged with any agile tags, but I still think that a key tenet of agile project management could help you. In your case, the problem seems to be that you are attempting to break down the work, and you are trying to decide how long various things will take. This is an anti-pattern even in traditional project management because:
- People aren't fungible resources, even when they're not over-specialized.
- What takes a given developer three days to accomplish may take another developer three weeks, so skill- or knowledge-based estimates aren't really fungible either.
- Complex work is rarely cleanly delineated along lines of specialization, especially in IT or knowledge work.
- Most work of any sizable complexity will require collaboration across areas of specialization or domain knowledge.
In short, you need more collaboration on your team in terms of both process and estimation. In comments, you give the example of front-end vs. back-end developers. My point isn't that these folks are interchangeable, or can do each other's jobs, but rather that they will need to collaborate in many areas such as design, testability, usability, and various other dimensions. Your planning should, at the very least, capture those areas of overlap and essential collaboration so that the whole team is thinking about those things when helping you to build the plan. Otherwise, you may end up with a critical path that lacks essential nodes, or estimates that don't take the overhead of interdisciplinary back-and-forth into account when planning.
As for the question of accuracy, in knowledge work it's practically a truism that any estimate not provided by the person doing the work is likely invalid. In other domains that's not always true, of course, but in IT you often run into problems when anyone other than the assigned resource provides the estimates. What you typically end up with then are management targets, not accurate estimates. If that's what's needed in your organization that's fine, but if you're actually looking to increase the accuracy of your estimation process then you need to involve the people doing the work when estimating.
By breaking the work down into milestones, you may be able to build your Gantt chart based on when the different work streams converge for a given project increment. In other words, given target date X for milestone Y, you can then build your dependency mappings and start-to-finish planning backwards from those dates and milestones. I won't say that this will be accurate to some arbitrary decimal place, but it will still give you a visualization tool that will help you determine whether a given work package is on-track or off-track.