It seems to me that your meetings are mislabeled: The 1 hour plus sit-down meeting with 20 people, most of whom neither affect nor are affected by your work, is not a Daily Scrum.
The Daily Scrum is this:
This isn't to say I am not involved with those 2 people I do actually work with. We usually talk before the meeting to "sync up", we let each other know if we have an issue, if someone is waiting on me I tell them what I'm up to and so on.
This simple chat with your direct collaborators, so brief that it doesn't even require sitting down, is a Daily Scrum. And since you have invented this tradition all by yourself (a self organizing team!) you already know the value this meeting brings to your day.
Appendix: Why is your long meeting not a Daily Scrum?
Scrum is defined in the Scrum Guide, which writes:
The Daily Scrum is a 15-minute event
... and that's a time box, i.e. if the team feels the meeting is done earlier they may of course leave earlier.
for the Developers of the Scrum Team
... so only the developers, not the entire team. And even the entire team should be
small enough to remain nimble and large enough to complete significant work within a Sprint, typically 10 or fewer people. If Scrum Teams become too large, they should consider reorganizing into multiple cohesive Scrum Teams, each focused on the same product.
If you only need to coordinate with two others the 3 of you should be your own team, with your own meeting!
And because Scrum Teams are
self-managing, meaning they internally decide who does what, when, and how.
the team gets to decide how it runs its meetings (possibly coached by the Scrum Master, who serves the team by
Ensuring that all Scrum events take place and are positive, productive, and kept within the timebox.
So if your Scrum Master was any good, they'd prevent hour long daily meetings full of irrelevant info ...
Appendix 2: Why daily?
Why do we need a daily meeting if we already have short meetings where the relevant people are getting aligned whenever the need arises
What distinguishes these short meetings from a Daily Scrum?
That the people involved are relevant? Like any meeting, the Daily Scrum should focus on topics of interest to its participants. Topics that affect only a small subset of participants should be discussed elsewhere (in this case, the Daily Scrum can still be useful to figure out who should be involved, and when to meet). Moreover, since Scrum teams are supposed to be cohesive, your work should usually be relevant to the rest of the team, so there should usually be something to talk about. And in the rare cases where there isn't, you can end the meeting early.
Or is it that the meeting is whenever the need arises? That can be a blessing and a curse. A blessing, because the meeting happens only when needed, but a curse, if a new meeting happens every time a need is identified! After all, every meeting interrupts the participants, causing information relevant to whatever they were doing before to be evicted from short-term memory, which will take some time to reconstruct before the work can proceed. By gathering these discussions in a single meeting, several topics can be discussed at the cost of a single interruption. And by having this meeting in the same time and place every day, scheduling is greatly simplified.
So why daily? Because that's easy to remember, and happens to be a good trade off between immediacy of discussion and frequency of interruption for many teams.