Matrix is not part of the fediverse, so that’s kind of a special case and doesn’t work the same at all as the rest.
What you describe sounds very simplified, but let me assure you that there is nothing simple about this problem (I say that as a software engineer that has studied ActivityPub, the protocol underlying the fediverse).
It sounds like that would require unifying the architecture of all fediverse platforms, which nobody is interested in and very much goes against the point (decentralization). Right now all of these platforms are written independently, with unique architectures and different programming languages.
Suffice to say that, while it’s a nice thought, what you’re proposing is not really realistic, nor is it actually desired.