There’s physics reasons that 5g is theoretically faster, while also having lower range. It’s related to the frequency used. All other things being equal, higher frequency radio waves attenuate faster in any given medium than lower frequency waves. (Their range would effectively be identical in a vacuum.) But, the total number of waves you can transmit in any given period is inversely proportional to attenuation.

As you approach a bitrate that is equal to the frequency, you start to run up against the limits of physics in terms of how much information you can transmit. Choosing a frequency arbitrary to illustrate: 900 MHz. At 900MHz, there are 900 million waves going out per second. In some perfect universe, you could turn the signal on/off up to 900 million times per second, giving you a theoretical bitrate of 900Mbit, or 112MB/s. In reality, you never approach this limit, due to error correction and a bunch of other things, so you might get a tenth of that speed as your practical maximum from an engineering perspective. So maybe 10MB/s. There are other tricks you can do, like MIMO on multiple adjacent frequencies, but that is a digression.

But if you go from 900 MHz to 2.4GHz, you automatically gain a reprieve from the physics, allowing you to potentially gain 2.7x the speed. Amazing! Only, now the signal is absorbed by materials 2.7x faster, and has trouble penetrating through walls as well and such. Yikes!

So you have a tradeoff. 4G is a lower frequency than 5G. 5G makes more sense in very dense open environments: stadiums, concerts, busy outdoor markets, etc. where you need a lot of bandwidth in a small area without a lot of obstacles. 4G will penetrate the walls of your house better, at lower theoretical top speeds. Phones can and should switch to whatever is optimal for the environment you’re in.

We’re near the limit of physics on these things and I don’t expect things will evolve a lot more from here.