The idea of the Dyson sphere, or actually Dyson swarm as it was originally proposed, assumes that there is no weird new physics that makes energy for free. If you have truly free energy then all bets are off for what you can do with it. If there are no new thermodynamics breaking discoveries then even with cheap fusion reactors making a Dyson swarm is the best long term way to get huge amounts of energy. With decent automation only a little better than we have now and a few centuries of time you could disassemble Mercury into space habitats with room for easily quadrillions of people. So without magic free energy why not do that?
That is already a thing and it’s called concentrated solar power. Basically aim a shit load of mirrors at a target to heat it, run some working fluid through the target and use that to make steam to turn a turbine. There are a few power plants that use it but in general it has been more finicky and disruptive to the local environment than traditional PV panels would be.