But then CFLs and general use LEDs are soft white for the most part
Because these are intended for the lower intensity home use, where the expected illumination level is in the area where these colors are fine. The thing is, for an eventual higher intensity use, where the colder temperature would be required, these CFLs and LEDs have too low light output, so the installation would have to use impractically high number of them to reach the higher levels.
Don't forget the rather typical F32T8 fluorescent emits nearly 3000lm, with most common two lamp per fixture it means 6000lm, while typical installation (e.g. over a single shop work bench) is two such fixtures. Having the same from incandescents (or equivalents) means 8x"100W", that is just too complex (once the fluorescents became available).
Normal application for the CFLs or similar is 1x or 2x per room way bigger than that work bench, so way less than 1/4 lighting levels. That is quite significant difference...
Plus unlike the workbench, at homes you usually want more relaxing atmosphere and for that the 4000K is still quite high.