I'm thinking of the most effective (as for the luminous flux) CFL geometry. (I'm not considering lowest space requirements, mechanical strength or fashion/personal preferences here.)

Let's say that the most effective theoretical CFL shape is an "1I" design - something like a linear fluorescent, attached at one end only, with 100% of its effective surface obscured by nothing.
Real CFL geometries "2U, 3U narrow - 120deg rotated, 3U wide - parallel, 4U, 5U curved - making a sort of bulb shape, a spiral..." have a relevant ratio of their effective surface hidden - radiating inwards. A fraction of the light emitted goes outwards through the gaps between tubes while the most of it (already filtered) has to go through phosphors again. (I mean two phosphors on the opposite tube.) That means relevant lossess, I think.

So, what's the most effective geometry? I think "2U and 3U wide - parrallel" designs have the most of their tubes visible directly, so I guess they're the most effective ones while "spirals and 5U curved" are the worst.
I wonder if it wasn't better to coat inwards radiating parts of tubes by silver (instead of the phosphor), thus reflecting the discharge towards the outwards radiating phosphors.

Well, as you wrote, for the brutto efficacy the most efficient would be the linear tube. It would be not only the most efficient, but the cheapest to make as well. But it will be, the least practical shape... But such thinking is a nonsense, because the lamp is not the complete illumination device. What is important is the total efficacy of the complete system, so lamp plus fixture. There are only few fixtures, where the fixture plays no role at all.

As the CFL main applicaton is in the place of an ordinary incandescent, you should evaluate both the incanescent fixture and the lamp together. So when the lamp does not optically match the fixture, you get light losses. But losses are losses, so you need to generate more lumen for the same task. And more lumens mean more input power. As the performed task remains the same, it means more power for the given task and that is lower efficacy.
Of course, the question is, how much is the lower optical efficiency compensated by the higher efficacy of the light source and vice versa, how much the worse lamp efficacy is compensated by the better optical efficiency of the fixture.
So thereis a compromise: Longer straight tube sections mean higher efficacy, but their larger size means worse matching with the optics, so more light losses in many fixtures.
The spirals maybe not that efficient by alone, but their compactness make their compatibility with the incandescents the best, so the least amount of the already generated light get lost afterwards (in the fixture, or by wrong beam pattern), so at the end with many fixtures thay are the most efficient CFLs.

I've looked at spirals and U-shaped CFLs closely and I can tell you that the DIAMETER of the tubes and the thickness and color-tone of the phosphors play a major role in the amount of light provided. Warm toned 2700K CFLs are grosely under-efficient for example as compared to cool or daylight phosphors. Things are quite complex, as there also seems to be an ideal diameter for the quickest max light output in very cold temperatures. U and linear are much slower to achieve max light in the cold.

Indeed, the narrow diameter and modern phosphors do mean higher efficacy than their predecessors, but these techniques would boost the efficacy of all shapes in the same way. So still a straight tube will have the highest and so on.

And the runup time won't be the longest with the straight tube, on the contrary, it will be the shortest.
The thing is, the tube fill (mainly the amalgam reservoir) has to be designed to ensure the peak efficacy at the steady state temperature, whichever it is. So a tube optimized for 50degC (that would be the case for a thin straight tube) would have at the 50degC about the same efficacy as a tube designed for 130degC (a tight packed spiral) and operated at 130degC. The thing is, each tube has to reach it's optimum temperature in order to be efficient. As for the straight tube it means heating up just to 50degC, it would reach that way faster than the spiral, which has to heat up to 130degC. And moreover the room temperature of 25degC is just 25degC off the optimum temperature for that straight tube, while the spiral has to heat up to 105degC to reach the same efficacy the straight one has immediately upon the ignition.

@lights*plus: "Warm toned 2700K CFLs are grosely under-efficient for example as compared to cool or daylight phosphors."
This is true for LEDs but not for CFLs (surprisingly).
Philips GENIE 11W WW  E27 220-240V 1BC: 2700K / 600lm
Philips GENIE 11W CDL E27 220-240V 1BC: 6500K / 570lm
Osram Duluxstar Stick 15W: 2700K/840lm
Osram Duluxstar Stick 15W: 6500K/800lm

@Medved: "Indeed, the narrow diameter and modern phosphors do mean higher efficacy than their predecessors, but these techniques would boost the efficacy of all shapes in the same way."
Making the tubes narrower while keeping original tube distances could help as inward radiating parts of tubes are directly visible in a higher ratio, due to bigger gaps.

Making the tubes narrower while keeping original tube distances could help as inward radiating parts of tubes are directly visible in a higher ratio, due to bigger gaps.

That could be, but the narrower tube is better utilized to make thelamp more compact (so squeeze them more), so better compatible with the incandescent, so the fixtures designed for incandescent would be more efficiently redirect the light where needed. The tubes are already in rather optimum spacing, spreading them more would make only marginal improvement, but the complete assembly would get significantly bigger.
And quite significant cause of a light loss won't change: Thespiral shape does complicate the  phosphor distribution over the tube surface: As the phosphor is usually applied as a suspension in a liquid carrier, thecarrier has to be at some moment drained from the tube. But the gravity causes the liquid to exitthe bottom wall later, so deposit there more of the phosphor. With a linear tube this could be tackled by very simple way: Just rotate the tube or keep it vertical, so there isno preffered side forthe phosphor to form too thick layer. But that is not possible with the spiral shape. Normally it yield thicker deposition on the side towards the lamp socket, but that mean the lamp shines mostly straight up (with the base down referenceposition), the light going towardsthe base get blocked. They are few methods to reach even coating even with the spirals, but each has some drawbacks:
One possibility isto coat the tube before coiling. That lead to a thicker layer on the inside, which is more an advantage, as it tendto reflect the light outwards. But the shaping means exposing to heat and that heat damages the phosphor to some extend, so reduces it's conversion efficiency.
Other possibility is to propell the phosphor dust by a gas (air, nitrogen,...) and let it adhere (using an adhezive coat upfront and an electrostatic charge) Similar method uses some special oil instead of the gas, the electrostatic adhesion is then the same. But both of these electrostatic methods are not ableto deposit thicker layer, so the phosphor deposited in that way tend to be just on the minimum thickness (too little phosphor means too little UV get converted into the light)
So compare to straight tubes, there is no method to apply homogenous layer of any arbitrary thickness (so the thickness could be really optimized for maximum efficacy without other constraints).

The reason for the higher efficacy of thethinner tube is the fact, the mercury is opaque for it's own radiation (the same energy level spacing cause the excited atom to radiates photons on exactly the same energies the unexcited one absorbs the most). And with thinner tube the photon has to travel shorter distance through the mercury filled gas, so higher chance to not get cought by some unexcited mercury atom, so at the end more of them reach the phosphor...

@lights*plus: "Warm toned 2700K CFLs are grosely under-efficient for example as compared to cool or daylight phosphors."
This is true for LEDs but not for CFLs (surprisingly).
Philips GENIE 11W WW  E27 220-240V 1BC: 2700K / 600lm
Philips GENIE 11W CDL E27 220-240V 1BC: 6500K / 570lm
Osram Duluxstar Stick 15W: 2700K/840lm
Osram Duluxstar Stick 15W: 6500K/800lm

Oops, I remembered one tone lower than the other but the converse.
Actually not surprizing if you convert invisible UV radiation to red and/or green via phosphors.

For LED's, it is rather easy to understand: The blue component is the direct light from the LED die itself, only the green till red part is converted. And the conversion mean losses. So as higher CCT means more of the unconverted blue, it means less losses.

With the CFL's it is not that straight forward, as all of the visible is in fact converted, so it depend on efficacies of the individual phosphor components. But apparently the phosphor components for the blue part have lower efficacy than the red ones, so more blue and less red means lower overall efficacy. Part of that is, the eye sensitivity curve used to determine the lumen output have it's maximum closer to the red side.

@lights*plus: In times when tri-phosphors haven't been invented yet (AFAIK, they actually came with LCD screens), it was more difficult to make the fluorescent light warm white. So I guess, with these phosphors, you could be right concerning colour temperatures and efficiencies. Moreover, both warm and cold/daylight colours were of lower CRI (or perceived light quality), compared to today's tri-phosphors - human skin and some other object looked a bit pale.

@Medved: It's interesting, I didn't know about these phosphor distribution concerns. If they wanted to make smaller CFLs, it's strange why didn't they rather make 5Us (thus making sort of a close cylinder) instead of making spirals. (In fact, a "tulip" shape is this 5U with additional curving, making roughly a bulbous shape. I've seen a few of them but these aren't very popular.)  A fashion trend?

Are you sure there are eye sensitivity concerns taken into account when lumens are being specified on lamp boxes?

The eye sensitivity curve (well, a standardized, average among supposedly all people) is used (according to the lumen definition) to recalculate the power radiated at each wavelength to the corresponding contribution to the light output in lumens. So e.g. the green (the peak sensitivity wavelength) contribute by about 550lm for each W of the radiated power, some deep red components by just e.g. 5lm/W,...
And that is the reason, why 90+ CRI fluorescents have about 30% lower efficacy than the 80+ regular tri-phosphors: For the 90+ CRI you need to radiate about 30% of the radiated power in deep red and deep blue parts of the spectrum. But because the eye sensitivity is there low, that 30% of the radiated power practically does not contribute to the lumen output figure.
So it is by far not true, than these high CRI have lower EFFICIENCY in generating light (they generate about the same radiated power), they do have lower EFFICACY (it is less light). So they do not generate any extra heat as well...
And that is the reason, why you should really distinguish between the EFFICIENCY (= useful radiated power in watts divided by input power in watts) versus EFFICACY (light flux in lumens divided by input power in watts) and never mix these terms up.

Good point!

According to Wikipedia:
In some systems of units, luminous flux has the same units as radiant flux. The luminous efficacy of radiation is then dimensionless. In this case, it is often instead called the luminous efficiency, and may be expressed as a percentage. A common choice is to choose units such that the maximum possible efficacy, 683 lm/W, corresponds to an efficiency of 100%. The distinction between efficacy and efficiency is not always carefully maintained in published sources, so it is not uncommon to see "efficiencies" expressed in lumens per watt, or "efficacies" expressed as a percentage.
The luminous coefficient is luminous efficiency expressed as a value between zero and one, with one corresponding to an efficacy of 683 lm/W.

And a press release at Osram.com (that breaks this rule):
OSRAM Opto Semiconductors has set a new laboratory record of 142 lm/W for the efficiency of a warm white LED light source. With a correlated color temperature (CCT) of 2755 K the LED achieves a good color rendering index (CRI) of 81.
...
“If we explore this technical approach further and allow deviations from the Planckian curve we should even now be able to achieve higher efficiency values of up to 160 lm/W for a correlated color temperature of 3000 K (cx 0.45/ cy 0.44)”, said Dr. Norwin von Malm, Predevelopment Manager at OSRAM Opto Semiconductors.