What components from halophosphor and triphosphore are dangerous by themselves? I know that early tubes contained berilium.
First their heavy contamination with mercury, but I guess the question was placed with hypothetical assumption they are not yet contaminated from the lamp operation.
First generic for all chemicals, include those that are not toxic by themself: Unless they are not certified as non toxic (food or cosmetic grade,...), there is no guarantee their production process wont leave there toxic contaminants (residues from other components of raw materials, catalysts used during production reactions, side reactions forming certain percentage of unintended compounds that could be toxic,...). An example is sodium hydroxide: It is safe and approved for food use (as pH regulator), but only when manufactured by using the membrane electrolysis. The other common method for NaOH production is using mercury electrode electrolysis. Although the main product is the same NaOH, obviously when produced using the mercury, it is almost guaranteed the product is contaminated by the mercury (or the contamination can not be excluded).
From this aspect whatever was in the flourescent, should be treated as containing mercury, so toxic just because of that.
And other aspect is, the composition of the main phosphor materials is quite broad, selected with the light output and generaly performance as lamp phosphors as the main objective so to me it is very unlikely all components are really safe.
The thing is, there are not only the main materials, but as well the dopants forming the different colors.
And if you take asbestos example, a material could be 100% chemically inert in the body (as the asbestos is), but its physical properties could make it extremely toxic anyway (the tiny needle crystals in the asbestos case).
So unless proven otherwise with the exact material, I always treat these as potentially dangerous.
If you prevent inhaling/ingestion of safe substsnce, nothing bad could happen, just a bit of inconvenience of using the PPE when mabe not needed.
But if you wont prevent a real toxic substance from entering your body, you could be in big, unrecoverable troubless very fast.
Plus all this is irrelevant in the case of fluorescent remains, asthere the phosphor dust definitely is contaminated by Hg, so it becomes toxic regardless the principal chemicals of the phosphor alone.