I've noticed these particular tiny anemone's population can fluctuate with the quality of the water but even at their max population I've never found them at all irritating.
I would not necessarily agree with this. While I agree that they are rarely an issue, I have seen many tanks with plague-like quantities (thousands to tens of thousands). Combined with how strong their stings can be, I would never dismiss their presence. Just keep an eye on them.