If a predator bites a bird or mammal then the animal is likely to die. A poison is unlikely to save the organisms life. Almost no birds are poisonous, and mammals are very rarely poisonous.
A poisonous bird or mammal might through its poison save the lives of its relatives, perhaps even its children. So the poison would confer some evolutionary advantage on the animal, but developing an immunity is likely to provide even more of an advantage for the predator. So in an evolutionary arms race, the predator might win. The predators would evolve immunities faster than the prey evolved poisons. On the other hand for the colonial animal or plant, the prey survives because of the poison. So the prey might win the evolutionary arms race. All of this helps to explain why plants and colonial animals are so commonly poisonous and birds and mammals so rarely poisonous.
There are many examples of this phenomena. Plants are like colonial animals in that removing leaves or a branch does not necessarily kill the organism. So this hypothesis suggests they should be poisonous, and indeed all or almost all plants are. Humans are immune to many of their poisons, herbivores, like deer, are immune to many more plants, but strict carnivores, like cats, will die if they eat many of the plants deer and even humans safely eat.
Colonial animals, for example, sea anemones, jellyfish, coral, and most sponges are poisonous. Bees, wasps, and ants are also colonial and they also have a poisonous sting. The colonial insects are very active while sponges, coral, and sea anemones are not. So they are very different, but the fact that both are colonial means they both use poison or venom.
Starfish and other members of the phylum echinoderm are often poisonous. Like the colonial animals, if they suffer injury, they can regrow damaged parts. Unlike many mammals and birds they do not rely on flight or speed to survive. So the first bite of the predator does not doom the animal. If the predator quickly realizes its mistake the animal can survive.
It is often asked, why aren't all animals poisonous. This hypothesis may provide part of the answer. It is not the whole answer. Obvoiously, there are many animals that are poisonous and are not colonial.