First of all we should consider the amount of toxin present in the mushroom and then its relative potency. Few substances in the natural world are so toxic and present in such high concentration (and these the layman is quite unlikely to encounter) that one need worry about getting it on your hands or even tasting a smidgeon. Incidentally, when I taste a mushroom, I literally do just that, not swallow it. Mushroom toxins work by being adsorbed through the intestinal tract, not through the skin. Then the toxins need to be transported to places like the liver or the central nervous system to hurt you. If you have just picked a few Amanita virosa for the sorting tables, you needn't worry about sitting down and eating a sandwich with your bare fingers because the lethal amanitin toxins are simply not present in that great of a concentration in the fungal tissue.
It is commonly accepted that A. phalloides is one our most toxic mushroom. It has been found that the lethal amanitins are present in only 3-5 parts per thousand and indeed, some specimens of A virosa and A. verna have no detectable levels. In other words one ounce of fresh death angel might have less than a fifth of a gram of toxin, an amount meaningless unless we know something about how potent the toxin is. Relative toxicity is often expressed as an estimation of the least amount of poison that would cause death, or the minimum lethal dose (MLD). A better method is to express the dose in a statistical way to minimize the fact that individual people will vary in their susceptibility to a poison. Such an expression is calculated on how much toxin would be needed to kill half of the people, if each person ate the same amount, each weighed the same, and none had any predisposing illnesses. That amount per person would be called the lethal dose 50 % or LD50. Expressed in weight of toxin per weight of individual ingesting the toxin, the LD50 for amanitoxins comes out to be in the neighborhood of about one 2 ounce mushroom for a 150 pound man (LD50 = O.1 mg/kg). It would take 10 small Galerina autumnalis to equal this same dose. In other words, if you ate this much of one of these two mushrooms, you would have a 50-50 chance of succumbing. Much less would be needed of course to make you violently sick, but this would still be appreciably more than what trace amounts might stick to your fingers when handling A. phalloides.
Even so, I do not recommend tasting such species although I wouldn't hesitate to do so (BUT underline that word "taste"). IN SUMMARY: By all means exercise care in identifying, eating, and handling various mushrooms, but at the same time, use some common sense to avoid unwarranted fear of them. Simple handling of even the most toxic mushrooms won't hurt you.