The story is called THE TRANSFER by the mysterious, mystical Algernon Blackwood, and at its heart is the psychic vampire known as Mr. Frene:
For this Mr. Frene was a man who drooped alone, but grew vital in a crowd-- because he used their vitality. He was a supreme, unconscious artist in the science of taking the fruits of others' work and living--for his own advantage. He vampired, unknowingly no doubt, every one with whom he came in contact; left them exhausted, tired, listless. Others fed him, so that while in a full room he shone, alone by himself and with no life to draw upon he languished and declined. In the man's immediate neighborhood you felt his presence draining you; he took your ideas, your strength, your very words, and later used them for his own benefit and aggrandizement. Not evilly, of course; the man was good enough; but you felt that he was dangerous owing to the facile way he absorbed into himself all loose vitality that was to be had. His eyes and voice and presence devitalized you. Life, it seemed, not highly organized enough to resist, must shrink from his too near approach and hide away for fear of being appropriated, for fear, that is, of--death.
So this was how I saw him--a great human sponge, crammed and soaked with the life, or proceeds of life, absorbed from others--stolen. My idea of a human vampire was satisfied. He went about carrying these accumulations of the life of others. In this sense his "life" was not really his own. For the same reason, I think, it was not so fully under his control as he imagined.
Frene is a man who destroys your vitality by being in your presence, and he plays a key role in the book Voice of the Undead.
Give the Transfer a read online-- or look for it in the collection The Vampire Archives.