I can relate to your confusion. When I was younger, I don’t really feel things a lot. I can always distance myself from my emotion and seems very reserved and cold. But thinking back, probably it was because I had my personal struggle and I don’t trust anyone. Unconsciously, I built myself a wall to protect my feelings.
But INFJ is a natural empaths. For me, I need to be surrounded by my own ‘people’ and feel secure first, before I can truly be emphatic. Reading a lot from people is a gift and liability because you absorb the negative energy. You need to be strong enough to take that. That’s why when I’m in pain, I lost the ability because I don’t want to take anymore emotional burden.