Thanks for your consult at DoctorSpring.
From the description that you have provided, it seems that you might be getting folliculitis which is a bacterial infection of the skin.
The best treatment for it would be application of neosporin cream for a week.
However, its difficult to diagnose a condition without actually seeing it. Hence if you could send a picture, it would be very helpful.
In case you can't send a picture, I would request you to visit your dermatologist if you get no relief with the cream.
Please let me know if you have any queries.