Thank you for your query at DoctorSpring.com
There are only 3 diagnostic criterias for tuberculosis. Firstly the Mantoux needs to be positive, however 30-40 % of the adults will turn Mantoux positive, only because of a subclinical infection in their childhood. So the results of Mantoux is not considered relevant.
The next is Sputum AFB, and that will definitely be negative for you, considering there is no recent infection. It does not depend on past infection.
Now the most important is Chest X-RAY. Since the doctor said chest x ray is negative, there is no way they can find out about a past TB infection.
So please do not worry.
Feel free to discuss further,