Testicles are very sensitive and even minor pressure or injury can cause pain out of proportion. I cannot exactly say that there might be any problem with your younger son in this case without physical examination. His crying occasionally might be related to examination by your wife.
But as young children cannot properly explain symptoms, it is better to get him evaluated by a doctor in this case. Testicular pain or tenderness can be due to Infections like mumps which may cause orchitis. Some unnoticed injury in play may cause inflammation and tenderness. If in case of severe pain, testicular torsion should be considered and is a surgical emergency. If you notice any lump or swelling on examination, consult your doctor for examination.
Physical examination when done gently does not cause any problem and there is nothing to worry about fertility. Avoid tight clothing and exposure to heat.
Hope this helps. Do get back to me with any further queries. Thank you