From what Tom has said thus far, it's clear that whatever happened at the Brennan's old home affected every member of the family, not just Daniel. This begins to show that, per the logic of the novel, families are a cohesive unit that experience failure and success together, not as individuals. Tom's belief that Father Vincent is staring at them shows that Tom is carrying a fair amount of guilt and is self-conscious about whatever happened, given that there's no indication that Father Vincent is actually staring.