If people think that Evelina is a prostitute, this could seriously damage her reputation. While Sir Clement claims to care about Evelina, really, he only cares about getting his own way. Instead of trying to help her when she was lost, he tried to take advantage of her situation. Lord Orville, in contrast, is genuinely concerned about Evelina and tries to remove her from the situation for her own sake, rather than to trying and get something from her.