I've thought about this a lot and watched it twice, just to be sure, but I have to agree with you. I think both of them were a little shady, but when it comes right down to it, Simon constructed this foundation of lies for their marriage - that he could NOT have children, and from their conversations, he knows that Daphne knows NOTHING about sex, which means he is willfully misleading her.
That, in itself, is a major violation - to tell a woman you are infertile and then have unprotected sex with her, even using the oft-unsuccessful withdrawal method is a serious violation when she is the one who would have to carry and birth the child. Granted, no one seems to care about this because Daphne DOES want that, but still - she should be able to experience pregnancy will full consent and disclosure.
If I hadn't watched it with the captions on, I would not have heard any "Wait" from Simon - it kinda sounds like he utters it at one moment, but again, that's unclear if he's saying "Wait, I don't want to come inside you," or "Wait, I don't want to come YET."
Honestly, I'm dismayed by how many reviews I've seen, even on major media outlets, that are calling this rape and labeling Daphne as a villain. I didn't even really like Daphne, but I can't say that she did anything worse than maybe some low-grade manipulation in response to being sent into a marriage with no education about childbearing or sex by her mother, and also deceived by her husband. I wouldn't call her completely innocent, but she's definitely not guilty.
I don't understand why the show had to create such a complicated moral tapestry that weighs heavily over something that seemed to have been intended as a fun fantasy. Seems odd to me.