Chances are that they are not compatible with each other. It might also be that they are secretly in love with someone else and are yet being forced to marry someone else. In such a situation it is natural for the male or the female to stray away from the person they have been forced to marry and spend time secretly with those whom they really love. Love at first sight is something that should be relegated from out society. It is not love that bowls us over when we first come across an attractive person of the opposite sex... it is desire.
Having a passion for someone of the opposite sex and being attracted to them at first sight does not mean that it is love. Chances are that the other person might not be compatible in the long run... post marriage. It is then that philandering takes place. There are countless reasons why married men and women stray away from each other and enjoy the company of someone else. The main cause is compatibility. They never spend enough time with each other to get to know each other well enough. By the time they find out that they are not made for each other, it is too late... and this is when they start straying away from each other to enjoy somebody else's company. This is why people often seek a sex friend.