Nintendo Switch lagging with 2 man group

I play D-3 on a nintendo switch. When I group with my brother one of us lags to the point that the game is unplayable. If I start the game and invite him, I am ok but he lags…If he starts the game and invites me, he is ok but I lag… Playing solo there is no problem. We have tried different characters at different times and we have both taken turns starting games and inviting each other and it’s always the same thing. The person doing the inviting is ok and the person being invited lags. What is going on? Please don’t refer me to the minimal post that I have already read…They did no do anything to answer my question and fix my problem…How can just 2 people lag a game to the point that it’s unplayable for the one that was invited. please help…Thank you

Unfortunately my husband and i also play on the switch and we are having the same issue i can’t figure out how to fix it.