Multiplayer on Xbox?

I got D3 today because I want to play it with a colleague. However, neither he can join me nor I can join him so we can play together. I can invite him and he can also select join, but it only loads briefly before apparently giving a connection error. Both of our connections are good, so it shouldn’t be that. I play on an Xbox series x and he plays on an Xbox series s. There is no error code on the error. Just “A problem occurred while joining the game”. What can we do?

You can check your router settings. I had similar problems, when I was behind double NAT, and my colleague too. When I connected my XBOX to different network (mobile hotspot), I was able to host the game.