Imo there should be an optional message for the invite so you can easily group up for different activities without unnecessary chat.
Alternatively there could be a party description which players can see when they are invited. With this simple change random party invites wouldn’t be random anymore. Because of the party description you know exactly what’s going on and it’s easier to decide if you want to join the party or not.