There area few technologies at play when it comes to how servers interact.
First, you have connected realms. These are, in effect, merged servers. (But in a way that didn’t cause collision between characters on different servers with the same name.) Connected realms share an AH, you can freely trade and mail stuff with players from servers connected to yours, and you can join guilds from connected servers. The easiest way I’ve found to see which servers are connected is to look a the forum’s realm forum list, as connected servers have combined forums. Generally, the larger servers are not connected to any others, and most connected realm clusters fall more in the medium range for population.
Next, you have sharding/CRZ (cross realm zones.) While technically, the two are a bit different, for practical end-user purposes they are used interchangeably most of the time. This technology allows Blizzard to maintain target population levels in zones by pulling players from a pool of servers. So, in lower population zones, it can put people together from different servers. In high ones, it can split people up to keep servers stable in high load times. While you can group up with players across realms, you can’t trade with them or join their guilds. (Think of it like LFD or battleground groups.)
Sharding still obeys realm type. So, RP realms only combine with other RP realms, and normal realms with normal realms. (Also, they appear to also group by data center, so the LA and Chicago servers shard separately.) Also, RP servers do not use sharding in non-current content. This is to allow for easier RP events, and sharding generally caused issues for these. (Although sometimes there is collateral damage, as it appears that all of Pandaria is currently sharding, even though only the Vale of Eternal Blossoms has current content.)
Finally, there are cross-realm groups. These are player formed groups, and can be joined by players from any server in a region. (So, you can group with players regarldess of data center or realm time this way.) You can do pretty much everything in these groups. The one exception is cutting edge Mythic raiding. Mythic raids only open up to cross-realm groups after 100 Horde and 100 Aliiance guilds have killed the final boss of the raid on Mythic.
So, what all of that means is that the main things that servers matter for are the AH, the variety of guilds available to join, and Mythic raiding.