This is generally the line on “defenders” in battlegrounds:
I’ve definitely defended towers before in AV, especially as a rogue even one person hanging back can make a big difference.
But, if your whole team is calling to have everyone come down for a big push and a person or a few people are still hanging back despite not having much more in the way of an objective that would help move towards a win to do or that won’t be impactful to the game, they might decide that you’re AFK. Basically, it comes down to if other people in the BG think that you’re deliberately not helping and going along with what they need you to do, and are doing nothing instead, they might report you as AFK, and Blizzard might agree.
If the rest of the team is calling on you to come help, it’s better to go help than stick to doing your own thing, because BGs are a team activity.