Unfortunately, it’s best for matchmaking to sometimes substitute you into a match as a backfill. With that said, we would like to make some improvements. These are tricky and take time.
We did recently fix an issue that has not gone live yet. Once you serve as a backfill, there is supposed to be a cooldown on subbing you in again. But currently if you switch modes a lot (i.e. from quick play to mystery heroes), you can be backfilled multiple times in a row. In an upcoming patch, this will be fixed.
Again, we want to do more improvements to backfilling over time, but simply removing the system is unlikely to happen.