Would this trading system work for D4? It combines free trade with BoA

General Discussion
61

Just because the D3 approach has issues doesn’t mean that using an AI is the answer.

That said even with the AI people will get frustrated with nerfs. People simply don’t like nerfs, even though they’re needed.

If you’re going to design an AI to do a job and then go in and reverse what it did, it seems like the AI wasn’t super useful in the first place.

That’s kind of the problem. If you let the AI make everything equal then everything being equal is what removes being rewarded and making choice not matter.

and if you go in and touch things up by hand, then you could have just let the developers do the job since they needed to do half the job anyway.

62

Yeah, somewhere perhaps in conversations with others, it seems a concensus was built that AI balancing would just make everything homogeneous and the same power levels - I think that’s a misinterpretation of the core sources with both the SC2 project and the reddit D4 post.

The D4 game would just expose APIs where the AI system would let Blizz staff set any power levels they wanted on various builds - they could set imbalances, as there are now, if they wanted. Just as easily as they could set many of the builds to be on par.

In that sense, it’s just a tool/system that saves them a ton of time because it can go about simulating all the various changes in-game without the need for tons of human hours spent playtesting. It could also alleviate the need for PTR in general. There are games that don’t need a PTR although I would still have one just to be safe until it’s proven out.

63

Yet it’s the way to keep a game fresh and competitive without adding any new content. I’ve said many times a well designed aRPG don’t need any content at all. It’s just a hamster wheel that’s changing form/shape each day. If they decide to add extra stuff in the wheel - the better, but the wheel has to work without extra stuff always.

What I quoted on reddit aren’t mine writings although I spoke about how AI can change aRPGs many times in the past, the AI vendor trading idea dates back to 2018. We’ll see what they learned in D4, and if we don’t - well, we’ll see it somewhere else (there’s usually more than one person discovering the meta, sooner or later).

What you describe about Roguelikes is exactly the Seasonal experience in aRPGs, lasting a few hours or months is not the point here.

I’d say from one side such randomization would help them if your game is well designed, from other side it might bury them if it’s badly designed. It will nevertheless make the tutorial way more interesting each Season since the player would be taught new things due to the randomized variables.

You need both approaches and although you yourself don’t enjoy the WoW balancing style it doesn’t mean all players are like that.

An optimal design would include the following:

  • Developers setting goals/rules/margins how builds/items have to perform so that we have adequate power tiers serving for most engaging MTBU curve for players going through the game
  • Developers together with data analysts examining players’ participation/gameplay moving builds among the different power tiers
  • AI guaranteeing the design/progression is as intended by adjusting the power level of the builds/items going off the margins
  • AI further fine balancing the builds/items among each power tier so players are presented with a real choice

An optimal design for a next gen randomization aRPG would further include:

  • Developers setting margins for variables in the game
  • AI shifting the variables for each account each Season
  • AI guaranteeing ultra weak/powerful combinations are excluded

The competition in the next gen aRPG then would consist of top 1k players all receiving the same variable combination for the last Season each year, which will be like an yearly world cup.

I completely agree. It’s also a tool that can be used in different projects/games and one that it might happen for you to need it most when you don’t actually expect. Going in that direction is something a wise person would do.

64

Lasting a few months or a few hours is a pretty important distinction in the context of this discussion, though.

Diablo is not a roguelike despite a few inspirations it took. Constantly randomizing all the stuff in the game wouldn’t be all that great. There’s a reason why the game got away from even randomly dropped spell books.

It’s also going to make it more difficult to enjoy things.

Did you have fun with that build you used last season and want to play it again? It doesn’t exist anymore because the AI decided to change the items and skills around.

Finally learn how to play your Demon Hunter well? Sorry, it’s basically something entirely different now. Start over.

What I’m hearing is that the AI isn’t really going to save time because you’ve still got all your developers there, and you still have to actually develop and maintain the AI itself.

You’re basically using it to try to artificially do more or less the same job the developers would do without removing the developer from the equation because they’re still needed to maintain the AI.

The AI isn’t something that takes no developer resources after all, not even after you’ve finished it.

1 Like
65

I would like Diablo to take a bit more inspiration from rogue likes/lites. But even rogue likes are not doing what Skelos is arguing for here. All the skills etc. work exactly the same way each run. The randomization comes from what you have access to. But what you learned in run 1 is still useful in run 100, as the Sword weapon still works the same way.

Randomizing the rules in a rogue like would kill it just as much as it would kill Diablo.

66

You can do this in NS. On Season if you decide to play it again it might be buffed/nerfed depending on how the randomization went.

67

I feel like Diablo is at maximum roguelike right now. I want to be able to plan out my character, which you can’t do in roguelikes because you have to make do with what you get.

That’s not something that sounds fun for a character I’m going to play for several months.

It sounds like a game where the meta is going to be rerolling characters until you get a perfect or near perfect roll and just abandoning most characters half way through leveling them.

So after all that your plan is to just use the AI to randomly increase/decrease a few values?

68

Yes, in the given by the devs margins. And of course the AI vendor trading, which would also be there together with the balancing.

69

Well that’s just rather basic.

Not that it’s a good idea, but you certainly don’t need an AI to do that. Any developer can write you a randomization script that can intake parameters pretty easily.

70

You also have balancing and item evaluation. Why we go circles here? We get you don’t like this, but your arguments are weak. AI is needed.

71

I’ve already come up with arguments against the other areas you want to use the AI but you just keep insisting that’s not how the AI would work because you’ve never clearly defined what the AI’s job exactly is going to be.

and you’ll just fall back on “but somebody else will do it first!” like that is supposed to mean anything.

72

The characters should definitely not themselves have any random elements. Mostly talking about not being sure which items you find (and no trading etc. to give you exactly what you want), so you have to “play with what you get” as in rogue-likes.

73

Even then I’d prefer gear to be a bit more integral to my build which means needing to be able to target very specific items.

Playing with what you get just doesn’t work as well when a playthrough might last over 1000 hours, imo. At that point even if there is randomness in it, I want to be able to build towards very specific items.

74

I looked at some deep reinforcement learning (RL) white papers and RL seems to be a good fit, for many of the topics you’ve discussed.

With RL, you have goal-oriented algorithms that can reach goals using multiple layers of neural networks, by weighting actions leading to the goal. In the case of Starcraft or Diablo 4, Blizzard exposes an API on their Agent (the character) for programmatic control.

An example, using D3, might be beating Belial. If the RL Agent is standing in a poison pool and dying it would decrease the weight for those actions leading to dying, eventually weeding them out. If it’s using a particular skill or timing to cast a certain skill that is used on the way to beating Belial, it would increase the weight to strengthen that skill or timing. Very simplified but the Agent can “learn” how to beat objectives by repetition and weeding out actions that don’t work and promoting actions that do work, like step-wise refinement, it just gets better and better by gradually weighting actions up or down.

On topic, with RL Agent, they could have all the items and sets and gear in the game as part of the system to be used by the Agent. The Agent, on it’s own, could use all the various sets for Demon Hunter for example to see which ones work best for which encounters, in T16 rifts, in GR120+, and so on.

Because the RL Agent is in a scripted virtual environment (with some RNG affecting boss moveset and monster movesets, etc) it would actually, based on what I’ve read, train extemely quickly by just looping over and over until it’s reached it’s success goal of defeating the rift, GR, boss etc or failed, knowing that it’s power level or gear is not sufficient to reach the goal.

By doing the above with all the classes, and items, and gear combinations, it would be able to show, as a rating metric which builds are better in a very precise way for T16, GRs, and so on. Goals for the RL Agent with T16 might include keys, DBs, gold, gems and other factors as well beyond just beating the content as quickly as possible.

As it’s going through all the gearing combos it would have on Sage’s or Cain’s at some point for T16 goals as example. And those sets would lead to strengthening the weights on using them because they yield more DBs and keys and so on.

I’m confident that this RL system would come up with a very accurate rating of items and gear in the game because it has hard data of their effectiveness at beating objectives. Afterall, it’s used every class, every set, every item, combination in the game to beat objectives. It knows what works and what doesn’t work.

So there’s your item “effectiveness” rating system - the product of an RL Agent’s work.

And there’s your tiering because the tiering is just dividing up the items and their ratings in rough tiers - those ranked highest in top tier. Those ranked lower in lower tiers. This exercice also showed that categories would be needed for T16 and GR push and PvP and so on. Different goals are best served by different gear.

So by adjusting my OP to include your AI idea in the specific form of an RL system to rate items, I’ve eliminated the balance team needing to rate items.

We now have an automated system for trading with accurate item ratings and tiers. It’s BoA, it allows free trade, and you have to have something good to get something good. There is at least, some level of RMT deterrent. There will always be RMT on some level with FT but this system dramatically reduces the incentive.

The core precept of needing something good in the first place to get something good in return, seems to hold up well.

75

Lets say we have item A, and item B. Both items are really good end-game items, that can help builds do the hardest content and similar efficiency.
However, item A is used in 1 million different builds across all classes. Item B is used on 4 different builds (out of 1000 billion builds or whatever).
Should the AI consider these two items to be in the same tier, since they have proven to be equally powerful? Or does it need to take demand into account (and also supply for that matter, as in, if item A drops 1000 times more than item B, should it affect its trading value?)

As well as the aforementioned issue of what builds are difficult to play. Or even fun to play. An AI might struggle a fair bit determining that.
Should item C and item D be valued the same, if they have the same power, but item C works in a easy to play build, while item D only works in a build that basically no human being is able to play properly.

76

Once we have such AI system in place we don’t need open player trading at all since it would be x10 times faster for the player to just trade with the town AI vendor who basically can sell you any item (you specify what you want from a list choosing properties etc) and who buys anything from you for the respective price. This way the bartering aspect is totally eliminated and RMT is dead. This was the initial idea from 4 years ago.

It will need some time to adjust once the Season starts. After that it should take anything into account and the more advanced your character is the better prices you’d get on the items since that would essentially be your upgrades in the end game.

77

I like 2 aspects of vendor idea - defeating RMT completely and the speed. But I think trading is mostly a social thing for some people especially in D2 community. In baal runs, something might drop in the waves before Baal that’s nice and tradable - people could trade right there. Or trade in your D4 clan - there’s a more social element. And at least for D2, D3 I think people are used to vendors having useful items but nothing like uniques or set items etc. Interesting though.

With an accurate item-ranking, I think they woud base their drop rates primarily on the item rank. More powerful items would drop less frequently. A D4 Mythic for example would be incredibly rare compared to a blue or yellow and so on.

So the drop rate and the actual in-game item quantity may not even be needed for tiered-trading using the trading window. And with that simplicity, perhaps players intuitively understand it better. The tiers are just based on item-ranking done by the RL Agent.

Couple other things came up looking at some other projects that used RL Agent systems. One of them was the DOTA OpenAI project which, beyond solo play, involved teams playing other teams in PvP. Which brought up needing up tp 4 Agents in the case of D3 to simulate 4-man runs which are just PvM instead of PvP obviously.

So the RL system could use 4 Agents and would just vary the 4-man class and gear combinations until it found those that completed objectives most efficiently. In the process it might find that certain items were strong for buffing the party, pulling or grouping mobs, and so on, ranking them accordingly.

So again with classification we might see some items ranked highly that might suck for solo play but would be top-tier for 4-man, and vice-versa.

One other thing is that D4 team has a hard requirement for items: “We want players to get the vast majority of items by killing monsters”.

So they don’t want people trading too much - they’ve learned their lesson from RMAH where players played RMAH more than the game itself.

With that as a hard requirement, you might only allow a certain number of trades before your trade window becomes greyed out. Or items could only be traded a certain number of times (requiring new drops to come into circulation since old drops could no longer be traded) and so on. You might even have to find some kind of mat to place into the trading window to allow a trade - if you don’t have one, no trading. You would then have to literally kill monsters or beat objectives to get mats that are needed to trade - something that solves the “trading too much” problem in a direct way by requiring killing monsters to trade lol.

The easy or hard to play build deal - another good point and very subjective one that might be different for different people. Might have to leave that unaddressed and allow that discrepancy to play out, see how it goes.

78

That would presumably be hard to do, with rare and legendary items, due to their random affixes. Unless you want every single permutation of an item to have its own unique droprate.
Like Legendary A scales well with crit chance, so crit chance has a low chance to appear on it. Legendary B scales badly with crit chance, so crit chance has a high chance to appear on it. Etc. Doable, but it sound like an awkward system.

Even more so when balance changes over time (each night according to some people).
That high dropate item of today might be the best item in the game tomorrow, when the AI realized how terrible the item was. Smart traders would hoard these items, so they could trade easy to get items later on, in case they randomly increased in value.

79

Yes, on theory we can make the same town vendor mechanic work with P2P trade too efficiently (without players losing much time) if we have a proper currency in the game.

Say, you just got Shako and I have Ist, we open the trade window and the AI evaluates that I must add 500 gold besides the Ist to get the Shako.

Through the AI town vendor I can more easily buy the Shako with say worse stats only for an Ist.

We can make both work for sure, but I think that once players realize the comfort the AI vendor gives they won’t ever P2P trade lol.

80

Yeah, I think random affixes and their rolls are just simple calculations though. A D3 rare ring with 500 main stat, 50CD, 6CC, sock obv being higher dps tier as example.

Tiers, in one sense, are just percentages of power or utility - they could be grouped fairly tightly or loosely.

2 rare rings for example may be placed in same tier if they are 450main stat/40CD/5cc/sock and 480ms/45cd/6cc/sock - they would have to choose percentages (how close in power items have to be) for each tier. 450/30cd/4cc might be in lower tier than those other 2. They might allow items to be farther apart or need them to be closer together to be in same tier - staff could choose as they see fit. The looser the tier grouping the more incentive for RMT.

With item ranking, just using D3 as example, they would make new set and item changes in between seasons and use RL Agent to rank items so as not to affect leaderboards during season. Since the RL Agent’s ranking is accurate - it’s not going to change during the season. It’s already gone through a million iterations of ranking before the season started and so on.

The staff would have to intro a new item or change an item or set in order for the RL Agent to need to change rankings.

In the last few comments I’ve been talking about item ranking for the trading solution rather than balancing and all that. So for this thread being trading, I’m more focused on trading and ranking.