I’ve done my fair share studying and analyzing AlphaGo, years ago, and I stumbled upon the theorized weakness. In AlphaGo, the AI’s goal orientation was about winning, not winning by a large gap, in which that was the normal standard of winning.
In AlphaStar, is it similar to that style? Does it play based on just win, regardless of margin, or does it try to win while maximizing in win margin?
Because, I’ve asked myself, the same question. IF someone told me to win against another Starcraft player, I never ask by how much of a margin should I win (that kinda sounds somewhat condescending and arrogant), but I try my best to win.
Few problems occur when you create a certain win scenario.
Win by big margin = big risk -> big reward, which is risky gameplay.
Win by small margin = low risk -> low reward, which is more conservative.
These two are both Win-Oriented scenarios, but an AI doesn’t “get” to choose to win by playing it’s best, since it’s all done in calculation.
This was the biggest flaw in AlphaGo, after Google’s Deepmind team figured out that has these Win-scenarios.
Is there a link or group, specifically for AlphaStar questions, that anyone can direct me towards?
I’m very curious.
Alphastar is basically just a really complex statistical model, that’s all that machine learning really is, and Starcraft 2 is very difficult to model. As a result, the strategy that AlphaStar employs is to simplify the game into things that it can model easily. Having good macro is something that is extremely easy to model so naturally it is very good at macro. Micro is also extremely easy to model ( at least well enough to beat humans) so it focuses on that. However, when it comes to even basic decision making in the game the ai falls flat on its face because it can’t model the more complex aspects of the game.
As an example, I saw a game that it played as zerg against a Terran player and the Terran player went for proxy two racks. The decisions that made were completely absurd. There was a time for example when the bunkers were completely empty and it just sat there, scared of the bunkers.
It doesn’t actually understand how bunkers work and it’s statistical model have not been trained with what options are available for that scenario and which are the best options so it just couldn’t handle that situation. The thing about reasoning is being able to take knowledge from a different situation and extrapolate it into a new situation and this ai cannot do that even in the slightest. This AI is simply a mapping inputs to the statistically most likely outputs.
I wonder what AlphaStar’s league looks like.
because, we as humans understand that bunkers is a placeholder building concept.
Bunkers possesses only 1 micro ability, target fire. But it seems like AlphaStar categorized bunkers as a building/production, that it doesn’t produce any units, and believes that it serves no purpose.
Or, it’s scared of bunkers because it does not know what units are in it, that could potentially destroy its army. like PvT, alphastar may think there is a ghost in it. Or in a surprising situation, AlphaStar has a specific knowledge about bunkers, we currently don’t know about.
My understanding of bunkers, it’s not worth attacking them in a financial sense, because if you destroy it, they lose 100 minerals, but if you attack it and lose 1-2 units in the process, and Terran decides to salvage it, Terran comes out free of loss, while dealing more damage.
Well i think it is logical alphastar should aim to maximize win probability, without risking much as possible, because alphastar cannot assess its opponents how good, or bad they are and how they play, unless they allow it to analyze peoples replays how it would know to do calculated risks. That it would be interesting how it would choose to play against specific player. I don’t see much point of risking otherwise… But as he said it cannot yet maximize magin, or risk that well atm even if it wanted, as it didn’t encounter all scenarios yet and learnt. We will see later i guess.
that’s in starcraft, and without calculation.
When you sell something on eBay, do you sell something using an RNG from google, or do you try to calculate how much money you should get back in order for you to be satisfied, selling gtx 1080ti for 100 dollars, or 500 dollars? There is a big or small margin. if you try to sell something for like 500 dollars, while brand new is like 450, you are taking a big risk that someone is dumb enough to purchase something for 500 dollars. because time is an issue, and you can’t hold an inventory for an exorbitant amount of time. (selling for 100 vs 500 dollars, is still getting money aka winning as you actually get money, but at some point you are in the reds even though you made money).
Here, we have a very similar scenario. Go risk big like 4 base expand, or go for 2 base conservative expand build. if you go 4 base expand and you end up winning a massive margin, like having more than 8k resources, and having 200/200 supply vs 150/200, or would you rather go 2 base expand and you end up winning in a base trade scenario where the supply is 12 vs 12.
In both situations, you win. By your quote, of what you specifically said, “In Starcraft there is no such thing as win by a big or small margin.” I’m going through a more detailed situation than just “win/lose situation.” Because, how you plan to win is the starting point of where you end up losing.
If you’re trying to win, you’re not only trying to win, you’re also trying not to lose.
Therefore, there are two values (which are pretty similar to truth values btw.): the one is “win” (W), the other one is “loss” (L).
You can basically look at an AI’s “decision making” as a big, vivid decision tree. In a certain game situation, there are a lot of possible decisions. From the get-go the AI doesn’t know which consequences a decision will have, but those consequences are binary, they either result in W or L. If a decision eventually results in W, it will be prioritized afterwards, if it results in L, it gets marked as less valuable. Thereby, the value of decisions get’s established - decisions which lead to W most of the times get “stronger” (the AI is more likely to chose them), decisions which lead to L most of the times get “weaker” (the AI avoids them most of the times).
Thus, you get a certain balance, a kind of a “dialectical” decision making: the AI tries to win as much as it avoids to lose. It tends to create the highest possible W-value of his decisions over the course of a game, but that also means it tends to create the lowest possible L-value. Therefore, it doesn’t “take a risk” at all: it simply considers if a decision results in a win or if it results in a loss most of the times and if there are other decisions which resulted in a win more often than the eligible decision.
From what I’ve seen, Alphastar seems to have a really hard time against cheese and early pushes. Either it’s because of the lack of information in the early game, especially, when scouting is denied; or it’s not “used to” it, since it tends not to finish the game early and might therefore not have collected enough data during his training and testing period and is therefore unable to “rate” such situations properly.
Well that’s just a simple matter of probability. Let’s say that how much a customer is willing to pay for a product is logistically distributed. If the product price is too low they will think something is wrong with it and if the price is too high then they won’t be willing to pay that price. So as you get farther and farther away from the mean price it’s going to take longer and longer to sell because there are fewer and fewer people willing to buy under those circumstances. Ergo, by maximizing your profits per item you are also maximizing time that it takes to sell because you are restricting your pool of potential customers and if you restrict yourself to say 1% of potential customers and then it will take 50x longer than if you could price it at a point that would appeal to 50% of potential customers. That’s maximizing price on a per-item basis but on a multi item basis it may hurt your profits due to storage costs and lower total number of items sold.
Systems like these are generally very easy to solve with an algorithm such as the Simplex algorithm. You have to Define your viable region by defining what values for variables are unacceptable, then with the variable region defined you can find the maximum points for the viable region which basically represent the best scenarios that can occur within the viable region and each with different trade offs. Using eBay as an example once you have your viable region defined you can calculate the best way to make the most money per item sold or the most money total or the least cost invested etc.
Something similar applies to Starcraft although the exact trade off is a little different. Generally it is expensive to try to end the game because of the defender’s advantage which means if you want to maximize your win rate you needed to take fights that don’t engage the defenders advantage. That usually means trying to starve your opponent and forcing them to come out and secure new income and trading against them out in the open as they try to move out and secure a base. As a consequence you end up prolonging the game which means that as a general rule of thumb to maximize your winrate you also need to prolong the game as long as possible. This is especially true for a player that has fallen behind as he doesn’t have the bank to be able to trade so securing new income is very risky since he has to move out in the open and take hold of a new base. His best odds of winning are to never ever leave his base. Thus we end up with a stalemate where the person who is ahead is strongly incentivized to never attack and the person who is behind is even more incentivised to never attack.
Well it seems to me that these Bots only have enough complexity to be able to model a simple build order. In fact it can hardly handle such a simple build order that even a slight hiccup in the build results in the game being in a scenario that the bot is not trained for. I am sure they can train their Bots to be able to win in any given scenario but piecing all that together into one super bot that can do it all, that can do a build and also defend a two racks, is much harder than training it to do a simple build order.
I am guessing that the computational complexity of the task scales astronomically and that they have picked a value for the internal complexity of the bots to be as high as possible while still being practical, e.g. capable of being computed in a reasonable amount of time on their hardware. In other words this is clear evidence that even with the supercomputer they are running over in their lab the solution is so complex they can’t solve it - they can make their statistical models correctly Model A simple build order but nothing beyond that.