The difference between this bot and a traditional bot is that a traditional bot is programmed by a human while these bots are programmed in mass by another program but the bottom line is always and forever will be that they are programmed. They don’t think on their own. They do what their programming says to do. There is no effective difference in intelligence. There is a difference in cost to produce the bot. It’s a lot cheaper to run a program on a dataset than it is to pay a team of programmers.
Well, the human brain can theoretically be modeled as a set of rules… but I have to agree with you. AI vs. Humans is like comparing steel to plastic. The AI is stronger on sheer computing power. But a steel ziplock bag is never going to work. Starcraft is a situation where sheer power can beat out flexibility. But at the end of the day, there is still a pretty heavy reliance on power.
You are getting into some deep philosophical concepts but let’s focus on the practical differences.
The first difference is that the human brain is self-adapting. The brain itself teaches itself to model new data and it can do this from a ludicrously small amount of data. This AI does not do that. It needs a ludicrously large amount of data and it is trained by a separate AI. They said that each bot needs the equivalent of 200 years worth of SC2 training when a human can learn it in a few months.
The second difference is that the human brain can abstract, e.g. it can take something it’s learned from one scenario, and re-apply the relevant aspects to a new scenario. This bot cannot do that AT ALL. Watch Reynor do a bunker rush. The bot just sits there, clueless on what to do. When reynor finally salvages the bunker, thinking a nydus is coming, the bot TAKES A THIRD instead of taking its natural.
One of the challenges of Autism is abstraction. An autist can learn a kitchen, but when you move one chair it is like a completely new place to them because they can’t disentangle all the various aspects of its previous model to account for the chair. The model is a whole in other words. The child will have to re-learn the entire kitchen again because one chair moved. This bot has that problem. It has ZERO ability to abstract. It can’t take scenarios where it has learned how to deal with bunkers, and apply them to the bunker rush scenario. It can’t do it, at all!
1 Like
But wouldn’t it be familiar with bunker rushes by now?
Anyone gone through the replays yet? Anything funny in there?
So it looks like there were two rounds of agents on ladder: the initial barcodes that got “scouted” after playing their 50 games per race, and the final version which only played 25 games per race.
The final MMRs for the first round were:
Protoss: 6066
Terran: 5621
Zerg: 5920
The MMRs for the final (and strongest) version were:
Protoss: 6275
Terran: 6048
Zerg: 5835
These are SC2 caster MMRs, not pro player MMRs, so maybe tone down the hype a bit?
They are also based on a sample of only 25 games per race, which is very low, like a day’s worth of practice for a pro player. For a new account, we need a minimum of 25 games to gain some confidence in the MMR, and the results from AlphaStar have much wider confidence intervals than the MMRs of the human players it faced, as befits a preliminary MMR value.
Given that there was practically no chance to study and prepare against the weird builds of AlphaStar (marines without upgrades and mass banshees for the win?), these MMRs are a high water mark, and would go down fast once players know what they’re up against. And no, it wouldn’t be unfair to study replays from AlphaStar, since its supervised learning phase relied on 971000 replays from 3500+ MMR players.
1 Like
That’s exactly what AlphaStar doesn’t do afaik.
That’s why I think it’s trash.
It’s pretty easy to create an AI with perfect macro and perfect blink-stalker micro
… but adapting to something during a game because of scouting or changing the strategy because you scout something your normal strategy doesn’t work against (because of your comparably bad level of micro) and then winning… can AlphaStar do that? I don’t think so.
1 Like
I did an analysis on this back when they announced they are going to tackle Starcraft 2. I predicted that the Bots would be inclined to win on the grounds of micro and mechanics and to ignore strategy as much as possible, and that it would be an enormous challenge for them to incentivize the bot to win on strategic grounds when micro & mechanics are so much simpler to model.
yeah i read it. it was interesting to say the least. Its one facade of AI they are testing. I was reading about how they are making up AI to study plants and identify them.
Only 25 games? That’s nowhere near enough. It needs to play at least a few thousand for it to contribute to the meta enough for opponents to even start profiling it. Maintaining a win rate over a small number of games is very different from maintaining a winrate over a large number of games because as you play more your opponents will adapt to how you play. You only maintain your rank if your play adapts at least as fast. Furthermore, 25 games is nowhere near enough games to get a good selection of varying opponents when there are thousands of opponents and hundreds of playstyles.
Blockquote
Only 25 games? That’s nowhere near enough. It needs to play at least a few thousand for it to contribute to the meta enough for opponents to even start profiling it. Maintaining a win rate over a small number of games is very different from maintaining a winrate over a large number of games because as you play more your opponents will adapt to how you play. You only maintain your rank if your play adapts at least as fast. Furthermore, 25 games is nowhere near enough games to get a good selection of varying opponents when there are thousands of opponents and hundreds of playstyles.
I agree with you there. I think it would take a lot more games for the testing to take hold if the A.I. is performing how they hope.
I imagine since it got into the top 15 or so % of players on a server, it stands to reason they probably didn’t want Alphastar to take away from players on the ladder as far as placement.
However, I do believe it played 91 games and won over 50% of them.
Can you imagine or even fathom, if it were possible to adapt and change and make decisions based an ever changing meta for players, that an A.I. this capable would be able to as well had they kept it in longer?
We might never know now.
https://en.m.wikipedia.org/wiki/Lotka%E2%80%93Volterra_equations
The winrates of a strategy will shift over time, so you can only measure a winrate over a large period of time.
Hi rabiDrone, have you thought of the reason as to why you play under a barcode account yet? Previously you were telling me that you play under a barcode 'because it’s no different than playing under any other alias’, which is a sentiment you seem to disagree with suddenly. Get back to me on how this is different, maybe something to do with the sample size of AlphaStar’s games?
These statements only seem contradictory on the surface. Even if you give AlphaStar a recognizable, non-barcode name it still would be hard to recognize it since anyone can have that name as well. SC2 is an anonymous game and tracking accounts on the ladder is very difficult and guessing that it is AlphaStar by the name alone would be a best-guess scenario at best. However, that’s basing your guess on the loading-screen information alone, and no other factor. With in-game information, you can much more reliably guess who your opponent is and what they are going to do (assuming you have played a large number of games vs them - large enough to recognize the player from their play).
The fact that recognizing accounts is difficult is complimentary to my argument in this thread. The key here is the volume of games played by AlphaStar. Tens of thousands of games are played each day, yet over the course of the past months AlphaStar has played like a few hundred on the ladder, aka a minuscule amount. Nobody in their right mind would even bother accounting for the possibility of facing vs AlphaStar, let alone have any information to work with to profile it which again feeds back into what I was saying earlier in that the bot is being shielded from being profiled.
It needs to play a VERY large number of games to see if it can maintain it’s win-rate. Maintaining a win-rate over a small number of games is very easy as it means you had a build that was good vs the current meta, but the meta is always shifting and as that shifts what builds are good also shifts. Maintaining high MMR over a long period of time is a matter not of having a good build, but of being able to keep track of the meta, and create builds that counter the meta and to do this better than all the other players.
At their full power, they rekted Mana 5-0 . Due to popular whining, they were nerfed to “behave” like human instead as an “AI”.
Winning entirely on the merit of micro, which existing AIs already could do.
1 Like
Yes, that’s what make them “AI” instead of Human. AlphaStar was suppose to be the amplified version of AI with Human like decision making. Yet they nerfed the AI part and make it more Human due to being too OP.
Not like I don’t understand why they nerf it but it just defeat the purpose of making this AI in the first place: To see the true limit of SC2 when playing perfectly. It is not everyday where you see Stalker of all unit countered Immortal/sentry/Archon.
That’s not the goal. There are AIs that already do that that cost a thousandth as much $$$ to make. The goal of this AI is to test the learning capability of their algorithms. Micro and mechanics are two very simple things from an algorithmic perspective, so the AI is naturally inclined to try to win games on the merit of those two factors alone and that’s basically what has happened, but it is contrary to their goal which is why they tried (and failed) to negate it. It still wins games purely off of mechanical and micro-management merits. When it comes down to strategy and decision making, it completely falls on its face like it has never played the game before.
There are many examples of this. I saw one where it had won the game, but it was too dumb to go kill the Protoss’ fourth base so it went back home after clearing out the main and natural and waited for the player to just leave. Symptoms such as these are not symptoms of a bot that understands the game in even the slightest. vs that bot you could literally expand to your 5th or 6th bases and it would never know it. It would sit there making roaches while you maxed on 200 supply of carriers. In that game, even with the protoss rallying void rays from the 4th base location the bot was STILL unaware that there was a base there. This is how stupid these bots are.
1 Like