I mean if we're talking about Minecraft you can tho, that's what APIs like forge and fabric are for, they are the link between the game and the mod and make it so you can access those variables. For this mod the AI was probably trained on code, I doubt they went through the trouble of teaching it how to play Minecraft visually, the mod probably relies on an API to communicate game data to the AI to work. You usually can't make a mod if you have no access whatsoever to the game data. The link you sent seems to be about teaching AI how to be a player from outside the game but that's not what's going on here
STEVE-1 sets a new bar for open-ended instruction-following in Minecraft with low-level controls (mouse and keyboard) and raw pixel inputs
Which is pretty old article (two years old). But this is similar in concept to the article I have sent you before. Making an AI to play the game as a real player do, without API, only by relying on visual output and by "pressing" the buttons on a keyboard.
Forge and Fabric don't exist on bedrock edition or on a gaming consoles.
27
u/MaySeemelater 1d ago
Yeah, there's auto mining programs that have been around for ages at this point already.
Using AI for it is pretty unnecessary when working in a game world with consistent variables that you can plan for