War Stories: Crash Bandicoot - Ars Technica - Memory was so short in Crash Bandicoot that I took to stealing little bits and pieces of extra memory from the Sony libraries. I would like just try erasing parts of them that I thought I wasn't using and see if things still worked. If they did, I would mark them as available and I just hacked their code by just changing the byte codes. I'm like, you can do this. Look, I fixed it. If they wouldn't fix it for me, I was just gonna like edit their code. It was free memory. [laughing] The memory was finite. But you were definitely not supposed to do that. Hi, I'm Andy Gavin, co-founder of Naughty Dog Inc and lead programmer on Crash Bandicoot. We set out to make the first 3D character platform action game, to do it right, and literally hack the hardware. [upbeat music] It was part of the Naughty Dog philosophy to leave no stone unturned, no cycle of CPU or GPU or byte of memory that you could use unused. So if it existed there in the machine whether we were supposed to use it or not, whether you had to use some crazy trick to use it or not, we would figure out how to like make the most of it. In the summer of 1994, my partner Jason Rubin and I were wrapping up Way of the Warrior, which was our 3DO fighting game and we were trying to figure out what kind of game do we wanna make next. The biggest genre on consoles at the time were platform action games, games like Super Mario World and all sorts of other classic 16 bit platform games. And at the same time in the arcades, there was a new 3D hardware and certain genres were making the transition from their traditional 2D state to 3D. For example like Street Fighter II and Mortal Kombat were still all the rage, but now there was Virtua Fighter. That was using 3D graphics and they were just cool and it was clear that things were going that way. New consoles that are coming out, they're gonna be 3D. Can we make a 3D platform game? No one had ever done it, but like, but it's got to happen. Think about that world or like speeding through a Sonic the Hedgehog doing those loop tubes in 3D. What is that gonna look like? Prior to fall of 1994, Naughty Dog was just Jason and Andy and it was the synergy between Jason and I. We both did all the creative stuff, we were best of friends, we were roommates, and Jason was a phenomenal artist and I was a game programmer, I think a pretty good one. [laughing] Between the two of us, like he would try to make it look incredible, I would try to make the technology really work and we both tried to make it really fun. Jason and I had sold the distribution publishing rights for Way of the Warrior to Universal Studios to a new unit called Universal Interactive. They're basically like, come out to California, move out to Universal's lot next to Steven Spielberg and we'll give you like a bungalow for free and utilities for free and you can just do whatever you want and all you have to do is show it to us. Got the dog, the Naughty Dog Morgan and ourselves into my Honda Accord and we drove across the country, so we had lots of time to talk. So we're like, well what would it be really like to do a platformer in 3D? We're talking about like Sonic the Hedgehog. Imagine you're going through loops, you're running and you're jumping platforms and collecting stuff in 3D. The camera's always behind you. You see the world and that looks all great, but it's like the Sonic's ass game. All you do is see Sonic's ass. But the front of the character is where the expression is. Could we do levels where you come at the camera? But how are gonna see where you're going? These eventually led actually to the boulder levels in Crash Bandicoot. Should we do levels sideways, should he look over his shoulder towards the screen? We're simultaneously designing the game that became Crash Bandicoot and sort of shopping for a platform for it. We knew because it was gonna be a 3D game that we wanted to do something on one of the new 32 bit platforms. So there's the 3DO which we had already made a game on, but the machine was sort of clunky half 3D machine. It was also very expensive and wasn't selling very well. Then there was the Atari Jaguar. We just sort of took it as a bit of a joke. And then you had the big guys all about to release new machines. You had some mystery machine from Nintendo but we had no way of knowing what Nintendo was up to. They just basically didn't talk to Americans. Sega had two things going on. They had a 32X which was coming out that fall which was some kind of like way in which you souped up the Genesis. Then they were making this new machine that I don't remember if it was called the Saturn yet or, but it became the Saturn. And then there was this wildcard Sony. They hadn't done video games before. We heard they had this powerful new machine, so we contacted both Sega and Sony and got the information on the machines which they gave us. You have to sign your life away and your firstborn and all that and then you can order the machines if you gave them the deposit and stuff. And then Sony had this new machine which was like all 3D. It could do 2D pretty well too and it was like a new clean design very similar to high end commercial 3D hardware like the Silicon Graphics hardware but with a bunch of simplifications to make it much more economic and there were two main custom chips in the PlayStation 1, a graphics unit and a sort of custom MIPS CPU. But the GPUs today are wildly more complicated and they also do a lot of the math work that old GPUs in the 90s did not do. PlayStation 1 GPU just drew triangles on the screen but it was pretty good at it. It could draw about 120,000 polygons a second of triangles which was phenomenal. If you were using a PC at the time, they had no 3D graphics hardware. They had like VGA boards. You'd be lucky to get a couple hundred polygons because you had to do them all in the software. But here was the PlayStation which was gonna be like 199, 299, whatever it was. - 299. - And it was like a complete machine with a CD drive and with the memory and everything. Console machines like compared to PCs at the time, a world of difference. We're talking Windows 3.1 and DOS. To run a game, you usually had to have a boot disk with like custom autoexec bat and config.sys and you had to open up your machine. This was not a very game friendly world on PCs. But PlayStation or Genesis or a Super Nintendo, you just shoved in the cartridge or the CD, pushed on, boom, it booted. I liked the Sony best. It was neat and clean and powerful and intended for 3D. I really thought it was the only one of those machines that'd actually do what we were sort of aiming at. The core idea behind Crash Bandicoot as a game, it was gonna have a mechanic like a game like Donkey Kong Country. You were gonna go through levels with successive timing challenges and enemies and jumps. It was gonna be platforming and it was gonna have a cartoony animal character and we wanted to not make it look like a video game exactly. We wanted it to look like a Looney Tunes cartoon where the character was highly animated, fluid. If he got mushed by a giant stone roller, he got turned into a flat thing that waddled around. We wanted a world that looked like a cartoon world with the kind of sensibility that was classic cartoons or sort of, it was being reinvented at the time with TV shows like Duck Tales, Simpsons was big. Animation was cool again. And Crash was in a frame, there were 30 frames a second and we got about 1,500 polygons on the screen. 600 of them went to Crash. That's how important he was to us. Like, most people, they'd use like 80 and so their characters would look like weird walking blocks. We wanted him to look like a real cartoon character. That required a lot of detail, so he gets a third of the entire budget. With Crash, we were forging into new ground everywhere. Like no one had ever made a 3D platform action game. While it was completely obvious that PCs, which were the standard thing that people would use for development machines were just not gonna cut it. Again, it's Windows 3.1 and no 3D graphics and the things had like effectively 640k of RAM. We made the plunge to buy for everyone in the company which was about five people at this stage, Silicon Graphics workstations, mostly Indigo2 Extremes. These are like 75 to $100,000 workstations. They had 3D graphics. The software to do the 3D graphics was also like about $75,000 per machine. This was what people had done in recent, very recent memory at the time, Terminator 2 or The Abyss or even Jurassic Park. And so we settled on using as our software Alias PowerAnimator, which is one of the three choices [laughing] at the time. So we got this clunky big box which was the early PlayStation prototype and it turned out to be a pretty good machine. But first you got to like understand it. It comes with a bunch of manuals which are incredibly badly translated from the Japanese and are mysterious and you'd have amusing debates between the programmers what they actually meant by the funny choice of English words. The way to really figure out what to do is to test things empirically. You do that by taking the individual pieces and you write test code to do certain things on the machine and put it through its paces, so like taking a car out on a track and seeing how fast it can actually corner. There's a bunch of different graphics modes and so it's like half speed in the high resolution mode but in the mid resolution mode, which was clearly just sort of an afterthought, it's the same speed as the low resolution mode. Jason already made this Crash Bandicoot character who looks pretty much the same as the final Crash Bandicoot character and so I had that drawn on the screen and we shrank it down to about the size it would be on the screen. I ran some code to actually calculate the number of pixels that each of his polygons occupied on the screen and it turned out they were like, the average was like 1.2 pixels. So I'm like, well why would we texture these polygons? We decided for characters, let's not use the texture mode. Let's just use the faster, easier to use shaded mode which most people didn't do. Most people just used textures. And part of this is because we ran the tests and found that the non textured mode ran twice as fast. If you didn't run the test and just sort of like used the numbers in the book, just gave you one number. So we figured out that PlayStation could actually draw a pretty decent number of polygons per seconds from the graphics hardware, but you have to do the math to figure out where those polygons are gonna be. These days like almost every computer and even your phone have advanced GPUs and they have vertex units and they do tremendous numbers, gigaflops often of multiple adds. In those days, most computers just did multiplies and adds one at a time on the CPU and they could do like hundreds or thousands, not billions. This was a serious problem. [upbeat music] There was this fundamental problem in the PlayStation sort of hardware software interaction where its math side was just not up to snuff. But we kind of knew that this was a software issue, that Sony just wanted us to use these libraries that they had written but they weren't really like using the machine to its fullest. Somewhere in here, the designers had put hardware that was designed to do multiply adds because you need to do millions of them but this was all hidden behind a bunch of C programming language libraries that Sony had which you gave them the numbers and it multiply added them and those things performed terribly. While the graphics unit could get 100, 120,000, somewhere in that range polygons per second drawn, the libraries could only transform the math for them for maybe like 5,000, 10,000, which was just not going to cut it. So the base tested performance on Vertex math on the PlayStation 1 using the official Sony way of doing it, calling their graphics libraries, was like at least an order of magnitude off, like 1/10th of what it should be. Just didn't run fast enough. [upbeat music] We kind of did what any sort of good scientist does and you just sort of take it apart and got enough of that figured out that I knew that there was like actually some real horsepower in there but they were kind of like hiding it away. With that particular problem through a series of campaigns through some of the guys I knew at Sony, the creative solution on both the sides turned out to be, here's how it works, two pieces of paper slid across the desk. You didn't hear it from me. [laughing] But that was actually enough. We just sort of systematically worked through what was on there and it turned out that there was this little sort of math side brain, it's called a co processor, inside Sony's custom CPU which actually could do this very specific and limited math that was needed to do the Vertex transformation and could actually do it just about the speed that the GPU could consume it. In pretty much every dedicated gaming hardware since the beginning of arcade games, there's always been two pretty much always there main brains. You've got the graphics unit and you got the CPU. CPUs, like pretty good old CPUs move individual numbers and that allows them to do anything that computers can do but it's like they have one thought at a time. The graphics unit because graphics is a more game specific area, it was some way in which it renders its graphics in hardware. Early 80s games, even something like Galaga or whatever where like the aliens are sweeping down, those are what are called sprites. The CPUs at the time could not draw sprites. These little graphics units were designed to do sprites and scrolling backgrounds. Basically all of computer graphics technology has been driven by the video game. But you always had this balance in these machines between their sort of graphics brain and their sort of general math brain. Graphics units nowadays can do in parallel sometimes thousands of these same operations and that's why the actual sort of processing power on a GPU is way more than a CPU. Like they have many gigaflops, teraflops on some of them. They don't do as general stuff. [intense music] Let me step back and explain gameplay in the 2D platformer. Let's take Donkey Kong Country. You can move forward and backward in the level, so to the left and right and you can jump up and down or jump up on a platform, but essentially it's kind of linear. Usually go to the right and obstacles come to you. This goes all the way back to like Pitfall in 1980. You might swing on a vine to swing over a bunch of spiky things. Monsters may be crawling on the platforms or bees flying above. It's a linear progression. You can see where you're going and it's sort of very fast paced. It's like jump jump strike, jump jump strike. So it's all kind of nicely lined up in 2D. The designer measures out this almost rhythmic musical progression of the game and this is one of the things that makes these games really addictive and fun. Your progress in getting better at the game is to first learn the controls, get good at being Donkey Kong and then to learn the specific movements of the enemies and the objects in the level and sort of memorize a route. It's almost like playing back like in a rhythm game. But in 3D, everything's different. You've added an extra dimension. You've now got this side to side too. If you got in Donkey Kong Country, Mario, whatever, if three turtles are coming at you, you have to jump over or kill or knock away each of the turtles in succession. In 3D, you can just go to the right and avoid them entirely. You've added a whole extra dimension to the space and so there's too much empty space. This had the effect of sort of diluting the ratio between choice and conflict. We had to figure out how to compensate for this in order to maintain the intense action pace. [upbeat music] We had this realization that basically we had added a dimension so the simplest strategy was take out a dimension, but take out different dimensions in some way. An example of that was say like the boulder level. The dimension we're actually taking out there is time. It's not one of the three spacial dimensions. Because of the hairy nature of it, that the boulder's gonna smoosh you if you don't move like now now now, you don't have the luxury to pick around those other dimensions, so it drives you through the otherwise sparser thing so fast that it feels just as intense. And the hog level is a reverse of this but instead of the thing moving at you, you keep moving at the things. Like you're on the hog and you can't control how fast he goes. He's a wild hog and you have to navigate mostly left and right and then we had the more normal 3D level. In something like Insanity Beach, we put walls of jungle up in order to sort of narrow that dimension down not fully, but partially. And then there's other things to help narrow it down like for example the enemies, say like a crab or skunk in those levels, they track you laterally. That doesn't mean we can't use the new dimension if we wanted too because you can have a box on the left, a box on the right, and then you can make interesting things where they have to choose to face this peril and go over to that box or if you're in like something like Hog Wild, we would deliberately do this. It might be that the spiky thing is over on the left with a box and the clear path is on the right. So the simple way to survive that obstacle is to go all the way over to the right and just ride past the spiky thing. Then you leave the box unbroken. To get the box, you have to like go to the last minute right in front of the spiky thing and then slide over so it gives you this choice or timing elements like on some of those 3D type levels like the rollers, they go in there and then you can time together things like you got these roller ones and then you have like platform forwards. You have to jump but then you can add to the tension by making the platform drop. It's like a timed platform. You get on it and it goes and drops. And then we had other format levels where we took away a dimension. For example, there's a bunch of levels that are kind of like say the wall, it's essentially kind of 2D. I mean, you can come out but we've just made a somewhat 2D level by fixing the camera looking to the side. And then there's another type of level where the camera's up like this and Crash moves in sort of a grid but there's not too much play with the up and down and instead of being wide open, you're above something deadly on walkways. In each of these cases we're like adding constraints or removing degrees of freedom to try to narrow it down and increase intensity. Because we have this extra dimension, the game is more varied than a traditional 2D one would be because we're removing different dimensions in different levels. We have like seven, 10 different strategies for removing dimensionality. Boxes too actually were designed to fill in the void. There's not actually that many enemies on screen because we wouldn't support that many but we filled in the empty spaces to a large extent with boxes. And then they can drop down in the stacks and create puzzles out of and so they had for what's a simple object, you know, a few polygons and a few rules, they're incredibly versatile. [intense music] We totally knew we wanted to do this kind of animation never really been seen in video games like the sort of Looney Tunes style, distorting animation. It's a very stretchy, rubbery style of animation which is done in traditional cell animation. That meant that characters had to really animate. Traditionally in sort of low polygon 90s era graphics, you would make your characters by constructing a very small number of bones, upper arm bone, lower arm bone, head bone and you would stick all the art that pertained to that character's upper right leg on the upper right leg bone. And the bone was like a rigid thing. It was like a joint. It could be rotated or moved. You could kind of go like [imitating robot] and if you want fingers, you had to have like lots of bones. That was just too much math for the PlayStation to handle. We just knew it could never handle tons of bones. And you certainly couldn't do something like say he gets his hand hit by hammer in a cartoon. Well what happens? It swells up like a balloon or it gets flattened into a pancake and something like that. We just couldn't do that at all with the whole bone system. A game like Virtua Fighter in the arcade used the classic bone system. This worked okay for a fighting game because there's not a lot of ton of deformation but the characters were a little stiff. With bones, it's almost impossible or very different to do facial animation too and we knew we really wanted facial animation. What kind of cartoon character doesn't like smirk or wink? Is it even possible to get that kind of animation like on a Silicon Graphics workstation? Jason made this Crash model and he set about trying to do cool and wacky animations with him and he found with some work, you could get the PowerAnimator to actually do it because it had these sophisticated like bone and vertice weighting tools and these distortion fields and like the way that PowerAnimator did those things was just not possible on the PlayStation. These are all things that used the expensive floating point hardware that was on the Silicon Graphics workstation and it couldn't even do them in real time. It had to render them. But a video game is real time. You have to do everything in 1/30th of a second. This was another big problem because he wanted that animation really really bad. [upbeat music] I'm like, well we can read the positions of all the vertices. If you know where the positions of all the vertices are in every frame, then it doesn't matter what kind of bones. The SGI can use 1,000 bones and it's all the same. You're just drawing the polygons and the vertice positions and it can be done even quite fast. You don't have to do any bone work so you don't have to spend any CPU on bones. But the problem with that strategy is then you have animation where it's 30 frames a second and Crash is like 500 vertices and maybe like 600 polygons and you got to store the position in every frame of animation for every vertex. That's a lot of data. But at the same time, we also are already committed down this crazy memory strategy to thirty fold our data, so well, we can handle a bit more data than other people can. So we would bake down the animation data using all of Jason's fancy effects in PowerAnimator. I did it and it worked because the PlayStation development unit had like eight or 32 megs of memory. So you could have a much bigger thing than would ever work on the real machine. So we're working with this for months and always I'm thinking, we're gonna have to fit it in there somehow. It's just so huge. It's gonna have to get smaller. But I had this hunch that there really wasn't so much data in a mathematical sense, that we could write a domain specific compressor it's called for this. So what that means is data in a sort of computer science theory has a certain complexity. There's a pattern to it and limits to it and with like the animation data, it can tolerate some data loss. The same theory is why jpeg works and why jpeg totally changed the internet and totally changed how images are stored. For jpeg, you transform the image into frequency space, a difficult mathematical transform, and you throw away the high frequency junk, flip it back and depending on how much junk you threw away, it looks better or worse, but it's still like the most important parts of the image and it got like 1/20th or 30th the size because you took out the unimportant stuff. I was pretty convinced that this animation data was like that, that you could take out the high frequency animation, you know, like vertices waist, it wouldn't move up and down that much. Mark [Cerny] wrote this program where he analyzed, he took a bunch of the animations and analyzed every component and every vertex and calculated the kind of dynamic range or the amount of change that occurred across animations and found that indeed the information change was quite low, like they just didn't do much. So it turned out we could use this very specialized thing where it would analyze a particular animation and then it would figure out the range and all these things and it would have a map at the beginning that said like vertex seven Y low information, so it only needs two bits and it moves over this range and this one needs this many bits. The end was something like 50 or 81 compression, like the data got like a 50th the size. [upbeat music] In looking at the original PlayStation 1 design, while it was well designed and balanced, the machine had two megabytes or RAM and one megabyte of VRAM. But it also had the CD drive which was 640 megabytes of read only storage and that's a lot more. The ratio is very big. So I had the theory it would be possible to make levels much bigger than two megabytes. Like in a normal game, you have a game like Twisted Metal or something, you would go to the Eiffel Tower level and it would load the Eiffel Tower level into say one megabyte. So they would have a certain amount of art in them and animation which is about one megabyte's worth. Now you could do various things to try to squeeze that down, but there's no way getting around this basic thing and people didn't really try. Here's how it basically works. The computer, the CPU and the GPU could only really access stuff that's in the memory right now. Every 1/30th of a second in a video is a right now to render my frame. Anything that you're actually going to draw right now, any animation you're gonna be using right now, any sound you're gonna be playing right now has to be in memory 'cause you can get to the memory quickly. The CD takes about 1/3 of a second to move its head to any specific spot on the CD. It takes some time to actually load the data. It could load a megabyte in maybe six seconds or something. So you can't just go draw one frame and go get a new different megabyte off the disk because it's gonna take you eight seconds to get that different megabyte off the disk. Well, what are you gonna do, sit there do nothing for eight seconds? You can do that between levels. So when the level switches, that's why a typical game would go to a loading screen and they'd load that whole megabyte or whatever off the disk. The CD, it's just a further storage. It's bigger but it's slower. It's further away from being able to use it, but why can't you use it? It's there. On an early PlayStation game, they have like on megabyte levels or two megabyte levels and the CD is 640, that means that could fit 300 levels on the CD. Did they have 300 levels? No, that would take them like 10 years to make the game. So the CD's just mostly empty or they fill it all with music because music is kind of fat or video or something, but a lot of times, it's just empty. Games like Twisted Metal might only be 50 megs on the disk. And so every games' levels are gonna look like roughly a megabyte, maybe a megabyte and a half of data. Now that was sufficient. Many many PlayStation games were made, but it was part of my theory that that didn't have to be so. [gentle music] My idea was that I would use advanced virtual memory techniques to swap chunks of data in and out from the, like basically if the level was let's call it 30 megabytes, well maybe you only need at any one moment in time one megabyte but the level's actually 30 megabytes. I would chunk the entire level into 64k pages and then there were chunks of data. They could be something, anything from a Crash animation to the Crash code, a piece of background. Chunks had to be less than 64k, like pile them in 'till the page was almost full but they couldn't go across pages or you'd have to break the chunks into smaller chunks. Then the level consisted of 30 megs of pages, 16, 18 pages that could fit in memory. The problem is can I range the chunks in the level such that at any one point in the level, I never need more than 16 pages worth of data and you can actually take the same chunk, duplicate it into multiple pages if that makes the whole puzzle work out better. The game is constantly, it's figure this out in advance, but it's constantly figuring out which pages it's gonna load in if you're going this way and which page it's gonna load in if you're going that way and it throws away old pages that it doesn't need and loads new ones into their place. Any page can replace any other page as long as you never need more than 16 active ones at once. Crash textures are all pretty sharp with a lot of color and detail and Tomb Raider's are like all washed out and pixelly and whatever 'cause they only have so much memory. They don't have any place to put all the extra texture. We had 20, 30 times the amount of space for it. Or the number of polygons in the level. Something like Tomb Raider, it's pretty blocky, like square corridors and whatever and Crash has like weird shapes and whatever 'cause we have many more polygons. There were a bunch of different technologies that serve this but the memory was a huge one. So this was one of my sort of like, patched out a whole bunch of patents on this actually. I was definitely one of the first people to see that you could really use that storage on the CD or the disk drive, it didn't really matter, as a dynamic part of your game to sort of expand it. That's done all the time now. - Fear not Plumber Boy, the Bandicoot will come. I'm coming to get you, pal. - I think one of the core legacies of Crash Bandicoot is that games can have their own distinct stylistic personality and art style, and yes, to some extent many did before. I mean, Mario has its own sort of Mario style but it's very much an old timey video game style. But Crash had its own consistent world which draws obviously from American cartoon style. And so but the entire Crash product, by that I mean our first four games, are consistent within that style. The originals still play quite well and they still look pretty good even like on a PlayStation 1 because the style sort of transcends the specifics of the pixels. Early on in making it, we had this totally arrogant half hope, half idea and it was another major factor in choosing the PlayStation was that Nintendo has Mario and Sega has Sonic. Sony doesn't seem to have a mascot so if we make a mascot style game on the Sony machine, we might not have the same competition. And then there was this fairly immediate buzz where they're like, we have to have this game. Like, we have to make sure it's only on the PlayStation. Like we deliberately set up to be in the position to maybe do it and then we sort of instigated it. But that it worked out was like kind of miraculous. But our dream was to become the mascot. Now Sony never even even when they bought it, they never like official, they're like, oh it's not a mascot, we don't have a mascot like and whatever but like everyone assumed it was a mascot and they got so behind it that they just bought the rights from Universal. - Hey Plumber Boy, Mustache Man, your worst nightmare has arrived. - Crash Bandicoot started off selling well but it just kept on going and going and going. Crash Bandicoot the first one sold better in its second year, maybe in its third year, and I'm pretty sure that when we shipped CTR in the Christmas week of CTR, Crash Bandicoot won, sold more copies in that week than it had ever sold previously. First of all, it was a really broad appeal game, like everyone could kind of play it whether you were a young kid or you were a hardcore gamer. It wasn't highly violent but the characters were appealing, it was funny and the gameplay was like pretty intense and approachable. You could just sit down and play it and this was one of the things that we really wanted for it. I wanted a game that you didn't have to think too hard about. Crash Bandicoot was really the crucible in which the Naughty Dog philosophy was forged, that every element that goes into the game needs to be great because we wanted to make a truly great game, a game that was gonna be a hit, a classic, and in order to do that, we came to believe that everything in it had to be great and you needed the best technology possible to make all that happen. The better your technology, the better the game could feel, the better it could look. If you got enough of these things all right, you could sort of transcend mediocrity to become like a gaming masterpiece. [upbeat music]