Tricky Bits with Rob and PJ

Moving and Grooving to RISC-V

• Rob Wyatt and PJ McNerney • Season 2 • Episode 1

Share episode

Enjoying the show? Hating the show? Want to let us know either way? Text us!

Exploring NVIDIA's Move from ARM to RISC V in GPUs

Rob and PJ return to discuss NVIDIA's strategic shift from Falcon and ARM microcontrollers in their GPUs to the open-source RISC V. The conversation delves into the history, technical considerations, and licensing challenges of microprocessors. They examine why ARM's longstanding presence is being challenged by RISC V, especially for embedded controllers, its flexibility, and cost advantages, alongside security concerns and the future potential of open-source hardware.

00:00 Welcome Back to Tricky Bits
00:27 NVIDIA's Shift to RISC V Microcontrollers
02:02 The Role of Microcontrollers in Modern Devices
04:33 Technical Challenges and Limitations
06:18 Designing Custom Microcontrollers
16:33 RISC V: The New Contender
18:00 Comparing ARM and RISC V
24:13 Apple's Strategic Choices
31:48 RISC V Maturity and Market Penetration
33:58 Challenges and Security Concerns
36:53 Financial Incentives and Reference Models
40:48 Comparing RISC V and ARM
42:25 Security Concerns with Chinese RISC V Chips
49:22 Microcontrollers and Their Ubiquity
54:34 Future of Processor Diversity
58:28 Open Source Innovation and Ecosystem

PJ:

welcome back, everybody. It's Tricky Bits with Rob and PJ. We know it's been a while and we apologize. We had a few things going on, but we missed you. We know you missed us and we're happy to be back. Now, Rob, today we're going to be kicking off the discussion a bit around. NVIDIA replacing its Falcon and ARM microcontrollers within the GPUs with RISC V or RISC V, I think there's a really interesting question case study here with microprocessors in general, in terms of like, Hey, we used to be all. X 86 and arm risk fives come onto the scene. And I think there's some interesting questions around both technical areas, as well as the licensing areas. So let's kick it off with Rob. why is Nvidia replacing these Falcon and arm microcontrollers within the GPUs? Are they working just fine?

Rob:

Yeah, I guess they were working just fine, but they're old, and at least especially the Falcons, the Falcons have been in NVIDIA GPUs since the original GeForce days. So they're getting kind of dated. They need modern microcontrollers to be able to do what they want to do with. I mean, the amount of work these microcontrollers do in these bigger devices is quite significant. So having a 20 year old embedded controller that you're kind of having to coerce into doing these things is, is not the best thing. And if you're going to replace it, obviously ARM is the obvious place to go, but then you've got all the licensing issues and like that, that you have to deal with, where if you go to RISC V, do what you want.

PJ:

So let's talk a little bit about the microcontrollers themselves. These aren't the compute units that are being used inside the GPUs. What are the Falcon and Arm microcontrollers being used for today? Like what is their purpose?

Rob:

So. NVIDIA GPUs, iPhones, all Apple products, like all NVIDIA products, all Google products have a significant number of these embedded controllers throughout the device a, on a GPU, for example, the. You send these command buffers to the GPU DMA'em, to the command buffer, you DMA, the command buffer to the GPU and it processes it. It's really, you're sending it to a microcontroller and that microcontroller is reading each command one by one and sending it on to its next destination, which could be another hardware block or controller or whatever it may be. And that's what these things do. And there's a lot of them. Each engine might have its own in an iPhone. There's probably multiple 20, 30, 40 of these things embedded across the device. They are literally everywhere. The ISP has a couple in it, inside the camera stack, a lot of what the logic that's happens is done in software. All the audio systems have multiples of these things stacked up and. Apple currently use ARM, but they are, at least when I was there, they were looking at moving to RISC V for audio and then other things afterwards. So these things are everywhere and it's not nothing to do with the that you see as application processors. They're the big processors, obviously, that you run your applications on and the GPU is. thing that gets the spotlight, but behind the facade of GPU, like what is the bits in the GPU? Yes, there are compute engines, there are shader engines, there are a bit of fixed function for rasterizing, but it's all glued together by these embedded microcontrollers. if you look, I like the Linux code. There's a folder called firmware, and that's the that loads into these microcontrollers.

PJ:

So, these are the glue that really Makes the compute happen. They, they, they're traffic controllers. They're the ones who shuttle this stuff around.

Rob:

They're not doing any real work. They're just sending data to where it needs to be, reporting errors, flagging various things or whatever it may be. They're not doing any real work as you see it as an end user.

PJ:

from a technical perspective, where are we reaching the limits right now, where it's like, Hey, this Falcon, this arm microcontroller, you know, it needs to be faster, it needs more bandwidth or what are like really the technical limits that are. Causing there to be some concept of we need to make a change and we have this decision gate. We need to talk about whether it's designing a new falcon chip or licensing a new arm core or going to risk 5. What is it? Technically that is that is motivating this.

Rob:

Features, as, as always, if like you couldn't, couldn't run a modern application on a 6502, you wouldn't even attempt it. Memories, not enough, what it can access, how fast it can do it are all big issues. And obviously, 8851, which was existed alongside the 6502 still exists to this day, Intel used them everywhere and they've modified it and added things to it. But, Ultimately it has limits. These embedded controllers need to be able to see address space. And if you're a 32 bit controller, and now you've in a 64 bit machine, then how do you present memory across that? And we just make the embedded system 64 bits so we can see everything it needs to see? Do you page it in piece by piece, move it around? It's all obviously part of the design, but at some point it just becomes the problem itself. Is this embedded controller on this? Getting the data you need through it becomes more and more problematic. And when that happens, do you extend it, make it bigger, make it faster, it 64 bits, or do you just go, why design a microcontroller? This one already exists.

PJ:

Well, let's talk about that. So I'm NVIDIA. I'm Apple. I have the premier chip designers in the world working for me. Why not just design my own microcontroller? They did it with Falcon. Why not do it again?

Rob:

Sure. You can put verifying it, testing it and maintaining it is. super expensive and takes time. Today it's, and toolset, all of that plays a factor. You need, now you need a software team to go and modify LLVM so it can generate code for your processor. All that's already done if you take ARM or RISC V.

PJ:

Okay. So I'm an NVIDIA or Apple RISC V is really the new kid on the block. We're talking about something that maybe is only, you know, 15 years old at this point in time. Arm's been around forever. It's power efficient. has really stolen the thunder in terms of like the mobile space. It has done tremendous stuff in terms of Apple changing over to that instruction set for Apple Silicon. Why not just continue to choose arm at that point in time? That feels like a no brainer.

Rob:

So ARM has been around for a while, yeah. I mean, I was working on the original ARM 2 in 1985, 86, something like that. So it's It's a 40 years old. So arm is no spring chicken. People like to compare it to x86 and it's only 10 years newer the most. arm is It's been around a long time and. Obviously, Arm's licensing model plays into all of this is Arm don't actually make processors. They might make a few reference

PJ:

Silence.

Rob:

One of them, two of them, 10 of them, doesn't matter. in there and add the

PJ:

going to go ahead and start the recording, and I'm going to go ahead and start the recording.

Rob:

is. And that's when you'll see things like A72s. That's

PJ:

Okay.

Rob:

Qualcomm, Apple and video, all of what's called architectural licenses, and can design their own ARM processors. So it is, which is why Apple Silicon isn't an A72 and the. The Qualcomm processors are not known ARM cores because internally architecturally designed by Qualcomm. Now they have to play along with ARM's rules. Part of the licensing agreement is there's a book called the ARM ARM, which is the, the Bible of ARM. It tells you of every single instruction side effects and the signals it needs to generate and things like this. And there's a whole test suite that goes through that. As long as your chip passes that test suite and it's now a legal ARM chip, you're good to go. So there's no connection between all these big ARM processors. They're all designed internally. They all have architectural licenses. That's kind of where it ends. So that, that accounts for the big application processes, the Apple silicon processors, these embedded controllers, you still have to license them. some of these companies only have an architectural license for say 64 bit. So they can't make a 32 bit version of the processor. They can buy an off the shelf one or call, but it costs. And if you have 20, 30, 40 of these things in a device that are just doing mundane jobs, then that cost starts to significantly add up. And yes, the cost of pennies on some of these things, but it adds up not having that control of your own destiny is also a factor for some of these companies. For example, You're not allowed to add instructions to ARM. They very much frown upon you, the base and adding new instructions.

PJ:

Well, let's do they frown or they just disallow it as part of the licensing agreement? I guess I want to be clear on this one.

Rob:

other end of that license is really what it comes down to.

PJ:

I say, okay, so that's.

Rob:

have relaxed that a little bit in the last couple of years. Because of the obvious issues that it creates. It's traditionally it was always, I want to have this extra feature to the processor. And could do it as a co processor and then use the existing co processor instructions within ARM to communicate with it. But if you wanted to add that actual new instruction to the core instruction set, you weren't allowed to. And this came up for Apple to some extent because they added AMX which is the matrix extensions. And they added that as a co processor and then you have to. They added a few instructions to communicate with it, but there is a bank of instructions where they are vendor specific, but that's all you get. So modifying the core architecture, if you wanted to change out the ad instruction worked, like I want add to do something different to the current ad instruction is. I mean, what you would do, I don't know, but you can't. If you want to re implement how ad works, that's fine. As long as you pass the ARM checks at the end. If you want to make the multiplier be a single cycle integer multiplier instead of a multi cycle integer multiplier, that's absolutely allowed because you're not changing the

PJ:

Okay. Okay.

Rob:

designated

PJ:

Okay.

Rob:

instructions. That's all you get.

PJ:

So, so effectively you're allowed to change the implementation to your heart's delight, but you cannot change the

Rob:

license.

PJ:

got it if you're an architectural license, but you cannot change the interface regardless of license is what you're telling me.

Rob:

Yeah. And yes, I mean, you're saying regardless, but did it a little bit. So

PJ:

Sure.

Rob:

there are ways and means by playing by the rules as much as you can, and then do the minimal what you need to do. For example, if you took some AMX

PJ:

It's good.

Rob:

ran on an Apple Silicon, it wouldn't work on any other device. And I think that's what ARM as a whole want to avoid. They want to avoid fragmentation of the Silicon, fragmentation of the tool chains and things like that.

PJ:

Got it. I mean, I was going to say with Apple, it's, it's good to be a 3 trillion company. It gives you some leeway.

Rob:

And

PJ:

Now,

Rob:

I mean, they they almost bought ARM. And it's probably reasons like this that it would have made a lot of sense for NVIDIA, or even Apple, to own ARM. Business wise, globally, may not be such a good idea. Internally, once they had that control, they could do things like this and they could technically do it anyway, but they'd be in violation of that license. And I mean, Arm aren't shy of taking these people to court as Qualcomm's fine enough. Yeah,

PJ:

right, well, what's interesting is that we're talking right now about a licensing agreement that prevents you from doing what you want to do technically. So ideally, Apple would have loved to place these matrix extensions directly like they wouldn't have to design a separate coprocessor and then use the. Special magic to send instructions to that coprocessor. I would want it right in the processor to be able to do what I want and not spend the overhead technically to actually like execute these instructions.

Rob:

And then you get into, I mean, these are all of the application level processes. It's still kind of way today. The small embedded arms today, you can modify the instruction set. At least add to it. You can't modify it, but you can add to it. it's these embedded controllers that we're talking about. We've obviously Apple Silicon and AMX are all of the big application things. small embedded ones. Sometimes I like, I need a cortex. I need a dual core cortex M0, but I need atomic operations so that can communicate between the two dual cores. Cortex M0 doesn't have atomic. Instructions. now you're going to switch an interrupts off and things like that. So you can update one without the others being able to do it at the same time. Like old fashioned multiprocessor things, like this. Like I can add atomic instructions to this core, but I'm not allowed. I can add it, whatever this may be, but I'm not allowed. I could add this different bus but I'm not allowed where if you take your own core and video with Falcon or. Something that they might want to design in house. They can add all this as they wish and modify it as they wish. No one cares if X or processor Y, or they will have the same name or even the same thing. Like, none of it's public. It's all in deeply embedded. It's,

PJ:

much. Well, let's, I just want to stay on, yeah, we'll get the risk on Wednesday. I just want to stick on one more point. So you're talking about extending the, the arm microcontrollers. Do I need a separate license for that? Cause we talked about, Hey, I've got a six.

Rob:

exists for that.

PJ:

Oh, for the microcontrollers. You don't need to,

Rob:

can,

PJ:

you don't need to actually.

Rob:

but an architectural license for microcontrollers is, I don't think anyone does it because this

PJ:

Okay.

Rob:

incident license is so expensive and it's so hard to redesign these processes. Why would you do it for a tiny microcontroller? Especially a

PJ:

Okay. Got it.

Rob:

your hands are tied.

PJ:

Do we end up in a spot where it's like, I can't. Make the changes I want to make because it's not worth it. But these are starting to become the problem we talked about earlier. They're the bottleneck that I need to actually make a change to.

Rob:

can't make the changes you want. The license doesn't really exist. If I could modify it, I'm basically designing this tiny little controller from scratch. Might as well just make my own. And that goes to the original point of why not just make your own? The reason you don't make your own is because RISC V exists.

PJ:

so 15 years ago, I didn't really have much of a choice, right? I basically like was stuck with arm and I don't know, Whomever else I might want to try and like snag a microcontroller from. Nowadays, we've got this new shiny open source RISC V. What makes it appealing? let's talk about the license of it. It is open source, correct?

Rob:

a license,

PJ:

So,

Rob:

you take,

PJ:

I mean,

Rob:

an ISA, which is defined. That's the open source part.

PJ:

yeah.

Rob:

if you want to modify the ISA, you really need to be part of the consortium that runs RISC V. You can't just submit a PR saying, I need a new instruction. There's a lot of big companies involved now. They're designing a very clean, very modern instruction set, and can take that and implement it as you wish you want to add to it, add to it. Do you want to remove things? Remove things. If you want to make it open source back to the public, do so. If you want to give your designs back to the public. Do so, you don't have to, if you make a really fast custom

PJ:

Um,

Rob:

and I get my little feature that I, that I need. why wouldn't you go down that path? It's already been, it's an instruction set that's been tested. Again, it has tool support, it has debug support, it has fabrication layout support. why would you start from scratch? I

PJ:

Okay, risk five. It's ready to go. Like we've got the, as you said, the tool chains, we've got the fabs, we got everything like ready for it to action. Why isn't it then like the natural choice for everyone just to jump on? Like, are there any actual questions about, do I still pick arm today? Or do I You know, create a RISC V chip. what would be the blocker in doing this? Other than, oh, we know ARM works. We have these, these long, deep supply chains for this stuff. What prevents us from like jumping to RISC V? Because it seems like it solves a whole lot of problems all at once.

Rob:

think a lot of that depends on what you're aiming for. If you're looking at these small microcontrollers and you're paying a license fee per, then obviously there's a financial incentive to use ARM for these microcontrollers. If you're looking at the application side, then RISC V is only just getting up to speed at multi gigahertz processors. They're very expensive to lay out. They're very hard to lay out at that speed. And it takes somebody to go do it. So people are,

PJ:

Okay. You

Rob:

point, things like that. And that's it. It's the knowledge base of knowing how to work with ARM and knowing how to do this. That's lacking in these processors. And a lot of these big companies are not laying out these massive, there's no Apple sized company who's saying we're all in on RISC V. We're going to make a multi gigahertz chip, multi core. We're going to compete with Apple Silicon with the RISC V design. Nobody's saying that.

PJ:

So, so, and is that purely a maturity question or is there anything particular about risk five that makes it difficult to do this multi gigahertz support, we need time to literally bring it up to speed.

Rob:

Some of that and. maturity, there's things like, how does multi core work on with atomics? How does SIMD or wide vector instructions work? Some of that's in RISC V, some of it's still getting added. so ARM is definitely ahead of the game as far as modern requirements, but a lot of it is in design of these chips. Like, okay, we've got the ISA, how do I make silicon that runs the ISA? ARM's been around for a long time. They've been doing this for a long time. There's a lot of people who know how to do it with ARM.

PJ:

this is actually kind of a fascinating pattern, and when ARM first came on the stage, I mean, obviously, it's been around since the 80s, so it's been around for quite a long time. I feel like it really hit the cultural zeitgeist with mobile devices. And people really got tuned into the fact that, Hey, this is efficient, low powered processors that I'm using to run my mobile phones. x86 couldn't shrink down appropriately to become as power efficient to compete in that space. Risk five in this case again seems to be targeting these low powered microcontrollers, but instead of necessarily being a big technical advantage, it seems like the main advantage is, hey, I can get the power I want. I can get the processing I want out of these microcontrollers. I just don't have to pay a lot on the licensing side. Is that a fair statement or do you think that there's more to it technically?

Rob:

it's very much that. It's the same as the Linux argument of how much would Microsoft have loved to be in the server that was on every single. server on the planet.

PJ:

Yeah.

Rob:

It's like, but it wasn't going to be the people were like, we don't want to pay the license fee. The license fee is a barrier to entry. And how about the same thing for Linux? It runs everywhere. Yeah. It runs in cars. It runs in set top boxes, routers, phones, desktops, servers, pretty much everywhere you need a full fledged operating system, you can take Linux, modify it as you wish. Make it work. Can you do that with Windows? Absolutely not. You take what they give you and that's what you get. And

PJ:

Okay. Okay. Okay.

Rob:

If

PJ:

Okay. Okay. You

Rob:

you'll use to do that will be very different for the high speed processor engineers who will do it at a much higher, higher clock speed. And they're not even close to the same.

PJ:

Again, it's fascinating because what we're talking about here is that RISC V has this ability to really nip at the heels of ARM at the low power section simply because you can eliminate that licensing fee. It sounds like ARM doesn't have a lot to worry about near term that these high powered chips, you know, in my phones where it's like the main processor still need to be ARM. The one that's in my, my. Like Macbook is still going to be an ARM chip for the time being, at least until they start to figure out how to make these multi gigahertz RISC V processors actually work,

Rob:

ARM obviously have an edge in the game because why did Apple pick ARM when they went to Apple Silicon? They could have just been like, we'll make our own thing. We're already going to do the entire internals ourselves. If we have to stick to this licensing agreement, Then it's kind of a hindrance to them. But there's also a lot of gains you get from it. So, I mean, Apple were in a position where they were already going to switch instruction sets. They could have just been, we're going to make something completely new. Apple don't give their processes away to anybody else anyway. So if it was

PJ:

right?

Rob:

It wouldn't have made any difference to anybody outside except the compiler would have changed. it is just the way it is. I mean, even with AMX now, it's barely documented and what is documented is all been done by reverse they already have these in house and it's very Apple to not document anything and not share anything. They could have just said, no, we're not doing ARM. We're just going to do Apple Silicon is Apple Silicon. It's nothing to do with ARM. It's not x86. It's not ARM. It's what we came up with inside.

PJ:

Okay. Let's let's press it.

Rob:

of

PJ:

Why not do that?

Rob:

lot of architecture and a lot of

PJ:

Hmm.

Rob:

who come from experts. And anyone can sit down and design an instruction set. It's just how good will it be when you pull it into a real application? And that's why these have to be designed by real people and not just. Designed by, oh, look, I'll have an instruction that does this and this and this. You end up with x86 if you do that. Again, asks the question of why did Apple go with ARM? There's obviously a reason.

PJ:

it's, it's sort of a fascinating. Progression because I could look at the, you know, if you look at, at, Hey, there's the Motorola days for Apple transitioning to, to the Intel days. And in many ways I could, I could almost look at those as being incredibly similar in the sense that it was an instruction set and a chip that was fabbed for me in either of those cases. And then,

Rob:

And none of Apple's history,

PJ:

yeah,

Rob:

68, 000, they went to PowerPC, which was the Motorola connection.

PJ:

right.

Rob:

to Intel x86 and then And then ARM.

PJ:

Right.

Rob:

good at transition architectures. They are actually very good at that. They never got in a position before where you could do that. You can't take an Intel chip and redesign it as you wish. You can't take the Intel instruction set. And make your own execution engine for that instruction set. do, because they have a historical connection to Intel, a second sourcing and things like that, where it goes all the

PJ:

Right.

Rob:

been lots of lawsuits between them too. And they're on better terms these days. But others have done it. I mean, Trans Meta

PJ:

Syr Syrix!

Rob:

did it, there's a handful, but they all have to go through the huge hoops to do it. So it is

PJ:

Yeah.

Rob:

but why? If you're going to do that, for something that's far more modern. And ARM is the child of choice today, but that's likely to change in the future. And

PJ:

talk a little bit about that. So, arms gotta stay.

Rob:

those old processors. It's always

PJ:

Okay, let's do that.

Rob:

if you want a 68, 000, became an embedded controller in the end, but it took a long time. And you only got a 68, 000. If you want to make a fast 68, 000, no one ever did. You always

PJ:

Hmm.

Rob:

68, or 1 0, or 2 0, or 3 0, blah, whatever these extended 68, 000s were. They're all designed by Motorola. You could get them as a cell you could put into your own design as that became a thing early on. That wasn't even a thing. You'd have your 68, in one package and you'd have your glue in another package. It wouldn't necessarily be in the same piece of silicon. By the time got to laying out their own SOCs and fabricated them themselves. The 68, 000 had gone away, MIPS was on its way out. PowerPC is still around, but it was rubbish then, it's rubbish now. so it left the big players, left x86, which was. Dominated in the PC space wasn't going to go anywhere, but it's always been awful. Intel have spent so much money trying to make x86 go fast when it would have been, it would have been better just to get rid of it. And they tried, they went to Itanium and that was a huge flop. And again,

PJ:

Yeah.

Rob:

underestimate the power of the desktop space the legacy of everything that exists for x86. It's, it's really why you can't get rid of it. And it takes someone like Apple to actually get rid of it in a closed system.

PJ:

The other thing I think Apple has done well to make that possible is stuff like Rosetta. They've done a nice job of actually like being able to swap stuff over. So I think they've really thought through that transition plan. Probably better than anyone else.

Rob:

absolutely. I mean, they did Rosetta the first time around. Yeah. When they went to Intel, they

PJ:

Yeah,

Rob:

and now it's Rosetta 2 to go from

PJ:

right. Hmm.

Rob:

can say, we're only going to support 64 apps. And we'll jump from some hoops to get the atomics and things like that, which are hard to emulate to work. I mean, I technically Apple did change their Silicon. So the memory model, which is very hard to implementing software and things like the carry flag. Work the same way on arm to technically out of spec. So that out of spec. So Apple Silicon added a flag ago. This was probably the eight 12 or 13 or something like that. It was a long, long time ago when they first had this glimmer, they added Silicon. And a flag in the process registers where you could say, do the R, do the Intel memory model. So it was an ARM processor doing the Intel memory model, one being strong ordered and one being weakly ordered. So on the strong ordered model, you have some guarantees if like, if you do A before B, then A will happen before B. Where in a weakly ordered memory model, you could store two things and you'd have no idea which order that will actually occur in. So you can't really model one in the, from the other in software, it would be cripplingly slow. to guarantee the order when you can't

PJ:

Hmm.

Rob:

hardware. So to fix it, Apple was like, well, we'll just do ARM instructions with Intel memory ordering, which is not an ARM design. It's not in spec, but the processor can still pass the ARM arm and the reference designs and all of the tests. You just don't set that flag. So that flag is what I labeled them to. get Rosetta down because they don't have to deal with all the weird atomics that Intel can do and some guarantees like if I write this, then this, it's not an official atomic operation, but it's guaranteed to be atomic in result due to the memory ordering on an arm, you can't guarantee that. how would you emulate that in software?. That was their golden goose in making Rosetta work.

PJ:

Yeah, the pains of trying to guarantee that in software seems like you would be having or quarter the, quartering the speed pretty rapidly on, on a given device.

Rob:

a hundredth of the speed, because you've got to commit all those writes before you commit

PJ:

Yeah.

Rob:

to make sure the external visibility is true that flag was a genius move in some ways. And you've got to think, when you're emulating these processors, you're not just emulating the instructions. You have to emulate, if you have multi cores, you have to emulate the interaction between those cores, which is where the memory model comes into play more than anything.

PJ:

It's interesting. I mean, there's especially apple, but I have mentioned other people in the space are thinking through this. There's a tremendous amount of investment that's been done into arm. I have to believe they're looking to pay off that investment for yet another five to 10 years to come. But. I is, is really what we're looking at here, a maturity issue where. Hey, down the line, RISC V will start to eat, not just at the microcontroller level, but at every other level inside of these chips where, at what point in time is the next like set of processors going to fall to RISC V? Or will they? what are some of the issues that could prevent this beyond maturity? Is there anything we have to be concerned with security?

Rob:

cells available. Right now, V isn't giving out cells of like, this is a working processor in

PJ:

Oh, okay,

Rob:

RTL form, they're just an instruction set. So you have to make your own. I think currently do give out the architectural licenses. So you can go design your own. It's got to pass all these tests, but you can also just take an A72, put it in your design. A72, designed

PJ:

Okay. Let me go back. It's a good call. Let's go. I think it's going to be a great Yeah, I think it's going to be a great call. Great. I heard you all. Bye.

Rob:

designs of macro cells that you can put inside an FPGA that you can put inside RTL to go and get fabricated. can the ARM model with actual cores, rather than just the ARM model for the instruction set. These are standard cores and make them open source too. So if you want to modify them, you can. So you don't have to, it's not an all or nothing. It's like, I want to take your core, but I want to add this to it. Today, you're starting from scratch. If you want to do that with RISC V, there's a handful of people have done it. Not many people are, there's some commercial licenses for RISC be an open source license to really make it happen.

PJ:

Well, from a, from a financial sense,

Rob:

you take people who are making ultra high speed RISC V cores, whether it be Apple for some embedded controller, whether it be NVIDIA, whether it be Google, whoever it may be, all these companies are putting significant. Investment into these designs, they're probably going to keep it private because there's no, there's no law, even in the license agreement that says you've got to give your design back. It's only the ISA that's open source.

PJ:

well, it's interesting because it sounds like though what you're saying it's because I can't just pull an equivalent to an a 72 off the shelf to kind of put it in my design. This is a limiting factor for anyone who's not a giant company to implement a risk 5 design. So I, it is.

Rob:

that the microcontrollers have. It's.

PJ:

Yeah.

Rob:

It's the,

PJ:

this is capital intensive,

Rob:

Yes. It really comes down to that. On one side, you've got, we'll use RISC V because the license fee's a problem for ARM. When you have 40 of these devices in a, in a product, that's a lot of license fees.

PJ:

right?

Rob:

And so it's worth that capital expenditure get rid of those. On the big processors, then it's massive capital expenditure,

PJ:

Yeah,

Rob:

and we're not going to give that back. think RISC V, the consortium should start. releasing reference models, because that's more down the ARM path. but again, 99 percent of ARM chips that are sold are not architecturally designed ARMs. They are existing cores.

PJ:

right.

Rob:

I mean, Raspberry Pi, for example, Raspberry Pi is just an off the shelf ARM core. They put them all together in whatever package that they have for their current design. And, the Raspberry Pi. It's, if they had to design that from scratch, then they would never have had enough resources or it would have been rubbish. that's kind of the borderline there. You can design a microcontroller and it can be rubbish and it'll still work. It's just fine. You

PJ:

Sure.

Rob:

you can make a, an ARM2 or a Cortex M0, which is probably its closest relative, relative to the original ARM. can design them and of people have done it and designed their own. In FPGA and it works fine. You don't have to be a top notch silicon designer to make these microcontrollers. only gets really difficult when you get to the really high end stuff, when you've got everything out of order, multiple calls, multiple execution resources, you decode in instructions to micro ops, so you can utilize all of these internal resources. That's when it gets state of the art chip design is at that level. This is the, the new Intel level. This is the Apple Silicon level. That's not really in place yet for RISC V I could quite easily make a RISC V core that ran it. 20 megahertz, the interface to a few peripherals. I probably screw a few things up. It would work enough to get the job done. And that's all that matters. I could, and I can do that today. I can lay that myself. Doing that at 5 gigahertz. It's significantly more difficult when you've got all the silicon working against you. You have all of the execution working against you. Yeah. I mean, you can't literally run a processor at five gigahertz if you execute them in in sequence. You just can't do it. You can't take an original in sequence processor and clock it to five gigahertz. It just won't work.

PJ:

Well, it's interesting to think through this problem then from the financial incentives, because as you say, you want someone to start producing some reference chips where it's like, Hey, you can go out and use this. But. In order for someone to produce some reference chips, there has to be sufficient financial incentive to justify, doing everything needed to fabricate one of these things because it's going to be expensive. Like, we're not talking about, now I can get an ARM chip for pennies on the, like, pennies on the dollar. Like, we're talking about a significant amount of capital that needs to be dumped into this thing to actually produce just even one reference chip. Right?

Rob:

Yep, and that's kind of it. Like, we're talking about these license fees, like the super expensive. It's pennies per chip at the most, because you could buy a Cortex M0 for pennies. If you buy them in bulk, you can literally get them for a handful of pennies so the license fee itself can't be that much, but it adds up, I don't think Apple or Nvidia have a license where they can redesign these. I think they only have 64 bit licenses. So a lot of these embedded controllers can't even be designed under their architectural license. And obviously they could not tell anyone, but it's all comes back down to how badly do you want to break the license and how much do you want to get sued? But as far as I know, if, if, if like Apple or Nvidia use a M0, they just take it off the shelf one, pull it in. So they

PJ:

Right.

Rob:

the license and they pay him for that core design. RISC V should just start making core designs and be like, this is a implementation of our base instruction set for a minimal power microcontroller that runs up to 10 megahertz, basically, and then have one that will run up to 100 megahertz and then have a handful of different designs for the base instruction set, which is good for Microcontrollers all the way up to the full fledged thing that has floating point and everything else that runs at five gigahertz. It's like, that's it. But who pays that is the big question.

PJ:

That is it.

Rob:

cost those Apple silicon S level designs cost hundreds of millions of dollars.

PJ:

Right. And I could look at an analogy to this, which is. I could say, hey, the Kronos group did this for OpenGL back in the day, where they effectively define what the API needs to look like and everyone else is responsible for actually implementing this thing. But the Kronos group wasn't going to be manufacturing GPUs at that point in time, like that was

Rob:

And it's

PJ:

ATI, that was NVIDIA.

Rob:

Linux exists in this exact same space, but anyone can take a Linux and build it on a custom piece of hardware. It's not that difficult to build a working kernel. Now, imagine if you had to take, Oh, this is the Linux system calls, but you've got to go re implement a whole operating

PJ:

Yeah,

Rob:

it. Looks like Linux. Imagine taking, I want to take the Linux user mode stack, I'll take all of Ubuntu from the kernel up, and I'm going to rewrite the kernel in my own way, but all of Ubuntu is still going to work.

PJ:

right.

Rob:

That's the sort of a thing you're taking on when you start designing these high speed chips. Like, this is just the shell interface to the internals. Design your own internals. That's no different to saying, this is the syscall list for Linux, make everything else work. It's incredible amount of work, and it's an incredible amount of effort. And in software, you don't have to make these things. So we just take what's already there.

PJ:

Right. Sure.

Rob:

RISC V exists in the same world as Linux. Exists in, I mean, you could literally take risk five core run Linux and everything from the RTL all the way up to the application is open source.

PJ:

I, again, like, you know, I could take a Linux, I could toss it onto a lot of existing hardware. RISC V, like we actually need some hardware to be created to implement that. Right. Hmm. Yeah.

Rob:

computers with high-ish powered risk five processors, but they're not that high Power arms still much faster and they come in somewhat concerned, all these higher powered RISC V cores, not all, but a good majority of these high powered RISC V cores are Chinese. So you buy these Chinese single

PJ:

Let's talk on this front.

Rob:

Let's have a look at, see how fast they have RISC V up to.

PJ:

Yeah. We haven't even gotten to the security concerns, but I think this is a good time to turn into that topic.

Rob:

Some people are claiming like, Some of the best performing risk fives are maybe 30% faster and 20% more efficient than a Cortex a 55, which is a 64 bit arm and maybe have the same or better vector performance, and or a space mi or something like that. But the processor is getting there, but they're fairly old arms. It's like there's no a 78 performance cause

PJ:

but to be sure, when we say efficient, we're talking about power at that point in time, correct?

Rob:

yeah. It's, uh, power efficiency and overall performance are the two metrics, I guess, that we use today for how good these things are. Just because you're twice the speed, but you take 10 times the power, doesn't make you terribly useful because you're soon going to overheat and have to slow down. So you have to take these numbers with a grain of salt of like, are they peak

PJ:

Right.

Rob:

Or are they sustained numbers?

PJ:

So you, made a comment a second ago about there's concern with these higher powered risk five chips being produced we chatted a little bit offline about, Hey, are there potential back doors that could be baked into the hardware that couldn't be patched that would allow basically. You know, some foreign actor, let's say, to get access to these chips at the deepest level. How much of a concern is this for real?

Rob:

It's definitely a concern. There are media tech chips out there with a full hardware TCP IP stack in them. They're still ARM cores, but they have a full TCP IP stack in them in things like routers.

PJ:

Yeah.

Rob:

but that gives the access to the outside world. The big problem with Hiding this sort of thing in a processor is what's the chance of that going in a device that has any access to the outside world? routers, the obvious one where there are obvious security concerns that you have to look at. And there are some open source routers that use the media tech hardware stack of like. though it's open source, it might not be buying you anything because it could be, who knows what that's

PJ:

Right.

Rob:

can't We can't have the designs. It's all hardware. is the truly a hardware TCP IP stack? I doubt it. I doubt anybody would. to design TCP IP solely in hardware, because the logic gets

PJ:

Yeah.

Rob:

If logic gets then it makes more sense to do it in software, which is why all these processes exist as embedded devices, because replacing them with dedicated hardware means it's not patchable, it means it's not updatable and means that logic is huge. And at some point, the logic is more complicated than.

PJ:

And it's expensive, right? I mean, I can take something off the shelf.

Rob:

I have an ARM processor and it can do A, B, and C, I mean, imagine having, uh, any modern ish device, the buttons that are on the front of a CD player, for example, device, it's 30 years old, uh, all the buttons that you had for all trap skip and then memory and indexing and blah, whatever it may be. Imagine doing that in logic with gates.

PJ:

Yeah.

Rob:

It's to put a dirty processor in there writing software.

PJ:

yeah. Yeah.

Rob:

And if you get it just fix the software. It's much easier than going back and redesigning all this hardware. Nobody in their right mind would make a TCP IP stack and directly in hardware, may have some hardware

PJ:

Sure.

Rob:

with some of the ultimately it'll be controlled by software so concerns. Are there security concerns about this Chinese made RISC V process that's just in an embedded device that's not doing anything that's connected to the internet? There's certainly less of a concern

PJ:

leak

Rob:

concerns about hardware designs and things that could spy and you'd never know that it's spying. But this whole firmware, like it's not up to, I read an article where it's like, well, these chips are not updatable. Most chips are not updatable. I mean, Intel have replaceable microcode, but it's only for a handful of instructions. They can't. It's not like the chips, all software like that, microcode Patch can't do much. They couldn't patch, they couldn't fix Specter or meltdown with a Microcode patch. So if it's like, if it's this address in the microcode that you're trying to run, and these registers all match, then redirect to something in the which was put in by software. Um, That's kind of how it works. So you can't just go, Oh, I'm going to rewrite all the internals of the chip. It's, I'm sure over time it's getting more and more powerful, but we're nowhere near that level because now you've just got more software running inside a processor that's supposed to be hardware. So like, that's just totals all the way down.

PJ:

All right, so there's a little bit of FUD that's out there that's not specific to RISC V around, hey, this is less secure. It sounds like you can have, especially in the use case we've been talking about for these Embedded microcontrollers that are the traffic cops. They aren't going to be able to get contact with the outside world. And the likelihood of them being able to spoil anything is actually really low.

Rob:

Yeah. bought Chinese calls to do those things, like I said, they're not connected to anything. They can't do anything. do their job or they won't. And if they're monitoring what job to monitor. It's a processor. Uh, It just, what's it doing with it? And if it's like, say, if it's embedded in another chip that that chip has, it's in the GPU. And then ultimately, if that wants to get data out, there has to be a way for it to get all the way back across through the driver, out of the ethernet port, to whoever the Chinese overlords are. It's a bit of a far fetch without anybody spotting that much traffic going.

PJ:

I mean, you have to like say, Oh, this is a, I don't know. It's like the, the processors have been hacked and I've hacked this game to send this particular set of stuff into this. But if you've hacked the game, you've probably get the data in a much easier way. So at that point in time,

Rob:

hacking the microcontroller inside a

PJ:

right.

Rob:

I don't V is not Chinese in itself,

PJ:

Correct.

Rob:

it's designed in Berkeley, I believe, and Now it's hosted in Switzerland as a neutral country,

PJ:

Correct.

Rob:

can tariffs and, export Yeah, so it's like, if you pull a RISC V processor with a hardware stack in a router, that was made in China, then, potential, you've got to look at that pretty closely. You're also assuming that all Chinese chips are bad and they're not. if you're designing your own RISC V in a GPU and it's sending data home, then you did it. You designed it.

PJ:

and, you know, we should also point out that even if there was a vulnerability, that doesn't mean it was intentional just because shit happens and someone may have just made a mistake, right? Like it's like, does it necessarily mean because of the vulnerability and something that, Oh, this is proof positive that someone's trying to spy on us. It could just be like, Oh, I fucked up the instruction when I, when I implemented it.

Rob:

There's all sorts of things like that. And, and it does come down to like, what can you leak from that? If you have the, some bad implementation or you can hack the firmware in these GPUs of like, are they being used for crypto mining? Can I find keys and It's like, the answer is probably yes. Um, Why have tech securely seriously at all levels. is that any different than it is today? Not at all.

PJ:

Right. Exactly. From a, a typical user or programmer standpoint, this is still an almost irrelevant argument or kind of like happening because like these microcontrollers are not going to be noticed by the users and they're unlikely to be ever programmed by the the programmers.

Rob:

If you ask people like, what processes are in an iPhone, they'll go, oh, there's the six calls of the main CPO, but they don't realize that there's 35 other

PJ:

Right.

Rob:

around everything from USB C to is, uh, the, the image stack is full of them. The, they have arm controllers at the end of PCIE. Buzz is just a handle. Various things.

PJ:

What's fascinating to me is that, you know, these decisions about what processors to use really are going to be up to these high level chip designers who, and, and the finance folks to figure out what is the break even point for where it makes sense to replace our arms and our cores, uh, with a risk five design. Because these microcontrollers, like, it's like, it's enough of a savings going forward that. We don't want to be dealing with the the architectural license or even just the off the shelf license anymore We want to be able to like save like at this very lowest level that most people aren't gonna care about

Rob:

external users don't see any of

PJ:

Right

Rob:

I'm like, ultimately this chip needs to do ABC today. We could do this this way, but we'd like to get to a point where we could do these other things. And this leads towards. A given architecture, that given architecture might be RISC V, it might be ARM, it might be doing it in house. And that has this cost associated with it, this development time and either side saying we have to do this for cost. We have to do this for architecture. It's the whole plan. I mean, like I said, none of this is visible to the external All we're selling them is we're selling this product of a GPU. How we make that GPU internally is our business.

PJ:

Okay.

Rob:

now, uh, I don't know, but ARM is expensive. There's no, let's not, let's not beat around the

PJ:

Thank you.

Rob:

It's not cheap have your own ARM processor.

PJ:

Yeah, and it's, but it's fascinating to think about a world where, x86, we believe will still be a lot around for a while to come just because of how much stuff requires x86. ARM, I expect will be around for a while as well. And then maybe we just start to say like RISC V, you know, occupies this lower end space. So you have these sort of high, middle and low tiers for what processors end up doing. Silence.

Rob:

the Cortex A, R, and A series, like the M's are the embedded ones. The, A application processes are the The big heavy weights at the middle, uh, with slightly more power than an embedded controller, less than application processor kind of thing. You could put in a car and that sort of thing. there is this massive spread and yes, you could put a microprocessor in pretty much any device these days and you can get them for pennies they're just there to save logic. If I can put all of this in. Logic decision making into a piece of software and have this 50 cent chip, then I got a fixed cost and can add functionality and software where if that was adding hardware, we have to redesign, redesign, redesign, and even if that's. Yes, you could do logic in FPGAs, but they're crazy expensive.

PJ:

most of the time, cost effective,

Rob:

Getting interface like of buttons. And if you hold this one and you click this one, or you hold it into three seconds to reset it,

PJ:

correct?

Rob:

sort of behavior, it's software and it's running on a tiny, tiny. controller. However, ARM's not the only player in the game yet. The Chinese make some micro controllers, which you can get them for like three pennies and they have a full dev kit. PIC still exists. There's a whole bunch on the small side. There's a whole bunch.

PJ:

We

Rob:

controlled. the buttons on your microwave and your oven are all definitely software controlled because it's easier to do in software. to do the rest of the software. There's a whole gamut of processors that are under it that go all the way down to the smallest, most mundane tasks that you can imagine. And, and cars are a huge consumer of microcontrollers. These days because every single little device will have its own microcontroller because if it talks to the cameras, it needs a microcontroller. So these things are literally everywhere in cars. It's not just like there's the ECU and that's it. It's like there's hundreds of them scattered about doing all

PJ:

this actually then, you know, brings an interesting question because do we, do we actually get to more divergence of microprocessors than convergence? Because it seems like, you know, we, still got its old, uh, AIX machines, mainframe still running. We'll have x86 forever. We'll have ARM forever. RISC V's coming on board. PIC seems like it's not going anywhere. Like, do we just basically just keep multiplying out these processors? Like, for whatever things we need. And we never actually shrink them down.

Rob:

we're at a, point where no one's making

PJ:

Oh, fair.

Rob:

It's, like, if you want to operate, you'll use Linux or you'll

PJ:

Yeah.

Rob:

and you'll modify it. I mean, yeah, I exist and like all the old Unix y things still exist, but they've mostly been replaced by it's you're not going to, well, I'm really gonna, it's not going to write a new one. If you've of academic. Experience. You can go write all that yourself and it would be an academic exercise and you'd learn a lot. If you're a company, you want to hire people, then you need to use their experience and their experience is probably in Linux. And they don't want to build experience in your

PJ:

Right.

Rob:

fucked up system. They want to build. Experience and it's something that's going to serve them in the future. And that's where we are today with Linux. And RISC V going down. That's the exact same path of Everyone just knows how it works. Um, everyone just knows what to do in this scenario.

PJ:

Well, what's fascinating is like Linux, you know, obviously is not a monolithic thing. It splits into a lot of different flavors, but it's all fundamentally Linux. And I guess what you're like, part of what you're saying is that the same thing will happen where it's like, Hey, we want to reutilize our knowledge sets on risk five. So. Because it's open source, because it's following a similar pattern to Linux, that same set of people that basically just want to do and not have to fuck around with basically a new instruction set every single time, we'll just say, Hey, well, it'll be risk five. Maybe it'll be a slightly different flavor, but it's all the same.

Rob:

I mean, imagine if, if I was a RISC V designer working for some company, I've done a few designs and I go work for a new company, then I'm pretty much going to promote

PJ:

Yeah,

Rob:

V because I can And I think that's a lot where arm strength came from is because they're in that peak

PJ:

yeah,

Rob:

where everyone who designs arm purchases and risk fives just catching up. But ultimately it's the people who need to do this. If you look at it from a very personal point of view of like, I would do this, but you can't do the whole thing. You need help. A company needs to make money and it needs to be sustainable. And that's kind of where these trends ebb

PJ:

yeah, Yeah. I mean,

Rob:

financial incentive or a technical incentive. It's It's the whole ecosystem that will push it in the direction that it wants or needs to go. And the more easy you make it to adopt that ecosystem, more traction it will get. If like, if you have to fight arm every step of the way to do something, then people are just going to go, we'll just do it in this five, and at some point you go, you know what, this worked really well. This was a dead end. keeps that innovation going where arms. you start to get into these licensing models where you can do A, B, C, you can't do D, E, and F, it's kind of stagnated. It's like, okay, you've decided how I'll make my thing, without even knowing what it is. Well, at least in RISC V you can go, I'm gonna have a go anyway. And at some point you're like, Damn, that worked really, really, really well. Oh, that was a terrible idea. Whichever. It doesn't matter which path you go down at the time. It might, but in the, for the ecosystem, it doesn't matter. Everyone learns. And ultimately this gets folded back in and we all know not to do that again.

PJ:

it's exciting times in that sense. Like it means that we're back to the wild, wild west again with this and it opens up the door for possibilities.

Rob:

The ability to make the product as you want to make your product has never been better. You don't have to deal with the windows license. You don't have to deal with, can I add this to the windows kernel answers? No, you can't. How much innovation do these companies hold back not being willing to play along? And this happens to all companies. Nvidia are equally bad of when we were also on, this may have been the PlayStation. It's. Their engineers are going, you'll use the GPU this way, and we're like, no we won't, we'll do it this way. And it just makes more sense for games. So, you always get these, these clashes, and sometimes you work through them, and sometimes you don't. And, that goes across the board, whether it be, going back to the original Xbox, I want to put the Nvidia chip and the Intel chip in the same package. Not possible, can't cost reduce. can't be done. You're not going to have enough synergy between those two companies to get one of them to give up on the fabrication process and let the other ones fabricate so there's it's never just a technical solution or just a business solution, especially when you're asking other companies to invest a lot of money in your

PJ:

Right.

Rob:

Companies want to own it. And when they own it, you lose that innovation because you only get their innovation. Your innovation is irrelevant because it doesn't benefit them. Where if this stays fully open, everyone gets the benefit of everybody else. And shining

PJ:

but I think, you know, you have the larger point here of like at all levels of the stack where Linux is the example for the operating system. Unreal is the example for the game engine and RISC V is the example for hardware.

Rob:

Yeah.

PJ:

But it gets back to your point that because the possibilities are starting to be incepted today. Now, if you want to do something interesting, it's available to you. May not be the cheapest thing in the world, but it's possible, which is pretty cool. So let's see where this innovation goes. I think the open source up and down the stack is awesome. And then who knows, maybe one of these days we'll figure out how to get an open source GPU out there until then. Let's see where the visionaries in the economics take us.