LIVE from GTC: DGX Spark Insides First Look

Uh, hi. I'm here with, uh-

Hi

... Israel from NVIDIA. Welcome.

Hi.

Um, what is your role on the DGX Spark?

Well, I'm, I'm doing the tech marketing for it.

Yeah.

So, I'm the person that bridges the technical teams and the marketing teams. So I know enough about the hardware, but also in a way that it can be, uh, digested, I guess, by -

Yeah

... by the non-tech people.

Yeah. So maybe let's talk about that, right?

Sure.

This is a $3,000 device. Uh-

Starting. Starting at $3,000.

Starting at $3,000.

Yep.

Wait, what does it go up to?

4,000.

Okay. So like just l- extra upgrades or something like that.

That might be a good first question for you, right? What, what's the difference between the 3,000 and the 4,000 model?

Uh sure. I don't, I don't know actually.

It's-

Just extra memory?

Well, it's storage.

Yeah.

4, 4 terabytes on the four- uh, on the $4,000 one, and 1 terabyte on the $3,000 one.

And what, uh, part of the market is this designed for? Like, do you have a name for... Like, I guess prosumer?

Yeah. It, it's, uh, we, we call it a, a mini AI supercomputer, right?

Okay. Yeah.

But, uh, the, it is, it is really in many ways a miniaturized version of these things you see behind me here-

Yeah

... like the real data center equipment. So the idea is you're taking a, a nibble of the, that data center to your home, so you can develop. Uh, this is really a developer box. We're focused on, on individual developers so they can do work on a platform that is really 100% derived from, from these, right?

Right.

The architecture, the CPU instruction set-

Software stack, yeah

... the drivers, the network component, networking accelerations, um, even the cross-memory, right, that, that we have here, the shared memory, uh, layout, that, that's the same that you would find on a Grace Blackwell da- data center machine.

Yeah.

And you can... Once you, you learn how to develop here, you're, and you write code here, this code is ready to go there into production to large scale deployment without having to worry about is this gonna work, you know, uh, is, is my stack ready for, for that.

No, it's the same stack.

So is this for... How much is, is hobbyist usage and how much is that kind of workflow where they're actually prototyping for the big data center?

Well, uh, honestly, the, the amount of things you can do here, I wouldn't call it a hobbyist anymore.

Sure.

'Cause you, you can do-

Yeah

... uh, some really serious AI, uh, work fine-tuning, which is very demanding, right? And this is one of our key use cases when we developed this product. So, uh, this is... Yeah. I would, I would say this is one step further from a hobbyist.

And, and today there's so much you can do in, in terms of bu- real business, right? Um, over, uh, say a 32 billion LLM, right? Just by adjusting that to a certain demand using fine-tuning, th- th- that's already a product in many ways.

Yeah.

Now, the only thing that you cannot do here compared to these things is, yeah, this is not designed to serve-

High batch, yeah

... 500,000 clients at the same time, right?

Yeah.

So, um, you, you do this at home, you test, and you, and you, you run, or even on a small company, and once your, your stack is good, th- this can be pushed out to a, a larger, right, either on-prem or, or, um, a cloud resource that will run.

And like I said, the, the beauty is you don't have to tweak your code. It's, it's just gonna move over to the next step.

Yeah.

Right? Whether that is the workstation that we have there or a real data center, uh, grade equipment.

Yeah. I was actually kinda wondering, so I know about the Jetson Nano, which you released a few months ago, um, and then we have the 5090s over there. Um, what is the sort of per- the, the, the product suite?

Like, how would you sort of grade it from like the small end to the high end?

Yeah. This is a little hard to compare with Jetson because, uh, it's all new.

Yeah.

Right? This is a Grace Blackwell, so all the s- all the silicon here is, uh, newer than, than what you would find. Um, but the, the idea is that, uh, this would be your, your starting point for a Grace Blackwell setup.

Uh, let's pause. No, go ahead. Oh, okay. All right. He's, he's doing it. All right. You can, you can. Yeah. Okay. Cool. Sorry, keep going. Yeah, the idea is...

Yeah. You, uh, thi- this is really your starting point as a, as a developer for, for the G- Grace Blackwell architecture, right? Um, w- we will continue to support different ranges of products for different types of development, right?

So we have the IGX that's more oriented towards enterprise work, and, um, it's, it's more of a professional tool, right, than, than this. And this is really more oriented towards a, a home user.

Yeah.

One feature that you find here that, it's here we'll... I'm, I cannot wait for this to reach homes and people start playing with, but this is also a, a 5000 series Blackwell in many ways, right? It has DLSS, it has RTX, and it has a very capable display output subsystem.

And, um-

Right over here.

Right. So there, there are things you can do here that we're not even touching just yet, um, in terms of, of marketing, right? 'Cause for now we're focusing more, like, on ML Ops-

On the office

... and AI, right?

Yeah.

But, um, uh, this is still a very capable-

Gaming machine?

... home computer. Well, yeah, gaming comes with, with, you know... We're s- this ships with Linux, right?

Yeah.

So that's your... You start from there, and it's an ARM machine.

Yeah.

So code, right, it, it has to run, and it, it, it's not... Today, you know, the, the Linux gaming on ARM is, uh, still like a, this moving piece.

Yeah.

But, uh, we've seen effort from Valve, right? So th- there are companies that are working on, on, on that. But, I mean, this was not designed for it, but-

Sure, sure

... still, you know-

It's capable. It's got the hardware

... I'm telling you what's in here-

Yeah

... silicon-wise, and-

Yeah

... I'm pretty sure people will find interesting uses for that as well.

Yeah. So, um, I mean, this is the first time, like, usually we see these boxes. This is the first time you sort of o- opened up the, that I've seen sort of this opened up.

Yeah. The, this-

Anything interesting that we should look at?

This is literally the first time-

Yeah

... we're showing this board. Yeah, so the board, the board layout's actually very simple, okay? So on the top here, we have the GB10 SoC, uh, that we built in partnership with MediaTek.

Mm-hmm.

Around it you have the LPDDR5X memory modules for your shared 128 gigbit, uh, gigabyte memory space that these two chips can reach out to without having to create copies of the memory content, right? So that alone should give you an enormous advantage on certain workloads that you have a lot of transit between your, your RAM and your vRAM, compared to, say, even a powerful workstation th- but that you're still using a regular PCI Express card, right?

So it's DDR5, is that what it is? Y- uh, yeah, it's LP, LP 'cause, uh- Low power DDR ... this is more like a laptop memory- Yeah, yeah ... in the electrical sense than, um- So it's low power DDR?

Yeah. Okay. But the, the performance is great. It's actually a little faster than a socketed, um, DIMM. Yeah. Comparable with, like, using the same type of, uh, chip, right? Right. Right. And we have the C2C, uh, interconnect. That's the te- this is NVIDIA, right?

So it's our design that connects the GPU to the CPU, and allows for... We're estimating there's somewhere in the five times faster than PCI Express, uh, communications here between these two chips. So this design- So they share the same memory controller and everything, right?

Inside- Yeah. Uh, the, there, there's the, the memory controller is basically provides access to both GPU and CPU. Yeah. Okay. Yeah. This, this is, um, actually a very interesting topic, but, uh, it's very deep as well. So I, I, I can only get this- Yeah.

... this far, 'cause I'm a marketing guy. Yeah. So- What is the networking on it? Is that- So, yeah. Oh, that's another crazy part. Yeah. Okay, so yeah, we were just talking about this, right? Yeah. So far we're talking about this, so let's move over this part of the PCB here.

So we have a, this is enterprise grade ConnectX-7 dual port 200 gigabits per second ethernet. 200 gigabit per port? Per port. Okay. Yeah. Which is wild, right? I don't, from top of my memory, I can't think of a box this small with a network this fast.

Now, of course, we don't expect people to have 200 gigabit ports on their home network. That'd be actually nice, but, uh, yeah, that's not happening. But this is designed for the scale-out option. So using, um, this is basically a cluster- Oh, to, to network them.

Yeah. Yeah. Okay. So using networked distributed workload balancing with TRT LLM, and all of our software stack for that, which is the same enterprise software, you can develop something at home that would run on, say, the NVL 72.

So is it shipping with, like, Ubuntu? Is that- Yeah. So, uh, we call it DGX OS, but this is Ubuntu 24.04 LTS. Okay. The only thing is, yeah, we, we add a few extras to it. So we add, um, I don't know off the top of my head everything that we add to it, but it has some performance optimizations.

It has our repositories and our software preloaded, so, uh, driver, driver for the ConnectX-7. Yeah, sure. All of these pieces are there for you. Those modules. Yeah. Are th- So did you guys announce when it's gonna be available, or price?

I, I understand- Yeah. So it is available right now for a wait list, right? So people that wanna get on the wait list, they, they can get in. Uh, our partners are also enabling their wait lists. One thing that I want all of you, uh, media to be aware is, uh, the, what, what our partners, ASUS, HP, Dell, and Lenovo, these are the four partners we have enabled right now.

We're selling the same thing, okay? The, the only difference is they, they're gonna have different case designs, maybe different cooling solutions. But, uh, this board is the same. The feature set is the same. The only variable that we're, we have between models today is, uh, we, on the bottom of the, the board that you can't see 'cause it's glued in a base here, um, there's an M.2 for NVMe, right?

Right. It's PCI Express. Um, there's a one terabyte disk for the entry model, and a four terabyte version for the top model. Is that- That's only, the only difference ... how much memory?

Is it user upgradable?

Uh, I don't, I don't know if we're gonna state it that way.

Okay. But it's, it's standard.

It's M.2. Yeah.

Okay.

How much memory does it come with? Sorry. There was- The base model, one terabyte. And the- One terabyte of DRAM. Uh- No, how much DRAM? Sorry. Oh, okay. No, yeah. RAM. D- it's 128 for all trims. All trims 128.

Oh, okay.

Yeah. All right. That's what I'm saying here. This is, this is something that I really want people to be clear about.

So what's the largest model that will run on that?

Well, we're expecting, uh, for the single unit, somewhere in the 200, uh, billion in FP4.

200 B? Yeah.

Which is, yeah, try doing that on a laptop. It's gonna be really hard.

Yeah. It is.

And for the stack up, it's, it's 400, right, with, with two.

And all, like, the Python software for, like, the data scientists, that works, like the QDF stuff will work-

Oh, I'm, I'm gonna have a guy in the booth later today, afternoon, uh, when we're done with the press, that he is working on the Python optimization. So you can-

Oh, okay

... he can give you a much better answer than I. Okay. Cool.

Why is that a question for you?

Why is that a question?

Well, yeah, why is-

Just the, I mean, typically X86 is the platform for a lot of-

Oh, okay. All right

... that stuff.

Yeah.

Simple. Okay.

So just making sure that, like, if I am writing Python code and I need to move it to this-

This fellow

... it just works right.

Awesome.

How many can you run in parallel? Uh, well, so far we're supporting two. Two? Okay. Right? But, um, again, this is- Stack them up ... ethernet, and, and the scale out is using, I- is software based, okay? We're offering you the, the same software that we use for very large clustered distributed workloads on, on systems like the NVL 72 behind us here, to use here.

So that, that's the interesting part. You have a mini lab at your home that's using basically the best scale out technology available today. If you learn here, you can just go there and do the same with the same code, with the same everything.