Rendered at 06:30:17 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
jmward01 26 minutes ago [-]
I didn't see cost. If cost is similar to flash drives then this could be massive. Every GPU ships with a lot of ram still but for AI inference and games you have 10TB of this stuff to stuff all your textures into and all your model weights in. For fine tuning models this would also be great if using lora or similar.
sixdimensional 2 hours ago [-]
So like.. conceptually kind of like memory mapped files on fast flash persistent storage, IIUC? Or maybe it's more like GPU-managed demand paging, caching and DMA? That could get you the capacity and better I/O characteristics.
I'm curious about how the unifying architecture is going to evolve between CPU/GPU having direct access to a singular pool of memory/storage also.
I also keep wondering when memristor technology might enter the ring, because as I understand it, it would be like moving compute into the memory, which would potentially remove the need to move the data in and out of storage as much also.
It feels like computing hardware infrastructure is fundamentally evolving.
dlcarrier 24 minutes ago [-]
Intel really missed out, when they discontinued the Optane line, right before the RAMpocalypse.
hankbond 2 hours ago [-]
Necessity being the mother of all invention. I also thought the compute-in-memory approach was interesting re: https://mythic.ai
trollbridge 2 hours ago [-]
I've been wondering when we'd get around to having the equivalent of "memory that runs at very GDDR6 speeds for reading, but is much slower for writing", which is exactly what you need when working with an AI model. Versus current HBM which has the same speeds writing as reading.
Animats 1 hours ago [-]
Good use case for fast read, slow write.
wills_forward 2 hours ago [-]
stick enough floppy's in parallel and you could do the same thing
readthenotes1 2 hours ago [-]
Go ahead, take it further.
Imagine you are immensely intelligent beings who project into our space-time Continuum as mice. You need a lot of memory and processing but you don't need it quick. What do you do?
(With apologies to Douglas Adams)
cuu508 2 hours ago [-]
I have fond memories of Flash 5 in early 2000s. At the time I knew BASIC and Pascal. I could figure out how to draw things, how to animate them using keyframes and motion tweening, and how to make things interactive with ActionScript, all by poking around and reading the help manual (F1) on our underpowered offline PC.
I'm curious about how the unifying architecture is going to evolve between CPU/GPU having direct access to a singular pool of memory/storage also.
I also keep wondering when memristor technology might enter the ring, because as I understand it, it would be like moving compute into the memory, which would potentially remove the need to move the data in and out of storage as much also.
It feels like computing hardware infrastructure is fundamentally evolving.
Imagine you are immensely intelligent beings who project into our space-time Continuum as mice. You need a lot of memory and processing but you don't need it quick. What do you do?
(With apologies to Douglas Adams)