Palmer Dabbelt

Deepseek R1 crashed Nvidia stock by 16%. So I figured let's read the deepseek paper and what's this all about. It's surprisingly readable but light on details. I attached my annotated paper for you to read. Now coming to JOHNAIC, yes, we can run Deepseek R1 on our machine. My customers have already run it on their JOHNAICS. Just did some basic benchmarking, we get ~84 tokens per second. This is especially important for us because the R1 model will produce more tokens than other models. What's so awesome about deepseek is their training method. They've relied more on RL and synthetic data rather than standard human annotations. This makes model training significantly cheaper because annotations are super duper expensive. They trained a big model with this and then distilled this model into smaller models that we all can use. That's how we get smaller models that can be run on our own machines. Surprising thing is even 7b parameter models beats GPT4o!

118

13 Comments

Over the years I've heard a number of stories where startups had decided to redesign X from the ground up for Y. But they're not often as compelling as Brandon Lucia's idea to rebuild the general purpose processor for efficiency. Efficient Computer's Electron E1 is the result, a chip that strips away a lot of the infrastructure of a CPU yet can still do CPU things. (Thanks Todd Austin and Rakesh Kumar for the chat.) https://lnkd.in/es6qmKzh

127

2 Comments

Excited to share that our paper "RANBooster: Democratizing advanced cellular connectivity through fronthaul middleboxes" has been accepted to ACM SIGCOMM 2025! 📡 As 5G RANs shift toward virtualization and disaggregation, there’s growing room for innovation from smaller vendors and open-source projects — but challenges around programmability, data access, and control remain. 🧠 RANBooster tackles this by introducing a middlebox architecture that enhances RAN functionality via the open fronthaul interface, with no changes to existing components. 🚀 We demonstrate RANBooster across several RAN stacks and an enterprise-grade private 5G deployment, with 4 reference apps: • DAS • Distributed MIMO • RU sharing • Real-time radio resource monitoring 🙏 This work is the product of an awesome collaboration with Tenzin Samten Ukyab, Bozidar Radunovic, Sylvia Ratnasamy and Scott Shenker. 📄 Read our paper here: https://lnkd.in/dP5x_3PM 💡 Stay tuned for more updates about the open sourcing of the project! #5G #OpenRAN #RANBooster #SIGCOMM2025 #Networking #srsRAN

124

5 Comments

You see the pink-ish and purple-ish bits in the pic below? That’s the cache memory on an AMD Ryzen 5000 processor chip. It takes up pretty much 50% of the entire surface of the chip. Now imagine if the cache memory could be stacked on top of the Core CPU circuits instead of side-by-side. Half-size chips for the same processing power! Or, much more powerful chips in the same space. Cache memory is used by the CPU for calculations, and operates between 10-times and 100-times faster than RAM used for data and programme storage. The more powerful and faster the processor, the more cache you need, so space on the chip can really become a problem. Experienced ex-Arm engineer, Kauser Johar (and more recently, Head of Technology at Cambridge-based quantum computing startup, Riverlane), believes he has a solution. Along with semiconductor industry program manager Catriona Wright (also recently at Riverlane), they have just launched ‘stealth’ startup Chipletti to develop vertically-stacked cache memory ‘chiplets’. Chipletti (OK, it sounds like a brand of Italian French-fry but it ‘does what it says on the substrate’) is one of the UK semiconductor startups selected for the third cohort of the Government-sponsored #ChipStartUK programme, managed by incubator Silicon Catalyst.UK (see my prior post on the recent ChipStartUK Showcase). Vertically stacking memory chips is pretty ground-breaking stuff. Of the Tier 1 semiconductor companies, only AMD has yet demonstrated the technology. Intel and Apple are expected to follow suit. What Johar and Wright want to do is ‘democratise’ stackable chips for Tier 2 and Tier 3 semiconductor companies who don’t have the scale or capability to do it themselves. Having only started full-time at Chipletti in February, they have already raised a modest amount of angel funding to commence developing the chiplet technology design. They are now hoping to raise a further £1.5m to hire a small team to help them complete the design and then manufacture sample chiplets. It inspires me to know that the UK has entrepreneurs like Johar and Wright developing semiconductor technology with the potential to transform the design and performance of ever-more powerful computing chips. Good on them! #semiconductors #chips #cachememory

125

3 Comments

Are LLMs truly intelligent? The jury's still out—but they're undeniably well-educated. We recently put 13 popular LLMs to the test with 100 advanced signal integrity questions. The results reveal just how much technical knowledge these models have absorbed during training. 📊 What we tested: Advanced SI concepts across major commercial cloud-based models 🔗 Questions with answers are at https://lnkd.in/gXp5hs7H Coming next: We're analyzing 90 open-weight models (including several that run locally on laptops) with the same question set. Results in my next post! Your turn: Have you tested LLMs on specialized technical knowledge in your field? Share your experiences below! 👇 #SignalIntegrity #AIEngineering #PCBDesign #Simbeor

30

10 Comments

Robotics is at a turning point. The fundamentals—chips, connectivity, compute—have finally caught up, and with Viam, the tools to integrate them with hardware are widely accessible. That means engineers can stop asking, “Can I even get this to work?” and start asking, “What’s the right problem to solve?” We’re already seeing incredible answers emerging on Viam from startups like Gambit Robotics, Inc. building intelligent kitchen robots to enterprise teams driving new automation across industries. Thanks to Patrick Kulp for visiting the lab to see some of it firsthand and for the thoughtful feature in Tech Brew. Read the full article here: https://lnkd.in/ekKuNaCJ

75

1 Comment

The claims of reduced compute and energy demands for Deep Seek type models may be overblown. We began evaluating Deep Seek last week (like everyone else in the world) to assess its energy impact and optimization capabilities for our platform. Below are the results from our real-world inference test suite. I chose to compare Llama 3.3 70b to Deep Seek r1 70b here, as I can effectively stress a single H100 with each without extensive setup. There is an apples-to-oranges element here; while Deep Seek r1 70b is distilled from Llama 3.3, it is designed to provide a chain of thought and generate a much longer output window, whereas Llama 3.3 is tuned for chat and some reasoning. I am primarily interested in the energy consumption of each and will normalize the results (both have the same number of parameters, which helps control for that). The net outcome is that the two models have approximately the same energy efficiency (tokens/Ws) with slightly slower performance (4.8% tokens/s). However, Deep Seek r1 70b has about twice the runtime (due to generating so many output tokens), resulting in 87% more total energy consumption compared to Llama 3.3 on the same set of prompts. Even if the token output was constrained the energy efficiency per token will be similar meaning that the total energy impact compared to more commonly deployed models is likely the same (output constrained) or worse (longer outputs). While the consensus seems to be that training Deep Seek likely used significantly less energy than comparable models, we believe that inference will constitute the bulk of energy and computational usage in the future. Better models will be more useful, and even if we improve their energy efficiency, we will likely encounter Jevon’s paradox, where they will be even more widely used. Looking at the data I can only anticipate that these innovations are going to results in us staying on the same energy consumption growth trends as prior to DeepSeek v3/r1 being released.

346

75 Comments