Tricky Bits with Rob and PJ
Rob Wyatt and PJ McNerney discuss the latest and greatest news in the tech world and to figure out where things have been, where they are, and hopefully where they are going.
Tricky Bits with Rob and PJ
Why is Hardware Hard?
Enjoying the show? Hating the show? Want to let us know either way? Text us!
Hardware is where the digital meets up with physical reality.
In the modern world, software can be developed and deployed worldwide in timescales measured in minutes....but despite advances in development, consumer hardware at scale still presents many difficult challenges.
Although there are many modern avenues to prototyping hardware solutions, making the leap beyond RPi, Arduino, ESP level projects into mass market devices encounters challenges including funding, development, manufacturing, shipping, supply chains, and revenue models.
In this episode, Rob and PJ explore the world of hardware development, understand its pitfalls, and look at where it might be going.
Welcome back everybody to Tricky Bits with Rob and PJ. You know a lot of the discussions we've had are around software, around the advances, around new apps. But it's really important to remember that the software can't run in a vacuum. It has to run on something, and that something is hardware. I mean computing started back with vacuum tubes and primitive transistors primitive transistors and one of the hallmarks that I think has been really fascinating is that software, over time, has gotten easier to use. There have been easier programming languages, there are easier paradigms, there are ways to actually engage it at any age.
Speaker 1:Hardware, on the other hand, has always been hard. It requires, at scale, still a lot of capital expenditure up front. There's the production, there's the testing, there's the yields, there's a whole slew of things that still makes hardware hard today. And even though we've got this rise of a lot of low-cost hardware platforms like Arduino, raspberry Pi, esp8266, which have made things easier to play with, it's still hard to have consumer-level hardware that actually works. And, rob, I think it'd be really useful to dive into why is this still a hard problem and what makes making these physical devices so difficult?
Speaker 2:So one thing I'll say out there is hardware is easier than it used to be, and in some ways that's a blessing and a curse. Today you have all these little SOCs that you can get, which just boot up. They have internal clocks, they have all of the peripherals integrated, they have ADCs built in, they have DACs built in and things like that. Years ago a lot of this would have been all external componentry. It would have been a lot harder to bring these devices up. And if you go into the sensor world today you can buy finicky analog sensors that just output digital data and you just literally connect it to I2C, you connect it to SPI and it just works. All of the magic that's happening is in this little IC and it's just communicating with digital.
Speaker 2:30, 40 years ago that would have been a bare analog sensor. You'd have had to have designed 10 op amps around it and stabilize the voltages and do everything you need to do have your own analog to digital converter and then feed the result into some sort of digital interface, into the microcontroller. So across the board it's a lot easier than it used to be, but it's still not easy, and I think a lot of the open platforms like Arduino, esp, pi and things like that open up electronics to the masses, which is great. People start to understand electronics more and that bit's kind of gone down the same path as software went down. And then people make these cool products with Arduino and you get to where, okay, I want to commercialize this, I want to make a commercial hardware product. And that's where it gets hard, and that may be harder today than it's ever been.
Speaker 1:It's fascinating. I agree that these small, low-cost products Raspberry Pi, arduino have made it seem a lot like software development, where it's like, hey, I could just pull off this compiler and throw a few things together and, oh look, now I've got something connected to Wi-Fi, bluetooth, and I can turn on a light In the software world. I could just deploy this to some cloud. I could deploy my app to some cloud and trust it's getting replicated a thousand times, but let's get into what you're saying. Why is it harder than ever, though, to make that leap from my little Arduino to something I want to sell at Best?
Speaker 2:Buy. Every single step of the way in hardware is kind of fraught with problems, starting at the VC level. If you have a good idea for software, then finding VC funding is quite easy and the cost to scale in software, like you say, is zero. You just deploy a few more Amazon servers and set up your scalability and you're good to go. And that's not as simple as it sounds, Don't get me wrong. That's difficult there and there is some cost associated with that, for sure, but it's not like the cost associated with hardware and VCs tend to back away from hardware. They don't like the idea that. Okay, so now you've got a finished product and it still can't be sold because you have to make it. And and what's the real cost of making hardware? And most hardware devices you buy today are subsidized by their back-end services. Nobody makes hardware today.
Speaker 2:That's just hardware. That's what Arduino is for. It's if you buy a camera, if you buy a doorbell or whatever it is. Everything has a service, and that service is typically a subscription, which is how they got the product funded in the first place. Without that subscription, no VC would have been like yeah, we'll fund that. So everything always comes back to the software.
Speaker 2:Backend Hardware is literally a means to an end as far as commercial hardware is concerned. If you're funded yourself, you can do anything you want. But if you go into VCs you better have a software play. That's going to be the scalable part or the sustainable part where we can take subscription revenues and all that. And this whole thing gets into the argument of, like who really owns the hardware at that point. So let's go through the process of designing it. Yeah, you design it whatever your product is. Let's say it's a camera, and so you go pick all the components. You probably make 10 versions of it upfront to see like which one works best. You have to make a whole bunch of development boards so you can get access to all the test points. At some point you have to design for form, so you have to redesign that pcb again to get it to fit in its final form and just have to skip over the step.
Speaker 1:When I'm doing these development boards or redesigning the pcbs, I I need to go design this in something like eagle or send off an Eagle file to somewhere in China right to get these boards actually manufactured for me.
Speaker 2:It depends. Eagle is the tool of well, it was the tool of choice for the and I think Kaikan is the tool of choice these days For real development. It'll probably be something like Altium and that'll come into play later when you start doing noise analysis and EMI dissipation or you have high-speed tracks.
Speaker 1:But in terms of the physical manufacture of those development boards or the form-factor-friendly PCBs, those are still shipping off to China, right, they don't have to go to China?
Speaker 2:For the ultimate one, you probably will go abroad to make them. It's hard to make bulk PCBs in the US, at least cost effectively, for prototype boards. It really depends on how many you want. If you only want three or four, you get them made pretty easily, pretty quickly in the US. But if you're want three or four, you get them made pretty easily, pretty quickly in the US. But if you're making something like a platform that has developers, then you might need a few hundred of them or a few thousand of them, and now that's a small batch at that point. That's just. That's not a prototype account.
Speaker 2:Ideally, when you're in development, you'd make the prototype level. You'd make three or four of them for the handful of engineers who are working, actively working on it and then, okay, now it works, you might make spin a few of those. You might have to put bodge wires on them or whatever it is. But then you end up with a few working boards and these are not form factor boards, they're just like everything's broken out. Then you can make a bunch of them now, whenever they work. We could make maybe 50 of them and give them to developers or software developers or the app developers or whoever's working on this platform. But they're not final form. They're just on a piece of plastic on a desk with a bunch of wires hanging off them.
Speaker 2:At this point the hardware guys and I go back and make final form and go through the whole process again. In doing this starts to get hard now because you start taking debug connectors off the board and it's like I need that connector, and so you put test points on, or you put sometimes they're gone completely and you now solder into chip legs. So depends on what the end goal is as to whether these things get fully taken off or not. I mean, if you're making a game console, you probably don't want to leave the test points on the board at all, but it's kind of hard to make the final board without them. But then you, uh, you're down here, you've got all your software working, you've got all your hardware done. You now have to do all the regulation testing. There are no software regulations like you have to Regulations.
Speaker 1:Yeah, it's the Wild West.
Speaker 2:But yeah, it is the Wild West For some software. There are like mission critical stuff we're not talking about. We're just talking about I have a software product.
Speaker 1:We're not talking real time. We we're not talking about, we're just talking about I have a software product. We're not talking real time. We're not talking medical. We're not talking financial. Yeah, those are aviation or aviation military like these are all heavily regulated industries.
Speaker 2:Yeah, so all those have do have regulations. There's things like do 178 for the faa systems, which only applies to those systems, doesn't apply to general software. But now you've got to go. You've got to do emi testing. You've got to get fcc certification. If you have wi-fi, that has to be certified to make sure it's staying in its band.
Speaker 2:So a lot of people will be like well, I'll use a pre-existing wi-fi module because that's already pre-certified and I carry on that certification when I put it into my device, as long long as I obey certain rules. If you start messing with it you have to recertify it. But if you just drop it in as is and use the antenna and use it exactly as it was designed, you can carry the certification it already had. But that can add to the cost because now I can't use these low-cost Wi-Fi components. I've got to use this existing pre-made module which I'm buying as a unit. So there's a dilemma there, of like do we go through the certification process? Do we have the engineers who can certify Wi-Fi? Like, doing radio emissions at 2.4 of 5 gigs is not easy when you get the report back of saying, oh, you've got a weird lobe here that's interfering with something else. It's like what caused that.
Speaker 1:So most people will use pre-existingent wi-fi modules for this obvious reason give us a scale rob of like what does a low-cost wi-fi component versus a an all-in certified chip look like? I mean?
Speaker 2:cents dollars. It changes over time. The obviously this all-in certified chip has already been certified. So they've already paid a bunch of money to get it certified. So module bare wi-fi could be a few pennies right, grand scheme of things, because a lot of the digital stuff's in the soc potentially. And then all you have to do is the analog antenna sides. The pre-certified modules are typically a their own macro controller and in there they're dealing with all the analog stuff, so all the analogs designed it's on. It's under a little metal can.
Speaker 2:The antenna comes out and goes somewhere and then there's some interface to that and that could be spy for the low speed ones and sdio, pcie for for the much higher speed ones. But then you have to factor in like can I interface to it? Like, okay, this Wi-Fi chip is now okay, it's certified, so it saves us certifying it. But it needs SDIO to drive it. Does the other side that we're on our own embedded processor? Does it have SDIO? Does it have SDIO? Does it have PCIe? Does it have the interface? We need to talk to this. So as part of the design you have to pick all these components to interact based on the end result. So there's no point in getting all the way to the end and going, oh well, we can't actually have this wi-fi chip, we have to do it ourself.
Speaker 2:Um, but back to the original question of cost. The bare wi-fi doing it yourself, assuming you have the digital side can be pretty cheap. The modules today are also pretty cheap. They never used to be. They used to be like $5 or $10 a piece, which doesn't sound like much, but it is when you need a million of them. And things like the ESP32 and the RP2040 and some of the low-cost Wi-Fi solutions that they have now are certified and they're really cheap. So you can get them way under a dollar now for pre-certified Wi-Fi chips, and they usually have Bluetooth built in too. But that cost thing, let's go back to that cost.
Speaker 2:So you get to this end result you've got a product that's now certified by one means or another old. You've got a product that's now certified by one means or another and you still have to do FCC testing on your product. You can't pre-certify your design, sure? So you've got to know going in. Okay, these are high speed signals, they're going to cause problems. Let's put them on an internal layer of the PCB, let's shield them, let's do what we have to do, knowing that it goes in. So a lot of people will make hardware from the arduino path.
Speaker 2:They're not really hardware engineers and they'll get into whole world of earth when they get to the real production side. Because they just ran high speed tracks across the top of the circuit board and didn't realize that they're actually antennas, so got it lots. This happens a lot with kickstarter. You'll see hardware products on kickstarter and they never really get there. They keep making refinements and they keep doing updates but they never actually make that final product because they can't actually get it out.
Speaker 1:Because of the certification problem. Right, that's the crux of the issue.
Speaker 2:Oh, it just never quite works because of either certification of high-speed signals that aren't reliable, impedance-matched traces, which were never never impedance matched, which will stop things like usb working, hdmi, asata all these high-speed serial buzzes all have incredibly strict requirements as to how the tracks have to be laid out, and if you don't do it, it might work, it might not, it might not be reliable. It might not be reliable when you add in the errors of manufacturing. So some will work, some won't work. Happens to be the one you hand built did work, so things like that. It's like, at some point, if you're going to make a hardware product, you've got to take a step back and use a hardware team that knows all of these pitfalls and in the end, yes, it seems irrelevant in development, but in the end it will screw you. So you get development, but in the end it will screw you. So you get to where you are. So we've gone through all this process and there's a lot of development. All the software's working, hardware's working In the software world. When the software's ready, you just release it. When the hardware's ready, you now need to pay per unit to manufacture it, which is why VCs hate it.
Speaker 2:Yes, imagine, let's go to Apple scale. Let's say you're making a phone that costs. Say it costs them $500 to make physically make the phone casings, all of that and all the electronics, the screens and everything put together, assembly, all the things you have to do to get a finished hardware product. Packaging is included in there too. Let's say it costs $500 and you need 2 million of them or 10 million of them. Yeah, there's a few billion dollars in cost there that you have to pay up front to get those things made.
Speaker 1:And that's assuming we're assuming an ideal world. What I mean by that and you mentioned it earlier I can guarantee effectively that when I copy a piece of software from my computer to the cloud or my computer to your computer, accepting stray cosmic rays it will copy over perfectly, byte for byte, cosmic rays, it will copy over perfectly, bite for bite. When it comes to hardware manufacturing the situation you described where it's $500 a shot and I need 2 million units that is again presuming an ideal world where nothing in the manufacturing process fails. So there's a yield problem that is worthwhile to talk about here, right?
Speaker 2:There is a yield problem For final products. It's quite a good yield If not pick a Bella manufacturer.
Speaker 1:What's a good versus a bad yield at this point in time?
Speaker 2:A good yield is 99% of them in final terms, work. I mean, if you get a 1% failure rate on manufacturing, it's still quite bad. But it's like, where's the failure? I mean if you make it a finished product in a case, maybe it's machining that's the problem, maybe it's the way it clips together. A hardware problem also has don't forget, a whole other team of industrial designers who are making the case. The electronic engineers are not making the physical form factor, so all that is factored into the hardware product, so it's not just the electronics. So the failure could be oh, the case is not clipping together, the screws are pushing through. Maybe it's an injection molding problem on the electronic side.
Speaker 2:Pcb fabrication surface mount is quite reliable. But then are you buying pre-certified parts? Are the parts you're buying already been yielded, binned, for example? What I mean is are you taking on the yield errors of the chips being made or, as the manufacturer of those chips, already gone through them, tested them to make sure that these are the bad ones? These are in the bin, they'll never get sold. These ones are actually good. So when you put them on your board they will work and you think, well, it'll always be that case. But is it? If you're buying parts from suppliers who knows where they got them from? There's loads of reports of fake chips getting into military supplies, so you're getting bad parts that got pulled off the production line for some reason, you get them in your design. That's usually where the bad yields come from. If you're using like named parts with good suppliers, the ones you get should be good to dive into this one just for a second either can.
Speaker 1:I'd like it lovely if you could confirm or debunk a myth. I remember hearing that back in the day. I remember it was like the 486 dx chip versus the 486 sx chip and someone someone said eh, they're basically the same chip, except the yields went wrong, and so this one may not have its math coprocessor up and running. They just decided to sell it as this separate low-power unit or not as powerful a unit.
Speaker 2:Oh, absolutely happens, absolutely, to this day, it happens you take a 4090 and a 4080 is just a 1490 chip where some of the shader cores didn't work, and they actually design chips bearing this in mind, because you don't want to have a chip where these three have to fail. It can't be any of that fail. So they have like programmable routing of like okay, this one, this one and this one failed, we'll deroute this whole group and it now looks like it has X shader cores instead of Y shader cores and that's a 4080 instead of a 4090.
Speaker 1:The bigger the chip, the harder it is to make More of that can go wrong effectively.
Speaker 2:And we're not really talking about chip manufacturing here. Yeah, the more area you take on a given wafer, the more chance that there'll be a fault in that area, that of of your chip, and that chip will not work. So rather than saying the whole thing has to work or not work, we'll say like, well, all of this can fail and the chip's still good. There are areas that, if you, if there's a fault in those areas, then it's not usable, because that's maybe the control logic or the routing or the bits that are essential, but it increases the yields massively.
Speaker 2:And on these huge like Apple-sized chips or NVIDIA-sized chips, you do everything you can to get the yield and Apple do the same thing too. That's why you get Apple Silicon calls with a varied number of GPU calls. They're the same chip. They're not making different chips. It's the exact same piece of silicon. If you take a photo of them, they're identically the same, except this one.
Speaker 2:It might work, and there's been lots of reports of strategies to buy cheaper chips that you can overclock. I remember a story from back in the Xbox days of like Intel would take chips off the line and they'd fulfill the billing requirements of what the market needed. So they'd take a chip and they'd fulfill the billing requirements of the market needed. So they'd take a chip and they'd fill all the high performance ones first. So say, they'd test them at a gigahertz, okay, and all these work, and they keep taking chips off the line until they fill the quota for a gigahertz.
Speaker 2:When they've done that, they have all of these chips that are left over, but they'll fill the next bucket from the production line first. So these chips are coming off the production line, being tested at like 500 megahertz instead of a gigahertz and they were passing. So now a lot more are passing that criteria. But those chips were never tested at a gigahertz, so half the 500 megahertz chips would probably work at a gigahertz. They were just never tested at a gigahertz, so half the 500 megahertz chips would probably work at a gigahertz.
Speaker 2:They were just never tested, so knowing it's kind of a secret how they do the binning, to stop you figuring things like this out. So for lots of chips, especially like gpus and big pc chips, people will try to figure out like, oh, the way it was binned was this strategy. So therefore, if you buy this one, it'll most likely be the one that you can overclock to the high end one. Whether that still happens today I don't really know, but it's definitely something to consider how much of that is then disabled, though, at maybe the firmware level.
Speaker 1:Oh, lots of it's disabled.
Speaker 2:Now it's like when it's binned. They have like one-time programmable fuses and fusible links inside the chip and as they're testing them, they'll just have a laser in the factory that just burns something out Sometimes it's in software. Sometimes it's physically disabled. In hardware They'll just cut tracks in manufacturing when it's still a bare die and sometimes it's a fusible link that they'll blow in software later as they're programming like keys and things into it. They'll disable certain parts of the chip.
Speaker 1:So for the big players Apple, intel, nvidia, amd, et cetera they have robust enough lines that they can take these failures in yield and still sell them as product. Failures in yield and still sell them as product. But coming back to the, you know I'm a startup and I want to do hardware. In all likelihood I probably don't have that wide a band of product to be able to sell, so my yields actually need to be like it needs to be in that 99%.
Speaker 2:Yeah, well, I'll forget. We just talked about making silicon. No one making a startup is making silicon. It's way too expensive, so it's off-the-shelf components At best. You're doing new firmware for some of these components. Chances are you're not even doing that. Chances are you're just doing software for your application processor, for whatever device you're doing.
Speaker 1:So your yields are going to be very good. So, rob, someone will lend you the money to say I need a billion dollars to manufacture all these units. The economics that you talked about earlier, though, is that selling hardware unto itself is not a viable economic path, at least not anything that where you'd be modeling it on like a Raspberry Pi or Arduino. You need to have some sort of backing, ongoing subscription service or some other way to monetize this hardware once it's out there, correct?
Speaker 2:Ideally, I mean. That's why everything today is a subscription service. But before we go there, let's go back to that production cost.
Speaker 1:Let's go back.
Speaker 2:Yes, you make these million devices. At $1,000 each, there's a billion dollars. Obviously, a typical device is not going to be that expensive, but laptops and things like that. But let's use those numbers because they're nice and easy. So, first of all, someone has to give you the billion dollars to run the production lines and you make these thousand, these million devices.
Speaker 2:So so, now you have them, what do you do with them? You, there's a handful of options you can do just-in-time shipping. So you, basically, your warehouse is the shipping container, so they go to exactly where they need to be when they need to be there. The alternative is you put them in a warehouse and then you distribute them as they go, right, and who pays for the warehouse space? How long are they going to be in the warehouse? What if you don't sell them? How do you pay the billion dollars back if they're sitting in a warehouse? Also, hardware needs physical shelf space. If you're going to put it in Walmart or Target, who's paying for that physical shelf space? If it's drop shipped or Amazon shipped, then how?
Speaker 2:What's the implications of getting this piece of hardware to the end user?
Speaker 2:So the production cost is only a fraction of the cost that you're going to incur, and yes, that's an upfront cost, but you might be paying that back over a long period of time, depending on how long these things sit in a warehouse.
Speaker 2:The other thing as well, and the reason that there's so much software involved in hardware these days, is you can't update hardware. If it's broke, it's broke. Now we've got to go and fix all of these, and are we going to go back and open all the boxes and fix them and put them back in the boxes? That actually happens or we're just going to throw them in the trash or we're just going to go and make new ones? The best goal is to make your hardware as flexible as possible so you can put new software on it to do new functionality and get all the functionality into software which you can update and not hardware, and then you can add new features as you go, because you're adding them to the software, even if this means there's pieces of hardware which are not initially used. It's there for a future plan, a future upgrade.
Speaker 1:Which adds to the cost, right? I mean, if I'm tossing a ton of microprocessors everywhere that I should be able to update, I mean that's a lot more than, hey, I need to throw a few op amps in there and resistors to get the job done, because I want to be able to update this.
Speaker 2:It's probably not to that scale. It's not like there's going to be entire unused processors. There might be, depending on a $1,000 device there could well be, but a typical $50 device you buy from Best Buy is not going to have that much unused hardware. It's most likely the things like Bluetooth which is not yet enabled, which we have plans to use in the future. Maybe it's a camera and there's some other LEDs in there that can enable night vision and things like that which will be enabled in software later.
Speaker 2:But to do that the hardware has to be there. That hardware has to work. So part of the test plan is testing it works. Even if it's not viable for consumer use, it still needs to be tested before it gets out of the factory, goes into the warehouse.
Speaker 2:And certified. Right and certified so depending on what it is. If, for example, let's say it's that Bluetooth Today, bluetooth Wi-Fi, are in the same module, they'll be certified together. But if they weren't in the same module, you have to certify them separately. So at that point you've got to certify the Bluetooth module, even if you're not using it, because it's there and you might use it later.
Speaker 2:If you don't plan on using it later, why is it there If it can't be certified? If it fails certification, you have to disable it and never use it. So it's a very tangled web of how a hardware product gets to where it is, and there's companies like Arrow Electronics in Denver will help with a lot of this distribution, manufacturing and things like that To some extent. They're not going to be the one out of a billion dollars when you make these products and they don't ship. If it's a $50 million or $10 million, they'll be more willing to help when you get to the $1,000 device and a million of them. You're talking nvidia, apple, that sort of scale where they just write a check and go.
Speaker 2:I hope this works. Obviously, the amount of checks and the amount of double checks they do is insane, which might not happen for smaller devices and vcs know this. So, although you might have a great hardware team, if you've never worked together through an entire production, there's lots and lots and lots of missteps that could happen and it's another reason why VCs don't want to be there and VCs want nothing to do with this. Development cost of like yes, we'll fund you to make the product, but we're not going to fund you to build the product.
Speaker 1:Right, you have to come with us with a product that's even if it's prototyped Ideally.
Speaker 2:Yeah, you have a product and a path to make it, and at that point vcs would be more interested because they just see the end result.
Speaker 2:And now it's a software stack running on a custom piece of hardware which is a path to build got it. If you show up with an entire novel hardware product and don't have a path to build it, and it's expensive to build or has lots of like says lots of firmware and lots of things that could go wrong, vcs want nothing to do with it because they have the cost of development of the, the team, the typical vc expenses, plus there's this unknown back end of like how long are these things going to sit in a warehouse or on a ship or wherever? How? What if Walmart or Target pull out? What happens to the physical stock? Vcs want nothing to do with physical hardware and it's why there aren't many hardware products these days.
Speaker 1:So to double click a little more on that, that software isn't just a means towards the economic side. It's also a form of lock-in, and what I mean by this is that once I've manufactured a piece of hardware, it's relatively easy to copy that right.
Speaker 2:Yes, hardware is easy to copy. I can take your circuit board apart and make my own version of it. So there's a protection step you go down and it's also why a lot of these companies won't open source their hardware. It's you buy a ring doorbell and you run the ring software. Why would they open source it? To? Let you just take their hardware, which they may be losing money on, and let you do what you want to do with it.
Speaker 2:The back end is where they're making any markup which they didn't make on the hardware and the more they can make in services and software after the sale, the more they can subsidize that hardware and make it more cost effective to initially purchase to the point. Sometimes it's free and consoles are on this path too, like not as much as they used to be, but the game consoles they can sell at a loss just to get the devices out there so they can make it back on the software later. That's a well-known market, but all of these doorbell cameras and a YZ like house cameras that have a back-end service are on that path. And how much you get for free on the service versus how much you get when you subscribe is really kind of where they draw the line on their business model of like. If you buy, kind of a YZ cam and never subscribe and just use it to view inside your house, don't use any of the events, don't use any of the AI type features it can do, then you're actually costing them money because they still have to have the servers for all of that. But they obviously hope enough people subscribe that it covers the overall want and you'll see it all the time where you'll have some hardware device.
Speaker 2:There was a baby monitor at one point I think Lewis Rossman has a whole YouTube video on this exact thing where there's a baby monitor I can't think of its name, but it had all these services that were free and then, once people had got used to using it for free, they took away all the good features and put it behind a paywall so you had to subscribe. Without subscribing the device barely worked. So you kind of you're locked into this architecture. You've bought these devices, they're all in your house and then suddenly you now have to pay a monthly subscription to use it. And that's another reason why vcs don't like hardware, because this is all these big backlashes of now. What do I do? I have no option, I have to pay it and yeah, for a financial point of view it's good, but from a PR credibility point of view as a business it's awful.
Speaker 1:Well, it's very close to being a bait and switch. At that point in time where I bought a physical piece of hardware with certain understandings and yes, maybe you could fit this into the EULA to say, after your 30-day trial, we're cutting this off.
Speaker 2:Oh, it's not even a 30-day trial, this was just an on. That's after your 30-day trial, we're cutting this off. Oh, it's not even a 30-day trial, this was just an on. That's fine 30-day trial. You get to see everything and then you get to subscribe, but then you still have the hardware. As a consumer, I've already have the hardware. I get my 30-day trial. What's left if I don't subscribe? Is the hardware even usable? In which case can I send it back? I would hope so, because otherwise it's a brick that I just paid maybe $100 for, maybe $1,000 for. That's now unusable because I don't agree to your terms of service. I don't agree to having your subscription. So there has to be something that you can do with this hardware even if you don't subscribe. And how many people you get to subscribe, which is how many people stay in this. I bought the hardware I'm going to use whatever I can, and not subscribe seriously changes your business model, but it doesn't change that initial cost of making that piece of hardware, right, right, so it's just fraught with problems across the board. And and then, obviously, what? What's Rev2? Are we going to make new hardware? How much can we add to the software before we need a new piece of hardware and a lot of this fits into.
Speaker 2:Well, let's offload all the data. If it's a camera, let's do all the processing in the cloud and not on the local device, because then the local device could be real cheap. Internet's free these days. Let's just take those images, compress them on the local device, do a minimal amount of processing on the local device, send it to the cloud. Let the cloud do the big processing. We can have a big cloud presence. We can have a web portal and all of this and make it cloud-based, which was a buzzword for a while. But I think that table's turning slowly where people don't like cloud services. Don't forget, cloud just means someone else's computer and it's a good way to get more advanced features without having to update the hardware or the software on the local device.
Speaker 2:You can start doing AI. You can start doing image detection. You can do package detection things you couldn't do on the local device. You can start doing AI. You can start doing image detection and you can do package detection Things you couldn't do on the local device. All would require much more powerful hardware on the local device. Let's say you're making one of these smart cameras. Where would you do it? Ideally, you'd do it all on the local device. You'd open the hardware would cost what it costs. Your hardware team would get that cost as low as possible but still enable you to do all the processing local on the device. It wouldn't be connected to a cloud service. It wouldn't be connected to even the person you bought it from. It would be open source. You could run any streaming software you want it to be, but that would make the hardware incredibly expensive, and this hardware does exist. You can build for cameras in particular. You can buy cameras which are just it's just a video feed coming out.
Speaker 1:Do what the hell you like with it well, this is also just the old consoles too, right? I mean before the consoles actually connected to the internet. This is exactly what they were. I produced basically a complex set of stuff that I wanted to run?
Speaker 2:Not quite, because they always had the license fee to get the cartridge of the disc out there. So this would just basically be a piece of like. I bought an audio mixer. That's it, that that's what it costs.
Speaker 1:Oh, I see.
Speaker 2:You don't get to control what I mix with that audio mixer. Yeah, If I want to mix video game audio or porn yep.
Speaker 2:You don't get to pick or listen to what I'm mixing on it. If I'm making the next great new album, you don't get to spy on it. I'm not going to upload everything to you to be vetted to make sure I can use your hardware for the purpose you designed it for, and so this is more like a standalone piece of hardware. I have a camera. I have an SLR camera that has all of these features when I initially got it. Unless I update the firmware, it will always have those features Canon, nikon, sony, et cetera. Don't get to pick what I use that camera for. I can use their lenses for anything I want to use them for. If I can make a physical mount that takes their lens, they can't stop me using it. Yes, they can stop me using the autofocus, because I don't know the protocol and all things like that, but mechanically I can use your lens. There's nothing you can do to say I can't. Um, this is what this sort of camera would be.
Speaker 2:It's a camera you buy, it's a security camera and it just streams out after. At that point there's no cloud connection. I can connect it to my own cloud. I can connect it to my own local server. I can connect to it live. I can run my own ai processing. It's just a stream of video coming out of the camera, but that's hard to sell in today's world. You went to a vc, it's I'm going to make a camera. They're going to say, yeah, okay, but we're not going to fund you. Why Where's the selling point? Where's the ongoing revenue? Where's the customer base? Why would they buy yours over theirs? They already do this. And so that's where you get all these back-end services of bloating the platform to the point where it's a sellable product with all the buzzwords that you need today to get a product funded.
Speaker 1:Well, I want to connect this back to something you said earlier, because you said that hardware has. In one sense it's easier, maybe as an entry point, but it's never been harder than before. Is part of the hardness, or is the entirety of the hardness, in this back-end software stack cloud-based shenanigans that need to take place to justify the cost of the hardware.
Speaker 2:The hardness of hardware is in the initial hardware of what we talked about of making that product certified. That's a whole hard problem in itself.
Speaker 1:That's always been the case, right, I guess. What makes it harder?
Speaker 2:And then it's the hard again yeah, even harder, because now you need all these software services to justify the cost of the hardware. And if you're doing it yourself, fine, if you can self-fund this, you can get these new products out. If you can't self-fund this entire process, then you're going to have a hell of a hard time finding external funding.
Speaker 1:So are we in a catch-22 where Apple is clearly a hardware company like full stop. They sell hardware. They're increasingly selling services. Google has hardware lines, obviously. It has Android, which is more of the software, but it has Nest. Are we in a situation right now where you have to be a gigantic either tech company or just company simply to put out any sort of ecosystem or product? We can play with it in the Arduino and Raspberry Pi world and have some fun here to create like small units or even small scale production, but when it comes to like I want to do something which is the Nest ecosystem, you have to be Apple, amazon, google to actually pull this thing off. Microsoft.
Speaker 2:It helps, for sure it helps cover those massive costs that you're going to incur along the way. A lot of these bigger companies can look at it as a loss leader to get the data at the back end, like Amazon with the ring and Google with Nest and what do they do with all that data they get? I mean, we know Amazon sells it to law enforcement and things like that. There's lots of other business opportunities when you're operating at that scale where small people may not have that opportunity.
Speaker 1:Going back to the development side, because I know we jumped further down the pipe. I do think it's interesting to talk a bit that when I'm developing software, a lot of times I can either put a breakpoint in or I can use a printf statement. With hardware I may not be able to do that, especially if I'm outside of the chip where I am connecting a whole bunch of stuff, so it then becomes leds and maybe probes as my printf statements right yes, and no.
Speaker 2:Part of the initial bring up of a chip, especially a complicated one, is to as quickly as possible get into a place where you can do printfs and proper debugging as quickly as possible. So initially, imagine you're bringing up this happened for me with the Atari BIOS stuff is you boot and there's nothing available. But you do have the traditional PC LED display or seven-segment display which is on, physically wired to a port. So your boot or you can display a one and then you display a two and then you display a three as you're going through various segments of the boot code. So if you see it gets stuck on number six. You know it's in this section of code and and that modern pc still have that two digit, four digit, seven segment display on the board and you can see the bios rapidly going through numbers based on what it's doing and obviously the boot code is like okay, it crashed here, something's wrong with memory and initially that's all you have for debugging on the, these big chips. They do have gpios. You could use an led, but it's much easier to use this display. But the very first thing you would do is bring up a u-volt so now you can get text in and out of these devices. And this goes for an arduino too. It's like the very first thing you do on an embedded microcontroller is open up the u-volt so you can do printf style text, get text in, get text out.
Speaker 2:And the small chips? That might be where it ends because they don't have any real application-level debug support, but they do. All the small devices have. If they're ARM, have SWD and call site debugging so you can just connect a hardware debugger to them. If you have an AMD chip, they have hardware debuggers. They're just expensive and hard to use. They all have JTAG. They all have JTAG support for debugging so you can do debug from the reset vector.
Speaker 2:But it tends to only be used briefly and not used at all sometimes. And the goal is obviously to get it to a point where you can boot higher level code and you can then run debuggers. Application debuggers are a long way from what we're talking about here too. So we're a long way from running GDB Sure. But then again I say that, but they're all like bare metal GDB stubs that people have written and they get by. I mean they do work. If it's just a serial protocol, if you've got the UART working. Then you can do GDB over UART and you can take the commands it's giving you and you can report the results. And if you can report registers and you can read and write memory, most of the GDB will work. At that point. It doesn't take much to get it to work.
Speaker 1:How much of the hardware development has effectively become software development. Because in this example, where we're able to effectively put any sort of debugger onto the microprocessor, makes total sense. But are we really talking about SOCs? Talking to SOCs, and that's primarily what we're seeing nowadays, rather than hey, I need to put an LED in here to see if the voltage is right across these capacitors. Well, you would never.
Speaker 2:Obviously, that level of debugging still exists. Analog's still analog, analog's still hard, very hard. But for digital electronics you tend to like, okay, power supply works, and as you bring it up the first boards you don't just populate the whole board and power it up and watch the smoke come out. Yeah, you'll. First of all you'll get a bare board and you'll do continuity checks between the ground and power to make sure, like, okay, the board's not shorted. You'll do some continuity checks to make sure a is connected to b, not connected to c, things like that. So now you're pretty sure the board's good, and then you'll pretty much put the power components on and power it up with the voltage rails there, and then you'll go around and measure analog voltages. Okay, I've got 3.3 volts here. I've got 1.1 volts here, I've got 5 volts here, where they're supposed to be. At which point you can do some load testing, like if I load these power supplies down, do they stay stable? Okay, we're good, it looks like we've got good power rails on this board, nothing's shorted.
Speaker 2:Then you start adding other components, and in whatever order. Maybe it's based on how expensive they are. I don't want to blow up this giant processor unless I have to. So I'm going to bring the board up piece by piece, testing it, testing by piece, testing it, testing it, testing it. At which point you get to the ending you're like, okay, we're good. The whole thing boots it all, powers up, we're good to go. So that level of debugging is very early on. It's typically not done on production boards, it's done when they don't work and typically you'll bring the development boards up one by one and that's why they have all these extra test points and things like that so you can get easy access to these voltages which may be on inner planes. You can't actually get a test probe on them, so you just pop it up to the top level so you can put a test probe on.
Speaker 2:That's why the development boards exist. You'll bring them up and then, once you've brought them up, you're like, yeah, bought them all. Be like, yeah, it actually the design works. Now we have faith that if we auto populate these boards with a pick and place machine in a factory, they would actually work. And then, when they don't work, you still have the test points to go around, figure out why it didn't work. That board might be dead because you killed it, but at least you get the option now to go and test. Okay, the power supply failed, or this chip's bad, or it's not on properly, or whatever it may have been that caused that board to fail. You can go reiterate on that problem. If it's your problem, is it a production side problem? Does it keep happening? Maybe you need to pick a different vendor to make the board. Maybe it's your problem that you did a stupid thing and blew the boards up, and that's that design iteration.
Speaker 1:But it's no different to the software design, but it can be longer because if I'm making, it can be longer, it can be more expensive, right, depending on what?
Speaker 2:you're blowing up. Yeah, if you're letting the magic smoke out of amd apus all the time, that gets to be expensive. The boards for that level of complexity are expensive. They're eight 16-layer boards. You're not going to get them delivered in 24 hours yes. So the whole production process overlaps with what the worst-case scenario is the big bubble in the pipeline where, okay, I now need to wait three weeks while these boards get sent back. That's what you want to avoid, Ideally. You know you have to wait these three weeks to get these boards back.
Speaker 2:You pipeline it such that I can keep working on the old boards until the new boards get here, but sometimes the old boards are so bad you can't actually do that and you just have to sit around and twiddle your thumbs for a few weeks.
Speaker 1:Stepping back to what we talked about with software. When we were coming up in the 80s and 90s and playing with this stuff, you necessarily had those things tied together in the sense that I needed a physical piece of media to go get my software. It was my floppy disk. So there was actually fundamentally a distribution problem. That was there and as we're talking, it occurs to me that hardware still has that distribution problem. I still need, you know, to get it, I need to manufacture it and then I need to get it from point A to point B, adept enough to actually be printing out hardware, even simple hardware, where it's like hey, here's my schematic. Go print it out and hook it up to a battery and you're good to go.
Speaker 2:Do you think that we're on a path for that? You can do that today. I mean, I have a CNC machine and a laser cutter in my shop and I have made circuit boards with both of them, but they're very crude circuit boards. Yeah, I can't do four layer board. How do I get the inner layers? I can make two two layer boards and stick them together. That seems a bit crap.
Speaker 2:Uh, the other thing is all the plating. It's like yeah, I can take a two layered piece of copper and I can etch it with a cnc machine or I can burn off the copper with a laser, but they're not through hole component. Yeah, the the veers, the holes you drill aren't plated, which is part of the reliability of modern boards. I would say no, we've gone away from that. That's where we used to be. We used to make boards with ferric chloride and things like that and we make them with lasers and we make them with cnc machines. But they are very crude, very prototype boards. It's so easy and so cheap to just submit a, a chi-cad eagle ultium file to jlpcb or osh park. If you want to do them in the US, you can get them back in a few days. It's simply not worth building your own, and they can do eight 16-layer ports super quick.
Speaker 2:So no, I don't think we're going back to the days where you make complex PCBs at home. They're nasty things too. All of the metal depositing you have to do to get them to work. It's just an industrial process. It's not something you want in your house. Even ferric chloride is nasty. It's not that nasty actually, but it's what you do with it.
Speaker 1:Disposal problem.
Speaker 2:No one will take it so it's like chemical waste at this point. So avoiding that in your house is probably the best way and you'll get much better results letting the pros do it, Especially if you get the small things. I want traces that are fractions of a millimeter apart and I'm doing bgas with bga breakout with micro traces. It's in micro vias. I'm going down half a layer like uh blind vias that go between two layers of a board. Don't go all the way through. You're not making that at home.
Speaker 1:Yeah, got it. Okay. So we're not going to be going down the path of huge amounts of custom hardware that is distributable in the same way software or YouTube is at this point in time.
Speaker 2:At best you would make 100 units of JLPCB and you would hand build them. I have friends who make analog synthesizers and that's exactly what they do.
Speaker 2:They. They make euro rack components and it says they'll get a hundred boards made and they might populate 10 of them. Sell them, populate 10 more. Sell them whatever their business is, however fast they sell. Sometimes they populate 100, but but it's basically done by hand they'll. They could populate them with pick and place.
Speaker 2:Some people have their own pick and place machines, some people build pick and place machines, but again, it's it's why it's probably easier just to get them made. And the problem is then is how many do you have to get made to make it viable? People like jlpcb you can make five. So the days of hand building boards are also going away, because as long as they have the components, they will build them. And some of these chinese production houses you can send them components. You can take a digi key order, send it to them and they'll pull it in their warehouse with your name on it and won't use it for anybody else. So custom components, that's kind of what you have to do. Or you can go, okay, populate these and then do not populate these, and then get them back and put your components on when they get here. So it's a bit of hand assembly. But all those tiny surface mount resistors and capacitors were already put on for you and all the other generic chips and generic and that's done for ip reasons as well, right potentially.
Speaker 2:I mean they can copy your board regardless. Yeah, of course so, but if you're not, if they're not programming it, then it's not a big deal. The ip is all in the software that the board kind of always been open, always has been. They can be hard to reverse engineer due to tracks being hidden on the inside of a board, but with enough time you can figure out what connects to what and a lot of times it's obvious. It's like okay, this op-amp is connected this way because it's an amp and I just know it's this way and you can just knowing what each component on the board is doing and why it's there, you can quickly figure out how it's all, how it's all connected together.
Speaker 1:Uh, without the software of course, your hardware potentially useless but you can know, based on the board, this is what it's all going to do.
Speaker 2:Yeah, you can know what it's doing and you could write new software uh for it, which gets you in the whole hardware security side of like secure booting and things like that, because people like Nest and Ring don't want you putting open source stacks on their hardware. Nope, because they need the back end to make money.
Speaker 1:Nor does Apple right.
Speaker 2:Nor does Apple, nor does Sony, nor does Microsoft. So how hard these things are to take over is an arms race. It goes back and forth. I'm sure it's much easier to take over is an arms race, it goes back and forth. I'm sure it's much easier to take over a Nest and make it do what you want it to do than it is to take over a PlayStation 5 or an iPhone.
Speaker 1:One of the things that strikes me as I reflect back to again the late 90s, early 2000s, is that I used to have this litany of devices, this litany of hardware devices a camera, a CD player, eventually an MP3 player, a mobile phone and even maps, like some sort of GPS map system. All of these have all been consolidated hardware-wise into my phone, so it's basically my phone. Really is the nexus of all the hardware I used to use, with a huge software stack on top of it yeah, but take all that hardware you used to have.
Speaker 2:That's all the hardware you can't make today, all the hardware which has no follow-on services. You buy a cd player, you buy cds, but the manufacturer doesn't benefit from that. Yeah, you buy a GPS Potentially there's a bit of on sale because you can sell new maps. A camera there's no ongoing fee to the camera manufacturer, no matter how many photos you take. You might buy SD cards, you might buy consumables, but that money doesn't go to the original manufacturer. That hardware model there is what got us to where we are today, where everything is subscription-based. Everybody wants the ongoing fee, so everybody adds all these bullshit software services that make you subscribe it's.
Speaker 2:I wish we still have hardware where it was just a piece of hardware, and that does still exist. Like I said, audio mixers and things like that, but they're not consumer devices In the consumer space. Well, the audio mixers and things like that, but they're not consumer devices In the consumer space Well, the oscilloscope, you've got right, but it's not consumer hardware, correct? There's lots of test equipment and things that are just one-off purchases. There's a whole rack behind me right here of all pieces of hardware, studio hardware that's needed for other industries it still exists as a standalone piece of hardware. It's changing over time. You can get subscriptions for oscilloscopes and things like that. So I think in another decade or so there won't be much hardware that doesn't connect to the internet and doesn't have some sort of ongoing free, because all these companies want that ongoing money, especially if it allows them to sell their hardware for cheaper and makes it more competitive in the initial purchase phase.
Speaker 2:It's all important, but for consumer hardware there's pretty much nothing left in the space of oh, I have a handheld, one of these, and maybe hi-fi components still exist. You can still buy.
Speaker 1:Again, that's almost prosumer at that point in time.
Speaker 2:We're getting pretty high yeah you can buy CD players and DVD players and things like that, but again, they're not that many of them. There's not as many of them as they used to be.
Speaker 1:That's actually where I was going to go to, which is that it seems like where we're at from a hardware consumer space is that we are moving at the speed of our phones now, meaning that if it isn't a part of our phone, then we shouldn't expect there to be much more in terms of an expanse of hardware capabilities. No, everything's a phone. Everything has an app of hardware capabilities.
Speaker 2:No, everything's a phone. Everything has an app. I think it's ridiculous. I don't want an app to unlock my house, as you had an experience with yesterday.
Speaker 1:Yeah, let's talk about that one. So I did mention that I wanted to get into it because at the higher level we've talked about the difficulty in manufacturing hardware, digital hardware, hardware that's connected to the internet and the experience that I had is Yale. Yale is a huge lock company. They've been doing it forever and I, admittedly, have been a big home automation guy. I know, rob, you and I get into this every once in a while. You're not, and I know I am, but for a good reason I think you're not.
Speaker 1:So I got into my house, I was able to unlock the door using I don't know it was either Wi-Fi or Bluetooth, I'm not sure what protocol it was using Got in, it was fine, I left, locked it and then when I left I noticed that it said, hey, device unavailable. So I went back to the house and then couldn't get back in because I didn't have the key and it said that the lock needed to be recalibrated or something like that. So it definitely showcased this shortfall where, as a consumer experience of a lock a digital lock, which has worked very well for me in times past there's this real pathway of failure that exists. Is it somewhere in the software or the hardware somewhere in between, I don't know, but it definitely has a negative consumer experience for me, consumer experience for me, and I think it goes that even large companies, even large companies that are experts in these areas, do get it wrong and don't have good failure paths.
Speaker 2:But they're not experts. That's the point. They're experts in locks. They're not experts in user interfaces to a lock. For 500 years, a lock's had a metal key that unlocked it. Now we have all these lock people Yale, for example. Your lock was a Yale.
Speaker 2:Do they have app developers? Who's writing this software? It's like they don't have experience. They're trying to digitize their existing business and make them all appified and gamified, but they don't have the developers to do it. They see the buzzword. They go oh, we could make an app.
Speaker 2:First of all, I don't want yale's shitty app on my phone. I'd love to go and see how much data they gather. Enough like why do you need my health data? You're a fucking lock. It's. It's like no. That's why I don't like any of these apps. They're all data grabbers. Basically, it's like you're not having anything.
Speaker 2:I have basically no apps on my phone. My phone's three years old and it pretty much has the default apps that it came with. Um, I hate apps on my phone and everything gets. So my phone still works at full speed, by the way. So if you don't, if you want your phone to last years, don't install any apps. Uh, but I don't want your shitty app on my phone and I don't want things like this. I'm like, if you're going to tell me to recalibrate, tell me while it's unlocked where I can get in and out, where I've not accidentally locked myself out of the house and who does the recalibration. What is the recalibration? Have you looked online or anywhere to see what do you actually have to do or do you have to replace the load?
Speaker 1:oh no, I've looked. The instructions are. You know, take out the batteries, take out the module, do a little dance and you see if it goes. All goes back together again so that's not acceptable.
Speaker 2:See if it goes back together again as a hardware guy is not acceptable. And if you're gonna do that, do it while you're unlocked. Do it while you're in a state where you can access the whole thing and not accidentally like, oh, now I've locked myself out of the damn house.
Speaker 2:Well, I think it's a really good point, which is that even commercial products, tons of commercial products which are hardware products, have got these big gaps in them, and that'll be from companies like Yale, who decided to appify and digitize their existing business and aren't actually very good at it, and their digital locks are frankly not very good.
Speaker 1:Well, it's fascinating because, you know, we have these legacy manufacturers like Yale or Honeywell, who got into the digital thermostat game after Nest did, and that's, on one hand, is legacy folks that are trying to digitize themselves and on the other hand, you've got folks like Google, who are digital folks that have been trying to become more hardware by the acquisition of Nest.
Speaker 2:I think it comes down to consumer perception. If Nest had a good idea, I still think it's a bad idea, but a lot of people thought it was a good idea is you can have a thermostat. That's smart and you can have this really cool-looking device on the wall and it detects when you're there and not there and turns the heat on, turns the heat off and learns oh, I'll turn the heat on at 6 am because that's when you're going to be around. Blah, blah, blah. For me personally, I don't care. I'm more than capable of getting open the house being cold and I turn the heat on and I turn it off when I leave. The worst is I forget and I leave the heat on all day. I still think my approach is more efficient because it's the heat's not on if I'm not there at all.
Speaker 2:Well, that's just my opinion doesn't matter for this story. But they had a good idea. People generally thought nest was a good idea and they they went through all these problems of getting that thing made and there's lots of blog posts out there of how Nest got made pretty much what we just talked about. They had to do the software stack they had to do the app.
Speaker 2:They had to do all the connectivity. They had to get it connected to thermostats in place of a thermostat, and various thermostats work different ways. Some are millivolts, some are switches. Various thermostats work different ways. Some are millivolts, some are switches. It was a market that hadn't changed for years and years and years. Thermostats were thermostats. No one cared Like the thermostat I have is the one I'll always have. I don't care, but lots of people thought it was a good idea.
Speaker 2:There are lots of good use cases for these smart thermostats. They went through the whole design process. They made the app, they made the prototype, had various interfaces so they could connect to a majority of heating systems in your house and they have a heat and AC and this whole thing, and they added some stuff. This is a good example of sensors you might put in the device that you aren't using yet, of the sensors that detect proximity of people walking past. The Nest has to have them so it knows you're there, because it's not the app doing it, it's the device doing it. Initially they might not get used, they they're there. There's plans to add new firmware support later and new app support, so all of this could all work together, so they do that and it's a huge hit.
Speaker 2:And then all of these companies realize that, oh, we're missing the board. You get the incumbents that decide we need a part of that. That's decimating our market. We have nothing to compete with it. So, on one side, they'll make a shitty version because they're not capable of making a good version, because it's all new tech apps and digital electronics, which is funny because honeywell make avionics, so they have the staff. They just don't have them in the right building at the right place in the country. Yeah, so they'll do a shitty version and it'll be a shitty app. And they got caught with the pants down and now they're reacting to it.
Speaker 2:Google, on the other hand, when they look at someone like that, they don't have any consumer space. They're not Apple, they don't have anything in the house. They see it as an opportunity to get data and they see it as an opportunity to get data, and they see it as an opportunity let's just go buy them. And that's how they got there. So, also caught with the pants down, simply because they don't have a consumer presence in the hardware space, and that's kind of how it is. But nest is a great example of one of these hardware products that comes out of nowhere does very, very well.
Speaker 2:There's lots of examples in the like says camera spaces of home cameras, wing is another one. And like who think you need a camera in the doorbell? Are they useful? Again, I don't have one, but people think they are, uh, useful things to have. They're willing to give up a little bit of that privacy for the service that it gives useful things to have. They're willing to give up a little bit of that privacy like the service that it gives Useful things to have. They're willing to give up a little bit of that privacy. If I have a device in my house that I want to communicate with from my phone, if I'm in the house, I can talk to it directly.
Speaker 2:I can connect to Wi-Fi on the same network. If I'm not in the house, then how do I do it? And the way you do it is you have the device connect to a server and that's how you can always connect out of your house.
Speaker 2:You can't connect into your house and fire off the data to the server device, and then your phone can get access to the device connects to the server and asks for data, like do you have anything for me? And the server says no, and it does that every minute or so, or 10 seconds, or live, whatever that period is, and at some point your app on your phone will go oh, turn the thermostat on, so it'll send a message to the cloud. When the device next asks, like, do you have anything for me, it'll be oh, yeah, turn on the heat. And that's how you do it. So that's the basis as to why a lot of these smart devices have a cloud component is because it's to circumvent network issues, and from there it just grows. So, yes, I see why there's a cloud component, but that's where it gets fuzzy for me, because what are you logging?
Speaker 1:And it's fascinating because, as you said earlier, this is where the economics of all hardware are at. At the end of the day. How can I grab as much data as possible? And let me put these little devices in your home that allow me to find out more information about you.
Speaker 2:Yeah, and it's like are they grabbing more than they need? Most are yes, because that data may be valuable. It may add a product feature later, so they'll grab everything they can grab, and that's most apps on phones too. Most apps on phones basically grab as much data as they can grab.
Speaker 1:Data is the new gold. So, rob, like we talked about, the development of hardware, even though it's still hard, is easier than it's ever been. That's what we talked about at the very top. Distribution remains a hard problem. It's always been a hard problem, and this added onus of needing the cloud, needing the software, needing the flexibility of the hardware also makes it hard. Do you think that there is some sort of breakthrough that's needed effectively to make the hard parts, the hardest parts of this, the distribution, the manufacturing easier? Like, is there a pathway that we have where we actually can get hardware to a point where it may not be as easy as software but can take me from hey, I'm a hobbyist who's just fucking around with my Arduino to a product I might be able to sell a few thousand units on? Or do you think that the economics are so radically flipped up that that's actually an impossibility?
Speaker 2:Well, you've got to bear in mind hardware is hardware. It physically exists. You can't transmit it. It needs to be shipped, so there's that limitation. There have been some improvements in this area, things like just-in-time shipping. So you don't have a warehouse, you can make them on demand, you can ship them on demand and the supply chain is the warehouse. Effectively, you never put them anywhere to store them, they're only ever in transit at exactly the right time. There's a lot of software and backend services which can manage this. There's a lot of software and back-end services which can manage this. There's a lot of manufacturers who can manufacture it just in time. And it really depends on how complicated your device is. If it needs firmware and testing and programming, then that just-in-time gets a lot more difficult. If they have to reconfigure production lines to accommodate you for weird test processes, then again just-in-time gets more complex. So if you're designing with just-in-time in mind, the things you have to do which you wouldn't normally do and or might not want to do, but it it enables this just-in-time path and if that's what you need for the distribution side, then it's something you have to consider early on, when you're making your hardware so
Speaker 2:that's one thing you can do. The other thing is you can work with companies like Arrow Electronics who have programs which enable this and they absorb a lot of the initial cost and they have all the partners already established and suppliers and things like that, and ultimately it might be more expensive for a huge consumer product. But if you're doing a few thousand, it's the way I would go. I would work with one of these partners who could make them. The caveat there is if you intend to make a few thousand and accidentally sell a few million, then you've dug yourself a hole you might not be able to get out of because you're so tied into their systems.
Speaker 2:And it's now so expensive because you made a million and you only spoke to make a thousand that you might have to go back and start again and redesign it for a totally different process. It's usually a decent problem to have. It's like we're selling more than we thought, and I guess I haven't been in that situation. So I assume people like arrow have a plan for like if this happens.
Speaker 2:This is sure, and you can always cap the number of units you're selling, like if you're selling them at a loss you can, yeah, but if it's if you don't, if you was losing money and there was a niche market, you probably would do that. But if it's like we're making a bit of money but it's not ideal, or the hardware is way more than we expected, because we planned budgeted for for 1,000 and the way we made them is for 1,000, but we're making a million, then it's a big deal and you've got the whole thing of like. How do you make them Like? Who's the contract manufacturer? Who's actually pulling these things together? And are they doing just electronics? Who's making the case? How does the electronics get to where the case is made? Are they all made in the same place, the same factory? Who's making the box? Who's shrink-wrapping the box? Are they made in the same place? Are you shipping from A to B to put them together, then shipping from B to C to put them in the box? That way yield and hardware comes into play, because they'll get broken along the way. Coming off the production line, they tend to work quite reliably. Coming off the production line, they tend to work quite reliably, but a few will get lost along the way before they get put on a pallet, put onto a ship. So it's a big problem and it's not an easily solvable problem.
Speaker 2:These massively vertically integrated manufacturers can help when they can do packaging, casing, electronics, surface mount, hand solder, testing, verification and shipping so it's all made in one place.
Speaker 2:There's a lot of people out there who've been down the path and a lot of people who can help with your product at any scale. And VCs if you're going to VC fund a company, expect a big battle and there are some VCs who specialize in hardware so they understand this path a lot more. A lot of the software vcs just don't want to talk to you because they don't understand the process. Um, there are vcs out there who do understand the process and who do fund hardware and also have access to like manufacturing funds, so short-term loans, so you can pay the manufacturing and recoup it, sure, at the back end and sell them as quick as possible. This obviously still risks involved of what if you don't sell them? You've already paid for them to be made. But there are people out there who can help and I would dig through the vcs and only go to the ones who know about hardware if you're a hardware product and I can imagine there's very few especially who want to do the consumer space.
Speaker 1:Again. There's probably specialized industries like medical devices where it's like, hey, I've got the next great cancer sensor. Or military applications where it's like, hey, I want to create the next great avionics package. There are definitely specialized industries for which there's both a need maybe a captive customer and money for it. But I can imagine that the consumer space again, given the big players in the world, is a really hard space to get VC funding, unless you're Tony Fadell and you just did the iPod.
Speaker 2:Absolutely. I mean, I have a hardware background with the Xbox and the PlayStation and things, things like that, and I did the game board, but it's still very hard to talk to these, to these people. They just see numbers of like it's a good idea, but I just don't see it being a a mass market thing. And hardware is just hard. It's the way it is and that's why most things that were hardware are now software, and in some ways it's nice because you get the phone, for example going back to your example of the phone replaced, your standalone camera, standalone GPS flashlight, the phone itself is all one, and then you get the synergy between those devices. If I have an app that can use GPS, which wasn't possible before, then you get this broader scope and more rich experience because of the integration between all of these things. And it is better how it is today. It's just not good.
Speaker 2:If you're trying to break into that market, if you're going to design hardware, you've got to pick your market and be very honest with yourself, as how many are you going to make, which fundamentally changes who you talk to. You want this infinite scalability For hardware. That infinite scalability doesn't exist, and even from small to medium to big scale. Scalability doesn't exist because you would use components and processes on a hundred units that you wouldn't do on a thousand units, which you definitely won't do on ten thousand, and it's impossible on a million, like if you're in a million units. They're robot assembled. There are things you have to do so they can be put together by a robot.
Speaker 2:If you're making a hundred, you can hand solder them all, even the smallest surface mount components. If you're making ten thousand, maybe you hand solder them all, even the smallest surface mount components. If you're making 10,000, maybe you hand solder the power connector and everything else is pick and placed, but the case is still screwed together by hand. No one touches a iPhone. It's literally put together completely by a robot Making that many that quick. You cannot have a person involved, so you have to be dead honest with yourself as to how you're going to make these things, and some of it's self-limiting.
Speaker 2:if you're making a thousand foxconn, aren't going to talk to you and secondly, just the setup costs to get their production lines up and running is millions and millions and millions of dollars which you're not going to fund if you're going to make 10, like you say hardware is hard.
Speaker 2:But yeah, brutally honest with yourself is what you have to be of like, realistically, what's the market size of this? Because that'll dictate how you make them. And if you, if that changes and you're lucky, then you'll change your plans and that might mean redesigning the whole thing from scratch. And if you're unlucky, you pick your aim too high and you're not selling as many. You now have a warehouse and storage problem and potentially a long-term update problem. So matching your production capability, your expectations, is a big reason why hardware files.