As a former AI accelerator employee (laid off), I'm kind of happy I was laid off because I realistically don't see a need for specialized hardware anymore.
Large companies can afford Nvidia. Nvidia's software stack is best in class. There's no business need here and the model execution is increasingly becoming possible on single consumer GPUs.
The only place where I see specialized chips excelling is on the edge or if they are truly revolutionary (in which case they're only an acquisition target for Nvidia).
The truth is... The large language models are likely excessively large.
Power is the main reason to do custom ASICs. I’d be curious as to your opinion of Recogni given they are claiming a 10x power reduction per unit compute.
Unfortunately, I've worked at several players which promise power reductions. It doesn't matter though. People don't care about cost at this point. If you are cost-sensitive you're not doing the kind of revolutionary AI work these companies need to create a competitive moat. And once your model works on NVIDIA and is trained, how much are you going to spend on ML engineers to make it work on something else? Because that cost better be less than the marginal cost reduction on electricity. Plus, NVIDIA et al will likely get more and more efficient.
The only exception is running things on mobile. There is demand for porting models to run natively on mobile, and somewhat reasonable support for doing this.
Large companies can afford Nvidia. Nvidia's software stack is best in class. There's no business need here and the model execution is increasingly becoming possible on single consumer GPUs.
The only place where I see specialized chips excelling is on the edge or if they are truly revolutionary (in which case they're only an acquisition target for Nvidia).
The truth is... The large language models are likely excessively large.