AI Is Reshaping Your Entire Stack, Not Just Applications
·8 min
The episode opens with a question about how CIOs should prepare for the next five years of chaos in IT and infrastructure due to AI. The speaker emphasizes that AI must be viewed as an end-to-end infrastructure and application problem, not just an application layer issue. Historically, technologies did not change core infrastructure, but AI is transforming everything from hardware to application execution. The compute side now requires efficient GPU utilization instead of focusing on x86, with integrated GPU and CPU offerings on ARM becoming important. New hypervisors and Kubernetes platforms on ARM are needed for deploying AI applications, driven by power considerations. Networking and storage are affected as I/O is offloaded to GPUs, with specialized cards initiating storage and networking, changing the role of traditional NICs. Hypervisors must optimize I/O with DPUs and ensure virtual machines achieve bare metal performance, a concept termed "bare metal virtual machines." Data management is a major challenge because enterprise data is often unclean and lacks proper security boundaries, hindering AI feeding and knowledge graph creation. AI itself involves choosing models and agents, with security concerns requiring agent gateways to control access and actions based on identity. The speaker calls for a coherent enterprise strategy involving CIOs, CDOs, CAIOs, and CTOs, possibly through an AI committee, to avoid isolated efforts. Economics of AI are critical, as not all use cases are viable, and exploratory projects can be expensive. Use case-driven AI is recommended, with learning from peer successes to implement proven tools and allocate budget wisely for exploratory projects.
