SoftBank Corp. Announces “Infrinia AI Cloud OS,” a Software Stack for AI Data Centers

Software stack manages GPUs, Kubernetes, and AI workloads at scale

TOKYO–(BUSINESS WIRE)–SoftBank Corp. (TOKYO:9434, President & CEO: Junichi Miyakawa, “SoftBank”) announced that its Infrinia Team*¹, which works on the development of next-generation AI infrastructure architecture and systems, has developed “Infrinia AI Cloud OS,” a software stack*² designed for AI data centers.

By deploying “Infrinia AI Cloud OS,” AI data center operators can build Kubernetes*³ as a Service (KaaS) in a multi-tenant environment, and Inference as a Service (Inf-aaS) that provides Large Language Model inference capabilities via APIs, as part of their own GPU cloud services. In addition, the software stack is expected to reduce total cost of ownership (TCO) as well as operational burden compared with bespoke solutions or in-house development. This will enable the rapid delivery of GPU cloud services that efficiently and flexibly support the full AI lifecycle—from AI model training to inference.

SoftBank plans to deploy “Infrinia AI Cloud OS” initially within its own GPU cloud services. Furthermore, the Infrinia Team aims to expand deployment to overseas data centers and cloud environments with a view to global adoption.

Background of “Infrinia AI Cloud OS” Development

The demand for GPU-accelerated AI computing is expanding rapidly across the generative AI, autonomous robotics, simulation, drug discovery, and materials development fields. As a result, user needs and usage patterns for AI computing are becoming increasingly diverse and sophisticated, and requirements including the following have emerged:

• Access to infrastructure that is fully managed by GPU cloud service providers, abstracted GPU bare-metal servers
• Cost-optimized, highly abstracted inference services without concerning with GPU management
• Advanced operations in which AI models are trained and optimized on centralized servers and deployed for inference at the edge

Building and operating GPU cloud services that meet these requirements requires highly specialized expertise and involves complex operational tasks, placing a significant burden on GPU cloud service providers.

To address these challenges, the Infrinia Team developed “Infrinia AI Cloud OS,” a software stack that maximizes GPU performance while enabling the easy and rapid deployment and operation of advanced GPU cloud services.

Key Features of “Infrinia AI Cloud OS”

Kubernetes as a Service

• Reduces the operational burden of managing the physical infrastructure and the Kubernetes software layer by automating the entire stack (from BIOS and RAID settings to the OS, GPU Drivers, networking, Kubernetes Controllers and Storage) on state-of-the-art GPU Platforms such as NVIDIA GB200 NVL72
• Software-defined dynamic, on-the-fly physical connectivity (NVIDIA NVLink) and memory (Inter-Node Memory Exchange) reconfiguration, as the customers create, update and delete their clusters to suit their AI workload needs
• Automatic node allocation based on GPU proximity and NVIDIA NVLink domain to reduce latency and maximize GPU-to-GPU bandwidth for highly distributed jobs

Inference as a Service

• Enables users to deploy inference services simply by selecting Large Language Models, without working with Kubernetes or the underlying infrastructure
• OpenAI-compatible APIs, enabling drop-in integration with existing AI applications
• Seamless scaling across multiple nodes in core and edge platforms such as NVIDIA GB200 NVL72 and other platforms

Secure Multi-tenancy and High Operability

• Tenant isolation through encrypted cluster communications and separation
• Automation of operational maintenance, including system monitoring and failover
• API environment for connecting to the AI data center’s portal, customer management systems, and billing systems

These key features allow AI data center operators with customer management systems, as well as enterprises offering GPU cloud services, to add advanced capabilities that enable efficient AI model training and inference while flexibly utilizing GPU resources, to their own GPU service offerings.

Junichi Miyakawa, President & CEO of SoftBank Corp., commented:

“To further deepen the utilization of AI as it evolves toward AI agents and Physical AI, SoftBank is launching a new GPU cloud service and software business to provide the essential capabilities required for the large-scale deployment of AI in society. At the core of this initiative is our in-house developed ‘Infrinia AI Cloud OS,’ a GPU cloud platform software designed for next-generation AI infrastructure that seamlessly connects AI data centers, enterprises, service providers and developers. The advancement of AI infrastructure requires not only physical components such as GPU servers and storage, but also software that integrates these resources and enables them to be delivered flexibly and at scale. Through Infrinia, SoftBank will play a central role in building the cloud foundation for the AI era and delivering sustainable value to society.”

For more information on “Infrinia AI Cloud OS,” please visit the website below:

https://infrinia.ai/

About SoftBank Corp.

Guided by the SoftBank Group’s corporate philosophy, “Information Revolution – Happiness for everyone,” SoftBank Corp. (TOKYO: 9434) operates telecommunications and IT businesses in Japan and globally. Building on its strong business foundation, SoftBank Corp. is expanding into non-telecom fields in line with its “Beyond Carrier” growth strategy while further growing its telecom business by harnessing the power of 5G/6G, IoT, Digital Twin and Non-Terrestrial Network (NTN) solutions, including High Altitude Platform Station (HAPS)-based stratospheric telecommunications. While constructing AI data centers and developing homegrown LLMs specialized for the Japanese language, SoftBank is integrating AI with radio access networks (AI-RAN), with the aim of becoming a provider of next-generation social infrastructure. To learn more, please visit https://www.softbank.jp/en/corp/

*1: The Infrinia Team is an Infrastructure Architecture & Systems Team established within SB Telecom America, Corp., a wholly owned subsidiary of SoftBank Corp., as part of the company’s broader initiative to advance next-generation AI infrastructure. The Infrinia Team is based in Sunnyvale, California USA.

*2: A software stack is a set of software components and functions used together to build and operate systems and applications.

*3: Kubernetes is an open-source system for automating the deployment and scaling of applications and for managing containerized applications.

• SoftBank, the SoftBank name and logo are registered trademarks or trademarks of SoftBank Group Corp. in Japan and other countries.
• Other company, product and service names in this press release are registered trademarks or trademarks of the respective companies.

Contacts

Kyoko Shimada

SoftBank Corp.

Corporate Communications

+81-3-6889-2301

sbpr@g.softbank.co.jp