In partnership with
The GPU Daily · #034 · covering Sunday 16 August
The physical layer of AI, every day. Here's what moved on Sunday 16 August 2026.
In this issue
Sponsored
Today's issue is brought to you by Rafay
The GPU stays independent because sponsors cover the bills, not because they shape the copy. Rafay works the orchestration and GPU-management layer beneath all of this, so if you build, buy or fund the physical stack of AI, they are worth a look.
AI Infrastructure Leadership Summit
Hosted by Rafay, join fellow tech executives for this two-day summit bringing together leaders from the world's most innovative neoclouds, telcos, NVIDIA Cloud Partners, sovereign AI initiatives, and enterprise AI platforms to share real-world strategies for commercializing AI infrastructure, expanding margins, and building the next generation of AI businesses. *Registrations subject to approval
Top story · SOFTWARE & AI
China · ▲ Bullish · 15 Aug 12:15 UTC · Newsroom · 3 sources
Alibaba's open-weight AI models crossed 3 billion cumulative downloads, surpassing both Meta and Google on that metric and making Alibaba the largest open-source model distributor by download volume. For GPU infrastructure operators, that scale of adoption means a growing share of inference workloads running on self-managed or third-party compute rather than proprietary cloud APIs, which compresses hyperscaler lock-in and expands the addressable market for independent inference providers. Watch whether Alibaba's download lead translates into enterprise deployment contracts outside China, which would be the harder test of whether this is genuine adoption or domestic pull-through.
SOFTWARE & AI · 15
China · ▲ Bullish · 10 Aug 00:00 UTC · Company/PR
Moonshot AI released Kimi K3 on 27 July 2026 with 2.8 trillion total parameters, 896 experts, 16 active per token, 76.8% SWE-Bench Verified; full weight matrix 1.56TB at MXFP4 requires minimum 64 GPUs for production; official API $3.00/M input, $15.00/M output tokens.
Why this matters: Largest open-weight model released to date; production deployment requires 64+ GPU supernode, establishing new baseline for inference cluster sizing and sustained demand for large-scale inference infrastructure.
Global · ▼ Bearish · 16 Aug 20:45 UTC · Trade press
A three-stage cascade architecture for retrieval-augmented generation routes only 10-15% of queries to LLM inference, reducing costs by 6x compared to all-LLM baseline while improving consistency.
Why this matters: Inference cost optimisation technique that reduces GPU utilisation per query; if widely adopted, could compress inference-per-dollar economics and shift demand toward cheaper inference endpoints.
Global · ▲ Bullish · 10 Aug 00:00 UTC · Company/PR
Prompt caching reduces inference costs 56-90% for workloads with 70%+ stable prefix; Anthropic charges $0.30/M cache-hit vs $3.00/M standard input (90% reduction), 25% write penalty, 5-minute minimum TTL; system prompt caching saves $54/day on 10K daily requests.
Why this matters: Establishes cost structure for inference workloads with repeated context; incentivises prompt engineering and static-first design patterns, shifting inference economics for RAG and multi-turn agent systems.
Global · ▲ Bullish · 10 Aug 00:00 UTC · Company/PR
Llama 3.3 70B requires 40GB KV cache for 128K context on H100; PagedAttention enables 2-4x more concurrent requests; prefix caching reduces effective input cost 60-90% for RAG; INT4 quantization reduces storage 4x with 1-3% degradation.
Why this matters: Establishes inference optimisation techniques that reduce per-token compute cost and GPU memory requirements; adoption across inference platforms compresses cost-per-inference and increases cluster utilisation.
US · ▲ Bullish · 11 Aug 13:00 UTC · Company/PR
NVIDIA Nemotron 3.5 Lightning, 30B MoE with 3B active parameters, available on GMI Cloud serverless API as of 11 August 2026; achieves 54.3% SWE-Bench Verified (vs 21% for Nano), 4x higher throughput, 30% faster task completion.
Why this matters: Optimises inference for high-volume agent sub-tasks; open-weight release under OpenMWD licence enables custom domain fine-tuning and reduces inference cost for tool-calling workloads.
US · ▲ Bullish · 12 Aug 20:00 UTC · Company/PR
SpaceX AI released Grok 4.6 on 12 August 2026 with 500K-token context window and configurable reasoning; pricing $2-$4/M input tokens (under/over 200K), $6-$12/M output tokens; company-stated performance matches GPT-5.6 Sol on Artificial Analysis Index.
Why this matters: Extends context window to match Claude and GPT-4 Turbo; pricing undercuts GPT-4 Turbo on long-context input tokens, compressing inference cost for RAG and agent workloads.
China · ▲ Bullish · 15 Aug 16:21 UTC · Company/PR
Versatile Media produces finished two-minute videos for approximately ¥250 each using MOKE, a node-based workflow consolidating model calls through Atlas Cloud's 400+ model inference API.
Why this matters: Production-scale inference platform adoption for video generation, sustained demand for multi-model orchestration and cost-efficient inference endpoints.
Global · ▲ Bullish · 16 Aug 23:30 UTC · Trade press
ClickHouse reported customer base growth driven by AI workload demand, company-stated.
Why this matters: Adoption of columnar databases for AI analytics and inference logging, a secondary demand signal for GPU infrastructure supporting data pipeline workloads.
China · ▲ Bullish · 16 Aug 05:48 UTC · Company/PR
Alibaba's Wan 3.0 generates native 1080p video at $0.20 per second in single pass; Seedance 2.5 upscales from 720p at $0.30/second with visual instability in fast motion requiring 3.2-to-1 shoot ratio.
Why this matters: Compresses inference cost per second for video generation workloads and demand shift toward native high-resolution output, affecting GPU utilisation patterns in inference clusters.
US · ▲ Bullish · 16 Aug 20:30 UTC · Newsroom
University of Phoenix and OpenAI announced a partnership to address the AI skills gap through educational programmes.
US · ◆ Neutral · 14 Aug 08:51 UTC · Company/PR
Veo 3.1 native single-pass generation maxes at 8 seconds (4s/6s/8s for 720p/1080p/4K), with sequential extension passes adding approximately 7 seconds per pass up to 148 seconds total.
Why this matters: Establishes inference cost structure for video generation workloads; extension mode requires multiple sequential passes, increasing per-video compute demand and GPU utilisation for longer-form content.
US · ◆ Neutral · 13 Aug 08:22 UTC · Company/PR
Veo 3.1 operates freemium with 50 daily credits (Lite 10, Fast 20, Quality 100 credits per pass), paid subscriptions $1.29-$99.99/month, and Vertex AI API pricing $0.03-$0.60/second depending on variant and resolution.
Why this matters: Establishes pricing floor for video generation inference; third-party aggregators like Atlas Cloud offer unified access at $0.05-$0.20/second, margin compression in inference platform markets.
US · ▲ Bullish · 14 Aug 01:15 UTC · Company/PR
Baseten made DeepSeek V4 Pro 0813 available through its Model APIs as of 14 August 2026 via OpenAI-compatible endpoint.
Why this matters: Expands inference platform coverage for open-weight reasoning models; demand for multi-provider model serving infrastructure.
China · ◆ Neutral · 12 Aug 11:42 UTC · Company/PR
MiniMax H3 video model runs exclusively on pay-as-you-go billing at $0.13/second 2K and $0.08/second 768P with no volume discounts; excluded from Token Plan subscriptions and Video Packages.
Why this matters: Establishes premium pricing tier for video generation; lack of volume discounts margin protection strategy and may constrain adoption for high-volume production workloads.
Global · ◆ Neutral · 10 Aug 00:00 UTC · Company/PR
HARDWARE · 4
US · ▲ Bullish · 13 Aug 17:51 UTC · Company/PR
AMD CDNA 5 powers MI455X with 432GB HBM4 memory and 23.3TB/s peak bandwidth versus MI355X's 288GB HBM3E and 8TB/s; Helios rack integrates 72 MI455X GPUs with 31TB aggregate HBM4 and 260TB/s scale-up bandwidth.
Why this matters: Delivers 4x low-precision throughput improvement over prior generation; compresses price-per-FLOPS gap with NVIDIA H200 and establishes AMD as viable alternative for large-scale training clusters.
US · ▲ Bullish · 14 Aug 16:59 UTC · Company/PR
Lightmatter's Passage L200 delivers 32 to 64 Tbps aggregate bandwidth using co-packaged optics for frontier-scale AI training, addressing a 240x growth in model parameters against only 2x improvement in interconnect bandwidth over three years.
Why this matters: Addresses electrical interconnect bottleneck in large-scale GPU clusters; adoption could shift training economics for models exceeding 100 trillion parameters and influence hyperscaler cluster architecture decisions.
Global · ▲ Bullish · 16 Aug 04:45 UTC · Trade press
AMD acquired Taalas, a company focused on AI inference optimisation, to expand its inference compute capabilities and compete with NVIDIA in the inference-per-dollar market.
Why this matters: AMD's strategic push into inference acceleration and software optimisation; acquisition suggests AMD is building inference-specific IP to differentiate MI-series GPUs from NVIDIA's H-series dominance.
US · ▲ Bullish · 12 Aug 23:21 UTC · Company/PR
Lightmatter's Passage M1000 3D photonic interposer delivers 114 Tbps aggregate bandwidth across 4,000mm² for rack-level AI training, addressing widening gap between 240x model parameter growth and 2x interconnect bandwidth improvement.
Why this matters: Targets electrical bottleneck in dense GPU racks; adoption could enable higher GPU-per-rack density and reduce training time for frontier models.
HYPERSCALER · 2
Global · ▲ Bullish · 11 Aug 00:00 UTC · Company/PR
IDC study commissioned by Microsoft found 88% of organisations expect AI as primary competitiveness driver by 2030, with production AI use rising from 23% to 51% in 24 months; CPU inference commercially unviable, GPU break-even against public cloud in under four months.
Why this matters: Quantifies enterprise demand signal for GPU inference infrastructure; establishes financial case for dedicated GPU servers over cloud, sustained capex for neocloud and hyperscaler inference capacity.
Europe · ▲ Bullish · 16 Aug 11:15 UTC · Newsroom
A Microsoft executive told Fortune that workforce skills, rather than technology access, will determine which Southern European nations lead in AI adoption.
Why this matters: Hyperscaler perspective on regional AI competitiveness; suggests Microsoft's European expansion strategy prioritises talent and partnerships over pure infrastructure.
ENERGY & POWER · 1
US · ▲ Bullish · 16 Aug 04:00 UTC · Newsroom
NVIDIA will commit $3 billion to SB Energy, a renewable power company, to support OpenAI's data centre and AI infrastructure buildout.
Why this matters: NVIDIA's willingness to fund power supply infrastructure directly, reducing capex friction for hyperscaler AI cluster deployment and establishing a precedent for chip makers backing energy procurement.
REGULATION & POLICY · 2
China · ▲ Bullish · 16 Aug 04:00 UTC · Newsroom
Hangzhou province is launching talent retention and recruitment programmes targeting founders and engineers from DeepSeek and Moonshot AI, competition for domestic AI talent.
Why this matters: China's strategic focus on building indigenous AI capability and retaining engineering talent, with implications for the pace of domestic GPU cluster development and model training capacity.
US · ▼ Bearish · 15 Aug 21:29 UTC · Trade press
A woman filed suit against xAI claiming her stepfather used Grok to generate over 7,000 explicit images from a photo taken when she was 11 years old, joining earlier class action by Tennessee teenagers.
Why this matters: Raises regulatory and liability risk for inference providers offering unrestricted image generation, potentially triggering content-filtering requirements that increase inference latency and computational overhead.