THE DISTRIBUTED INFERENCE FIELDBOOK

The cloud is moving into the rooms we already own

An entrepreneur’s field guide to Darkbloom, idle compute, cybercafés, and the next inference layer

Research edition · 24 August 2026
Prepared as an independent market investigation

Working thesis The opportunity is larger than idle Macs, but the winning company is unlikely to be a generic decentralized GPU marketplace. The stronger opportunity is a trust-aware operating layer for managed, distributed inference fleets.

The signal in the noise

Hiten Shah’s observation is simple: computers people already own are becoming part of the inference layer. Darkbloom is the clearest live example—a network routing inference to hardware-verified Apple Silicon machines, with an OpenAI-compatible API and provider payouts.

The underlying market is not “cheap AI compute” in the abstract. It is the intersection of four forces:

1. Inference demand is expanding faster than centralized capacity can comfortably absorb.
2. Inference prices are falling, making utilization and cost structure decisive.
3. Open-weight models are good enough for a widening set of repetitive workloads.
4. A growing number of organizations want stronger control over where sensitive prompts execute.

The entrepreneurial question is therefore:

Can distributed hardware provide sufficiently cheap, sufficiently trusted inference for workloads that do not need frontier quality or data-center-grade latency?

01 · What is already proven

OpenRouter is a demand observatory

OpenRouter’s 2025 State of AI study analyzed more than 100 trillion tokens across 300+ active models and 70+ providers, serving millions of developers and end users. More than half of usage originated outside the United States. Its data includes model, provider, token, timing, latency, streaming, cancellation, and tool-calling metadata.

That makes OpenRouter the most useful public window into real multi-model demand—not a survey of intentions, but observed API behavior.

OpenRouter currently says it routes requests across 80+ providers and exposes providers to 10M+ developers. Routing is performance-based: price, latency, throughput, uptime, and feature support determine traffic allocation.

Implication: the demand side already exists. The unanswered question is which portion of that demand will accept distributed execution.

Darkbloom has crossed from demo to marketplace

OpenRouter’s public provider table currently lists Darkbloom with two models and live token activity. The numbers are live and may use rolling windows, so they should not be compared mechanically with Darkbloom’s cumulative public claims. But the important fact is qualitative: Darkbloom is receiving billable marketplace traffic rather than only serving free test requests.

OpenRouter’s provider onboarding says:

• providers are paid through monthly invoicing;
• high-performing providers receive more traffic;
• providers below 80% uptime may be used only as fallback;
• TTFT, throughput, and uptime are publicly tracked.

Paid provider means: OpenRouter has accepted Darkbloom as an upstream inference endpoint and is routing real customer requests to it. It does not mean OpenRouter guarantees demand, profitability, enterprise SLAs, or privacy equivalence with a hyperscaler.

Diligence flag OpenRouter’s public provider table currently labels Darkbloom “Retains prompts,” while Darkbloom markets operator-blind privacy. That may reflect different layers of the architecture or a classification mismatch, but it needs to be reconciled before selling strong confidentiality claims.

02 · The demand map

The right customer does not need the best model

Distributed inference is most attractive when the job is valuable, repetitive, and tolerant of some latency.

The demand wedge is not “users accept worse AI.” It is:

Users want good-enough intelligence, at high token volume, without sending every request through a conventional API provider.

Stanford’s AI Index documents dramatic declines in the cost of achieving a given level of model performance. That expands the addressable market for smaller models: more workflows can use a 7B–30B model without a visible quality failure.

The likely early buyers

• AI startups with privacy-sensitive customer data
• Teams running open-weight models in production
• Batch-processing companies
• Developers building agents with heterogeneous task difficulty
• Enterprises needing regional or organizational control
• Model-evaluation and synthetic-data teams

The commercial wedge should begin with batch and privacy-sensitive inference, not real-time consumer chat.

03 · The supply map

100 million Macs is an upper bound, not a market size

Darkbloom cites more than 100 million Apple Silicon machines shipped since 2020. The useful supply is the intersection of memory, idle time, connectivity, willingness, and reliability.

There is no public measurement of this funnel. It is a scenario model. The practical conclusion is still clear: the addressable Mac supply is likely tens of thousands to a few hundred thousand machines, not 100 million.

The best early Mac providers are likely:

• high-memory Mac Studios;
• Mac minis used as servers;
• developer workstations left online overnight;
• university and office fleets;
• Apple-focused MSPs;
• homelab and self-hosting enthusiasts.

The average MacBook owner is unlikely to be the best initial provider. The rational provider already owns the hardware and has no important use for it during the serving window.

04 · The fleet-owner opportunity

Cybercafés are more interesting than consumers

A commercial database estimates approximately 157,644 internet cafés in Asia. This is not an official census and includes heterogeneous venues, but it is a useful directional indicator.

At 20–50 machines per venue, the theoretical fleet is roughly 3.2–7.9 million endpoints. If only 10% are technically suitable and willing to participate, that still implies 320,000–790,000 potential endpoints.

The number is not the main point. The operating model is:

• one owner;
• many machines;
• centralized deployment;
• existing power and broadband;
• sunk hardware cost;
• staff who already manage the fleet.

That makes a cybercafé a much more attractive supply customer than an individual consumer.

The cybercafé constraint

Most gaming-center machines are Windows/Nvidia, not Apple Silicon. The machines are also occupied during peak gaming hours. The right product is therefore not “always-on cloud.” It is:

Overnight inference for gaming fleets, with immediate preemption when a paying player needs the machine.

Best-fit workloads:

• batch inference;
• image generation;
• speech-to-text;
• evaluation;
• synthetic data;
• asynchronous document processing.

The first sale should be to the operator, not to the gamer.

Other sunk-cost fleets

Salad reports 60,000 daily active GPUs and 450,000 earning nodes. Vast reports 20,000+ GPUs across 40+ data centers. These claims are company-reported, but they independently demonstrate that distributed compute supply is already a commercial behavior rather than a theoretical one.

05 · Beyond Mac

The opportunity is bigger, but the privacy claim changes

Extending beyond Mac probably increases the supply market by an order of magnitude. Consumer Nvidia GPUs are more common in gaming centers, homelabs, and existing GPU marketplaces.

The tradeoff is that ordinary consumer GPUs do not offer the same hardware-backed confidentiality model as Apple Silicon or enterprise confidential GPUs.

NVIDIA’s formal confidential-computing support is concentrated in platforms such as H100, H200, B200, and selected RTX Pro products. The company describes GPU confidential computing as protecting code and data from even the computer or cloud-service owner. That is a different class of hardware from an ordinary RTX 3060, 3090, or 4090.

Therefore, a credible multi-platform network needs explicit trust tiers.

The product should make the trust boundary visible on every request.

06 · How to make the multi-platform version work

The opportunity is not a universal peer-to-peer marketplace on day one. It is a trust-aware fleet operating system.

Core control plane

1. Discover hardware, memory, driver, model fit, and network quality.
2. Benchmark every node using real model workloads.
3. Assign a trust tier and publish the evidence.
4. Keep approved model versions warm on suitable nodes.
5. Schedule by trust, price, latency, location, and availability.
6. Preempt immediately when a fleet owner needs the machine.
7. Measure TTFT, tokens per second, uptime, errors, and temperature.
8. Reconcile usage and payouts at fleet level.

Developer contract

Every request should specify:

Model:              gemma-4-26b
Trust:              managed or attested
Retention:          zero retention
Latency:            best effort / p95 target
Region:             selectable
Fallback:           allowed or prohibited
Price ceiling:      per-million-token limit

This is more valuable than hiding all infrastructure behind a vague “decentralized” label.

Cybercafé deployment flow

Fleet owner installs agent
        ↓
Agent discovers and benchmarks PCs
        ↓
Operator defines gaming hours and quiet hours
        ↓
Models are downloaded and verified
        ↓
Inference runs only while machines are idle
        ↓
Player activity preempts the worker
        ↓
Fleet receives aggregated monthly payout

07 · Economics

Electricity is not the primary constraint.

At $0.15/kWh:

• 50W continuous draw costs about $5.40/month;
• 150W costs about $16.20/month;
• 300W costs about $32.40/month.

The larger costs are:

• hardware depreciation;
• component failure;
• heat and cooling;
• lost gaming revenue during busy periods;
• network congestion;
• support;
• privacy and liability.

The economics are best when the hardware is sunk cost. A fleet owner who already bought 50 gaming PCs may welcome overnight revenue. A consumer buying a new GPU solely to serve inference is making a speculative hardware investment.

The most attractive supply-side customer is therefore:

An operator with many devices, low utilization windows, centralized control, and no need to buy new hardware.

08 · The entrepreneurial opening

Weak thesis

We will aggregate idle devices and sell cheap tokens.

This is vulnerable to existing marketplaces, falling inference prices, and provider churn.

Stronger thesis

We will make distributed hardware usable for serious AI workloads by providing trust-aware routing, fleet operations, attestation, policy, and evidence.

The strongest initial product is probably:

A trusted operating system for distributed inference fleets

Initial beachhead:

1. Gaming centers and cybercafés in Southeast Asia
2. Apple-focused MSPs
3. Universities and computer labs
4. Existing small GPU operators

Initial demand:

1. Batch inference
2. Open-weight model serving
3. Evaluation and synthetic-data workloads
4. Privacy-sensitive startup workloads

Only later should the network open to individual consumers.

The field conclusion

Darkbloom’s Mac-only wedge is technically elegant, but the larger market is likely a heterogeneous fleet market.

The supply opportunity is larger beyond Mac because gaming centers, homelabs, MSPs, and GPU operators already hold millions of potentially useful devices. The demand opportunity is credible because OpenRouter shows real multi-model usage at extraordinary scale. The missing layer is trust-aware orchestration.

The market will not be won by claiming that every idle computer is equivalent to a cloud server. It will be won by making the differences explicit:

• what hardware ran the job;
• who controlled it;
• what privacy guarantee applies;
• what performance was delivered;
• what happens when the node disappears;
• and how the operator gets paid.

Recommended next move: validate a 10–20 venue pilot in Southeast Asia. Install the fleet agent in 200–500 gaming PCs, run only overnight batch workloads, and measure actual contribution margin, preemption behavior, provider retention, and developer willingness to pay for each trust tier.

Sources and field notes

• Darkbloom — product, pricing, provider claims, privacy architecture
• Darkbloom open-source repository — architecture, provider hardening, trust model
• Eigen Labs: Project Darkbloom — project history and stated economics
• OpenRouter State of AI — 100T-token empirical usage study
• OpenRouter providers — live provider table and token activity
• OpenRouter provider onboarding — routing, uptime, payments, provider requirements
• Salad — distributed consumer-GPU supply claims
• Vast.ai — marketplace GPU supply and usage model
• NVIDIA confidential computing — supported confidential-GPU platforms
• Asia internet-café database — directional venue-count estimate
• Stanford AI Index — declining inference cost
• McKinsey: AI data-center demand — inference-capacity outlook