AWS and UC Berkeley's Infinity runs 75,000 AI agents on a Raspberry Pi by letting them sleep
Most AI agents spend their time waiting for tools. Infinity, a new open-source runtime from AWS and UC Berkeley, pauses them while they wait, so idle agents barely use any memory.
By ScoviAI Team
Key points
- Released by AWS under the Apache 2.0 license as part of the Hydro project with UC Berkeley
- Infinity suspends an agent during each tool call and resumes it when the result arrives
- AWS says 75,000 concurrent agents fit in the 8GB of a Raspberry Pi 5
- It's an early research release, shared to collect feedback
An AI agent is mostly a patient waiter. A coding agent might start a build that takes 20 minutes. A review agent might wait hours for the next pull request. An assistant might wait until tomorrow morning for someone to approve a step.
The problem, as AWS describes it, is that most agent software keeps the agent "awake" the whole time, holding a running process and its memory while nothing happens. Infinity, a new open-source project from AWS and UC Berkeley, takes a different approach: it lets agents sleep.
How Infinity works
Infinity borrows a trick from programming languages. When code in Python, JavaScript or Rust waits for something slow, such as a network request, the language can pause the function, save its state and free the thread to do other work. When the result arrives, it picks up where it left off.
Infinity does the same for agents:
- The agent calls a tool.
- Infinity sends off the call, saves the conversation and suspends the agent.
- When the tool's result comes back, Infinity resumes the agent from its saved state.
While suspended, the agent uses almost nothing.
Why it matters: thousands of agents on cheap hardware
The numbers AWS gives are striking. It says 75,000 concurrent agents fit into the 8GB of memory of a Raspberry Pi 5, a small hobbyist computer.
It also makes serverless hosting practical. On services like AWS Lambda, you pay only while code runs. Because Infinity suspends the agent during a tool call, the function can simply end. As AWS puts it, even if a tool call takes days, "nothing needs to run while the agent waits." The reference setup uses AWS Lambda, SQS and Aurora DSQL.
ADVERTISEMENT
Infinity also uses synthetic tool calls to deliver outside events to existing models, and the same mechanism for agents to send messages to each other.
What to keep in mind
AWS is clear that this is an early research release, shared to get feedback, not a finished product. It's published under the permissive Apache 2.0 license as part of the Hydro project, a collaboration between AWS and UC Berkeley.
The bigger picture
As companies move from one chatbot to thousands of agents running in the background, the cost of keeping each one "alive" becomes a real bill. Infinity is a bet that most of that cost is waste. It joins a busy month of agent infrastructure news, including NVIDIA's OpenShell blueprint for safe agents and Google's Gemini agent. New to the topic? Start with What is an AI agent?
Sources
- Introducing Infinity: a runtime for extreme agent concurrency (AWS Open Source Blog)
ScoviAI Team is the ScoviAI newsroom. We write every article ourselves from the original announcements and link our sources. Spotted a mistake? Tell us and we'll fix it.
ADVERTISEMENT
