• Skip to main content
  • Skip to secondary menu
  • Skip to footer

Technologies.org

Technology Trends: Follow the Money

  • Technology Events 2026-2027
  • Sponsored Post
  • Technology Markets
  • About
    • GDPR
  • Contact

Why Small Embedded Databases Are Winning the Edge

October 27, 2026 By admin

For twenty years the default answer to “where does the data live?” was a database server somewhere else. You opened a connection, sent a query over the network, and waited. That model still runs most of the world’s business software. But at the edge, on phones, in sensors, inside a browser tab or a serverless function a few milliseconds from the user, the answer is shifting. The data lives in a file, and the database is a library linked into the program that reads it.

SQLite has worked this way since 2000, and it never stopped shipping. Version 3.54.1 came out on October 16, 2026. DuckDB, the analytical cousin, shipped its 2.0.0 LTS release on October 21, built from close to 20,000 commits since 1.5. Turso is rewriting SQLite in Rust with concurrent writes. Cloudflare gives every SQLite-backed Durable Object its own database and builds D1 on the same foundation. The small, in-process database has gone from embedded curiosity to the storage layer people pick on purpose.

Why? Mostly physics.

The network hop is the expensive part

A query against a local SQLite file is a function call. Reading a row by key takes microseconds. The same read against a managed Postgres instance in another region goes through a TCP connection, TLS, a connection pool and a round trip that can easily cost 50 to 100 milliseconds. On a web page that fires a dozen queries, that gap is the difference between instant and sluggish.

Edge platforms make the problem worse before they make it better. If your code runs in 300 cities and your database sits in one, every request still crosses an ocean to get its data. Moving compute to the edge without moving data with it just relocates the wait. Cloudflare’s answer was to put the database next to the code. A SQLite-backed Durable Object keeps its own private database of up to 10 GB on the Workers Paid plan, stored on the same machine that runs the object, so reads are local disk reads. D1, Cloudflare’s managed SQLite service, caps each database at 10 GB as well, allows up to 50,000 databases per paid account, and now supports read replicas, each one its own Durable Object.

That cap tells you the design philosophy. Nobody is trying to build one giant database. The bet is on many small ones: a database per customer, per document, per chat room, per device. That pattern fits a single-file engine perfectly and fits a big shared server badly.

Offline first, sync later

The second reason is that the edge is often disconnected. A delivery driver’s phone loses signal in a parking garage. A soil sensor on a farm talks to its gateway twice a day. A factory line keeps running when the plant’s uplink goes down. Software in these places has to accept writes locally and reconcile them later, and an embedded database is the obvious place to keep them.

This is where most of the interesting work is happening. Turso has offline sync in beta, with local writes pushed to the cloud and remote changes pulled down when the connection returns. In August 2026 it opened an early preview of concurrent writes on Turso Cloud, using BEGIN CONCURRENT and an MVCC design adapted from Microsoft’s Hekaton, so writers that touch different rows don’t queue behind one lock. The open-source Turso engine, a ground-up Rust rewrite of SQLite, had shipped MVCC a couple of months before that.

The browser joined the party too. SQLite has an official WebAssembly build that can persist to the Origin Private File System, and DuckDB runs in WASM as well. A web app can carry a real SQL engine and query megabytes of local data without a server round trip at all. For dashboards, note-taking apps and anything that has to work on a train, that changes the architecture.

Size matters too, at the far end of the edge. A microcontroller with a few hundred kilobytes of flash can’t host Postgres. It can host a key-value store. LMDB, a memory-mapped B-tree from the OpenLDAP project, has been the quiet workhorse here for years: tiny and crash-safe by design, with very fast reads. RocksDB, Meta’s log-structured store, sits at the other end of the scale, embedded inside larger systems that need heavy write throughput. Both prove the same point. You don’t need a server process to get durable, transactional storage. You need a good file format and careful code around fsync.

DuckDB pushes the in-process idea into analytics. It reads Parquet and CSV directly, runs columnar queries on a laptop that once needed a warehouse, and in 2.0 added asynchronous I/O for remote and object-storage reads. It also polished Quack, its client-server protocol, which is a tell. Even the in-process champions are adding a server mode for the cases where one process isn’t enough.

The honest trade-offs

None of this makes the embedded database a universal answer, and pretending otherwise burns teams.

Concurrency is the first limit. SQLite in WAL mode lets many readers run alongside one writer. One. If your workload has many processes writing at once, they queue, and under load they time out. Turso’s MVCC work exists precisely because this is the most common complaint. It’s promising; it is also new.

Write scaling is the second. A single file on a single machine scales up, not out. Cloudflare’s 10 GB ceiling per database is a hard limit, and the documentation says it can’t be raised. The answer is sharding by design (one database per tenant), which is elegant when your data splits cleanly and painful when it doesn’t. Cross-tenant reporting means pulling data somewhere else.

Replication is the third, and the hardest. Read replicas are easy to bolt on. Multi-writer sync is not. Two devices editing the same record offline will conflict, and someone has to decide who wins: last-writer-wins, CRDTs, or application logic. Turso’s own offline sync beta shipped conflict detection before conflict resolution, which is a candid sign of how hard this is. Anyone selling painless offline sync for arbitrary SQL is skipping a chapter.

And there’s operations. A server database gives you one place to back up, monitor and patch. Ten thousand embedded databases spread across devices give you ten thousand places. The tooling is getting better, but it is still younger than the Postgres ecosystem by decades.

A few small projects worth watching

The space is wide open for smaller engines that pick one hard problem and work on it. Three open-source projects take different cuts at it.

AltSql targets IoT. It puts key-value access and SQL on the same records from a microcontroller to a gateway, with the device side at about 15 KB. It’s in alpha, tested in simulation, including 400,000 simulated power cuts without losing saved data, and compiled for a Cortex-M4 but not yet run on real chips. We covered its device-to-gateway sync idea in more detail here.

KvSQL is a Rust engine with no dependencies beyond the standard library. Key lookups read rows straight from the B-tree, SQL queries read the same rows, and both share one file and one transaction. It ships as an embedded engine plus a small HTTP server, kvsqld. Version 0.1.0 came out on October 22, so it’s early.

HyperCrux goes for breadth: SQL, key-value, graph links and vector search over the same records in one SQLite file, embedded in Go or run as a single binary. It’s at version 0.2 under Apache 2.0, with a pure Go beta engine in pre-release.

All three are young, and their benchmarks are self-reported. They’re interesting because they share the instinct driving the bigger names: keep the data next to the code, in one file, and make the different ways of reading it agree.

That instinct is winning the edge. The server isn’t going away. It’s just no longer the first place the data goes.

Filed Under: News Tagged With: Cloudflare, DuckDB, Edge Computing, Embedded Databases, SQLite, ioT

Footer

Recent Posts

  • Maxinsights Raises $30 Million Seed to Film Real Household and Hotel Work as Training Data for Robots
  • Nevis Raises $70 Million Series B With Schwab Backing as AI Agents Take Over Wealth Adviser Back Office Work
  • Apple iPad mini Gets an OLED Screen, A20 Pro Chip and IP67 Rating in Its Biggest Redesign Yet
  • An AI Agent Needs a Permission Model as Much as a Language Model
  • Why Small Embedded Databases Are Winning the Edge
  • inDrive and Pony.ai Target Kazakhstan for a Robotaxi Project Built on NVIDIA Technology
  • Applied Ventures Backs Albert Invent as Materials Research Becomes an AI Battleground
  • Pathos AI Raises $237 Million as BMS and Tempus Back a Broader Drug Development Push
  • Tractian Raises $130 Million Series D at $1.4 Billion Valuation for AI That Predicts Machine Failures
  • Jensen Huang Has a Strong Case Against the 2029 Memory Glut Theory

Media Partners

  • Market Analysis
  • BareProxy.com
  • App Coding
How to Tell an AI Infrastructure Order From an AI Infrastructure Story
Hybrid Bonding Is the Bottleneck Nobody Prices Into HBM4
EQT and Ovintiv Pay for Drilling Inventory While Johnson Controls Sells Its Alarm Business
Stocks Fall as Meta Capex Outruns Its Cash Flow, Chips Sink and Brent Tops $111
Why China’s Alleged Memory Chip Breakthrough Should Be Taken With a Grain of Salt
EnergyX’s Trafigura Deal Moves Lithium Extraction From Technology Promise Toward Commercial Scale
Marvell’s New Memory Expanders Target the Economics of AI Inference
AI Financing Fears Are Overrated: Hyperscaler Debt and Nvidia’s Vendor Deals Sit on Real Revenue
HyperCrux Market Analysis: Agent Memory and Local AI Drive Demand for a One-File Multi-Model Database
Small Infrastructure Primitives Run the $700 Billion AI Buildout, and Open Source Is How They Get Adopted
BareProxy Adds AI Crawler Control That Tells Real GPTBot and ClaudeBot Requests From Fakes
BareProxy Adds Bot Challenges That Make Scrapers Posing as Browsers Solve a Puzzle First
BareProxy Adds Cookie Consent That Holds Back Tracking Cookies Until a Visitor Says Yes
BareProxy Adds Markdown Serving That Turns Markdown Files Into Web Pages With No Build Step
BareProxy Ships Its First Nine Plugins, From Maintenance Pages and CORS to Geo Rules and Rate Limits
BareProxy 0.2.0 Runs WebAssembly Plugins Through the Proxy-Wasm Interface
BareProxy 0.3.0 Writes Its Plugins in Rust, Where a Plugin Adds 15 Microseconds Instead of 42
BareProxy Is an Operating System for Web Traffic: A Small Core and a Long Stream of Plugins
BareProxy Plugins Go WebAssembly, Starting With RenderCache and AI Crawler Control
BareProxy Runs on Linux, macOS and Windows, and One Plugin File Runs on All of Them
A Useful Software Primitive Should Be Easy to Explain and Hard to Replace
Apple's US Link-Out Fee Is Zero Until a Judge Sets It, and Apple Is Asking for Up to 15%
20 Primitive Project Ideas: Tiny Go Engines That Do the Essential 10% of Redis, Kafka and Etcd
40 Projects Worth Building: SQLite for X, Tiny Infrastructure, MCP Tools and Edge Systems
40 Radically Different Infrastructure Primitives, From Reach and Expect to Ignorance and Stop
Counterfactual Store vs. Markov Chains: From Branching State to Reach, an Engine for Possible Futures
Counterfactual Store: A Database That Stores Alternatives, Not Just Facts
Mobile App Performance: The Five Metrics That Decide Retention and Store Ratings
Screening 20 Infrastructure Primitives Money First: Why Distill and Precomputing Survive
Temporal KV: A Key-Value Store That Remembers History

Media Partners

  • AltSql.com
  • Technology Conferences
  • API Coding
AltSql Is a Native Hybrid of Key-Value and SQL, From the Sensor to the Gateway
AltSql Core and AltSql DB Are Now Open Source Under the Apache License 2.0
Introduction
AltSql DB 0.3 Adds Secondary Indexes and Statement Savepoints: A Lookup on a Million Rows Ran 198 Times Faster Than a Full Scan
How It Works
AltSql DB 0.2: SQL and Direct Calls Wrote Byte-Identical Files Over 100,000 Random Steps
AltSql DB Keeps a Whole Fleet in One File and Reads by Key 5 Times Faster Than SQLite Through SQL
AltSql Core and Six Prototypes Built Around It
AltSql Ask Puts One SQL Question to 500 Simulated Machines and Brings Back Only the Answers
Machine Monitoring Case Study: Overheat Alarm Handled on the Sensor While Wi-Fi Was Down
Conferences Are Theatre and the Hallway Is the Product
SoftDevStudio.com at Web Summit 2026, November 9-12, MEO Arena and FIL, Lisbon
Bodor Laser Brings Integrated Tube Processing to EuroBLECH 2026
Cloudflare Connect 2026 Reportage, October 19-21, Moscone West, San Francisco
International Test Conference 2026, October 11–16, Grand Hyatt San Antonio River Walk, San Antonio
AUSA Annual Meeting & Exposition 2026, October 12–14, Walter E. Washington Convention Center, Washington DC
SEMICON West 2026, October 13–15, Moscone Center, San Francisco
Fal.Con Europe 2026, November 2–4, Fira Gran Via, Barcelona
A3 International Robot Safety Conference 2026, November 3–5, Huntington Place, Detroit
Humanoids Summit Silicon Valley 2026, December 1–2, San Mateo County Event Center, San Mateo
RFC 9457 Problem Details Gives API Errors One Shape Instead of Fifty Ad Hoc JSON Bodies
Idempotency Keys: How One Header Stops a Retry From Charging a Customer Twice
A Tiny ETL Binary Competes With curl, jq and SQLite in a Cron Job, So Build It That Small
An AI Agent Flight Recorder Belongs in One Portable File, the Way HAR Did It for HTTP
Benchmarking MCP Servers and Stateful API Workflows: Latency, Throughput and Tokens per Call
Diagnose a Slow API From One Request: DNS, Connect, TLS, Server Wait and Transfer
Finding API Endpoints, Tables and Flags Nobody Uses Takes Code and Traffic Together
Fuzzing MCP Servers: Generate Bad Arguments From the Tool Schema and Watch What Breaks
Give Any API a History: Poll It, Hash It and Query Old Versions With SQL
Infer Your API's Real Contract From Traffic, Then Diff It Against the Docs

Copyright © 2026 Technologies.org

Media Partners: Market Analysis · Market Research · Referently · Photography