Databricks and Neon | Databricks Weblog

In the present day, we’re excited to announce that now we have agreed to accumulate Neon, a developer-first, serverless Postgres firm. Neon’s co-founders are among the many only a few individuals on the planet who may re-architect Postgres with true separation of storage and compute, constructed for contemporary builders and AI programs. Their world-class group of Postgres hackers and database veterans will be part of Databricks to ship serverless Postgres at manufacturing scale to builders in an AI-native world.

A developer-first mission, born from Postgres experience

4 years in the past, the Neon co-founders joined along with a imaginative and prescient to disrupt the database trade. They noticed that the foundations of database applied sciences have been largely designed for the 90s period. Their purpose was to construct a brand new database platform that will dramatically enhance experiences for builders, in some elementary methods:

1. They foresaw that Postgres would develop into the de facto customary for databases and had the imaginative and prescient to create a serverless Postgres platform.
2. They have been decided to make it doable to create a brand new Postgres occasion in seconds, so builders wouldn’t want to attend. 
3. They got down to simplify the operational side of database scaling by automating it as load modified, so builders may begin very small and never fear about over- or under-provisioning.
4. They needed to allow speedy experimentation and testing by supporting on the spot forking and branching of databases and creating totally remoted databases, as databases are sometimes one of the crucial tough modules to check in end-to-end utility testing.

A couple of years in, the Neon group engineered a brand new, modern database structure that decouples storage scaling from compute scaling, which in the end enabled all the above objectives. When Neon first launched, builders raved concerning the velocity, the simplicity, and the flexibility to department and fork their databases like Git does for code.

The shift towards AI brokers

As Neon grew to become GA final yr, they observed an attention-grabbing stat: 30% of the databases have been created by AI brokers, not people. Once they checked out their stats once more just lately, the quantity went from 30% to over 80%. That’s, AI brokers have been creating 4 occasions extra databases versus people.

For those who consider AI brokers as your individual large group of high-speed junior builders (probably “mentored” by senior builders), it’s then not stunning that the identical capabilities the Neon group centered on that made Neon nice for builders additionally made Neon nice for these AI brokers:

1. Postgres open supply ecosystem: All frontier LLMs have been skilled on the huge quantity of public info out there concerning the Postgres open supply ecosystem, so all AI brokers are specialists in utilizing Neon, which is constructed on Postgres.
2. Velocity: Conventional databases have been designed for people to provision and function. It was OK to take minutes to spin up a database. Given AI brokers function at machine velocity, extremely speedy provisioning time turns into crucial.
3. Elastic scaling and pricing: The decoupled storage from compute serverless structure permits extraordinarily low-cost Postgres cases. It’s now doable to launch hundreds and even hundreds of thousands of brokers with their very own databases cost-effectively.
4. Branching and forking: AI brokers will be non-deterministic, and “vibes” should be checked and verified. Neon’s skill to immediately create a full copy of a database, not just for schema but in addition for the information, permits all these AI brokers to be working on their very own remoted database occasion in excessive constancy for experimentation and validation.

Shared DNA

We have now lengthy recognized Nikita Shamgunov, Heikki Linnakangas, and Stas Kelvich – the founders of Neon. Earlier than Neon, Nikita began his profession engaged on SQL Server and later grew to become well-known within the database group as a co-founder and CEO of SingleStore, driving the startup’s enterprise to over $100M in annual income. Heikki has been a committer on Postgres for twenty years – he began his Postgres journey when he was bored on paternity go away and determined to look into Postgres internals for enjoyable. Stas is an ex-physicist who bought into Postgres hacking as a result of he wanted a greater R-tree implementation for his tourism search engine startup.

Databricks and Neon share the identical DNA in constructing hardcore technical improvements on the infrastructure layer. We additionally share the idea within the significance of open supply: we initially began the Apache Spark™ mission at UC Berkeley, which additionally occurs to be the unique birthplace of Postgres, the open supply database mission Neon builds on.

Trying ahead

OLTP databases symbolize a $100B market dominated by merchandise that have been constructed a long time in the past. We imagine it’s time for this market to be disrupted by builders and AI brokers. Along with the Neon group, we stay up for constructing probably the most developer and AI agent pleasant database platform.

What does this imply for present Neon clients and companions? We’re totally dedicated to the way forward for the Neon platform. The roadmap stays bold, and we intend to proceed making Neon the most effective database platform for builders. After closing, present clients and companions can anticipate continued help and innovation – now with the total backing and assets of Databricks. It’s an awesome second for present Neon customers, and an thrilling one for builders simply discovering Neon. Go launch a brand new Postgres occasion on Neon at the moment!

We’re equally enthusiastic about what this may imply for Databricks’ enterprise clients. We plan to share rather more on the upcoming Knowledge + AI Summit in San Francisco, June 9–12.

Try Neon’s weblog to listen to instantly from them about why they’re excited to hitch forces.