Boringly Simple: How 25 Contributors Change Our Data Platform With Almost No Review

In Nobody Codes by Hand, I told how the number of people contributing to our data platform at Optic 2000 went from 4 to 25 — most of them business people, working through a coding agent. I ended on an observation: with 25 contributors, review becomes the critical path. A platform team of four can't read everything 25 people and their agents produce.

Every change still goes through a Pull Request approved by a human. But in most of them, there's almost nothing left for us to review. Not because we trust agents more than we used to, but because we designed the platform so that a contribution can hardly express a mistake.

Our platform is deliberately boring. Adding a data source means writing a YAML file. Deploying a function means a Python file and a few lines of YAML. Publishing a data app means moving a file out of a folder. Everything that's hard — infrastructure, security, deployment, monitoring — is written once, by the platform team, in what I'll call the glue. Contributors never need to touch it, and neither do their agents.

This article describes the five principles behind that boredom, and what they change for a platform team. As in One Name, One Definition, the technical details are tucked away in collapsible sections, for those who want to replicate them.

Why Boring Is the Goal

A data platform contains two kinds of code. The configuration says what the company needs: this source and its tables, this function and its schedule, this role and what it can read, this data app and who owns it. The glue says how to deploy it: cloud permissions, warehouse grants, load balancer rules, container definitions, CI steps, alerts.

In most platforms, the two are intertwined. Adding a source means copying a Terraform module, adding a cloud permission, wiring a notification, declaring an alert. Every contribution touches the glue, so every review has to check the glue: is this permission too broad? Will this rule collide with another? Is the alert wired? That's specialist review. It doesn't scale to 25 contributors, let alone to their agents, which will happily write a perfectly plausible policy granting access to every bucket.

We made the opposite choice: contributors only write configuration, and the configuration can only say what the glue lets it say. What a contribution can't express, a reviewer doesn't have to check.

Data teams already know a tool built this way: dbt. You write a SQL model, and dbt handles incremental loading and execution order for you. We built the whole repository that way.

It matters even more with coding agents. An agent reproduces what it sees in the repository. In a platform where each pipeline is deployed a little differently, it has a dozen patterns to choose from, and picks one without telling you which. In a boring platform, the thirtieth source looks exactly like the twenty-ninth: the agent copies the previous one, and it's right.

Engineers know the advice to choose boring technology. We apply it one level up: we make contributing boring. Cleverness belongs in the glue, written once by the people who maintain it. Wherever a contributor goes, there should be nothing to figure out.

Principle 1: Separate Configuration From Glue

The whole platform is driven by a handful of YAML files. Terraform reads them and deploys everything they describe, in loops: nobody writes Terraform to add a source, a function, a web app or a role.

Here is everything a contributor writes to bring a new source from an API into the warehouse. A few lines in functions.yml:

get_crm:
  category: get_data
  schedule: cron(0 4 * * ? *)
  timeout: 600
  memory: 2048

A Python file, functions/get_crm.py, whose function calls the API and drops its records in S3 as they are, under one of our decorators. And the description of what lands, in load/crm.yml:

source_path: crm/
source_type: API
data_extract_lambda: get_crm
description: |
  Customer relationship data: accounts, contacts, opportunities.
format:
  type: JSON
destination_tables:
  - name: ACCOUNTS
    description: Customer accounts, one row per account and extraction.
    pattern: "accounts/accounts_.*\\.ndjson"
    columns:
      - name: ID
        type: varchar
        description: Identifier of the account in the CRM.
      - name: NAME
        type: varchar
        description: Name of the account.

Once the Pull Request is merged, the deployment pipeline does the rest: the function goes live with its schedule, a CRM schema appears in the warehouse with an ACCOUNTS table and its Snowpipe, and the files the function drops start flowing in. Not a line of Terraform, not a click in a console. Adding a source adds configuration, never glue: today, about forty source files and some forty functions are deployed by under 5,000 lines of Terraform that contributors never need to open.

Reviewing such a contribution means reviewing a declaration: a schedule, a memory size, the description of a table. The reviewer doesn't check whether a cloud permission is too broad or a Snowpipe wired to the right notification: there's no permission and no Snowpipe in the diff. Those were reviewed once, when the glue was written, and they're the same for every source.

When a need comes up that the configuration can't express, we don't make an exception: we extend the configuration, for everyone, with an optional field. A function that unzips large archives needs more disk: ephemeral_storage. A function calling an API with a strict rate limit must never run more than twice at once: max_concurrency. A table refreshed once a week mustn't raise a freshness alert every day: max_staleness_hours. A source that isn't producing yet, or is down while its supplier fixes it: muted_until, with a date, after which monitoring resumes on its own. Each field was added to the glue once, and is now available to every contributor, with no new pattern to copy.

And access to the warehouse is configuration too. Privilege sets are defined once, and each role combines them; a person is an email and a list of roles. No GRANT is ever run by hand: a missing privilege is added to the YAML and deployed like everything else. The YAML file is the access policy — versioned, readable by anyone, reviewed like any change.

locals {
  sources = {
    for file in fileset("${path.module}/../load", "*.yml") :
    trimsuffix(file, ".yml") => yamldecode(file("${path.module}/../load/${file}"))
  }
}

resource "snowflake_schema" "source" {
  for_each = local.sources
  database = "RAW"
  name     = upper(replace(each.key, "-", "_"))
  comment  = split("\n", trimspace(each.value.description))[0]
}

read: &read
  TABLES: [SELECT]
  VIEWS: [SELECT]

write: &write
  SCHEMA: [CREATE TABLE, CREATE VIEW]
  TABLES: [SELECT, INSERT, UPDATE, DELETE, TRUNCATE]
  VIEWS: [SELECT]

role_grants:
  FINANCE:
    warehouses: [FINANCE_WH]
    objects:
      MARTS: *read
      SANDBOX.FINANCE: *write
  MARKETING:
    warehouses: [MARKETING_WH]
    objects:
      MARTS.MARKETING: *read

human_users:
  jane.doe@example.com:
    roles: [FINANCE]
  john.smith@example.com:
    roles: [MARKETING, FINANCE]

Principle 2: Derive, Don't Declare

Every field a contributor fills in is a field they can get wrong. So we ask the same question of every setting: could the glue compute it? If it can, the field doesn't exist.

If two settings must agree, one of them shouldn't exist.

• Where a function runs. A Lambda function can't run longer than 15 minutes or use more than 10 GB of memory. Our functions declare their timeout and memory; when they ask for more than Lambda allows, the glue deploys them as a scheduled Fargate task instead — same Python file, same decorator, same status reporting. There's no backend field, so no backend can contradict the timeout.
• How much CPU. Fargate only accepts certain pairs of CPU and memory. Declaring the CPU would just be a chance to declare an invalid pair, so the glue picks the smallest valid CPU for the requested memory. A task that needs more compute asks for more memory.
• Where a web app lives. A web app's URL is its name, and its health check has a default path. Its load balancer rule needs a priority, unique across all web apps. Nobody picks it, and it isn't the web app's rank in alphabetical order either: adding a web app would shift all the ones after it, and the load balancer refuses two rules with the same priority, even for a moment. It's derived from a hash of the name, so adding or removing a web app never moves the others.
• What things are called. A source's warehouse schema is the name of its YAML file. A person's warehouse username is derived from their email, and their first role is their default role.
• What the menu shows. Our data apps — dashboards and small business tools, each a single Python file — are listed in a menu, but there's no menu file: the menu is the folder tree. A section is a folder with a short _section.yml, and each data app declares its own title, icon, description and owner, in its own file:

page.config(
    title="Supplier Invoices",
    icon="🧾",
    description="Invoices received from suppliers, by store and by month.",
    owner="jane.doe@example.com",
)

Adding a data app means adding a file; deleting the file removes it from the menu. There's no central list to update, and no menu entry pointing to a data app that's gone. Even publishing is a file move: a new data app starts in an under-construction folder, which the web app we serve to our stores never shows, and goes live there when the file moves out.

Derivation has a second benefit: the platform can change without anyone migrating anything. If Lambda's limits change tomorrow, we change one rule in the glue, and every function lands where it should at the next deployment.

Principle 3: One Place for Each Kind of Logic

Configuration takes care of infrastructure. But contributors also write logic: Python functions, SQL models, data apps. There, the same idea takes another form: each kind of logic has exactly one place where it can live, and the boundaries are strict.

• A function lands its source as is. Every field the source returns, under the source's own name, nested objects and arrays kept whole: no renaming, no filtering, no unnesting. All transformation happens in dbt. An extraction has nothing to decide, and nowhere to hide a business rule.
• Data enters the warehouse one way. Every source lands in S3 first, and Snowpipe loads it: no function ever writes to the warehouse directly. Every raw row can be traced back to its file, and any load can be replayed.
• Business logic lives in dbt, metrics in the semantic layer. A ratio is computed by the semantic layer as a ratio of sums, never averaged in a data app. Year-over-year comparisons — which period to compare with, which stores to include — are computed by a single engine, from a rule written once in our data portal's guides; no data app and no ad hoc query re-derives them. Even an agent answering an ad hoc question queries the semantic layer first, so its figures match those of the data apps.
• Definitions live in the business dictionary, attached to column names, as described in One Name, One Definition.
• Look and feel lives in a style guide. Data apps don't style their own indicator cards or tables: they call the helpers of a shared style-guide module, so that data apps written by different people look like one product.
• Explanations live next to what they explain. A function's entry in functions.yml holds no explanatory comment: what the function extracts, why that schedule, how its incremental load works goes in the docstring of its Python file, which the data portal publishes on the function's page. A metric's definition lives in its description in the semantic layer, and the portal renders it.

When every kind of logic has a single place, a reviewer no longer asks where something should be, only whether it's right. And an agent never has to choose between two places: there's only one.

Principle 4: Make the Safe Path the Only Path

A contributor shouldn't have to remember security, deployment or monitoring. When something matters, it's built into the glue, with no way to opt out — and what can't be forgotten doesn't need to be checked.

Security at the door, not in each data app

Our web apps sit behind a load balancer that requires the company login before a single request reaches them, and each one re-checks the user's identity through a shared library. A store manager only ever sees their own stores' figures, because the perimeter is applied by the semantic layer, from the user's identity, for every data app at once — whether the question comes from a data app, a Teams card or an AI assistant through our MCP server. A contributor building a data app can't forget a security check: there isn't one to write.

Agents get the same treatment. On a contributor's workstation, the agent connects to the warehouse with the contributor's own login and roles: a marketing contributor's agent can't read what marketing can't.

Configuration where its owner can change it

Put each piece of configuration where its owner can change it safely. Infrastructure belongs in reviewed YAML. Who can open which data app doesn't: it changes often, and it's decided by managers, not contributors. We first declared access in each data app's file, then moved it out of the repository altogether. Today it lives in a database, edited by a handful of managers from a dedicated data app. A data app nobody has been granted access to is open only to its owner and to admins. An access request arrives in Teams with a link that opens that person and that data app, and suggests the groups that would grant it while opening the fewest other data apps. A Pull Request can't grant access, so no reviewer has to check who gets to see what.

The same goes for a setting an operator must change without an engineer: it goes to a key-value store they can write to, not into the code. Secrets hold credentials, nothing else.

One deployment path

A merge to main triggers the only deployment path. Based on the files changed, the pipeline rebuilds what's needed, in a fixed order: infrastructure first, then the data transformations, then the code and web apps that read them — so code never ships ahead of the data it expects. Even metric definitions ship with the data: the dbt run publishes them next to the exports it produces, and every data app picks up the new version within minutes.

A Pull Request builds the models it changes in a schema of its own, where reviewers and agents can compare them with production. But it can't deploy anything, by accident or on purpose: the cloud only issues production credentials to pipelines running on main. And when a web app misbehaves after a deployment, rolling it back is one line in apps.yml: pin its image to the previous commit.

Monitoring built into every function

Every function runs under one of our decorators: scheduled, or triggered by a file, an event or an HTTP call. The decorator records each run's status, duration and logs. If the function dies before it can report — out of memory, out of time — a crash handler writes the error in its place. Every hour, a check compares each function's last run with its schedule; every morning, after a job has reloaded any file Snowpipe missed, another compares each raw table's freshness with its threshold: 24 hours, unless the source's YAML says otherwise. An error stays flagged until someone marks it as resolved in the portal, even if the next run succeeds. And since run statuses are exposed in the semantic layer like any other data, a scheduled Teams card lists the functions in trouble. When there are none, it says so: a card that stays silent when all is well can't be told from a card that's broken.

A contributor gets all of this for their function without writing a line of it.

Documentation generated from the configuration

Whenever the configuration changes, the data portal is regenerated from it: a page per source and per table, from the YAML files; a page per function, with its docstring, its configuration, its latest status and a button to run it; a page per dbt model; the dictionary and the metrics. Contributors document by filling in the descriptions of their configuration — there's no separate documentation to keep in sync.

Security, deployment and monitoring aren't items on a review checklist: they're properties of the platform.

Principle 5: Put the Rules Where the Agent Works

Configuration and glue remove most of the room for error. What's left — SQL, Python, data apps — is written by agents. For that part, the rules have to reach the agent at the moment it acts, and the machine has to check them.

• Instructions next to the code they govern, and a skill for each kind of task: the agent's context holds the rules of what it's doing, not the whole rulebook.
• Checks at write time, which send any violation back to the agent before a human ever sees the diff.
• Review agents before the Pull Request is even opened: when a human receives it, most of it is already clean.
• One golden rule: every review comment becomes a rule, so the same remark doesn't come back.

I detail these mechanisms, and the rules our incidents taught us, in The Platform That Learns.

Human approval doesn't disappear: every Pull Request is still approved by a person. But it comes after a first review, which has already checked the conventions. What's left for the human is the question no machine can settle: is this the right thing to build?

What This Changes

• Review shrinks to what matters. Most diffs are YAML declarations and code that follows a pattern a review agent has already checked. We read intent, not plumbing.
• Agents get it right the first time. A boring platform has one pattern per task. The agent copies the last source, the last function, the last data app — and it's right.
• Contributors build what they know. A financial controller can add a source, a model and a data app without ever hearing about IAM, VPCs or warehouse grants. Their expertise goes into the business logic, which is where we need it.
• The glue improves for everyone at once. When we improve crash detection, tighten a grant or change a default, every function and every role gets it at the next deployment. Nobody migrates their pipeline, because nobody owns a pipeline's plumbing.
• The infrastructure stays replaceable. Since the configuration says what and not how, moving a function from Lambda to Fargate — or the platform to another provider — is a change in the glue, not in forty contributions. That's what I meant, in my article on independence, by configuration is an asset, the metal is a commodity.
• The platform team becomes a product team. Its product is the glue, the conventions and the review agents; its users are the 25 contributors and their agents.

What It Costs

Boring for contributors isn't free for the platform team.

• The glue is harder to write. Deriving everything, supporting every optional field, closing every unsafe path: our Terraform is denser than a module per pipeline would be. It's written once, but it has to be written well. And when a derivation is needed outside Terraform — the portal also needs to know where a function runs — both implementations must change together.
• You have to say no. Every "just this once" exception is a second pattern for agents to copy. When a need doesn't fit, the answer is to extend the configuration for everyone, or to decline.
• The configuration must stay declarative. A YAML file that grows conditions and loops is a programming language without a debugger. Ours only says what; anything that needs logic goes into the glue, as a derivation.

Key Takeaways

Our platform is boring so that what people build on it doesn't have to be.

Everything hard — infrastructure, security, deployment, monitoring — is written once, in the glue, by the team that maintains it. Everything else is configuration that says what, derived wherever it can be, with a single place for each kind of logic and the rules waiting for the agent wherever it works. What's left for the reviewer is the only question that ever mattered: is this the right thing to build?

Ask yourself this about your own platform: when someone adds a data source, how many of the lines they write are plumbing? Every line of plumbing in a contribution is a line someone has to review — and with agents writing more and more of them, that someone won't keep up.

Want to open your data platform to business teams and their agents without becoming the bottleneck? Let's talk about it in a 30-minute data chat.