Why your data mesh is failing
It isn’t a Snowflake or Databricks problem. It’s a photography problem.
The average corporate data mesh is a camera roll of blurry iPhone shots with a thumb on the lens. We’ve sold the industry a beautiful lie: that decentralising data and giving every domain team the tools to build their own data products will magically create value. But we forgot one crucial detail. Giving someone a camera doesn’t make them a photographer.
I’ve spent three years in the trenches of data mesh implementation. The lesson that took me far too long to learn: this is a photography problem, not a platform problem. Data mesh doesn’t fail because of technology. It fails because teams confuse taking snapshots with owning photographs. A data mesh works when three things happen:
Perspective widens
Ownership becomes explicit
Teams resist the urge to photograph whatever is in front of them
Everyone can take a photo
I am chronically guilty of ‘spray and pray’ photography. When someone hands me a phone to take their picture, I mash the digital shutter button like a video game champion. I give zero thought to the lighting, the background or, if I’m being honest, sometimes even the people in the shot themselves. All we end up with are 20 identical, useless shots clogging up the photo gallery. None are the trusted memory being sought. In the data world, we do the exact same thing. I’ve watched a team rebuild an entire data product from scratch - reimplementing months of gnarly business logic - because they needed one extra column and didn’t trust the team that owned the original. In both cases, we created plenty of data but zero trusted memories.
The contrast became obvious when I hired a professional photographer for my wedding. I watched them work and learned that they didn’t mash buttons but intentionally used lighting, aperture and framing. They knew exactly what they wanted to capture and - more importantly - what they wanted to leave out. Building a data product requires the exact same discipline. A professional knows that a great photo is a deliberate, narrow slice of reality.
Every photo is a slice
A photograph is narrow by design. It captures one subject from one perspective. That’s the entire point. A data product should work the same way. Think of your smartphone’s ‘panorama’ feature where you take multiple overlapping shots to create a wider landscape. Have you ever actually taken a good one?
The same is true of the panoramic data product. Someone tries to capture everything - whether data or scenery - and ends up with a warped monstrosity with someone’s disembodied arm floating in the middle. The lesson? Split data products by subject area. Multi-domain data products are corporate panoramas. Crop ruthlessly.
Which photo goes into the album?
We have all been held hostage in that meeting. The one where three different leaders proudly present three completely different numbers for ‘active customers’. Instead of a strategic discussion about business value, it devolves into a 55 minute ‘robust discussion’ over what the word ‘active’ means.
That meeting doesn’t derail because you chose Snowflake over Databricks. It derails because your organisation confused the ability to snap a photo with the discipline of curating a gallery.
What you witnessed in that room is simply two teams who photographed the exact same scene from slightly different angles. Both datasets are technically “correct” in their own context. Neither is authoritative. To solve this, you need a person or process with the authority to decide which shot actually makes it into the company album. That’s governance, not bureaucracy. But governance only cleans up the mess after the fact. The better question is: what if teams shot with intention from the start?
Shoot with the end in mind
A great photographer doesn’t just take the shot - they understand their audience. Who is this photo for? What story does it tell them? People take photos at concerts just to prove they were there, not to create a masterpiece. Similarly, engineers build data products just to check a box, taking the path of least resistance.
Nobody wants to spend weeks building a data product, only to burn the final Friday manually typing out metadata into a dusty Confluence page. Neither do you pull out your phone at a concert and want to click through endless settings before taking the shot.
People will always choose auto mode. Every time. Our job isn’t to shame people into manual mode. It’s to make auto mode good enough. We must code the rules so the path of least resistance is also the path of compliance.
The EXIF data test
If you’re wondering whether your organisation has photographers or just people with cameras, here’s a quick exposure test:
Can a new team discover data products without a meeting?
Do data products exist with clear owners?
Are duplicates intentional and labelled as so?
Is there accountability for how it all fits together? (aka the big picture)
Say cheese
Data mesh isn’t a technical or architectural problem - it’s a perspective problem. Like photography, everyone thinks they can do it because they interact with data every day. But it fails because we confuse the ability to snap a photo with the discipline of curating a gallery. The real work of data mesh is widening people’s aperture - getting them to see beyond their narrow frame of reference.
Widening the aperture is uncomfortable. It means caring about things outside your frame. It means accepting that your “photo” exists in a gallery alongside others - and that the gallery needs curation. For most domain teams, this means caring about someone else’s problem. That’s the real ask of data mesh. Not better tooling. Not more autonomy. More perspective.
Are you building a data mesh with skilled photographers... or are you just handing out disposable cameras and hoping for a masterpiece?