[{"content":"How can an AI-ready data warehouse improve analytics without losing the business meaning behind the data? We’re restructuring a legacy data warehouse around a canonical core data model—and sharing the architectural decisions, lessons, and implementation using dbt and Microsoft Fabric.\nThis series explains why we began a major restructuring of our legacy data warehouse and how we implemented a new architecture and way of working. The series will also identify what we learned along the way, including what worked and what did not.\nFigure 1. The journey builds an AI-ready data foundation by making business meaning explicit and carrying it through modelling, generation, and documentation. The AI era # We have all read about ambitious AI promises. Machines will answer questions before we ask them, automate work, and remove people from processes. These predictions shape how organisations think about the immediate future. At conferences for data professionals, many presentations show what AI can do with data. AI can handle data and provide answers, but the demonstrations often use simple data structures that do not reflect the complexity of a real business environment.\nThe goal # The central question is how we can use AI to improve the quality of analytics work, increase the throughput of business requests, and reduce the time and cost involved. The practical question is what we must do to structure enterprise data so that it provides AI with the meaning it needs to deliver those benefits.\nThe medallion architecture # Figure 2. Bronze, silver, and gold define transformation stages; explicit information architecture defines data shape, relationships, and meaning. The medallion architecture is widely used by data practitioners. This architecture divides the transformation of data across three distinct phases:\nbronze: the layer where data is ingested with minimal conversion silver: the layer where source data is translated into business structures and transformed for BI and data science gold: the layer consumed by reporting, analytics, and other end-user tools The medallion architecture is a process architecture: it tells you where transformation occurs, not what the data should look like. What it lacks is an information architecture — a defined shape for each layer, with explicit rules, a specified input and output, and documentation that both people and machines can read.\nA medallion architecture is often implemented as a free-form transition from source systems to gold structures. For years, teams were told that this did not matter: structure only slowed delivery, which would hinder development of new features.\nLacking documentation # Legacy source systems often have cryptic table and attribute names. Relationships exist, but they are often not described. Knowledge of their meaning is assumed. Measures are sparsely described and can have different meanings in different contexts.\nThe medallion architecture has no inherent documentation requirements. Documentation was not a priority because there were always gatekeepers. The gatekeepers knew about the meaning of tables and columns. They made sure that the correct joins were made and the right measures were chosen. AI requires this structure and documentation to derive meaning from the data.\nWe need a new foundation on which to build.\nThe new architecture # The requirements of LLMs make what was once optional or an afterthought essential to successful operation. We need a clear set of rules:\nall the data needs to have a clear structure, with explicit meaning relationships between entities become first-class citizens all the data needs to be fully described in natural language This means that we must enhance the medallion architecture with the following requirements:\nThe bronze layer preserves source data in a predefined structure. It needs to be historised. The medallion architecture gains a canonical core data model that provides the required structure for AI tooling. The core data model uses universal business concepts, so source-system specifics are grounded in general concepts. Relationships are first-class citizens in the model. Documentation describes the structures and data in a form that links the data to LLMs and supports reasoning. Gold models use the core data model as their source, which requires additional transformation steps. The enhanced medallion architecture # What we end up with is:\nFigure 3. A documented core data model separates source-to-canonical mapping from the preparation of data for BI, data science, and end-user metrics. bronze: the ingestion layer, with source data held in a defined and historised format silver: the layer that translates source-system data into the canonical model and adds business meaning core data model: the fully documented canonical model where the new business structure resides gold engineering: the layer that transforms the canonical model into structures ready for BI and data science tools gold: the end-user layer, where fully described metrics and KPIs are available for consumption The additional layers make responsibilities and handoffs clearer for people and AI systems. They decouple the ingestion of data from the presentation of data, which leads to limited interdependencies. Teams can now work on the layers simultaneously.\nCentral to this enhanced medallion architecture is a data model based on universal business concepts and the principles set out by Len Silverston in his Universal Data Models.\nUniversal data model as the chosen architecture for the new core data layer # The core data model needs an architecture. For AI, Len Silverston’s work on data models offers a useful foundation. Len Silverston’s Universal Data Model (UDM) architectural principles focus on creating stable, reusable, and business‑aligned data structures that can endure change over time.\nThe core idea is to model the business in terms of fundamental business concepts—called subject areas—such as Party, Product, Agreement, Location, and Event, rather than around specific applications or reports. These models emphasise semantic consistency, meaning the same concept (for example, a Customer as a role held by a Party) is defined once and reused everywhere, reducing redundancy and integration friction. UDM promotes separation of core entities from their roles, states, and classifications, allowing flexibility as business rules evolve without restructuring the data model.\nBecause these models create clean, semantically consistent, and well-governed representations of business meaning, AI systems can more easily understand, relate, and reason over the data. The separation of core entities from roles, classifications, and states reduces ambiguity, which is critical for LLM-based reasoning.\nA simple Party example # Consider an organisation that is a customer and later becomes a supplier. A source-shaped warehouse would often create separate customer and supplier records, even though both records describe the same organisation. A UDM represents the organisation once as a Party and records each business context through Party Roles: Customer and Supplier. The roles can exist at the same time or have their own effective dates.\nThe same pattern applies to people. One person can be an Employee, an Applicant, and an Inventor without duplicating their identity or contact details. The Party entity holds the common identity and relationships; Party Roles record the context in which the person or organisation participates. A subtype is added only when a particular form of Party has durable attributes or relationships that do not apply to other Parties.\nThe journey # Figure 4. The journey builds an AI-ready data foundation by making business meaning explicit and carrying it through modelling, generation, and documentation. We began a project to prepare the data warehouse for this AI era.\nAdding a core data model # Our data warehouse lacks this core data model: the central layer that emphasises business meaning over source-system structure. Adding it also changes the way data is ingested.\nAll source-ingestion processes must route data into these structures. All gold models must use the core data model as their source. YAML structures used to model the core data model # The core data layer uses a lightweight, YAML-based implementation of conceptual, logical, and physical data models.\nWe wanted to make the project as portable as possible, so we used YAML and Draw.io to develop the new data model. Many legacy data-modelling tools are expensive, complex, and tied to proprietary data formats. Building everything in YAML allows us to achieve the same result, without the overhead and costs of these tools.\nThe conceptual layer YAML # The conceptual layer contains:\n- entity names - labels - descriptions - entity types - fully described relationships - relationship names - labels - source entities - target entities - cardinality The logical layer YAML # The logical layer is kept in sync with the conceptual layer. It contains all the information from the conceptual layer but adds:\n- derived-from-conceptual - primary keys - business keys - historisation rules - fully described attributes - attribute names - attribute descriptions - data types - confidentiality flag Automation with Python and AI # Plain YAML files, a helper application developed in Python, and AI skills support the creation of new entities and synchronisation between layers. The data model remains central, while AI and Python help automate the generation of physical structures and dbt models.\nDraw.io and Markdown describe and present the model. A generated OKF bundle grounds AI and serves as the authoritative description of the core data model.\nThis combination allows us to reduce dependence on legacy modelling tools while still working efficiently.\nMicrosoft warehouse with dbt as the physical implementation # The whole process is implemented in a Microsoft Fabric warehouse. The transformation processes are implemented in dbt.\nThe technology stack # YAML for data modelling Microsoft Fabric Warehouse as the data store dbt to generate the core data layer and perform transformations Python tooling to keep the model layers in sync and generate documentation in Draw.io and Markdown an OKF bundle to provide AI with the required contracts for the core data layer AI skills to support the modelling flows and the generation of dbt models The article series # This article series focuses on this new foundation and is based on a real transition. It details the architectural choices and the implementation, showing how the architecture was built with YAML, Draw.io, dbt, and Microsoft Fabric.\nThe series aims to help other professionals who want to make the same journey, but also hopes to encourage feedback.\nWhat we wanted AI to enable after the transformation # The foundation described in this series was designed to give AI the context and structure needed to work with enterprise data. We wanted it to enable:\nsemantic grounding for text-to-SQL and natural-language querying, reducing confident but incorrect answers AI assistance in designing and maintaining a high-quality data model faster source onboarding through source-to-concept mapping reducing dependence on scarce specialist knowledge, enabling broader participation in data-platform work dynamic gold models and on-demand analytical structures living documentation and a knowledge graph that remain useful to people and AI tools The tracks # The series is structured into four tracks:\nThe first track focuses on why particular architectural decisions were made. Many options are available, and this track explains the thinking behind the choices. The second track focuses on the model and the toolchain. It details how to create a YAML-based data model with conceptual, logical, and physical layers, and how that model can generate structures in Microsoft Fabric and dbt. The third track explains how AI is used in this new environment. The fourth track focuses on the dbt and Microsoft Fabric implementation. The articles are written from the perspective of a practitioner who has made the choices, lives with the consequences, and is still working through the details. Nothing here is presented as the only right answer. The goal is to share a coherent, working approach and to give other professionals enough to evaluate, adapt, and build on.\nThe next article in the series explains why the medallion architecture needs a canonical core data model.\n","date":"13 October 2026","externalUrl":null,"permalink":"/articles/the-journey-to-an-ai-ready-data-warehouse/","section":"Articles","summary":"How can an AI-ready data warehouse improve analytics without losing the business meaning behind the data? We’re restructuring a legacy data warehouse around a canonical core data model—and sharing the architectural decisions, lessons, and…","title":"The Journey to an AI-Ready Data Warehouse","type":"articles"},{"content":"","date":"13 October 2026","externalUrl":null,"permalink":"/tags/ai/","section":"Tags","summary":"","title":"AI","type":"tags"},{"content":"Articles on building analytics platforms that are ready for AI. Articles are grouped into series; each series is meant to be read in order.\n","date":"13 October 2026","externalUrl":null,"permalink":"/articles/","section":"Articles","summary":"","title":"Articles","type":"articles"},{"content":"","date":"13 October 2026","externalUrl":null,"permalink":"/tags/dbt/","section":"Tags","summary":"","title":"Dbt","type":"tags"},{"content":"","date":"13 October 2026","externalUrl":null,"permalink":"/tags/fabric/","section":"Tags","summary":"","title":"Fabric","type":"tags"},{"content":"","date":"13 October 2026","externalUrl":null,"permalink":"/tags/medallion-architecture/","section":"Tags","summary":"","title":"Medallion Architecture","type":"tags"},{"content":"","date":"13 October 2026","externalUrl":null,"permalink":"/tags/ontology/","section":"Tags","summary":"","title":"Ontology","type":"tags"},{"content":"","date":"13 October 2026","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":"","date":"13 October 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":" A series on restructuring a legacy data warehouse for the AI era: why it is worth the effort, and, most importantly, how to do it. # ","date":"13 October 2026","externalUrl":null,"permalink":"/series/the-journey-to-a-data-warehouse-in-the-ai-era/","section":"Series","summary":"","title":"The Journey to a Data Warehouse in the AI Era","type":"series"},{"content":" Consilium Consultants helps organisations to transform their legacy data warehouse for use with AI. A platform AI can rely on\nMany organisations have a data warehouse that delivers reports, but the business meaning behind the data is buried in code and in people\u0026rsquo;s heads. AI cannot use meaning it cannot see. I help organisations restructure their analytics platform so that meaning is explicit, and the platform becomes more reliable today and ready for AI tomorrow.\nMake meaning explicit A canonical core data model that defines what the business data means. Better quality, faster delivery Clear layer responsibilities and generated structures. Ready for AI A foundation AI can use with the context it needs. The article series\nRestructuring a legacy warehouse is a major undertaking, so I document the whole journey, including the architecture decisions, what worked, and what didn\u0026rsquo;t, and apply the same approach for clients. Start with the series The Journey to a Data Warehouse in the AI Era.\nRead the articles Get in touch ","date":"13 October 2026","externalUrl":null,"permalink":"/","section":"Your data platform works. Is it ready for AI?","summary":"","title":"Your data platform works. Is it ready for AI?","type":"page"},{"content":"","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":" George Schoofs is an experienced BI consultant, data engineer and data architect. He works with Microsoft Fabric, Power BI and dbt to help organisations make their data warehouse ready for AI\nQuestions about the articles, or interested in working together?\nLets Get in touch.\nTelephone: +31 6 41162117 Email: george.schoofs@consilium.nl LinkedIn: George Schoofs Medium: @george.schoofs Consilium Consultants BV # Gedempte Zalmhaven 817\n3011 BT Rotterdam\nNetherlands\nKVK: 24183428\n","externalUrl":null,"permalink":"/contact/","section":"Your data platform works. Is it ready for AI?","summary":"","title":"Contact","type":"page"}]