Three Reasons to Choose Data Warehouse

Automation Over ETL Tools.

by Ian Nicholson, November 2016. If you wanted to build a double-entry book-keeping system, could you build it in Excel? It’s possible. Excel is versatile enough to be able to build almost any kind of financial solution. A lot of people know Excel very well, so it wouldn’t be hard to find someone to build it for you. But WOULD you? Of course not! There are plenty of purpose-built applications for double-entry book-keeping: Xero, MYOB, Sage, QuickBooks. They are ready- made, and do the job of managing your accounts really well. Why on earth would you even THINK about building a solution in Excel? And yet, this is what happens every day in the world of Business Intelligence! ETL tools, like Excel, are Swiss Army knives for managing data. Essentially, they are programming languages that allow you to connect to a data source, extract that data (E), transform the data (T) and load it somewhere else (L). In the early days, ETL tools were the only means to build data warehouses for Business Intelligence solutions, and they really haven’t evolved much over the last few decades. But here’s the thing. A data warehouse is, by definition, an architecture – a way of structuring the data to make it accessible to Business Intelligence tools like Tableau, QlikView, PowerBI, Yellowfin, Cognos etc.   ETL tools themselves don’t build data warehouses.  Programmers using them do. And those programmers must first understand the architecture of a data warehouse in order to do a proper job. Unfortunately, it is a complicated job and even the most experienced ETL developer can spend months building a modest project. They simply do not deliver at any kind of speed desired by the business. Just like you wouldn’t use Excel to manage your accounts, you shouldn’t use ETL tools to build data warehouses. Introducing Data Warehouse Automation. Data Warehouse Automation tools are specialised applications for building data warehouses. Just like your accounting software knows how to post a receipt into the right account (following predefined rules) so a Data Warehouse Automation tool can structure your data and get it ready for BI. So here are your three reasons to choose Data Warehouse Automation over ETL. 1. Data Warehouse Automation is Faster. Building a data warehouse with a Data Warehouse Automation tool is fast. Very fast. A Data Warehouse Automation tool can deliver a data warehouse in hours or days – instead of weeks or months of ETL development. More importantly – a data warehouse project is a journey, not a destination! Change is a constant, and an ETL- programmed solution is brittle and hard to change. With a Data Warehouse Automation tool, changes to your existing data warehouse are a breeze, and can be delivered in minutes, not days. 2. Data Warehouse Automation is Cheaper. If you hire a data warehouse consultant for three months to build your data warehouse, that’s around 60 working days. Let’s say they charged a VERY conservative rate of $1000/day. That’s $60,000 – for ONE consultant – and your project may require a small team. A modern Data Warehouse Automation tool can be had for under $20,000, and one consultant could deliver in about 1/20th of the time of traditional ETL development. 3. Data Warehouse Automation is Better. Essentially, with a Data Warehouse Automation tool, the code is being generated instead of hand-written by an ETL developer. Generated code has no errors, so even a developer with a poorer understanding of data warehousing can still deliver a higher quality, industry-standard data warehouse built to best practices. Many Data Warehouse Automation tools also produce their own documentation, with data lineage, impact analysis and more. In Summary If you are still using ETL tools to deliver Business Intelligence, you are doing (or paying for) a lot of unnecessary effort. You are still waiting months for even the simplest solutions, and will probably still need more changes once you receive it. Data Warehouse Automation is here, and now. It delivers solutions at a speed that is acceptable to the business, and at a fraction of the cost.