Development of a back- end tool for loading data into data warehouse using open-source technology

Abstract

Almost every enterprise uses operational Database Management System (DBMS) for its infonnation management. Many companies have multiple branches in many places, each of which generates significant amount of data. These data may be a potential resource for the knowledge worker (executive, manager and analyst) to make better and faster complex business decision for the organization. For this purpose, historical and subject-oriented selective data are collected from different data sources and these data are stored under a unified schema in a data warehouse. It enables the higher executive to view summarized report over historical data of an organization. Vendor provided Structured Query Language (SQL) engines like Oracle, MSSQL Server, DB 18M2 provide integrated data warehousing tools. But small and medium enterprises do not take the advantages of the vendor provided DBMS due to its high Total Cost of Ownership (TCO). Although open-source SQL engine is available for free of cost, it does not provide data warehousing tools. As a remed:r of this problem, we have developed a back-end tool which extracts data from many sources and load into data warehouse using open-source DBMS MySQL. This tool also periodically refreshes data warehouse on updated data of operational DBMS connected through computer network. It contains an integrated PSTN network connectivity component that establishes dial up connection easily.