CHAPTER 3 Descriptive analytics Ii Business Intelligence and Data Warehousing Learning Objectives for Chapter 3 Understand the basic definitions and concepts of data warehousing Understand data warehousing architectures Describe the processes used in developing and managing data warehouses Explain data warehousing operations Explain the role of data warehouses in decision support Explain data integration and the extraction, transformation, and load (etL) processes Understand the essence of business performance management (BPm) Learn balanced scorecard and Six Sigma as performance Copyright C2018 Pearson Education, Inc
1 Copyright © 2018Pearson Education, Inc. Descriptive Analytics II: Business Intelligence and Data Warehousing Learning Objectives for Chapter 3 ▪ Understand the basic definitions and concepts of data warehousing ▪ Understand data warehousing architectures ▪ Describe the processes used in developing and managing data warehouses ▪ Explain data warehousing operations ▪ Explain the role of data warehouses in decision support ▪ Explain data integration and the extraction, transformation, and load (ETL) processes ▪ Understand the essence of business performance management (BPM) ▪ Learn balanced scorecard and Six Sigma as performance measurement systems CHAPTER 3
CHAPTER OVERVIEW The concept of data warehousing has been around since the late 1980s. This chapter provides the foundation for an important type of database, called a data warehouse, which is primarily used for decision support and provides improved analytical capabilities. We discuss data warehousing in the following sections CHAPTER OUTLINE 3. 1 Opening Vignette: Targeting Tax Fraud with Business Intelligence and Data Warehousing 3.2 Business Intelligence and Data Warehousing 3.3 Data Warehousing Process 3.4 Data Warehousing Architectures 3.5 Data Integration and the Extraction, Transformation, and Load (etl) Processes 3.6 Data Warehouse Development 3.7 Data Warehousin ng Implementation Issues 3.8 Data Warehouse Administration, Security Issues, and Future Trends 3.9 Business Performance Management 3.10 Performance Measurement 3.11 Balanced Scorecards 3.12 Six Sigma as a Performance Measurement System ANSWERS TO END OF SECTION REVIEW QUEST|ONs···· Section 3. 1 Review Questions 1. Why is it important for IRS and for U. S state governments to use data warehousing and business intelligence(Bi) tools in managing state revenues? Copyright C2018 Pearson Education, Inc
2 Copyright © 2018Pearson Education, Inc. CHAPTER OVERVIEW The concept of data warehousing has been around since the late 1980s. This chapter provides the foundation for an important type of database, called a data warehouse, which is primarily used for decision support and provides improved analytical capabilities. We discuss data warehousing in the following sections: CHAPTER OUTLINE 3.1 Opening Vignette: Targeting Tax Fraud with Business Intelligence and Data Warehousing 3.2 Business Intelligence and Data Warehousing 3.3 Data Warehousing Process 3.4 Data Warehousing Architectures 3.5 Data Integration and the Extraction, Transformation, and Load (ETL) Processes 3.6 Data Warehouse Development 3.7 Data Warehousing Implementation Issues 3.8 Data Warehouse Administration, Security Issues, and Future Trends 3.9 Business Performance Management 3.10 Performance Measurement 3.11 Balanced Scorecards 3.12 Six Sigma as a Performance Measurement System ANSWERS TO END OF SECTION REVIEW QUESTIONS Section 3.1 Review Questions 1. Why is it important for IRS and for U.S. state governments to use data warehousing and business intelligence (BI) tools in managing state revenues?
Revenues are complex and have many sources. This variety and detail make understand ing the data difficult, hampering efficiency. The use of bi tools allows for better analysis, understand ing, and governance What were the challenges the state of Maryland was facing with regard to tax fraud? The state was facing tax fraud from fraudulent returns as other states were. and the process of detecting and investigating potential fraud was time consuming 3. What was the solution they adopted? Do you agree with their approach? Why? The state implemented a data warehouse from Teradata that allowed them to examine data and identify/flag traits that were consistent with fraudulent return 4. What were the results that they obtained? Did the investment in BI and data warehousing pay off? The team was able to flag a smaller number of potentially fraudulent returns, but those that they did identify were significantly more likely to be false. this allowed the state to recover $7 million more, making the investment pay off. 5. What other problems and challenges do you think federal and state governments are having that can benefit from BI and data warehousing? Student responses will vary but could include ideas relating to voter fraud med ical use. and other tax issues Section 3. 2 Review Questions What is a data warehouse? a data warehouse is defined in this section as"a pool of data produced to support decision making " This focuses on the essentials. leaving out characteristics that may vary from one dw to another but are not essential to the basic concept The same paragraph gives another definition: a subject-oriented, integrated time-variant, nonvolatile collection of data in support of managements decision making process. This definition adds more specifics, but in every case appropriately: it is hard, if not impossible, to conceive of a data warehouse that would not be subject-oriented, integrated, etc How does a data warehouse differ from a database? echnically a data warehouse is a database albeit with certain characteristics to facilitate its role in decision support. Specifically, however, it is(see previous Copyright C2018 Pearson Education, Inc
3 Copyright © 2018Pearson Education, Inc. Revenues are complex and have many sources. This variety and detail make understanding the data difficult, hampering efficiency. The use of BI tools allows for better analysis, understanding, and governance. 2. What were the challenges the state of Maryland was facing with regard to tax fraud? The state was facing tax fraud from fraudulent returns as other states were, and the process of detecting and investigating potential fraud was time consuming. 3. What was the solution they adopted? Do you agree with their approach? Why? The state implemented a data warehouse from Teradata that allowed them to examine data and identify/flag traits that were consistent with fraudulent return. 4. What were the results that they obtained? Did the investment in BI and data warehousing pay off? The team was able to flag a smaller number of potentially fraudulent returns, but those that they did identify were significantly more likely to be false. This allowed the state to recover $7 million more, making the investment pay off. 5. What other problems and challenges do you think federal and state governments are having that can benefit from BI and data warehousing? Student responses will vary but could include ideas relating to voter fraud, medical use, and other tax issues. Section 3.2 Review Questions 1. What is a data warehouse? A data warehouse is defined in this section as “a pool of data produced to support decision making.” This focuses on the essentials, leaving out characteristics that may vary from one DW to another but are not essential to the basic concept. The same paragraph gives another definition: “a subject-oriented, integrated, time-variant, nonvolatile collection of data in support of management’s decisionmaking process.” This definition adds more specifics, but in every case appropriately: it is hard, if not impossible, to conceive of a data warehouse that would not be subject-oriented, integrated, etc. 2. How does a data warehouse differ from a database? Technically a data warehouse is a database, albeit with certain characteristics to facilitate its role in decision support. Specifically, however, it is (see previous
question) an"integrated, time-variant, nonvolatile, subject-oriented repository of detail and summary data used for decision support and business analytics within an organization. These characteristics, which are discussed further in the section just after the definition, are not necessarily true of databases in general--though each could apply individually to a given one As a practical matter most databases are highly normalized, in part to avoid update anomalies. Data warehouses are highly denormalized for performance reasons. This is acceptable because their content is never updated, just added to Historical data are static What is an ODS? Operational Data Store is the database from which a business operates on an on going basis Differentiate among a dM. an ODS and an EDw An ODs(Operational Data Store)is the database from which a business operates on an ongoing basis Both an EDW and a data mart(dm)are data warehouses. An EDW(Enterprise Data Warehouse) is an all-encompassing dw that covers all subject areas of interest to the entire organization. a data mart is a smaller dw designed around one problem, organizational function, topic, or other suitable focus area 5. Explain the importance of metadata Metadata, " data about data, are the means through which applications and user access the content of a data warehouse, through which its security is managed and through which organizational management manages, in the true sense of the word,its information assets. Most database management systems would be unable to function without at least some metadata. Indeed the use of metadata, which enable data access through names and logical relationships rather than physical locations, is fundamental to the very concept of a DBms Metadata are essential to any database, not just a data warehouse. ( See answer to Review Question 2 of this section above Section 3.3 Review Questions Describe the data warehousing process The data warehousing process consists of the following steps 1. Data are imported from various internal and external sources Data are cleansed and organized consistently with the organizations needs Copyright C2018 Pearson Education, Inc
4 Copyright © 2018Pearson Education, Inc. question) an “integrated, time-variant, nonvolatile, subject-oriented repository of detail and summary data used for decision support and business analytics within an organization.” These characteristics, which are discussed further in the section just after the definition, are not necessarily true of databases in general—though each could apply individually to a given one. As a practical matter most databases are highly normalized, in part to avoid update anomalies. Data warehouses are highly denormalized for performance reasons. This is acceptable because their content is never updated, just added to. Historical data are static. 3. What is an ODS? Operational Data Store is the database from which a business operates on an ongoing basis. 4. Differentiate among a DM, an ODS, and an EDW. An ODS (Operational Data Store) is the database from which a business operates on an ongoing basis. Both an EDW and a data mart (DM) are data warehouses. An EDW (Enterprise Data Warehouse) is an all-encompassing DW that covers all subject areas of interest to the entire organization. A data mart is a smaller DW designed around one problem, organizational function, topic, or other suitable focus area. 5. Explain the importance of metadata. Metadata, “data about data,” are the means through which applications and users access the content of a data warehouse, through which its security is managed, and through which organizational management manages, in the true sense of the word, its information assets. Most database management systems would be unable to function without at least some metadata. Indeed, the use of metadata, which enable data access through names and logical relationships rather than physical locations, is fundamental to the very concept of a DBMS. Metadata are essential to any database, not just a data warehouse. (See answer to Review Question 2 of this section above.) Section 3.3 Review Questions 1. Describe the data warehousing process. The data warehousing process consists of the following steps: 1. Data are imported from various internal and external sources 2. Data are cleansed and organized consistently with the organization’s needs
Data are loaded into the enterprise data warehouse, or b. Data are loaded into data marts If desired. data marts are created as subsets of the edw. or b The data marts are consolidated into the edw Analyses are performed as needed 2. Describe the major components of a data warehouse Data sources. Data are sourced from operational systems and possibly from external data sources Data extraction and transformation. Data are extracted and properl transformed using custom-written or commercial software called ETL Data loading. Data are loaded into a staging area, where they are transformed and cleansed. The data are then ready to load into the data warehouse Comprehensive database. This is the edw that supports decision analysis by provid ing relevant summarized and detailed information Metadata. Metad ata are maintained for access by It personnel and users Metadata include rules for organizing data summaries that are easy to index and search Middleware tools. Midd leware tools enable access to the data warehouse from a variety of front-end applications 3. Identify and discuss the role of middleware tools Mid d leware tools enable access to the data warehouse. power users such analysts may write their own SQL queries. Others may access data through managed query environment. There are many front-end applications that business users can use to interact with data stored in the data repositories, including data mining, OLAP, reporting tools, and data visualization tools. all these have their own data access requirements. Those may not match with how a given data warehouse must be accessed. Midd leware translates between the two Section 3. 4 Review Questions What are the key similarities and differences between a two-tiered architecture and a three-tiered architecture? Both provide the same user visibil ity through a client system that accesses a DSS/BI application remotely. The difference is behind the scenes and is invisible to the user: in a two-tiered architecture, the application and data warehouse reside on the same machine; in a three-tiered architecture, they are on separate machines How has the Web influenced data warehouse design? Copyright C2018 Pearson Education, Inc
5 Copyright © 2018Pearson Education, Inc. 3. a. Data are loaded into the enterprise data warehouse, or b. Data are loaded into data marts. 4. a. If desired, data marts are created as subsets of the EDW, or b. The data marts are consolidated into the EDW 5. Analyses are performed as needed 2. Describe the major components of a data warehouse. • Data sources. Data are sourced from operational systems and possibly from external data sources. • Data extraction and transformation. Data are extracted and properly transformed using custom-written or commercial software called ETL. • Data loading. Data are loaded into a staging area, where they are transformed and cleansed. The data are then ready to load into the data warehouse. • Comprehensive database. This is the EDW that supports decision analysis by providing relevant summarized and detailed information. • Metadata. Metadata are maintained for access by IT personnel and users. Metadata include rules for organizing data summaries that are easy to index and search. • Middleware tools. Middleware tools enable access to the data warehouse from a variety of front-end applications. 3. Identify and discuss the role of middleware tools. Middleware tools enable access to the data warehouse. Power users such as analysts may write their own SQL queries. Others may access data through a managed query environment. There are many front-end applications that business users can use to interact with data stored in the data repositories, including data mining, OLAP, reporting tools, and data visualization tools. All these have their own data access requirements. Those may not match with how a given data warehouse must be accessed. Middleware translates between the two. Section 3.4 Review Questions 1. What are the key similarities and differences between a two-tiered architecture and a three-tiered architecture? Both provide the same user visibility through a client system that accesses a DSS/BI application remotely. The difference is behind the scenes and is invisible to the user: in a two-tiered architecture, the application and data warehouse reside on the same machine; in a three-tiered architecture, they are on separate machines. 2. How has the Web influenced data warehouse design?