Data Lake Development with Big Data

English | 2015 | ISBN: 978-1785888083 | 164 Pages | EPUB, AZW3, PDF (conv) | 10 MB

Explore architectural approaches to building Data Lakes that ingest, index, manage, and analyze massive amounts of data using Big Data technologies
A Data Lake is a highly scalable platform for storing huge volumes of multistructured data from disparate sources with centralized data management services. This book explores the potential of Data Lakes and explores architectural approaches to building data lakes that ingest, index, manage, and analyze massive amounts of data using batch and real-time processing frameworks. It guides you on how to go about building a Data Lake that is managed by Hadoop and accessed as required by other Big Data applications.
This book will guide readers (using best practices) in developing Data Lake’s capabilities. It will focus on architect data governance, security, data quality, data lineage tracking, metadata management, and semantic data tagging. By the end of this book, you will have a good understanding of building a Data Lake for Big Data.
What You Will Learn

• Identify the need for a Data Lake in your enterprise context and learn to architect a Data Lake
• Learn to build various tiers of a Data Lake, such as data intake, management, consumption, and governance, with a focus on practical implementation scenarios
• Find out the key considerations to be taken into account while building each tier of the Data Lake
• Understand Hadoop-oriented data transfer mechanism to ingest data in batch, micro-batch, and real-time modes
• Explore various data integration needs and learn how to perform data enrichment and data transformations using Big Data technologies
• Enable data discovery on the Data Lake to allow users to discover the data
• Discover how data is packaged and provisioned for consumption
• Comprehend the importance of including data governance disciplines while building a Data Lake