This David Papkin page contains links on Microsoft Azure DP-200 course.<\/p>\n
Azure Batch<\/strong><\/p>\n Use Azure Batch to run large-scale parallel and high-performance computing (HPC) batch jobs efficiently in Azure. Azure Batch creates and manages a pool of compute nodes (virtual machines), installs the applications you want to run, and schedules jobs to run on the nodes.<\/p>\n Azure Batch Overview<\/a><\/p>\n Azure Bot<\/strong><\/p>\n Bots provide an experience that feels less like using a computer and more like dealing with a person – or at least an intelligent robot.<\/p>\n About Azure Bot Service<\/a><\/p>\n Quickstart – Create a bot with Azure Bot Service<\/a><\/p>\n Azure Cosmos DB<\/strong><\/p>\n Azure Cosmos DB documentation<\/a><\/p>\n Azure Cosmos DB introduction<\/a><\/p>\n Azure Cosmos DB Quickstart<\/a><\/p>\n Azure Cosmos DB vs Azure SQL<\/a><\/p>\n Calculating Cosmos DB Request Units (RU) for CRUD and Queries<\/a><\/p>\n CosmosDB Capacity Calculator<\/a><\/p>\n Choose the appropriate API for Azure Cosmos DB<\/a><\/p>\n Databases<\/strong><\/p>\n Different types of Databases<\/span><\/p>\n Document store<\/a> – <\/u>document-oriented database systems, are characterized by their schema-free organization of data.<\/p>\n <\/p>\n MongoDB<\/a> is a document database, which means it stores data in JSON-like documents. We believe this is the most natural way to think about data, and is much more expressive and powerful than the traditional row\/column model. The Benefits of Graph Computing (Apache Tinkerpop)<\/a><\/p>\n Example : Gremlin<\/p>\n Gremlin<\/b><\/a>\u00a0is the graph traversal language of Apache TinkerPop.\u00a0Gremlin<\/b> is a functional, data-flow language that enables users to succinctly express complex traversals on (or queries of) their application’s property,developed by Apache TinkerPop of the\u00a0Apache Software Foundation<\/a><\/p>\n Key-value store<\/a> – <\/u>simplest form of database management systems<\/a>. They can only store pairs of keys and values, as well as retrieve values when a key is known.<\/p>\n These simple systems are normally not adequate for complex applications. On the other hand, it is exactly this simplicity, that makes such systems attractive in certain circumstances. For example resource-efficient key-value stores are often applied in embedded systems or as high performance in-process databases.<\/p>\n Wide column store<\/a> (column base) – <\/u>store data in records with an ability to hold very large numbers of dynamic columns. Since the column names as well as the record keys are not fixed, and since a record can have billions of columns, wide column stores can be seen as two-dimensional key-value stores<\/a>.<\/p>\n Cassandra<\/a> – open-source<\/a>,\u00a0distributed<\/a>,\u00a0wide column store<\/a>,\u00a0NoSQL<\/a>\u00a0database<\/a>\u00a0management system designed to handle large amounts of data across many\u00a0commodity servers<\/a>, providing high availability with no\u00a0single point of failure<\/a>.<\/p>\n Azure CosmosDB<\/strong> – Azure Cosmos DB is a fully-managed database service with turnkey global distribution and transparent multi-master replication.<\/p>\n Azure SQL<\/strong> \u2013 relational DB<\/a><\/p>\n Azure Data Bricks<\/strong><\/p>\n What is Azure Data Bricks?<\/a><\/p>\n Azure Databricks<\/a> is the latest Azure offering for data engineering and data science. Databricks\u2019 greatest strengths are its zero-management cloud solution and the collaborative, interactive environment it provides in the form of notebooks.<\/p>\n Databricks is powered by Apache Spark and offers an API layer where a wide span of analytic-based languages can be used to work as comfortably as possible with your data: R, SQL, Python, Scala and Java. The Spark ecosystem also offers a variety of perks such as Streaming, MLib, and GraphX.<\/p>\n Data can be gathered from a variety of sources, such as Blob Storage, ADLS, and from ODBC databases using Sqoop.<\/p>\n Tutorial: Extract, transform, and load data by using Azure Databricks<\/a><\/p>\n These notebooks<\/a> show how to convert JSON data to Delta Lake format, create a Delta table, append to the table, optimize the resulting table, and finally use Delta Lake metadata commands to show the table history, format, and details.<\/p>\n Manage Notebooks<\/a><\/p>\n You can manage notebooks using the UI, the CLI, and by invoking the Workspace API. This topic focuses on performing notebook tasks using the UI. For the other methods, see\u00a0Databricks CLI<\/span><\/a>\u00a0and\u00a0Workspace API<\/span><\/a>.<\/p>\n Introductory Notebooks<\/a><\/p>\n Connecting to SQL Databases using JDBC<\/a><\/p>\n JDBC – Introduction<\/a><\/p>\n JDBC To Other Databases<\/a><\/p>\n Read and Write Apache Parquet file in Spark<\/a><\/p>\n How to view Apache Parquet file in Windows?<\/a><\/p>\n Azure Data Catalog<\/strong><\/p>\n What is Azure Data Catalog?<\/a><\/p>\n Azure Data Catalog documentation<\/a><\/p>\n Azure Data Factory<\/strong><\/p>\n Introduction to Azure Data Factory<\/a><\/p>\n Azure Data Factory<\/a><\/p>\n Create Azure Data Factory from Cloudshell<\/a><\/p>\n Create and configure a self-hosted integration runtime<\/a><\/p>\n Azure Data Lake<\/strong><\/p>\n Azure Data Lake<\/a> is an on-demand scalable cloud-based storage and analytics service. It can be divided in two connected services, Azure Data Lake Store (ADLS) and Azure Data Lake Analytics (ADLA). ADLS is a cloud-based file system that allows the storage of any type of data with any structure, making it ideal for the analysis and processing of unstructured data.<\/p>\n Azure Data Lake Analytics<\/strong><\/p>\n Azure Data Lake Analytics<\/a> is a parallelly-distributed job platform that allows the execution of U-SQL scripts on the Cloud. The syntax is based on SQL with a twist of C#, a general-purpose programming language first released by Microsoft in 2001.<\/p>\n Azure Data Lake Storage<\/strong><\/p>\n Azure Data Lake Storage Gen1<\/a> (previously known as Azure Data Lake Store) is an enterprise-wide hyper-scale repository for big data analytics workloads. Data Lake Storage Gen1 lets you capture data of any size, type, and ingestion speed. The data is captured in a single place for operational and exploratory analytics.<\/p>\n Data Lake Storage Gen2<\/a> is the result of converging the capabilities of Microsoft two existing storage services, Azure Blob storage and Azure Data Lake Storage Gen1.<\/p>\n Azure Data Lake Storage Docs<\/a><\/p>\n Quickstart: Analyze data in Azure Data Lake Storage Gen2 by using Azure Databricks<\/a><\/p>\n What is Apache Hadoop?<\/a><\/p>\n Azure HDInsight<\/strong><\/p>\n Azure HDInsight is a cloud service that allows cost-effective data processing using open-source frameworks such as Hadoop, Spark, Hive, Storm, and Kafka, among others.<\/p>\n Using Apache Sqoop, we can import and export data to and from a multitude of sources, but the native file system that HDInsight uses is either Azure Data Lake Store or Azure Blob Storage.<\/p>\n What is Azure HDInsight?<\/a><\/p>\n Azure HDInsight documentation<\/a><\/p>\n Cluster types in HDI<\/p>\n
\nGraph DBMS<\/a> – r<\/u>epresent data in graph structures as nodes and edges, which are relationships between nodes. They allow easy processing of data in that form, and simple calculation of specific properties of the graph, such as the number of steps needed to get from one node to another node.<\/p>\n