Trending Microsoft Certifications in Data, Artificial Intelligence, IoT, and Machine Learning

The cutting edge Data, Artificial Intelligence, IoT, and Machine Learning are here to stay for many decades to come. There are many trending certifications from Microsoft for these technologies. 

Artificial intelligence (AI) is predicted to increase economic growth by an average of 5 percent across many industries by 2050. IoT is changing the way companies all over the world are doing business. Gartner Says 5.8 Billion Enterprise and Automotive IoT Endpoints Will Be in Use in 2020. Gartner forecasts that the enterprise and automotive Internet of Things (IoT) market will grow to 5.8 billion endpoints in 2020, a 21% increase from 2019. By the end of 2019, 4.8 billion endpoints are expected to be in use, up 21.5% from 2018.

A recent global digital report published by 'We Are Social,' and 'Hootsuite' states that the number of people using the Internet to search has hit 4 billion people in 2018. Every second, there are approximately 40,000 searches processed, which equates to 3.5 billion a day, or an incredible 1.2 trillion searches per year. Each year, humanity spends the equivalent of 1 billion years online.

That's a staggering amount of data gathered every day, and it would be impossible to analyze without the help of machine learning.

These 13 Certifications are good to have if you specialize or would like to specialize in Data, Artificial Intelligence, IoT, and Machine Learning.

AI-100: Designing and Implementing an Azure AI Solution

Candidates for this exam analyze the requirements for AI solutions, recommend appropriate tools and technologies, and implements solutions that meet scalability and performance requirements.

Candidates translate the vision from solution architects and work with data scientists, data engineers, IoT specialists, and AI developers to build complete end-to-end solutions. Candidates design and implement AI apps and agents that use Microsoft Azure Cognitive Services and Azure Bot Service. Candidates can recommend solutions that use open source technologies.

Candidates understand the components that make up the Azure AI portfolio and the available data storage options.

Candidates implement AI solutions that use Cognitive Services, Azure bots, Azure Search, and data storage in Azure. Candidates understand when a custom API should be developed to meet specific requirements.

Skills measured:

1. Analyze solution requirements (25-30%)
2. Design AI solutions (40-45%)
3. Implement and monitor AI solutions (25-30%)

DP-100: Designing and Implementing a Data Science Solution on Azure

Candidates for this exam apply scientific rigor and data exploration techniques to gain actionable insights and communicate results to stakeholders. Candidates use machine learning techniques to train, evaluate, and deploy models to build AI solutions that satisfy business objectives. Candidates use applications that involve natural language processing, speech, computer vision, and predictive analytics.

Candidates serve as part of a multi-disciplinary team that incorporates ethical, privacy, and governance considerations into the solution. Candidates typically have background in mathematics, statistics, and computer science.

Skills measured:

1. Define and prepare the development environment (15-20%)
2. Prepare data for modeling (25-30%)
3. Perform feature engineering (15-20%)
4. Develop models (40-45%)

DP-200: Implementing an Azure Data Solution

Candidates for this exam are Microsoft Azure data engineers who collaborate with business stakeholders to identify and meet the data requirements to implement data solutions that use Azure data services.

Azure data engineers are responsible for data-related implementation tasks that include provisioning data storage services, ingesting streaming and batch data, transforming data, implementing security requirements, implementing data retention policies, identifying performance bottlenecks, and accessing external data sources.

Candidates for this exam must be able to implement data solutions that use the following Azure services: Azure Cosmos DB, Azure SQL Database, Azure Synapse Analytics (formerly Azure SQL DW), Azure Data Lake Storage, Azure Data Factory, Azure Stream Analytics, Azure Databricks, and Azure Blob storage.

Skills measured:

1. Implement data storage solutions (40-45%)
2. Manage and develop data processing (25-30%)
3. Monitor and optimize data solutions (30-35%)

DP-201: Designing an Azure Data Solution

Candidates for this exam are Microsoft Azure data engineers who collaborate with business stakeholders to identify and meet the data requirements to design data solutions that use Azure data services.

Azure data engineers are responsible for data-related tasks that include designing Azure data storage solutions that use relational and non-relational data stores, batch and real-time data processing solutions, and data security and compliance solutions.

Candidates for this exam must design data solutions that use the following Azure services: Azure Cosmos DB, Azure SQL Database, Azure SQL Data Warehouse, Azure Data Lake Storage, Azure Data Factory, Azure Stream Analytics, Azure Databricks, and Azure Blob storage.

Skills measured:

1. Design Azure data storage solutions (40-45%)
2. Design data processing solutions (25-30%)
3. Design for data security and compliance (25-30%)

70-761: Querying Data with Transact-SQL

This exam is intended for SQL Server database administrators, system engineers, and developers with two or more years of experience who are seeking to validate their skills and knowledge in writing queries.

Skills measured:

1. Manage data with Transact-SQL (40–45%)
2. Query data with advanced Transact-SQL components (30–35%)
3. Program databases by using Transact-SQL (25–30%)

70-762: Developing SQL Databases

This exam is intended for database professionals who build and implement databases across organizations and who ensure high levels of data availability. Their responsibilities include creating database files, data types, and tables; planning, creating, and optimizing indexes; ensuring data integrity; implementing views, stored procedures, and functions; and managing transactions and locks.

Skills measured:

1. Design and implement database objects (25–30%)
2. Implement programmability objects (20–25%)
3. Manage database concurrency (25–30%)
4. Optimize database objects and SQL infrastructure (20–25%)

70-764: Administering a SQL Database Infrastructure

This exam is intended for database professionals who perform installation, maintenance, and configuration tasks. Other responsibilities include setting up database systems, making sure those systems operate efficiently, and regularly storing, backing up, and securing data from unauthorized access.

Skills measured:

1. Configure data access and auditing (20–25%)
2. Manage backup and restore of databases (20–25%)
3. Manage and monitor SQL Server instances (35–40%)
4. Manage high availability and disaster recovery (20–25%)

70-765: Provisioning SQL Databases

This exam is intended for architects, senior developers, infrastructure specialists, and development leads. Candidates have a working knowledge of the various cloud service models and service model architectures, data storage options, and data synchronization techniques. Candidates also have a working knowledge of deployment models, upgrading and migrating databases, and applications and services, in addition to integrating Azure applications with external resources.

Skills measured:

1. Implement SQL in Azure (40–45%)
2. Manage databases and instances (30-35%)
3. Manage Storage (30–35%)

70-767: Implementing a Data Warehouse using SQL

This exam is intended for extract, transform, and load (ETL) and data warehouse developers who create business intelligence (BI) solutions. Their responsibilities include data cleansing, in addition to ETL and data warehouse implementation.

Skills measured:

1. Design, implement, and maintain a data warehouse (35–40%)
2. Extract, transform, and load data (40–45%)
3. Build data quality solutions (15–20%)

70-768: Developing SQL Data Models

This exam is intended for business intelligence (BI) developers who focus on creating BI solutions that require implementing multidimensional data models, implementing and maintaining OLAP cubes, and implementing tabular data models.

Skills measured:

1. Design a multidimensional business intelligence (BI) semantic model (25–30%)
2. Design a tabular BI semantic model (20–25%)
3. Develop queries using Multidimensional Expressions (MDX) and Data Analysis Expressions (DAX) (15–20%)
4. Configure and maintain SQL Server Analysis Services (SSAS) (30–35%)

70-777: Implementing Microsoft Azure Cosmos DB Solutions

Candidates for this exam are developers and architects who leverage Azure Cosmos DB. Candidates should understand fundamental concepts of partitioning, replication, and resource governance for building and configuring scalable applications that are agnostic of a Cosmos DB API. Candidates should also have basic working knowledge of the Cosmos DB SQL API.

Candidates for this exam design, build, and troubleshoot Cosmos DB solutions that meet business and technical requirements.

Skills measured:

1. Partition and Model Data
2. Replicate Data Across the World
3. Tune and Debug Azure Cosmos DB Solutions
4. Perform Integration and Develop Solutions

70-778: Analyzing and Visualizing Data with Microsoft Power BI

Candidates for this exam should have a good understanding of how to use Power BI to perform data analysis. Candidates should be proficient in connecting to data sources and performing data transformations, modeling and visualizing data by using Microsoft Power BI Desktop, and configuring dashboards by using the Power BI service. Candidates should also be proficient in implementing direct connectivity to Microsoft SQL Azure and SQL Server Analysis Services (SSAS), and implementing data analysis in Microsoft Excel. Candidates may include BI professionals, data analysts, and other roles responsible for creating reports by using Power BI.

Skills measured:

1. Consuming and Transforming Data By Using Power BI Desktop
2. Modeling and Visualizing Data
3. Configure Dashboards, Reports and Apps in the Power BI Service

70-779: Analyzing and Visualizing Data with Microsoft Excel

Candidates for this exam should have a strong understanding of how to use Microsoft Excel to perform data analysis. Candidates should be able to consume, transform, model, and visualize data in Excel. Candidates should also be able to configure and manipulate data in PowerPivot, PivotTables, and PivotCharts. Candidates may include BI professionals, data analysts, and other roles responsible for analyzing data with Excel.

Skills measured:

1. Consume and Transform Data by Using Microsoft Excel (30-35%)
2. Model Data (35-40%)
3. Visualize Data (30-35%)

Introduction to Azure Stream Analytics

Azure stream analytics are truely real-time analytics, from the cloud to the edge. Azure Stream Analytics is designed to analyze and process high volumes of fast streaming data from multiple sources simultaneously.

The simplicity of Azure Stream Analytics is that it uses familiar SQL syntax and is extensible with JavaScript and C# custom code.

Basics of Azure Stream Analytics

Azure stream analytics solutions will always have input, query, and output.

Basics of Azure Stream Analytics

Advantages of Azure Stream Analytics

• Ease of creating analytics pipelines

• Can be used for complex and large workloads

• As a cloud service, Stream Analytics is optimized for cost

• Can be used on the Edge

• Built-in machine learning (ML) models to shorten time to insights

• Azure Stream Analytics has built-in recovery capabilities in case the delivery of an event fails

• Azure Stream Analytics is a fully managed serverless (PaaS) offering on Azure

• Azure Stream Analytics encrypts all incoming and outgoing communications and supports TLS 1.2

• Stream Analytics can process millions of events every second 

• Stream Analytics can deliver results with ultra-low latencies

Adding and Updating Security Roles to a Migrated Business Process Flow

There is still an issue in case you want to "Enable Security Roles" for a migrated Business Process Flow in Dynamics 365 Version 9.

When you do a migration from a lower version of Dynamics CRM to Dynamics 365 and as part of the migration are existing custom business processes. If you open one of these business processes and select "Enable Security Roles", there are no options to either "Enable for everyone" or "Enable only for the selected security roles". 

"Enable Security Roles" Option for Migrated Business Process

In Dynamics 365 for new Business Process Flows and existing system Business Process Flows, there are no issues. You can find options "Enable for everyone" and "Enable only for the selected security roles". 

"Enable Security Roles" Option for Existing Business Process

The solution is to directly open "enable security roles" windows for a migrated business processes using a URL.

The format of the URL for a Business Process in my Dynamics 365 V9.0 environment would be:

https://ashishm1974.crm.dynamics.com/tools/dialogs/RoleAssignment.aspx?dType=1&oid=%7b6E9A821B-3CBC-4F04-9619-F2B723FE4880%7d

This will directly open the window to enable security roles for a business process. Replace the URL and Business Process GUID as in your Dynamics 365 environment.

Who are Data Scientists and What they Do

Now-a-days the buzz word is Data Analytics. Customers are savvy and want to know how we can help analyze their data.

I feel Data Analytics should be part of every medium to large Dynamics 365 project. This is first of many blogs I will start writing for Data Analytics.

Good news is that Microsoft is already leading the way in Data Analytics through extreme investments in Dynamics 365, Windows, Azure, Machine Learning, SQL Server and IoT. 

Data scientists are a special breed who work on data and have the art of making some sense out of it. Having said that it is not as simple as picking customer's SQL Server and start writing TSQL queries. 

Data scientists should have the following

• Constant learning of current state of any project or business

Customer's data is the representation of its business in the form of numbers, text, dates and images.

• Fully understand the expectations and deliverable 

Planning and the execution both depend on the end goal. A data scientist should be clear of what is expected so that the analysis and transformation can be performed accordingly.

• Knowledge of methodologies and tools 

A data scientist should be aware of various methodologies and tools available at his/her disposal. For example, Microsoft has many tools for data analysis and they work with all kinds of data.

• Curiosity, Persistent, Patient and Focused 

A data scientist sometimes need to churn Terabyte or Petabyte or Exabyte of data. Therefore they need to have constant curiosity and be persistent, patient & focused to explore, visualize, slice and dice data.

• Technical Savvy

Data scientists should know how to work with raw data. They should be able to transform it to an easy format & visualization so that correct analysis can be made and timely decisions can be reached. Data scientists should be good in math and statistics. Statistics is a branch of mathematics dealing with the collection, analysis, interpretation, presentation, and organization of data.

What does a Data Scientist Do?

• Data scientists spend most of the time preparing data

Data scientists need to clean, prepare and process raw data so that it can be analyzed. 

• Run software programs against data

Data scientists use various software programs and languages to analyze data. For example SQL query language, R, Python, Hive, Hadoop, Microsoft R Server, Power BI, Excel, etc.

• Prepare Reports and Visualizations

Data scientists produce reports, charts and tables for easy understanding.

Use of Option Set Vs Two Options in a Business Process Flow

Business processes within Dynamics 365 streamlines and creates a visualization of information flow through various stages. Each stage combine "data steps" to reach a business decision to either move forward or backward.

The data steps in a business process stage are fields used to capture information. Option sets and two options can be used as data steps. 

Two option should only be used if a data step is not required in a business process stage. The problem with two options is that they cannot have "unassigned value" as a default. Therefore even if a two option is marked as required on a business process stage, one can still move to next stage based on the default selected.

1) I have created 2 fields on Opportunity. These will be used as data steps to capture whether due diligence is done. 

One field is an option set with two values "Yes" and "No". The default value is unassigned.

Option Set Field

Second field is a two options with values "Yes" and "No". Default value is "No". The issue here is that the default value can only be selected as "Yes" or "No".

Two Options Field

2) Both these fields are displayed as data steps on the Opportunity Business Process "Qualify" stage.

Business Process on Opportunity

3) Open a new Opportunity, enter a "Topic" and "Save". This will create a new opportunity in "Qualify" stage. As seen the two fields are displayed as data steps in "Qualify". The top field is an option set and bottom field is a two option.

Testing these Fields

4) The both are required but only option set field will force a user to select a value (if nothing is selected) when moving to next stage. 

Option set can have an unassigned value as a default. Therefore always use option set over two options whenever it is required to capture data on a business process stage