Introduction to Azure Stream Analytics

Azure stream analytics are truely real-time analytics, from the cloud to the edge. Azure Stream Analytics is designed to analyze and process high volumes of fast streaming data from multiple sources simultaneously.

The simplicity of Azure Stream Analytics is that it uses familiar SQL syntax and is extensible with JavaScript and C# custom code.

Basics of Azure Stream Analytics

Azure stream analytics solutions will always have input, query, and output.

Basics of Azure Stream Analytics

Advantages of Azure Stream Analytics

• Ease of creating analytics pipelines

• Can be used for complex and large workloads

• As a cloud service, Stream Analytics is optimized for cost

• Can be used on the Edge

• Built-in machine learning (ML) models to shorten time to insights

• Azure Stream Analytics has built-in recovery capabilities in case the delivery of an event fails

• Azure Stream Analytics is a fully managed serverless (PaaS) offering on Azure

• Azure Stream Analytics encrypts all incoming and outgoing communications and supports TLS 1.2

• Stream Analytics can process millions of events every second 

• Stream Analytics can deliver results with ultra-low latencies

IntelliSense in SQL Server 2008 R2 Management Studio stops working (most of the time)

After installing Visual Studio 2010 Service Pack 1, the IntelliSense in SQL Server 2008 R2 Management Studio stops working (most of the time).

The fix is to install SQL Server 2008 R2 Service Pack 1.

The fix for this issue was first released in Cumulative Update 7. For more information on Cumulative Update 7  click here.

Because the builds are cumulative, each new fix release contains all the hotfixes and all the security fixes that were included with the previous SQL Server 2008 R2 fix release. I recommend installing SQL Server 2008 R2 Service Pack 1 or Service Pack 3.

I hope this was informative. Please feel free to leave comments, if any.

Microsoft Dynamics CRM - Global Option Sets (Picklists)

CRM 2011 was released with many new features. New features were added in the core application, configuration and extending CRM. One of the most appreciated new feature is Global option sets. The other name given to it is Global picklists.

In Dynamics CRM 4.0 there were local option sets. Let's say the requirement is to have a country option set in 10 different entity forms. In order to have the country list, we need the country option set in all the entities and so we need 10 country option sets. All these 10 country option sets had same items in it. If we change the name of a country or add a new country, then we need to make a change at all 10 places. Dynamics CRM 2011 has improved upon this limitation. We can have one global option set for a country. Then each country list in every entity can derive from this "country Global option set". In case any changes are there, it will be done at one place and all derived lists are updated.

I will show how to create and use a global option set. I will create a global option set for the country. I will add few countries and then create a local option set in the entity "Movie". The local option set on the "Movie" form will be derived from this global option set. For the clarity of this demo, I have marked important sections on the screen shots by a green rectangle.

1) I will go to my unmanaged solution "Movies". I am soon planning to write a blog on Unmanaged Solutions.

Click on the component "Option Sets" from the left navigation. This will open the list of all my option sets in this solution. All the option sets added here will be Global option sets. These global option sets are part of the solution "Movies" which later can be deployed from one environment to another as part of a managed solution. Click on New button.

Click New button to Add a new Global Option Set

2) The new Global option set window opens. "Display name" is the label of this Global option set. "Name" is the unique Schema name and Logical name for this Global option set. Add the countries as options (also called as items in CRM lingo). Each country I add will have a Label and a Value. I suggest to accept the default "Value" assigned by CRM.

New Global Option Set Window

3) I entered few countries to this Global option set. The Global option set window will look like below. Save and close.

Create a new Global Option Set for Countries

4) I will create a local option set for the entity "Movie". Click on New to create a new field for entity "Movie".

This new option set field for Movie is derived from the Country Global option set. This Movie option set will display all the countries defined in the step above.

New Country field for "Movie" Entity

5) Add the required fields. Pay special attention to the "Type". The Type of this field is "Option Set".
I will select "Yes" to the option of use existing option set.

Select the Country value against the option set. This is where our local field will reference to any Global option set we select.

Leave default value as unassigned. Save and close.

Add this new field on the entity form. I then click on "Publish all Customizations" for my solution. This will publish all the above changes and make the new field available on the form.

Configure the new field as Country Global Option Set

6) Go to any existing record for the entity. I have opened the Movie record "GoldenEye" and find the Country option list. I can select any country from the list. Any changes to the Global option set will be automatically reflected here and at all the places which refers to it.

Use the new field on the "Movie" Entity Form

I hope this blog about CRM 2011 feature of Global Option Sets (Picklists) was informative. Please feel free to leave comments, if any.

Unit Testing for CRM 2011; Plugins, Custom Workflow Activities and Custom .Net Code

Every time we write a plugin and a custom workflow activity, we should make sure it works as desired. The business logic we write within these plugins and custom workflow activities, should accept the business confirmed parameters and should perform the agreed operations.

The above holds true each time we change these plugins and custom workflow activities. Every time we make a change, the plugins and custom workflow activities again need to be tested and verified. Unit testing in Visual Studio 2010 (out of the box) helps us achieve this process. In unit testing we will always need to Mock the CRM contexts and the data.

“Moq” mocking library can help in writing unit tests for CRM 2011 plugins and custom workflow activities, by mocking the CRM context and makes it easier to pass parameters and define the results.

Moq (pronounced “Mock-you” or just “Mock”) is the only mocking library for .NET developed from scratch to take full advantage of .NET 3.5 (i.e. Linq expression trees) and C# 3.0 features (i.e. lambda expressions) that make it the most productive, type-safe and refactoring-friendly mocking library available. And it supports mocking interfaces as well as classes. Its API is extremely simple and straightforward, and doesn’t require any prior knowledge or experience with mocking concepts.

The below screen shot shows the skeleton of how the MOQ can help us in Mocking (the CRM 2011 context). After Mocking, we can unit test by changing input parameters as we do with any other .Net unit testing.

MOQ: Mocking the CRM 2011 Context for Unit Testing in Visual Studio 2010

I hope this was informative. Please feel free to leave comments, if any.

Who are Data Scientists and What they Do

Now-a-days the buzz word is Data Analytics. Customers are savvy and want to know how we can help analyze their data.

I feel Data Analytics should be part of every medium to large Dynamics 365 project. This is first of many blogs I will start writing for Data Analytics.

Good news is that Microsoft is already leading the way in Data Analytics through extreme investments in Dynamics 365, Windows, Azure, Machine Learning, SQL Server and IoT. 

Data scientists are a special breed who work on data and have the art of making some sense out of it. Having said that it is not as simple as picking customer's SQL Server and start writing TSQL queries. 

Data scientists should have the following

• Constant learning of current state of any project or business

Customer's data is the representation of its business in the form of numbers, text, dates and images.

• Fully understand the expectations and deliverable 

Planning and the execution both depend on the end goal. A data scientist should be clear of what is expected so that the analysis and transformation can be performed accordingly.

• Knowledge of methodologies and tools 

A data scientist should be aware of various methodologies and tools available at his/her disposal. For example, Microsoft has many tools for data analysis and they work with all kinds of data.

• Curiosity, Persistent, Patient and Focused 

A data scientist sometimes need to churn Terabyte or Petabyte or Exabyte of data. Therefore they need to have constant curiosity and be persistent, patient & focused to explore, visualize, slice and dice data.

• Technical Savvy

Data scientists should know how to work with raw data. They should be able to transform it to an easy format & visualization so that correct analysis can be made and timely decisions can be reached. Data scientists should be good in math and statistics. Statistics is a branch of mathematics dealing with the collection, analysis, interpretation, presentation, and organization of data.

What does a Data Scientist Do?

• Data scientists spend most of the time preparing data

Data scientists need to clean, prepare and process raw data so that it can be analyzed. 

• Run software programs against data

Data scientists use various software programs and languages to analyze data. For example SQL query language, R, Python, Hive, Hadoop, Microsoft R Server, Power BI, Excel, etc.

• Prepare Reports and Visualizations

Data scientists produce reports, charts and tables for easy understanding.

Microsoft Dynamics CRM 2011 - SSRS Multi-Series Chart and Display in Dashboard

One of the helpful additions to new CRM 2011 features is Dashboards. In dashboards, charts can give us concise view and an accurate idea of the information we are looking for. If we can always get access to the information easily and whenever needed, then getting to a calculated decision becomes easy.

I remember, in one of my previous projects, client wanted me to have a similar dashboard experience. The problem was Microsoft Dynamics CRM version 4.0. Back then dashboards were not out of the box. So I had to write a custom made solution which was a combination of out of the box entities, plugins, silverlight and custom workflow activities. Not anymore. Dashboards are built into CRM 2011 framework.

My blog on CRM 2011 Dashboards will be coming soon!!!

I will demo how to create a multi series column chart in SQL Server Reporting Services (SSRS). I will also show how to add & display this chart on a custom CRM 2011 dashboard.

1) First I will create a multi series SSRS chart in Visual Studio report project. Then I will display this chart in a custom CRM 2011 dashboard.

The chart will compare the total cost of production for movies and the revenue for these movies for each genre. This chart will compare cost and revenue for the movie genres of Horror, Thriller and Family.

CRM 2011 when released did not have the capability of multi series charts. This capability or enhancement was added as part of  Microsoft Dynamics CRM November 2011 service update. The chart wizard now supports the ability to specify multiple data series when designing charts.

Movie Records (used as data in SSRS Multi Series Column Chart) 

2) I have opened a movie record. This record has 2 fields "Cost of Production" and "Revenue". Both these fields are in millions (let's say currency is AUD - Australian Dollars).

Fields "Cost of Production" and "Revenue" will be used in Multi Series Column Chart

3) I will create this multi series chart outside the CRM 2011 environment. I will create this chart as part of a Visual Studio report project or SSRS project or Business Intelligence project. Open Visual Studio 2008.

Note: Even though we use Visual Studio version 2010 for CRM 2011 development, but this version cannot be used for reports development. Microsoft SQL Server 2008 or 2008 R2 installs the report project templates in Visual Studio version 2008 (instead of Visual Studio 2010). Visual Studio 2010 only supports developing local reports (.rdlc).

Visual Studio to Create a Report Project

4) Create a report server project. Name the project "MovieReports". Define the location for the project.

Create a Report Project

5) After the report project is created, add a new report. Name this report "CostVsRevenue". The report is an .rdl file. I can use this file in 2 ways. Later I can import this file in Microsoft Dynamics CRM 2011 or I can deploy it to a reporting server. For our demo, I will deploy this report file to SQL Server 2008 R2 Reporting Server and use the URL address for dashboard display.

Create a New Report

6) In this new report, add a data source. Data source is a connection string embedded inside the report. This connection string connects the report data to a SQL Server database.

Add a Data Source

7) Click Edit button on the data source window. This will open another window to select a SQL Server and a database.

Data Source Window

8) In this window select a SQL Server which hosts your CRM 2011 database. I have selected the default Windows Authentication. Select the MS CRM 2011 database. Click OK button.

Create a Data Connection

9) The connection string is created. I have named my data source "Movies". Click OK button.

Data Connection String as Data Source

10) Add a new dataset. Dataset is a TSQL query connecting back to the database and getting results. Dataset will feed the data to our multi series column chart.

Add a Dataset

11) I have named my dataset "CostVsRevenueSQL". Now I will create and test my TSQL query in SQL Server R2 Management Studio.

Define a TSQL Statement

12) Open the SQL Server R2 Management Studio. Below is the query which I have created and tested. This query returns the SUM of cost and revenue. It is grouped by movie genre.

Create and Test the TSQL Statement

13) I will now embed my TSQL statement in the dataset "Query" field. I have selected "Movies" as my data source which I created above (STEP - 7).

Save the Dataset

14) The new dataset appears under the Datasets folder.

Dataset Added

15) I am all set to insert my chart, as I have my data ready. Take your mouse anywhere on the report's design surface and right click your mouse. Select Insert and then select Chart. This will bring the chart window.

Add a Multi Series Column Chart

16) There are many types of charts like Column, Line, Shape, Bar, Area, Range, Scatter and Polar. I have selected the first Column type chart which can handle a multi series display. My demo also applies to many other types of charts. You can use my demo in the same way with other charts.

Select a Chart

17) This will create a chart skeleton on my report. Now I can start binding data to the chart and I can customize chart as per business requirements. 

The Multi Series Column Chart Added

18) Select and click over on the top right hand corner which has "Series 1" and "Series 2" displayed. This will bring the "Chart Data" window. Add values. Do this by clicking the green plus sign at the right corner. This will bring all the fields from the query.

Since we are comparing cost and revenue, I will select options "Cost_of_Production" and "Revenue". This will add these fields as values. Cost and revenue will be part of "Y-Axis" on the column chart.

Add the Numerical Financial Data Values

19) Now I will add the "Group By" data which will form part of "X-Axis" on the column chart. We are only interested in cost and revenue for movie genres, so we select movie genre.

Add the Movie Genre as a Category Group

20) Our final chart data selected looks like below.

Chart Values and Category Groups

21) I renamed the title, X-Axis and Y-Axis displays for my chart. I gave more meaningful names for what it represents.

Add and Modify Chart and Axis Label

22) I will go to Preview tab and run my report.

The report is running and displaying data as required. It is called as multi series column chart as we have 2 columns side by side displaying 2 different value types.

Cost Vs Revenue Chart Output for all the Movie Genres

23) Take your mouse over the project name in Solution Explorer in Visual Studio 2008. Right click the project name and select properties. This will bring the properties window as shown below. I have added the values as per my scenario. These values will be used to deploy my report to the required folder inside reporting server as per what I need.

Report Project's Deployment Settings

24) I will now deploy my report. Select the report and right click and select "Deploy". This will deploy my report on the reporting server. I can access my report through a browser.

Deploy the Chart

25) Open Internet Explorer. I will go to my reports area and which is "http://crm2011/reportserver". Then as per the project settings I had given above (STEP-23), I can see my report under the folder structure "/MyHome_MSCRM/CustomReports/MovieReports".

Test the Chart from the Reports Server in a Browser (Internet Explorer)

26) Click the report to run in Internet Explorer. The report runs fine. Copy the URL address of this report which is required to display report in the CRM 2011 Dashboard IFrame.

Run the Chart and Get the URL Address

27) I will open CRM 2011 in the browser (Internet Explorer). I will go to my unmanaged solution where I need this new dashboard added. I will go to Dashboards section and click on New.

Create a new Dashboard in Movies (Unmanaged Solution)

28) This brings "Select Dashboard Layout" window. I will select the dashboard of type "3-Column Focused Dashboard". This is helpful in case our dashboard component display is large in height and width.

In my case I would like to display the chart in a bigger cell.

Add a 3-Column Focused Dashboard

29) Selecting the type will open the new dashboard in edit mode. I will add my SSRS chart to the first cell. The best option is to add an IFrame to the first cell and then display the chart in this IFrame. Click on "Select Iframe" option.

In First Column Insert an IFrame

30) The IFrame window opens. I have named my IFrame as "CostVsRevenue". The URL address which I had copied in STEP-26, I will paste it in IFrame URL.

Pay special attention to 3 check boxes. I haven't selected "Pass record object-type code and unique identifier as parameters" check box. I have not selected "Display label on the Dashboard" check box. I have also not selected "Restrict cross-frame scripting" check box.

Define IFrame Properties

31) The IFrame "CostVsRevenue" gets created. I have named my dashboard "Movies". I will "Save and Close" my dashboard. 

Save and Close the Dashboard

32) I will publish my changes by clicking "Publish All Customizations". This will make my dashboard available. This dashboard will be visible through the CRM "Dashboards" navigation in the "Workplace" area.

Publish the Dashboard

33)  I will go to "Dashboards" (left hand side navigation) under Workplace.

CRM Dashboard Section

34) I will go to "Dashboard" (right hand side) select list. I will select my newly created dashboard "Movies".

Select "Movies" Dashboard

35) Here it is, my multi series column chart displayed beautifully in the dashboard.

"Multi Series Column SSRS Chart" Displayed in a Custom CRM 2011 Dashboard

As we saw above, multi series charts are very helpful and powerful. Microsoft Dynamics CRM 2011 brings to us extensive charting and dashboard capabilities, helping the users to analyse their data quickly and effectively. The chart control is rendered as a single image.

Multi series charts help us compare and analyse 2 different values for the same type.

I hope this blog about 'SSRS Multi-Series Chart and Display in Dashboard' was informative. Please feel free to leave your comments.