Microsoft Dynamics CRM 2011 - Event Execution Pipeline

The Microsoft Dynamics CRM event processing subsystem executes plug-ins based on a message pipeline execution model. A user action in the Microsoft Dynamics CRM Web application or an SDK method call by a plug-in or other application results in a message being sent to the organization Web service. The message contains business entity information and core operation information. The message is passed through the event execution pipeline where it can be read or modified by the platform core operation and any registered plug-ins.

Note: While there are several Web services hosted by the Microsoft Dynamics CRM platform, only events triggered by the organization and OData endpoints can cause plug-ins to execute.

Architecture and Related Components

The following figure illustrates the overall architecture of the Microsoft Dynamics CRM platform with respect to both synchronous and asynchronous event processing.

Event Processing Architecture

The event execution pipeline processes events either synchronously or asynchronously. The platform core operation and any plug-ins registered for synchronous execution are executed immediately. Synchronous plug-ins that are registered for the event are executed in a well-defined order. Plug-ins registered for asynchronous execution are queued by the Asynchronous Queue Agent and executed at a later time by the asynchronous service.

Important: Regardless of whether a plug-in executes synchronously or asynchronously, there is a 2 minute time limit imposed on the execution of a (message) request. If the execution of your plug-in logic exceeds the time limit, a System.TimeoutException is thrown. If a plug-in needs more processing time than the 2 minute time limit, consider using a workflow or other background process to accomplish the intended task.

Pipeline Stages

The event pipeline is divided into multiple stages, of which 4 are available to register custom developed or 3rd party plug-ins. Multiple plug-ins that are registered in each stage can be further be ordered (ranked) within that stage during plug-in registration.

Event: Pre-Event 
Stage name: Pre-validation
Stage number: 10    
Description: Stage in the pipeline for plug-ins that are to execute before the main system operation. Plug-ins registered in this stage may execute outside the database transaction. The pre-validation stage occurs prior to security checks being performed to verify the calling or logged on user has the correct permissions to perform the intended operation. 

Event: Pre-Event      
Stage name: Pre-operation      
Stage number: 20      
Description: Stage in the pipeline for plug-ins that are to execute before the main system operation. Plug-ins registered in this stage are executed within the database transaction. 

Event: Platform Core Operation      
Stage name: MainOperation     
Stage number: 30      
Description: In-transaction main operation of the system, such as create, update, delete, and so on. No custom plug-ins can be registered in this stage. For internal use only. 

Event: Post-Event      
Stage name: Post-operation      
Stage number: 40      
Description: Stage in the pipeline for plug-ins which are to execute after the main operation. Plug-ins registered in this stage are executed within the database transaction. 

Event: Post-Event     
Stage name: Post-operation (Deprecated)     
Stage number: 50     
Description: Stage in the pipeline for plug-ins which are to execute after the main operation. Plug-ins registered in this stage may execute outside the database transaction. This stage only supports Microsoft Dynamics CRM 4.0 based plug-ins.

Message Processing

Whenever application code or a workflow invokes a Microsoft Dynamics CRM Web service method, a state change in the system occurs that raises an event. The information passed as a parameter to the Web service method is internally packaged up into a OrganizationRequest message and processed by the pipeline. The information in the OrganizationRequest message is passed to the first plug-in registered for that event where it can be read or modified before being passed to the next registered plug-in for that event and so on. Plug-ins receive the message information in the form of context that is passed to their Execute method. The message is also passed to the platform core operation.

Plug-in Registration

Plug-ins can be registered to execute before or after the core platform operation. Pre-event registered plug-ins receive the OrganizationRequest message first and can modify the message information before the message is passed to the core operation. After the core platform operation has completed, the message is then known as the OrganizationResponse. The response is passed to the registered post-event plug-ins. Post-event plug-ins have the opportunity to modify the message before a copy of the response is passed to any registered asynchronous plug-ins. Finally, the response is returned to the application or workflow that invoked the original Web service method call.

Because a single Microsoft Dynamics CRM server can host more than one organization, the execution pipeline is organization specific. There is a virtual pipeline for every organization. Plug-ins registered with the pipeline can only process business data for a single organization. A plug-in that is designed to work with multiple organizations must be registered with each organization's execution pipeline.

Note: In Microsoft Dynamics CRM 4.0, there existed a parent and a child pipeline. These pipelines have been consolidated into one pipeline for this release. A pre-event plug-in registered in the parent pipeline of Microsoft Dynamics CRM 4.0 is equivalent to registering in stage 10. A pre-event plug-in registered in the child pipeline of Microsoft Dynamics CRM 4.0 is equivalent to registering in stage 20.

Inclusion in Database Transactions

Plug-ins may or may not execute within the database transaction of the Microsoft Dynamics CRM platform. Whether a plug-in is part of the transaction is dependent on how the message request is processed by the pipeline. You can check if the plug-in is executing in-transaction by reading the IsInTransaction property inherited by IPluginExecutionContext that is passed to the plug-in. If a plug-in is executing in the database transaction and allows an exception to be passed back to the platform, the entire transaction will be rolled back. Stages 20 and 40 are guaranteed to be part of the database transaction while stage 10 and 50 may be part of the transaction.

Any registered plug-in that executes during the database transaction and that passes an exception back to the platform cancels the core operation. This results in a rollback of the core operation. In addition, any pre-event or post event registered plug-ins that have not yet executed and any workflow that is triggered by the same event that the plug-in was registered for will not execute.

My above blog is based on Microsoft's Official information.

I hope this blog about 'Microsoft Dynamics CRM 2011 - Event Execution Pipeline' was informative. Please feel free to leave your comments.

Microsoft Dynamics CRM 2011 - Introduction to Windows Azure Integration

Microsoft Dynamics CRM 2011 has been integrated with the Windows Azure platform by coupling the Microsoft Dynamics CRM event execution pipeline to the Windows Azure Service Bus in the cloud. In essence, the Microsoft Dynamics CRM pipeline connects to the Windows Azure Service Bus enabling the data that has been processed as part of the current Microsoft Dynamics CRM operation to be posted to the bus. Windows Azure Service Bus solutions that are “CRM aware” can listen for and read the data that is posted on the service bus by Microsoft Dynamics CRM. The posted data is stored in a RemoteExecutionContext class instance that is an extended version of IExecutionContext passed at run-time to Microsoft Dynamics CRM asynchronous plug-ins. In addition to information on the requested operation that was processed by Microsoft Dynamics CRM, this class adds a OperationId property and a OperationCreatedOn property, which contain the same data as the AsyncOperationId and CreatedOn attributes of the related System Job (AsyncOperation) record. These additional properties facilitate sequencing and duplicate detection if the Windows Azure Service Bus post must be retried.

This integration between Microsoft Dynamics CRM 2011 and the Windows Azure platform provides a secure channel for communicating Microsoft Dynamics CRM run-time data to external cloud based line of business applications. This capability is especially useful in keeping disparate CRM systems or other database servers synchronized with Microsoft Dynamics CRM Online business data changes.

 

Identifying Key Elements of the Integration

The key elements that implement the integration between Microsoft Dynamics CRM 2011 and the Windows Azure platform are as follows.

Asynchronous Service 

The asynchronous service is responsible for posting the Microsoft Dynamics CRM remote execution context. Each post is performed by a system job of the asynchronous service. A user can view the status of each system job using the Microsoft Dynamics CRM Web application.

Asynchronous Plug-ins 

There are two kinds of asynchronous plug-ins supported by the integration feature: out-of-box (OOB) and custom. Synchronous plug-ins are not supported.

An Azure aware plug-in is provided with Microsoft Dynamics CRM 2011. This OOB plug-in executes in full trust with the Microsoft Dynamics CRM platform. When registered with Microsoft Dynamics CRM, the plug-in can notify the asynchronous service to post the current request’s context. A developer needs to register a step on this plug-in that identifies the target message and entity in order to enable the service bus posting functionality.

You can also write your own custom plug-in that is “Windows Azure aware”. The custom plug-in executes in partial trust mode in the sandbox and can call any Microsoft Dynamics CRM SDK methods. For a custom plug-in to initiate posting the Microsoft Dynamics CRM context to the bus, the plug-in code must include some standard lines of code that notifies the asynchronous service to post the request context. This code makes the plug-in “Windows Azure aware”.

Custom Workflow Activities 

Custom workflow activities can be written to post the current request’s data context to the Windows Azure Service Bus.

Windows Azure Service Bus 

The service bus relays the remote execution context between Microsoft Dynamics CRM 2011 and Windows Azure Service Bus solution listeners. The Access Control Service (ACS) manages claims based authentication security.

Windows Azure Solution 

For the CRM-Azure integration feature to work there must be at least one solution in an Windows Azure Service Bus solution account, where the solution contains one or more service endpoints. For a non-queued endpoint contract, a listener that is “CRM aware” must be actively listening on the endpoint for the Microsoft Dynamics CRM request on the service bus. For a queued endpoint contract, a listener does not have to be actively listening. A listener is made “CRM aware” by linking it to the Microsoft.Xrm.Sdk assembly so that type RemoteExecutionContext is defined.

The solution rules must be configured to allow the Microsoft Dynamics CRM 2011 remote execution context to be posted to the service bus. To enable this posting, ACS needs to recognize the Microsoft Dynamics CRM deployment as a supported issuer.

Important: To develop a solution listener for Windows Azure platform, you need to install the Windows Azure SDK on your development computer.

Describing a Microsoft Dynamics CRM to Service Bus Scenario

Let us now identify a scenario that implements those integration components identified previously. As a pre-requisite, ACS has been configured to recognize Microsoft Dynamics CRM as the supported issuer and the Windows Azure Service Bus solution configured with rules to allow Microsoft Dynamics CRM to post to the endpoint on which the listener is listening.

The following diagram shows the physical elements that make up the scenario.

Microsoft Dynamics CRM to Service Bus Scenario

The sequence of events as identified in this diagram are as follows:

1. A listener is registered on a Windows Azure Service Bus solution endpoint and begins actively listening for the Microsoft Dynamics CRM 2011 remote execution context on the service bus.
2. A user performs some operation in Microsoft Dynamics CRM 2011 that triggers execution of the registered OOB (out of the box) plug-in or a custom Windows Azure aware plug-in. The plug-in initiates a post, through an asynchronous service system job, of the current request context to the service bus.
3. ACS authenticates the claims posted by Microsoft Dynamics CRM. The service bus then relays the remote execution context to the listener. The listener processes the context information and performs some business related task with that information. The asynchronous service is notified, by the service bus, of a successful post and sets the related system job to a completed status.

Establishing a Contract between Microsoft Dynamics CRM and a a Windows Azure Solution

For each solution endpoint, you configure a contract that defines the handling of these remote execution context ‘messages’ on the service bus and the security that should be used on that endpoint. Service bus messages are received at an endpoint using a queued contract, a one-way contract, a two-way contract, or a REST contract.

Queued:

A queued contract, also known as a “message buffer”, provides a message queue in the cloud where messages are stored in the cloud. With a queued contract, a listener does not have to be actively listening for messages on the endpoint. However, the queued messages and the endpoint will be removed from the service bus if not used for a pre-configured length of time that typically is less than 10 minutes. For queues, there is a destructive read and a non-destructive read. A destructive read reads an available message from the queue and the message is removed. A non-destructive read does not remove a message from the queue

One-way:

A one-way contract requires an active listener. If there is no active listener on an endpoint, the Microsoft Dynamics CRM post to the service bus fails. Microsoft Dynamics CRM will retry the post in exponentially larger and larger time spans until the asynchronous system job that is posting the request is eventually aborted and its status is set to Failed.

Two-way:

A two-way contract is similar to a one-way contract except that a string value can be returned from the listener to Microsoft Dynamics CRM.

REST:
A REST contract is similar to a two-way contract on a REST endpoint.

Identifying the kind of security a contract uses is part of the contract’s configuration. A contract can use Transport security, which uses Secure Socket Layer (https).

Claims authentication is used for secure access to the service bus. However, Microsoft Dynamics CRM and the service bus use different claims implementations. The claim used to authenticate to the service bus is generated within Microsoft Dynamics CRM and signed by the AppFabricIssuer certificate specified in the Microsoft Dynamics CRM configuration database.

Management and Notification of Run-time Errors

If an error occurred after a post was attempted to the service bus, check the status of the related system job in the Microsoft Dynamics CRM Web application for more information on the error. If the service bus is down or a listener/endpoint is not available, the current message being processed in Microsoft Dynamics CRM will not be posted to the bus. The asynchronous service will continue to try to post the message in an exponential pattern where it will try to post frequently at first and then at longer and longer intervals. For an internal Microsoft Dynamics CRM error, message posts are not attempted. For an external service bus or network error, the related system job will be in a Wait state.

My above blog is based on Microsoft's Official information.

I hope this blog about 'Microsoft Dynamics CRM 2011 - Introduction to Windows Azure Integration' was informative. Please feel free to leave your comments.

Microsoft Dynamics CRM 2011 - Process Architecture

The process management system in Microsoft Dynamics CRM includes the Microsoft Dynamics CRM SDK, plug-ins, forms, and other components.

Process Architecture Diagram

The following diagram illustrates the high-level system architecture for Microsoft Dynamics CRM, and highlights parts of the system that are specific to processes.

This diagram shows the process architecture of Microsoft Dynamics CRM including the internal components, the external components, the infrastructure, and the data.

Internal Components. The internal components that support the Microsoft Dynamics CRM process programming model include Web services, shared platform, and business logic. The shared platform consists of common Microsoft Dynamics CRM components that provide registration, metadata cache, and data access services. Business logic contains the implementation of business logic for Microsoft Dynamics CRM business entities.

External Components. The external components are as follows:

• Windows Workflow Foundation object model, which contains a set of classes used to create and parse workflow process definitions in XAML format.

• Windows Workflow Foundation execution, which contains a set of classes used to execute workflow processes.

Process Infrastructure. The Microsoft Dynamics CRM process infrastructure consists of the following components:

• Process entity model, which contains a set of classes that use the Windows Workflow Foundation object model and expose Microsoft Dynamics CRM workflow activities.

• Process business logic, which implements business logic for process-specific entities.

• Process execution, which provides workflow execution services, such as workflow hosting and persistence.

Process Data. The Microsoft Dynamics CRM process programming model is supported by the following data:

• Business data, which contains information associated with Microsoft Dynamics CRM entities.

• Process configuration data, which includes workflow process definitions, compiled workflow processes, and workflow process settings.

• Process runtime data, which is required to execute workflow processes and implement workflow process features, such as persistence and notifications. 

Process and the Unified Event Model

 

Microsoft Dynamics CRM uses a unified event model that is used in both plug-ins (callouts) and in processes. This event processing subsystem adds more flexibility to the execution of processes and plug-ins by introducing the pipeline execution model.

Using this model, processes and plug-ins are executed based on their registration, message type, and a predefined set of configurable parameters. The core platform operations take part in the execution sequence to form a much more reliable and extensible execution model.

Process Life Cycle

 

The life cycle of a process describes the state transitions from creation through execution. A process can be in one of the following states: Ready, Suspended, Locked, and Completed. The events that occur throughout the lifetime of the process cause a transition from one state to another.

Workflows

The workflow life cycle is as follows:

1. When you create a workflow, it is in the Draft state. You must activate the workflow before it can run. When you activate a workflow, it subscribes to specific Microsoft Dynamics CRM events. When these events are triggered in the platform, a snapshot of the workflow dependencies and input parameters are created and a new asynchronous operation is added to the asynchronous service queue manager. The asynchronous operation represents a workflow execution job and awaits execution in the queue in the Ready state. 
2. When the asynchronous operation is processed, a workflow instance, associated with this operation, is created by the Windows Workflow Foundation run-time engine and the state of it is changed from Ready to Locked.
3. The asynchronous operation is updated with the workflow instance state status on each transition. When the asynchronous operation is blocked, the Windows Workflow Foundation run-time engine puts the workflow instance into the Suspended state and removes it from memory. When the Suspended state conditions are satisfied, the workflow instance is loaded back into memory.
4. The workflow execution resumes by putting the workflow instance into a Ready state and then into a Locked state. In the simple scenario, the workflow instance moves to a Completed state when all workflow activities have completed successfully.

The state of asynchronous operations can also be changed by the user. For example, an asynchronous operation that is in a Suspended state can explicitly be restarted by the user.

Dialogs

A dialog life cycle is as follows:

1. When you create a dialog, it is in the Draft state. You must activate the dialog before it can be run. A dialog can be run from its primary entity form and grid, or directly by using the URL of the dialog.
2. Every time that you run a dialog, a process session (dialog session) instance is created for the dialog. As you progress with running the dialog, the dialog session entity is updated with the actions performed during the running of the dialog.
3. If you completed running the dialog successfully, the corresponding dialog session record is created with a Complete status. If you canceled the dialog without completing it, the corresponding dialog session record is created with an Incomplete status.

Process and the Asynchronous Service: For Workflows Only

 

The asynchronous service enables you to execute, monitor, and manage various long-running operations, such as bulk import, bulk mail, and workflow processes. To improve performance, scalability, and reliability of Microsoft Dynamics CRM, these operations are run asynchronously. This means that a requested operation is not processed instantly, but added to a queue and processed by Microsoft Dynamics CRM at an appropriate time.

When an event is raised in the Microsoft Dynamics CRM platform pipeline, all workflows that are associated with the event are executed by the asynchronous service. The workflow event handlers are added to the asynchronous queue and processed according to the event execution order.

Process Persistence and Shutdown: For Workflows Only

 

A workflow can be a long-running business operation that might take hours, weeks, or months to be completed. It can be effectively idle for long periods of time waiting for input from users or other systems.

To improve performance, scalability, and reliability of Microsoft Dynamics CRM, long-running operations such as workflows use the asynchronous service.

The asynchronous service, as the host of the Windows Workflow Foundation runtime engine, cannot always cache and keep active all objects that accumulate during continued workflow activity. When certain conditions, such as restart or shutdown occur when a workflow is running, the workflow runtime engine uses a persistence service to save the state of the workflow instance onto the disk. The persistence service is also invoked when other conditions occur, such as when a workflow becomes idle and is waiting for some external event to occur. Persisting these idle workflow instances saves memory and greatly increases scalability. If a server that is running the asynchronous service is shut down or if the workflow crashes during execution, the workflow can be restarted from its last persisted point after the server restarts. When the workflow is no longer idle, the state of the workflow instance is restored in memory to the state at the last persisted point. 

My above blog is based on Microsoft's Official information.

I hope this blog about 'Microsoft Dynamics CRM 2011 - Process Architecture' was informative. Please feel free to leave your comments.