D365 FO DMF Data Export using Logic Apps (2024)

This Blog describes the method of implementing Dynamics 365 UO’s Data Management Framework Recurring Integration Module using the Logic App. The blog provides a technical implementation of queuing and dequeuing of the Jobs using REST API of Dynamics 365 UO’s Recurring Integration Module. Dynamics 365 for Operation provides two primary sets of APIs, Recurring Integration APIs & Data Management Platform (DMF) package APIs, to support file-based integration scenarios. These APIs allow DMF data files as well as data projects to be imported and exported from Dynamics 365 for Operations.
In this use scenario for Dynamics 365UO recurring integrations, data will be exported to a downstream system on a recurring schedule using Logic App and store that information in Blob Storage. It shows how to export a data package and save it to a Blob and notify the systems using the Azure service bus.

More on Recurring Integration can be found here

Dynamics 365 UO Recurring Integration API forExport (dequeue)

To return a data package that contains all the data entities that were defined in the data project, and that the client application can unzip and consume, use the following structure.

https://<baseURL>/api/connector/dequeue/<activity ID>

After the client downloads the data, an acknowledgment must be sent back to Finance and Operations, so that you can mark the data as received. In cases when there was no file uploaded to the blob, the dequeue API will return a response indicating as such.

1. The execution Id of the DMF Job has been returned to the client application, which can be used to monitor the progress of the execution of the Job.

The set up involves the following set and the API supports import/export of DMF Data projects

Authorization for the Dynamics 365 UO Recurring Integration API

The integration REST API uses the same OAuth2.0 authentication model as the other service endpoints. Before the integrating client application can consume this endpoint, you must create an application ID in Microsoft Azure Active Directory (AzureAD) and give it appropriate permission to the application. When you create and enable a recurring job, you’re prompted to enter the AzureAD application ID that will interact with that recurring job. Therefore, be sure to make a note of the application ID.

Set up a Dynamics 365 UO data project and Dynamics 365 UO recurring data jobs

Create a data project

1. On the main dashboard, select theData managementtile to open theData managementworkspace.
2. Select theImportorExporttile to create a new data project.
3. Enter a valid job name, data source, and entity name.
4. Download a data file for one or more entities. Make sure that each entity is added, and that no errors occur.
5. Target data format, selectXML Element (Could use any format)
6. SelectSave.

Create a Dynamics 365 UO recurring data job

1. On theData projectpage, selectCreate a recurring data job.
2. Enter a valid name and a description for the recurring data job.
3. On theSet-up authorization policytab, enter the application ID that was generated for your application, and mark it as enabled.
4. Expandthe Advanced optionstab and specify eitherFileorData package.
5. SelectSet processing recurrence, and then, in theDefine recurrencedialog box, set up a valid recurrence for your data job
6. SelectOK, and then selectYesin the confirmation message box.

Download data from Dynamics 365 UO using recurring data jobs

You can use integration REST endpoints to integrate with the client, submit documents (import), or poll available documents for download (export). These endpoints support OAuth.

Dequeue the Dynamics 365 UO recurring Export Job

Dynamics 365 UO recurring data jobs API for Export (dequeue)

Make an HTTP POST call against the following URL and In the message body, you can then pass the data as a memory stream.

The following approach shows the way to dequeue the export Job to recurring integration. This approach uses data package-based export. Recurring integration supports both Data package export and the file export. The following parameters will be used

• The D365UO environment
• The Legal entity Name
• The ID of the recurring Job which was created in the previous step
• Name or description for the Export Job

Logic App and D365UO Export

The following section describes the method to create the Logic App for downloading Data Package from Recurring integration

1. Sign in to Azure portal: https://portal.azure.com
2. Select Create a resource, then select Logic Apps and click on Create
3. Select the appropriate Resource Group and a Logic App Name and then click on Review+Create
4. On the Logic App page select the Logic app designer option and then select Blank Logic App
5. Search for Recurrence and select the Trigger.
6. Select the Interval and the Frequency (based on your demand, shorter the interval, the more expensive the integration will be)
7. Add Parameters for the Logic App
   “ContainerName“: { “value”: “The Blob Container to store exported files” },
   “Dyn365fOClientId“: {“value”: “Client Id user for the authentication” },
   “Dyn365foURL“: { “value”: “The Url of D365FO environment xxxxxx.dynamics.com” },
   “processQueueName“: { “value”: “The queue Name for The ExportJob” },
   “ExportJobId“: { “value”: “The batch job execution ID” }
8. Add a New step, Search for Key Vault and Add an Action called “Get Secret”
9. Add a Connection to the KeyVault, Enter the name of the Key Vault.

10. Add a new step and search for Until
11. Before you add value to control the loop, Add an action:

11. Search for HTTP. In Actions select HTTP action [ This is the action that will execute the export Batch Job that we have created earlier. Provide appropriate Name for the Action]
12. Fill in the Parameters
    The URI to the DMF dequeue endpoint
    Authentication Type: Active Directory oAuth
    Authority: https://login.windows.net/
    Tenant: Name of the tenant
    Audience: Url of the D365UO environment (without the / in the end)
    Client ID: The client ID registered in Azure AD and authorized in DMF Recurring integration)
    Client Secret: The Application secret to connecting to Dynamics UO {The secret should be stored in KeyVault}
    

13. Add a +New step and search for Condition
14. Now you can add a value to control the loop. On ‘Choose a value’ text field and select the OutputParameters Status Code is not equal to 200
15. Add a +New step and search for Until, Add the Until Action
16. Search for HTTP. In Actions select HTTP action [ This is the action that will download the Data Package File. Provide appropriate Name for the Action]
17. Fill in the Parameters
    Method: Get
    The URI would be the output from the Step 10 : replace(replace(body(‘HTTP’)[‘DownloadLocation’],‘http:’,‘https:’),‘:80’,‘:443’)
    Authentication Type: Active Directory oAuth
    Authority: https://login.windows.net/
    Tenant: Name of the tenant
    Audience: Url of the D365UO environment (without the / in the end)
    Client ID: The client ID registered in Azure AD and authorized in DMF Recurring integration)
    Client Secret: The Application secret to connecting to Dynamics UO {The secret should be stored in KeyVault}

19. Now define the condition for Until Action. If the Value output parameters for the HTTP Status Code is equal to 200.

20. The false condition indicates that the package can’t be downloaded. We will send an email with the message. Add an action and search for Send an email (V2):
21. Sign in with your Outlook credentials and include the email information:
22. Add an action for the True condition. Search for Create A Blob
23. Add a connection to the Blob for the Create Blob connection
24. Then Add New action to Send a Message to Azure Service Bus.
25. Acknowledge The package download to Dynamics 365 UO Recurring integration using HTTP action. The HTTP code looks like below

{ "inputs": { "method": "POST", "uri": "@{concat('https://', parameters('Dyn365fOURL'), '/api/connector/ack/', encodeURIComponent(parameters('ExportJobId')))}", "body": "@body('Get_the_Export_Job')", "authentication": { "audience": "@concat('https://', parameters('Dyn365fOURL'))", "authority": "https://login.windows.net/", "clientId": "@parameters('Dyn365fOClientId')", "secret": "@parameters('Dyn365fOClientSecret')", "tenant": "@parameters('Tenant')", "type": "ActiveDirectoryOAuth" } }}

26. Save the Logic App. The Flow should look like this

27. Run the Logic App
28. Go back to the D365FO portal and review the Job history in the Data management workspace. A new export job with the name ExpensesExport should be executed

The JSON for the logic App is described below

{ "definition": { "$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#", "actions": { "Condition": { "actions": { "Check_the_Download_is_successful": { "actions": { "Acknowledge_The_Package_Download": { "inputs": { "authentication": { "audience": "@concat('https://', parameters('Dyn365fOURL'))", "authority": "https://login.windows.net/", "clientId": "@parameters('Dyn365fOClientId')", "secret": "@body('The_Secret_for_D365UO')?['value']", "tenant": "@parameters('Tenant')", "type": "ActiveDirectoryOAuth" }, "body": "@body('Get_the_Export_Job')", "method": "POST", "uri": "@{concat('https://', parameters('Dyn365fOURL'), '/api/connector/ack/', encodeURIComponent(parameters('ExportJobId')))}" }, "runAfter": { "Send_message": [ "Succeeded" ] }, "type": "Http" }, "Create_blob": { "inputs": { "body": "@body('Download_The_Package_File')", "host": { "connection": { "name": "@parameters('$connections')['azureblob']['connectionId']" } }, "method": "post", "path": "/datasets/default/files", "queries": { "folderPath": "/@{parameters('ContainerName')}", "name": "@guid()" } }, "runAfter": {}, "type": "ApiConnection" }, "Send_message": { "inputs": { "body": { "ContentData": "@{base64(body('Create_blob')?['Path'])}", "MessageId": "@{body('Create_blob')?['Path']}" }, "host": { "connection": { "name": "@parameters('$connections')['servicebus']['connectionId']" } }, "method": "post", "path": "/@{encodeURIComponent(parameters('processQueueName'))}/messages" }, "runAfter": { "Create_blob": [ "Succeeded" ] }, "type": "ApiConnection" } }, "expression": "@equals(and(equals(outputs('Download_The_Package_File')?['StatusCode'], 200), not(equals(body('Download_The_Package_File'), parameters('EmptyFile')))), 200)", "runAfter": { "Wait_Until_The_Package_Is_Downloaded": [ "Succeeded" ] }, "type": "If" }, "Wait_Until_The_Package_Is_Downloaded": { "actions": { "Download_The_Package_File": { "inputs": { "authentication": { "audience": "@concat('https://', parameters('Dyn365fOURL'))", "authority": "https://login.windows.net/", "clientId": "@parameters('Dyn365fOClientId')", "secret": "@body('The_Secret_for_D365UO')?['value']", "tenant": "heiway.net", "type": "ActiveDirectoryOAuth" }, "method": "GET", "uri": "@{replace(replace(body('Get_the_Export_Job')['DownloadLocation'], 'http:', 'https:'), ':80', ':443')}" }, "runAfter": {}, "type": "Http" } }, "expression": "@equals(outputs('Get_the_Export_Job')['statusCode'], 200)", "limit": { "count": 60, "timeout": "PT1H" }, "runAfter": {}, "type": "Until" } }, "else": { "actions": { "Terminate": { "inputs": { "runStatus": "Cancelled" }, "runAfter": {}, "type": "Terminate" } } }, "expression": "@equals(outputs('Get_the_Export_Job')['statusCode'], 200)", "runAfter": { "Wait_Until_Export_Job_Is_Complete": [ "Succeeded" ] }, "type": "If" }, "The_Secret_for_D365UO": { "inputs": { "host": { "connection": { "name": "@parameters('$connections')['keyvault_1']['connectionId']" } }, "method": "get", "path": "/secrets/@{encodeURIComponent('\"SecretName\"')}/value" }, "runAfter": {}, "type": "ApiConnection" }, "Wait_Until_Export_Job_Is_Complete": { "actions": { "Get_the_Export_Job": { "inputs": { "authentication": { "audience": "@concat('https://', parameters('Dyn365fOURL'))", "authority": "https://login.windows.net/", "clientId": "@parameters('Dyn365fOClientId')", "secret": "@body('The_Secret_for_D365UO')?['value']", "tenant": "@parameters('Tenant')", "type": "ActiveDirectoryOAuth" }, "method": "GET", "uri": "@{concat('https://', parameters('Dyn365fOURL'), '/api/connector/dequeue/', encodeURIComponent(parameters('ExportJobId')))}" }, "runAfter": {}, "type": "Http" } }, "expression": "@equals(outputs('Get_the_Export_Job')['statusCode'], 200)", "limit": { "count": 5, "timeout": "PT1H" }, "runAfter": { "The_Secret_for_D365UO": [ "Succeeded" ] }, "type": "Until" } }, "contentVersion": "1.0.0.0", "outputs": {}, "parameters": { "$connections": { "defaultValue": {}, "type": "Object" }, "Dyn365fOClientId": { "type": "String" }, "Dyn365fOURL": { "type": "String" }, "EmptyFile": { "defaultValue": "", "type": "String" }, "ExportJobId": { "defaultValue": "{18FA7FDE-557F-4421-BF6C-2C4417963C69}", "type": "String" }, "Tenant": { "defaultValue": "heiway.net", "type": "String" }, "processQueueName": { "type": "String" } }, "triggers": { "Recurrence": { "recurrence": { "frequency": "Minute", "interval": 2 }, "runtimeConfiguration": { "concurrency": { "runs": 1 } }, "type": "Recurrence" } } }, "parameters": { "$connections": { "value": { "azureblob": { "connectionId": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/resourcegroups/RG-Tasubaki-Encoding/providers/Microsoft.Web/connections/azureblob", "connectionName": "azureblob", "id": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/providers/Microsoft.Web/locations/westeurope/managedApis/azureblob" }, "keyvault_1": { "connectionId": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/resourceGroups/RG-Tasubaki-Encoding/providers/Microsoft.Web/connections/keyvault-2", "connectionName": "keyvault-2", "id": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/providers/Microsoft.Web/locations/westeurope/managedApis/keyvault" }, "servicebus": { "connectionId": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/resourcegroups/RG-Tasubaki-Encoding/providers/Microsoft.Web/connections/servicebus", "connectionName": "servicebus", "id": "/subscriptions/9ce707d5-a33e-4243-ba8e-9a3826893136/providers/Microsoft.Web/locations/westeurope/managedApis/servicebus" } } }, "Dyn365fOClientId": { "value": "0ae2fd1b-7b98-457c-b1c5-108824f924ec" }, "Dyn365fOURL": { "value": "nl1hhs-acc.sandbox.operations.dynamics.com" }, "processQueueName": { "value": "ExportJob" } }}

• D365 FO: Priority based throttling forintegrations
• Monitoring and alerting for Azure KeyVault
• D365 FO: Set financial dimension value usingoData
• Azure Integration using ManagedIdentity
• D365 Finance and Operations integration usingBYOD
• Azure Service Bus and Logic App integration Pattern usingPeekLock

Published by Poojith Jain

Poojith Jain is an Azure Architect with good experience with software design and development. He has a thorough knowledge of Azure Integration and he is passionate about solving complex and challenging problems in the field of AzureView more posts