DataWeave Library extension

DataWeave Library Extension

Shyam Raj Prasad

6 min readApr 17, 2022

DataWeave Library Extension

The DataWeave extension adds language support that enables you to develop, debug, and test DataWeave scripts.

The DataWeave extension also enables you to quickly start creating DataWeave libraries by:

Guiding you through project creation and setup
Easing your development efforts by offering autocompletion, navigation, and reactors features
Facilitating testing by providing DataWeave Testing Framework (Beta) and debugger

Requirements

Java 8 or 11 provided
Apache Maven
Microsoft Visual Studio Code for your specific operating system
DataWeave 2.0 (BETA) library in VS code (Extensions tab)

Developing DataWeave Libraries

A DataWeave library is a reusable package of DataWeave module and mapping files, and resource files, such as JSON, XML, and CSV. To develop DataWeave libraries, first create a new DataWeave project in Visual Studio Code where you then create, preview, and test custom mappings and modules.

If you want to try out your mapping without creating a DataWeave project, you can either open your .dwl mapping file directly in Visual Studio Code or create a new one and use sample data and scenarios.

Understanding the DataWeave Project Structure

DataWeave projects use Apache Maven to build and declare project dependencies. The following diagram shows the DataWeave project structure:

Create a DataWeave Project

To create a new project:

Open Visual Studio Code.
Click View > Command Palette.
In the search bar, type dataweave.
Select DataWeave: Create New Library Project.
Follow prompts to complete an initial configuration for your project:

Organization ID (Group ID) -> Organization id of your anypoint platform account
Artifact ID -> com.mulesoft.learning
Version -> 1.0.0-SNAPSHOT
Project name -> dataweave-analytics-library

6. Choose a directory folder for your new project and Select Ok.

7. When prompted, select Yes or No whether you want to open the project in a new Visual Studio Code window.

The following image shows a new DataWeave project in Visual Studio Code with Hello world example for MyModule and MyModuleMapping:

Create a DataWeave Module, Mapping and Test Cases

DataWeave modules are .dwl files that define functions, variables, types, and namespaces. These files cannot contain output directives, body expressions, or — — separators between header and body sections. A DataWeave module enables you to share all the content defined in the .dwl file for reuse.

To create a new DataWeave module:

In Visual Studio Code, click View > Command Palette.
In the search bar, type dataweave.
Select DataWeave: Create New Module.
Add a name for the new module dwl file and press enter.
Dataweave Module: This creates a new module .dwl file under the src/main/dw folder. Add the Below function to calculate the mean and frequencies of the array.

fun mean(values: Array<Number>): Number | Null = 
  if (isEmpty(values))
    null
  else
    sum(values) / sizeOf(values)

fun mean(values: Null): Null = null

fun frequencies<T>(values: Array<T>): Array<{value: T, occurrences: Number}> = do {
  var uniques = values distinctBy ((item, index) -> item)
  ---
  uniques map ((item, index) -> {
    value: item,
    occurrences: sizeOf(values find item)
  })
}

fun frequencies(values: Null): Null = null

6. In this module file, there is option to create unit test and generate dataweave documentation.

7. Unit Test: When you click create unit test, it will create a Module test file. You can modify the description and input payload to test the module dataweave transformation. Change the ??? to [1,3,4,5] and result should be Number. you can run this test and check the result.

8. Mapping File: At the top of Analytics Module file, you can see create integration mapping(check image available in point 6). When you click on this it will create a mapping file to test your dataweave module. You can provide the input and test it out the results.

Note: This mapping won’t be shared through your library, but you can use it to try out your module and create integration tests.

9. Sample Data: In the integration mapping file, you can see Define Sample Data | Run Mapping in above image. Click on that and define the sample data as payload. It will create a payload json file and you can add the sample data in payload.json file.

10. Once payload.json is created, you can change the ???? to payload and run the integration mapping file and test it out the results.

11. Debugging: Add a breakpoint in frequencies function and try run and debug option.

12. Integration Test for Mapping file: Now you can go to command pallete and type dataweave: create mapping test, it will generate the sample integration test case. You can change the input and output files. You can run the integration test and test it out the results.

13. Update the Org Id: Change the groupId to org id of anypoint account in pom.xml file. Uncomment the repository and distribution management in pom.xml file. Also replace these ORGANIZATION_ID to org id of anypoint account.

<distributionManagement
        <repository>
            <id>exchange</id>
            <name>Exchange Repository</name>
            <url>https://maven.anypoint.mulesoft.com/api/v3/organizations/ORGANIZATION_ID/maven</url>
            <layout>default</layout>
        </repository>
    </distributionManagement>
   
    <repositories>
     
        
        <repository>
            <id>exchange</id>
            <name>Exchange Repository</name>
            <url>https://maven.anypoint.mulesoft.com/api/v3/organizations/ORGANIZATION_ID/maven</url>
            <layout>default</layout>
        </repository>>

14. Maven Settings: Create a maven settings.xml file in root folder and update the credentials of anypoint platform in maven-settings.xml to publish the dataweave library. Both repository id and server id should be same.

<settingsxml
 <servers>
   <server>
        <id>exchange</id>
        <username>anypoint username</username>
        <password>anypoint password</password>
    </server>
 </servers>
</settings>

15. Publish the dataweave module to exchange : Go to terminal and run the below maven command to publish the dataweave module to anypoint exchange.

mvn deploy -s maven-settings.xml

16. Login to anypoint platform and verify the dataweave library in exchange.

17. Go to dependency snippets, and copy the maven dependency for this jar. In the below snippets, I have changed the group id to ORGANIZATION_ID, it should be the org id of your anypoint account.

<dependency
  <groupId>ORGANIZATION_ID</groupId>
  <artifactId>dataweave-analytics-library</artifactId>
  <version>1.0.0-SNAPSHOT</version>
  <classifier>dw-library</classifier>
</dependency>>

18. Create a sample project in anypoint studio and add the above dependency in pom.xml. Add a transformation and test the mean and frequencies module in DataWeave transformation.

In this article, we have seen how we can create and publish the dataweave library. Here is the github repo of dataweave-analytics-library. Please reach out to me if you have any questions or doubt.

https://github.com/shyamrajprasad/dataweave-analytics-library

References:

https://blogs.mulesoft.com/dev-guides/dataweave-libraries/

https://beta.docs.stgx.mulesoft.com/beta-dataweave/dataweave/2.4/dataweave-share-reuse

DataWeave Library extension