Administrating and Extending WSO2 ESB

  • Kasun Indrasiri
Chapter
First Online:

Abstract

This chapter is all about how you administrate your ESB deployment and how you can extend it for any custom requirement that is not support out of the box.

Keywords

Custom Requirement Proxy Service Error Code Error Handling Custom Connector 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
Download chapter PDF

This chapter is all about how you administrate your ESB deployment and how you can extend it for any custom requirement that is not support out of the box.

WSO2 ESB Analytics

You learned about three main components of the ESB product in the previous chapters—runtime, tool, and analytics. In this chapter, we focus on the ESB analytics component. In the same way you download the ESB product or tool, you can download the ESB analytics server. The ESB analytics provides two main functionalities related to your ESB runtime.

  • Statistics : Coarse-grain and fine-grain statistics of your integration/mediation flow.

  • Message tracing: Enable tracing for a selected message flow and trace through the message path.

The way that ESB runtime and analytics runtime works is that ESB publishes events/data to the analytics server via the Thrift protocol. The configuration of the connectivity between ESB runtime and the ESB analytics server is configured in <ESB_HOME>/repository/deployment/server/eventpublishers/MessageFlowConfigurationPublisher.xml. In order to collect ESB mediation statistics and message tracing data, you need to enable statistics and tracing as follows.

ESB analytics data collection can be configured by setting the following properties in the <ESB_HOME>/repository/conf/synapse.properties file. You can choose the option of collecting payload and properties if needed.

mediation.flow.statistics.enable=true
mediation.flow.statistics.tracer.collect.payloads=true
mediation.flow.statistics.tracer.collect.properties=true

However, this configuration parameter doesn’t enable statistics for all artifacts by default. If you need to collect statistics or/and tracing information for a particular artifact such as a proxy service, REST API, sequence, endpoint, etc., that has to be enabled at each artifact level (If you want to, you can use mediation.flow.statistics.collect.all=true to enable statistics for all artifacts, so that you don’t need to enable it at each artifact level.) By default, when you download the ESB runtime and ESB analytics server, you just need to enable these parameters on the ESB side and they will automatically connect to your local machine. However, if the analytics servers are configured in a remote machine, they need to be configured in the event publisher configuration discussed earlier.

Suppose that you installed and enabled statistics and tracing at the synapse.properties file for the use case shown in Figure 11-1.
Figure 11-1.

A sample service orchestration scenario with numerous message transformations

We have selected this use case as it has multiple endpoints and several message transformation logics. Now suppose that you enable statistics at the synapse.properties level and at the artifact level and enabled statistics and tracing. This is done by using statistics="enable" trace="enable".

You can configure it to collect statistics or/and trace messages in a given artifact. Statistics collection is lightweight but message tracing consumes resources, as it keeps the message payload at each point where we change the payload.

<api context="/atm/locator" name="ATMLocatorRESTAPI" statistics="enable" trace="enable" xmlns="http://ws.apache.org/ns/synapse">
    <resource faultSequence="ATMLocatorFaultSeq" methods="POST">
        <inSequence>
            <log level="full">
                <property name="API_Log" value="=== Req Received at API Resource ==="/>
            </log>
            <datamapper config="gov:datamapper/Req2S1.dmc" inputSchema="gov:datamapper/Req2S1_inputSchema.json" inputType="JSON" outputSchema="gov:datamapper/Req2S1_outputSchema.json" outputType="JSON"/>
            <call>
                <endpoint key="Geo2PostalEP"/>
            </call>
            <datamapper config="gov:datamapper/Postalcode2ATMs.dmc" inputSchema="gov:datamapper/Postalcode2ATMs_inputSchema.json" inputType="JSON" outputSchema="gov:datamapper/Postalcode2ATMs_outputSchema.json" outputType="XML"/>
            <header name="Action" scope="default" value="urn:PostalcodeToATMLocatorService"/>
            <call>
                <endpoint key="Postalcode2ATMEP"/>
            </call>
            <datamapper config="gov:datamapper/ATMLocation2Coordinates.dmc" inputSchema="gov:datamapper/ATMLocation2Coordinates_inputSchema.json" inputType="XML" outputSchema="gov:datamapper/ATMLocation2Coordinates_outputSchema.json" outputType="XML"/>
            <call>
                <endpoint key="ATMGeo2Address"/>
            </call>
            <datamapper config="gov:datamapper/ATMAddress2Res.dmc" inputSchema="gov:datamapper/ATMAddress2Res_inputSchema.json" inputType="XML" outputSchema="gov:datamapper/ATMAddress2Res_outputSchema.json" outputType="JSON"/>
            <respond/>
        </inSequence>
        <outSequence/>
    </resource>
</api>
Listing 11-1.

ESB Configuration of the Service Orchestration Scenario

Once you send requests through this REST API, you can log in to the ESB analytics server dashboard (https://localhost:9444/portal/dashboards/esb-analytics/) and observe the statistics and message-tracing portal.

The overview sections show the overall statistics of your ESB instance. Overall TPS, message count, successful and failed counts, usage of each ESB artifact, and so on, are shown in the overview section, as illustrated in Figure 11-2.
Figure 11-2.

ESB Statistics - Overview

Figure 11-3.

Artifact level statistics

Then you can further drill down to the analytics of each artifact type. Since we are using a REST API , we can select the REST API and observe the statistics collect against that API.
Figure 11-4.

Statistics of a given REST API/HTTP Service

Also you can observe all the messages that go through the API along with its message ID with status (success or failed).
Figure 11-5.

List of messages that are processed by ESB and their message path through WSO2 ESB

Then at the bottom, the overall message flow diagram is shown with the average processing time at each mediator/endpoint level.
Figure 11-6.

Drill-down mediation statistics of a given message across different message processing component

For any specific message ID , you can trace through the path by clicking on the message that you want to trace.
Figure 11-7.

Tracing the message flow of a selected message

For example, if you observe one of the failure messages , you can find out that it failed during the invocation of the first service, at which point the fault sequence was triggered.
Figure 11-8.

Message tracing UI for a given message

You can also view the payload associated with each mediator in that particular message path of the failed message.
Figure 11-9.

Using message tracing of a given message ID and observer the message payloads and message attributes throughout the ESB message mediation flow

You can enable statistics for all the artifacts while tracing can be applied for a selected artifact for a limited period of time (due to resource consumption). Often message tracing is useful for identifying the reason for various request failures in a production environment.

For high load, you may have to scale up the ESB analytics server. Since the ESB analytics server is based on the WSO2 Data Analytics Server (DAS), you can refer https://docs.wso2.com/display/DAS310/Deployment+and+Clustering for details on how to set up a high-available ESB analytics cluster.

Monitoring

When you run the ESB instance in your production environment, you have to monitor it for any abnormal behaviors that could affect the live traffic. For that, WSO2 ESB exposes the runtime status through JMX (Java Management Extension). JMX is a common method to manage and monitor the runtime parameters of a remote server. You can observe almost all the details related to ESB mediation flow statistics through JMX as well. You can use JConsole and connect to the ESB runtime to obtain this information.
Figure 11-10.

Mediation statistics are also exposed through JMX

In addition to the high-level statistics of ESB artifacts, WSO2 ESB also exposes the low-level latency information of its transports.
Figure 11-11.

Low-level HTTP transport details are exposed through JMX

You can find more details of these MBeans at http://docs.wso2.com/enterprise-service-bus/JMX+Monitoring .

Extending WSO2 ESB

WSO2 ESB provides most of the standard features that you will need in most of the integration scenarios. However, it is not uncommon that you may come across certain requirements that are not supported out of the box by WSO2 ESB. This is where you will have to use extension points in WSO2 ESB.

Class Mediator

One of the most common extensions that you have to implement is the class mediator. It is quite useful when you want to inject custom message processing/mediation logic inside an ESB sequence.

The class mediator creates an instance of a custom-specified class and sets it as a mediator. The class must implement the org.apache.synapse.api.Mediator interface or extend the AbstractMediator. If any properties are specified as part of the class mediator configuration, the corresponding setter methods are invoked once on the class during initialization.

In order to create a class mediator, you can select Mediator Project and it will create a class mediator skeleton, as shown in Listing 11-2.

package kasun.panorama;
import org.apache.synapse.MessageContext;
import org.apache.synapse.mediators.AbstractMediator;
public class MyClassMediator extends AbstractMediator {
        public boolean mediate(MessageContext context) {
                // TODO Implement your mediation logic here
                return true;
        }
}
Listing 11-2.

Class Mediator Structure

You can access all the information related to the incoming message through the message context object and you can implement any message processing logic inside the mediate method.

Script Mediator

The script mediator is used to invoke the functions of a variety of scripting languages such as JavaScript, Groovy, or Ruby. You can simply write the message processing logic using any of these languages and refer to the script from the script mediator inside a sequence.

<log/>
<script language="js"
    key="repository/conf/sample/resources/script/test.js"
    function="testFunction"/>
// test.js (included as a registry resource)
function testFunction(mc) {
     var symbol = mc.getPayloadXML()..*::Code.toString();
     mc.setPayloadXML(
        <m:getQuote xmlns:m="http://services.samples/xsd">
           <m:request>
              <m:symbol>{symbol}</m:symbol>
           </m:request>
        </m:getQuote>);
}
Listing 11-3.

Script Mediator Example

Similarly, any other script can be included in this manner. There are specific methods such as getPayloadXML(), getPayloadJSON(), and so on, that are supported as part of message context API associated with script mediator. For all the available API methods and examples, refer to http://docs.wso2.com/enterprise-service-bus/Script+Mediator .

Custom Connector

Although there are hundreds of ESB connectors available with WSO2 ESB, you may have to address any specific custom requirement. For such requirements you can create your own connector to integrate with any cloud APIs or any internal legacy system. To build your own ESB connector, you can use the following Maven archetype:

mvn archetype:generate -DarchetypeGroupId=org.wso2.carbon.extension.archetype -DarchetypeArtifactId=org.wso2.carbon.extension.esb.connector-archetype -DarchetypeVersion=2.0.0 -DgroupId=org.wso2.carbon.connector -DartifactId=org.wso2.carbon.connector.helloworld -Dversion=1.0.0 -DarchetypeRepository=http://maven.wso2.org/nexus/content/repositories/wso2-public/

This creates the org.wso2.carbon.esb.connector.helloworld directory in the current location of your machine, with a directory structure similar to Figure 11-12.
Figure 11-12.

Skeleton of the connector development project

You simply have to create the required ESB templates and the required class mediators to handle the specific scenario. This will output a connector archive, which can be deployed into an ESB. Refer to https://docs.wso2.com/display/ESBCONNECTORS/Writing+a+Connector for more details.

Other Extensions

There are various other types of extensions you can use with WSO2 ESB. Tasks, custom message builders and formatters, custom inbound endpoints, etc. are some of the other commonly used extensions.

Error Handling

Any mediation logic that you develop should have proper error-handling semantics and logics. In many ESB artifacts, you can associate a fault-handling mechanism known as a fault sequence.

A fault sequence is a collection of mediators just like any other sequence, and it can be associated with another sequence or a proxy service. When the sequence, proxy service, etc. encounters an error during mediation or while forwarding a message, the message that triggered the error is delegated to the specified fault sequence. Using the available mediators, it is possible to log the erroneous message, forward it to a special error-tracking service, and send a custom error message to the client. Proper error handling in the fault sequence prevents ESB from going to abnormal states and it also helps to maintain proper resource utilization.

Whenever an error occurs in WSO2 ESB, it attempts to provide as much information as possible about the error to the user through the ERROR_CODE, ERROR_MESSAGE, ERROR_DETAIL, and ERROR_EXCEPTION properties. These values can be added to logs via the log mediator. ESB has defined a set of error codes. For an example, if you have observed an error in your fault sequence with error code 101504, that means “connection timed out.”

All the available error codes and the reasons for each error can be found at http://docs.wso2.com/enterprise-service-bus/Error+Handling .

Summary

In this chapter you learned:

  • How to use the ESB analytics server with ESB runtime to obtain statistics of ESB mediation flow and to trace messages.

  • JMX monitoring.

  • Commonly used mediation extensions and how to implement them (class mediators, script mediators, and the custom connector).

  • Error-handling techniques in WSO2 ESB.

Copyright information

© Kasun Indrasiri 2016

Authors and Affiliations

  • Kasun Indrasiri
    • 1
  1. 1.San JoseUSA

Personalised recommendations