User Tools

Site Tools


replicator_agent_sdk

Replicator Agent SDK

The Replicator Agent SDK contains Java classes that simplify the process of writing Sharers and Subscribers or Producers and Consumers of replicated data leveraging the Perspectium Replication Framework.

Agent Release Notes

Installation

Once you have obtained your Perspectium Replicator SDK Account (contact support ), you will need to add the Perspectium Maven Repository URL and login credentials to your SDK project pom.xml file and your maven settings.xml file.

settings.xml

Because the Perspectium Nexus server requires authentication, you will also need to place your credentials in the Maven settings.xml in a servers element. There are two locations where a settings.xml file may live:

The Maven install: $M2_HOME/conf/settings.xml A user’s install: ${user.home}/.m2/settings.xml See http://maven.apache.org/settings.html for more information about the settings.xml file.

Here's a sample servers element using the default username and password for the deployment user:

<servers>
    <server>
      <id>perspectiumRepo</id>
      <username>your_username</username>
      <password>your_password</password>
    </server>
</servers>

pom.xml

Next you need to create a pom.xml file for your SDK project. See http://maven.apache.org/pom.html for more information about the maven pom.xml file.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
 
    <groupId>com.yourcompany</groupId>
    <artifactId>test-perspectium-sdk</artifactId>
    <version>1.0-SNAPSHOT</version>
 
    <dependencies>
        <dependency>
            <groupId>com.perspectium</groupId>
            <artifactId>perspectium-replicator-agent</artifactId>
            <version>3.27.0</version>
        </dependency>
    </dependencies>
 
    <repositories>
		<repository>
			<id>perspectium</id>
			<name>Perspectium</name>
			<url>https://nexus.perspectium.net/repository/public/</url>
		</repository>    
    </repositories>    
</project>

General guidelines

When you leverage the Agent SDK to write either a sharer or a subscriber you're writing a Java based handler that is configured or plugged into the Perspectium Replicator Framework for performing various data replicating tasks.

The various handlers leverage the Message class to represent the message or data that's being replicated and communicated via the Perspectium Message Bus.

Message

The Message class represents a message that is being shared or subscribed to via the SDK. The class has the following properties:

  1. Topic
  2. Type
  3. Key
  4. Name
  5. Value

The meaning of these properties and their values are context dependent and typically coordinated between a producer and consumer.

In most cases the sharer or producer will drive the value and meaning of each of these properties. Furthermore, in most cases, topic, type, key and name are mandatory.

The Value property represents the message's payload.

Name Overloading

The Message class provides the getName() method which is used to retrieve the Name value. In some cases the handler or sharer of data leverages this property to mean more than one thing. For example, when Sharing a table from a ServiceNow instance, the value is structured such that it contains the name of the table followed by a '.' period separator and then the action associated with the table row contained within the Value keys value. For example getName() could return a String such as 'incident.insert'. The row contained within the Value key's value is from the incident table and the action associated with the row is that it was inserted.

Writing Handlers

A handler is the actual software or Java class that's defined or plugged into the framework by using the <handler> directive to perform the replication.

Each handler will have access to its <task> section of the overall configuration. As already stated, there are two types of handlers that can be configured within the <agent> tag, the subscriber <subscribe> and the sharer <share>. Each of these directives will have zero or more nested <task> elements. Each <task> element will likely have unique directives that are required so the handler can perform its work. Each of these directives will be documented along with the handler.

Writing a Sharer

Each Sharer handler you write must extend the abstract class ASharer and implement the method void processMessages()Bold Text.

As a Sharer type handler you're sharing or producing data that is being replicated. As such, the data that you're replicating is placed into a Message and then shared to the Perspectium framework using the publish(<Message>) method available as part of the SDK.

* SDK Sharer Example

Writing a Subscriber

Each Subscriber based handler you write must extend the abstract class ASubscriber and implement the method public boolean processMessage(Message message) throws SubscribeException.

As a Subscriber type handler you're Subscribing to or Consuming data to be replicated. The Perspectium framework schedules your handler to run based on the type of scheduling that gets configured at the <task> level. The framework calls your processMessage(Message message)Bold Text method with an available Message from the Message Bus. The Message you receive has already been unencrypted for you by the framework.

* SDK Subscriber Example

Quick Start

To embed the SDK agent in your own Java code, make sure you have the correct configuration documented here Agent Configuration, create an instance of the ReplicatorAgent class and call the start() method:

ReplicatorAgent replicatorAgent = new ReplicatorAgent();
replicatorAgent.start();

So for example, in a test main java class:

package com.perspectium;
 
import org.apache.commons.configuration.ConfigurationException;
import com.perspectium.replicator.ReplicatorAgent;
import com.perspectium.logging.PerspectiumLogger;
 
public class Main {
    public static void main(String[] args) {
	PerspectiumLogger Log = PerspectiumLogger.create(Main.class);
	ReplicatorAgent replicatorAgent = new ReplicatorAgent();
	try {
		replicatorAgent.start();
	} catch (ConfigurationException e) {
		Log.error(e.toString());
	}
    }
}

to stop the the SDK agent call the stop method:

replicatorAgent.stop();

Next, make sure that you have the correct configuration documented here Agent Configuration. Create a folder named conf in the base directory of your source path, and create the following agent.xml in that folder.

<?xml version="1.0" encoding="ISO-8859-1" ?>
<config>
	<agent>
		<share>
			<task>
				<message_connection user="sdk_user" password="sdk_user"
					queue="psp.out.replicator.sdk_user">amqp://sdktest-cluster-1-amqp.perspectium.net
				</message_connection>
				<task_name>sdk_echo_sharer</task_name>
				<handler>EchoSharer</handler>
				<polling_interval>30</polling_interval>
				<encryption_key>The cow jumped over the moon</encryption_key>
				<echo_message>This is a simple message that is being shared or echoed.</echo_message>
				<topic>replicator</topic>
				<type>echo</type>
				<key>echo</key>
				<name>example_echo_sharer.insert</name>
			</task>
		</share>
		<subscribe>
			<task>
				<message_connection user="sdk_user" password="sdk_user"
					queue="psp.out.replicator.sdk_user">amqp://sdktest-cluster-1-amqp.perspectium.net
				</message_connection>
				<task_name>sdk_echo_subscriber</task_name>
				<handler>EchoSubscriber</handler>
				<max_reads_per_connect>10</max_reads_per_connect>
				<polling_interval>20</polling_interval>
				<decryption_key>The cow jumped over the moon</decryption_key>
			</task>
		</subscribe>
	</agent>
</config>

If you are embedding the SDK agent in your own Java code you can override the location of the configuration files using the following:

System.setProperty(CommonConfiguration.PERSPECTIUM_CONFIG_DIRECTORY, "/my/config");

Reporting

As of the Dubnium Release the agent supports reporting errors and acknowledgements back to the sender. This feature includes an extended SDK API for customizing reporting.

Setup

To include reporting in your SDK application, import the following package.

import com.perspectium.replicator.reporting.Reporting;

Instantiate a new Reporting object.

Reporting reporter = new Reporting();

Set the batch size limits in your agent.xml underneath your specified <task>. Once these limits are set the agent will send out a batch error or batch acknowledgement. Default value: 1000

<error_batch_size>42</error_batch_size>
<ack_batch_size>42</ack_batch_size>

Sending Errors

You can send back custom errors to your sharing ServiceNow instance. For instructions on leveraging this method in your SDK application, please refer to the below java-doc.

/**
 * Report an error for the specified message.
 * @param error: The error string you would like to report.
 * @param message: The message that the error belongs to.
 */
public void reportError(String error, Message message) throws ReportingException;

Handling Acknowledgements

Reporting also supports acknowledging messages through our existing Receipts functionality.

/**
* Process message and create acknowledgement to send back to ServiceNow sender.
* @param message: the message you are acknowledging.
*/
public void processAck(Message message) throws ReportingException;

Reporting Example

Here is an example implementation that illustrates how to use both new features.

package com.perspectium;
 
import com.perspectium.api.Message;
import com.perspectium.logging.PerspectiumLogger;
import com.perspectium.replicator.ASubscriber;
import com.perspectium.replicator.SubscribeException;
import com.perspectium.replicator.reporting.Reporting;
import com.perspectium.replicator.reporting.Reporting.ReportingException;
 
public class TestSubscriber extends ASubscriber {
	final static PerspectiumLogger Log = PerspectiumLogger.create(TestSubscriber.class);
	private final static String TASKNAME = "test_task";
	private final static String QUEUE = "psp.test.errors";
	final Reporting reporter = new Reporting(TASKNAME, QUEUE);
 
	public void postProcessMessage() {
		Log.info("POST Process TestSubscriber");
	}
 
	@Override
	public boolean processMessage(Message message) throws SubscribeException {
		if (message.getValue() == "") {
			String errorMsg = "Message value empty!";
			Log.error(errorMsg);
			reportError(errorMsg, message);
			return false;
		}
		processAck(message);
		return true;
	}
 
	private void reportError(String errorMessage, Message message) {
		try {
			reporter.reportError(errorMessage, message);
		} catch (ReportingException e) {
			// Reporting exceptions are thrown when reporting fails
			// You can log it, try again, etc.
			Log.error(e.toString());
		}
	}
 
	private void processAck(Message message) {
		try {
                        Log,info("Sending acknowledgment back to ServiceNow");
			reporter.processAck(message);
		} catch (ReportingException e) {
			Log.error(e.toString());
		}
	}
}

Download

Download the complete example EchoSharer/Subscriber test project here: sdktest

After downloading:

Unzip the attached.
cd to the directory where this was unzipped.
mvn clean install 
./start.sh


* Agent SDK Known Issues

replicator_agent_sdk.txt · Last modified: 2019/01/29 14:25 by timothy.pike