User Tools

Site Tools


update_set_troubleshooting

Troubleshooting

Receiving Data/Updates When Subscribe is Inactive

There are two different factors that can be at play if you are encountering this situation.

The first is relates to messages being consumed but not acted on. When you create a subscribe configuration you will check the actions that you want to accept (Create, Update, Delete). If a message is pushed to your queue for a table that you do not have a Configuration set or the message is an action that you are not accepting you will still consume the message from the queue but you will not act on it. This can be seen in the Inbound messages tab, where the message will read “Skipped” in the state.

The second is that you are Subscribed to a table and its parent table. For example, you have created a Subscribe for the table Incident and another Subscribe for Task (Incident extends Task). ServiceNow handles this hierarchy like Java hierarchies, i.e. an Incident is also a Task but a Task isn't necessarily an Incident. ServiceNow will also have a copy of the record in both tables, they will have the same fields, sys_id, and everything. Then at some point you set the Incident Subscribe to “inactive” but leave the Task Subscribe as “active”. There are several cases where if someone pushes a Task record update, and that record happens to be an Incident you will consume and act on that message.

To avoid this second matter you could either remove the blanket Subscribe for the parent table or specifically set the parent Subscribe to ignore certain classes within the Filter portion. For example : This condition within the Task Subscribe will make it so you will ignore updates to Problem and Incident.

Dynamic Sharing

Duplicate Outbound Replicator Messages

For the cases where two duplicate outbound Replicator messages are created, check to make sure there are not two Perspectium Replicate business rules on the table with each rule being in a different domain. The same rule in two domains (especially if one is in global and another in a lesser domain) can cause both rules to run to create Replicator messages when dynamically sharing a record.

Business Rule Exception

If you get the error:

Exception (TypeError: Cannot convert null to an object. (; line 1)) occured while evaluating'Condition: 
PerspectiumReplicator.isReplicatedTable(current.getTableName(), "share", current.operation().toString())'
in business rule 'Perspectium Replicate' on cmn_location:Test; skipping business rule

This is generally due to there being recursive Business Rules called. If there is a Business Rule which calls current.update(), then the Business Rules will recursively: update itself, trigger the Business Rules, update itself, trigger the business rules, update itself - until ServiceNow's recursive handling will kill it. Since our Dynamic Share is triggered on the Business Rules it will work for the true correct Business Rule call, but, it will fail on the recursive calls. So you will see errors there of the failed run, however, the correct call would correctly replicate the data.

A little more specifically, they are happening because on the Business Rule condition we check for the current.operation().toString() to help validate you have the share configurations correctly set up and we aren't replicating erroneously. Within a normal Business Rule call current.operation() returns the operation as expected (insert/update/delete). On a recursive Business Rule call current.operation() is null resulting in this error.

There are two ways you can narrow down on this problem:

  • Type “Debug Business Rule” in the filter navigator and enable the “Debug Business Rule (Details)” to start the ServiceNow trace of Business Rules and repeat the test to trigger these errors (edit and save a record). You may need to be in the form view of the record instead of the list view. Then copy and paste the Business Rule trace at the bottom of the form and send it to support@perspectium.com. This will help confirm that this is the case and narrow down on the rule.
  • Review Business Rules on the table to be shared for any current.insert() or current.update() being called as those will be the ones that would cause the recursive issue.

Clearing the Cache

Sometimes after installing the newest Perspectium update set, the ServiceNow instance may still be using the old version of the included Perspectium scripts. To fix this you can add “/cache.do” to the end of your Instance URL like so:




You should be taken to a page that looks like this:



This will clear the instance cache so that it will use the newest versions of the included scripts. Then just use the back-arrow in your browser to return to your instance.

Kill Long Running Perspectium Background Jobs

Sometimes the Perspectium scheduled job running in the background (such as the Bulk Share job to bulk share records) will get stuck running. In these cases, you will want to kill the job so it doesn't affect your instance's performance.

You can try performing the following steps to kill the job. In the below example, we will be killing a bulk share scheduled job against the incident table. Bulk share scheduled jobs will have the name “Perspectium Replicator Bulk Share <table_name>” where <table_name> is the name of the table in the bulk share configuration. Note that the schedule job's name field is only 40 characters max so for a bulk share on the incident table the name will be “Perspectium Replicator Bulk Share incide”.

1. Go to User Administration > All Active Transactions and see if the Perspectium job is there. If so, delete it.

2. Go to System Scheduler > Scheduled Jobs > Scheduled Jobs and see if the Perspectium job is there. If so, delete it. For a bulk share scheduled job, this will be a Run Once trigger type scheduled job.

3. Go to the sys_cluster_state table and look to see if a node has this job running. You can find the node that has this job running by filtering on the stats field, searching for where the field contains the job name. For example, using the above “Perspectium Replicator Bulk Share incide” job, you can filter on “Bulk” in the stats field:

4. If you find a node that has this job, click on the node to view its details in form view and then click on the “Run script” UI action at the bottom:

In the Run script dialog that appears, enter the following script to run:

Packages.com.glide.sys.WorkerThreadManager.get().shutdown();
Packages.com.glide.util.GlidePropertiesDB.set("glide.worker.startup", "com.glide.schedule.GlideScheduler");
Packages.com.glide.sys.WorkerThreadManager.get().init();

If the above doesn't work, please contact ServiceNow support to have them kill the job.

Troubleshooting Report

v3.13.0

We have included a module in the Perspectium update set to easily gather up the common configurations and logs that the Support team requests during troubleshooting. Running this module at the time of the initial issue request will help speed up the troubleshooting process.

To run this module, go to Perspectium > Control and Configuration > and select “Troubleshooting Report”.

This will create messages in the outbound queue that will be sent to Perspectium to help with troubleshooting. The following messages are created:

  1. A message containing Perspectium warning logs
  2. A message containing Perspectium error logs
  3. A message containing sample Replicator messages in your outbound and inbound queues (these messages remain encrypted as we are more concerned with seeing what type of messages are stuck in your queues and could possibly be causing issues)
  4. A message containing the following Perspectium configurations:
    1. Dynamic share configurations
    2. Bulk share configurations
    3. Subscribe configurations
    4. Shared and Subscribed Queues
    5. Perspectium system properties

Your Replicator messages, Perspectium password and encryption keys included in these messages are all encrypted and thus will not be viewable by Support.

2016/08/18 15:37 · Paul Nguyen

Perspectium Tables Being Cloned

When requesting a clone of a ServiceNow instance, you will need to select the “Exclude audit and log data” option to not clone Perspectium tables and their data:

Though the Perspectium tables are listed in “Exclude Tables” list, per the Exclude audit and log data option's description, this option also needs to be selected for this list to be honored.

2017/02/23 10:04 · Paul Nguyen

Error Processing Shared Queue java.net.SocketException: Broken Pipe

If you are seeing this error in your Perspectium Logs the issue is caused by improper naming of the endpoint url of the shared queue that you are using for bulk or dynamic shares. To verify this is the case and resolve the issue you can use these steps:

  1. The message should have the name of the queue that is generating the error. Ex: Error processing shared queue psp.out.yourQueue on http://yourEnpoint/: java.net.SocketException: Broken pipe (Write failed)
  2. Access the configuration of this queue by searching for Shared Queues in the ServiceNow Searchbar and click the shared queues link that appears.
  3. Now click on the shared queue that was named in the java.net.SocketException error.
  4. Once in verify that the endpoint URL contains http. This is the cause of this error.
  5. Change the http to https and update your shared queue.

Once this is done you should see your outbound messages being sent out. If you don not see their state changing from “ready” to “sent” you can manually start the Multioutput Processing job by:

  1. Search for the All Scheduled Jobs link under the Perspectium app in the ServiceNow menu and click on that link.
  2. Click on the Perspectium Multioutput Processing job
  3. Click the Execute Now button.

You should see your messages begin to be sent out. If you are still seeing this error after these steps, please contact Perspectium support for further assistance.

Subscribe Performance

For issues, with subscribe performance in your ServiceNow instance, review the following:

  • Check the Slow Query Logs to see if any of the subscribed tables or Perspectium tables are listed and what queries are being run.
  • Check if auditing was enabled on the Perspectium inbound table (psp_in_message). This table does not come with auditing enabled by default.
update_set_troubleshooting.txt · Last modified: 2018/04/08 21:21 by paul