User Tools

Site Tools


replicator_snc_bulk_share

ServiceNow Bulk Share

The Bulk Share allows a ServiceNow instance to share a pre-filtered range of data all at once. The consumer of this data can be another instance of ServiceNow, Replicator Agent, or any number of the other applications that Perspectium can integrate with.

This job can share multiple tables through the table hierarchy, run through advanced filters and scripts, share out related/auxiliary records, and more.

This page documents each of the core components of the bulk share and the following features related to it:

  • Creating a Basic Bulk Share
  • Additional Settings
  • Filter and Enrichment
  • Security
  • Notes
  • Advanced Options
  • UI Actions
  • Related List Tables

Creating a Bulk Share

To start configuring a bulk share, go to the Replicator > Bulk Share module under the Perspectium app.

By selecting the above module, you will be presented with a list of currently defined bulk shares. If this is your first time, there will not be any existing entries. Go ahead and click the button at the top of the list. You will now be presented with a form to define a bulk share.

Initial Setup

For basic functionality the form requires a name, a table, and a target queue. The name is just for cosmetic purposes, but we recommend giving it something that makes functional sense to you and your team. The table field is for the table that you would like to replicate records from (for example, selecting the incident table will replicate incident data when the share is executed). And the target queue is the queue that these messages will flow through to your Replicator Agent, ServiceNow Instance, etc.



Configuration


Additional Settings

Through the Bulk Share you can also share out any records related to the initial table records. This includes any:

  • Attachments
  • Embedded Images / Videos
  • Journal Fields
  • Audit Log
  • History Set
  • Referenced Fields D Release

Of the main record. By default this will share all the related records per table. The attachment option will share out the sys_attachment (metadata) as well as the sys_attachment_doc (data) records for the attachments.

You can read more about the sharing of Journal and Audit Records here as well as embedded images / videos here.

Sharing Child vs Parent

The Bulk Share and Dynamic Share can be used to replicate out ServiceNow data with respect to the parent child hierarchy. If you are familiar with how ServiceNow handles the parent child hierarchy this will make a little more sense.

Say you have a Windows Server [cmdb_ci_win_server], this extends Server [cmdb_ci_server], which extends Computer [cmdb_ci_computer], which extends Hardware [cmdb_ci_hardware], which extends Configuration Item [cmdb_ci], which extends the base cmdb table [cmdb].


When you are looking at a record in ServiceNow you are traditionally looking at the “leaf” or the most child like version of this class. So you would see the Windows Server version of this record as opposed the Hardware version of this record. In reality every windows server is a server, which is a computer, which is hardware, etc…. So you can configure your sharing to honor this hierarchy.

By default you will have “Share Child Class Only” selected. This means that if you are sharing [cmdb_ci] you will receive Windows Servers, Linux Servers, Network Gear, etc. You will receive the record as it is defined (by the sys_class_name) in the single table it is defined by (Windows Server, Linux Server, Network Gear)

If you uncheck this you are sharing the base table only. This means that you are explicitly sharing the version of this record for the table that you will select. So if you are sharing [cmdb_ci] you will receive all records as the base [cmdb_ci] record into the [cmdb_ci] table with the columns isolated to this table.

If you choose “Include All Child Tables Only” this will share out all the layers of the data. For our Windows Server example above that means a share on Configuration Item [cmdb_ci] will share out a Configuration Item, Hardware, Computer, Server, and Windows Server record, into the respective 5 tables.

You can read more about the handling for the ServiceNow Hierarchy here.

Include Referenced Field Records

D Release

A checkbox to indicate if referenced field records related to this table should also be shared.

You can read more about this feature Include Referenced Field Records here.

Insert Only

By default we send all our messages as bulk or upsert messages to the receiving instance. The receiving instance will determine if it needs to do an insert or an update. This option explicitly says to send every message as an insert. This can be done as an optimization when you are doing the initial share of a table and you know there are not going to be any conflicts.

If the receiving instance receives this insert call and the record already exists (generally by sys_id as the primary key), it will error out and throw away the message.

Table Map

Table Maps are a way to modify the structure and payload of the message you are sending out of the instance. By default this is the XML Serialized version of the record with display values appended if selected.

This outbound record can be manipulated by the Table Map to add/remove/modify columns and their data, modify to a JSON output, utilize our Common Document Format, etc. If you are familiar with how ServiceNow utilizes Transform Maps to manipulate data coming into ServiceNow, Table Maps are meant to represent the same logic but manipulating the data going out.

You can read more about Table Maps here.

View Name

By default, Bulk Shares will share all fields available in a table. There are times when you want to control the number of fields being shared from your ServiceNow table. By specifying a UI Form View in the definition for Dynamic Share or Bulk Share you can limit the fields to only those that appear on the form view that you have personalized. This can potentially slow down the sharing of data, because more work has to be done to remove fields before the data is sent. For information on views in ServiceNow see Personalizing Form Views in ServiceNow

You want to use the actual Form View name, not the Title for the View Name value. For example, use ess instead of Self Service.

You can read more about this Sharing Database Views here.

Run Schedule

You can schedule bulk shares to run automatically in the background. Regular bulk shares can also be assigned a run schedule based of a current scheduled bulk share. Essentially, this feature allows you to change a normal bulk share into a scheduled bulk share.

Cipher

Perspectium Bulk Shares hav several cipher options for securing your data. Currently we offer TripleDES, AES128, Base64 Encode only, and Unencrypted options. Make sure that the option you select is supported at the endpoint.


Filter and Enrichment

Share Only Selected Fields

Under the Filter and Enrichment tab you can select to share only selected fields. This is useful if you do not want to create a custom view, or you do not have the permissions to do so.

When the box is selected, a “Share Fields” tab will appear at the bottom of the page where you can specify which fields you want to share out as part of the XML.

There is an “Add all table fields” button in case you want to share a majority of the fields so you can add all fields first and then delete the ones you don't want versus having to add each one individually.

You can read more about this feature Sharing Selected Fields here.

Share only Sys IDs Listed

Much like the Share only selected fields box the “Share only Sys IDs listed” option will allow you to share only the sys ids specified in the bulk share. This way, instead of having to specify conditions to find certain sys ids, you can list the actual sys ids here and then the bulk share will only share those records.

You can enter in (sys_id - is one of - …..) into your filter condition to easily set up the sys_id sharing as well but this option is intended to be a more automatic way to do so.

A “Sys IDs” related list is provided in the Related Links section where you can enter the sys ids you want to bulk share (“Record Sys ID” is the record's sys id in the table).

You can read more about this feature Sharing sys_ids listed here.

Condition Builder

You can also add filter conditions to the bulk share in order to specify which records from the tables get shared.

You can add multiple Filter Conditions, as well as “OR” clauses in case you want two sets of conditions that allow for replication.

Share Scripts

There are three share script options available when configuring your bulk share, Before Share Scripts, Before Bulk Share Scripts, and After Bulk Share Scripts.

Before Share Scripts

This is a script that gets executed right before the shared object is being queued up. The object is in GlideRecord type and can be accessed via the globally available variable called current. For an example of how to use this script, go here.

Before / After Bulk Share Scripts

The “Before bulk share script” and “After bulk share script” sections allow you to specify a script that will run before the bulk share starts sharing out records and after the bulk share has completed sharing records. This can be useful for when you want to run a script using the Perspectium API to populate the bulk share with the sys ids of the records to be shared and/or create PSP outbound messages to send before the bulk share runs or after it is completed.

Both sections expect server side javascript, and is preconfigured with a globally available variable called bulkshare_gr that is the GlideRecord object corresponding to the bulk share configuration itself (i.e. you can use bulkshare_gr.u_name to access the name of the bulk share).

Share Updates Since Then

Users can select the option “Share updates since then” to only share records that have been inserted or updated since the last bulk share ran which is shown in the “Last share time” field. This option will also be applied on: Scheduled Bulk Shares, Run another bulk share like this and Execute now.

Limit Number of Records Shared

Users can limit the number of records that get shared in a bulk share using the “Limit number of records shared” field. The bulk share will stop sharing records once it reaches the limit that the user input. Bulk shares will evaluate the condition first before applying the “Limit number of records shared”.

For example, the configuration below will only share 50 incident records.


Security

Run As

Bulk shares automatically run as Admin users, and thus have access to all records in a table. if you only want certain records to be shared, or avoid sending records that you don't have access to, you can choose to run the bulk share as another user.


Notes

Notes

The “Notes” box allows you to write notes for communication purposes. The notes field is not mandatory. Once any changes have been made it will automatically save the changes to the notes box.

Team

The Team field will be available on the dynamic share, bulk share, scheduled bulk shares, subscribed queues, and subscribe form views. You can select a team that was created on the Multi Team Administration module. This is really just for cosmetic / organizational purposes.

You can read more about Multi Team Administration here.


Advanced

This tab is only available if the “Advanced” box is checked

Distribute Bulk Share Workload


When selected, the “Distribute Bulk Share Workload” option will schedule a bulk share job to run on a different node of your instance than the one the last bulk share ran on. This will help to optimize performance so all your bulk shares are not run on the same node of your instance.

Scheduled Job Priority

You can specify the priority of the scheduled job that executes the bulk share under the advanced tab in the bulk share form.

By default, scheduled jobs are created with a priority of 100 and specifying a value here allows you to override this with a different value. If the Advanced section is selected and no value is specified, the bulk share will run with the default value of 100.

Node To Run On

C Release

This option allows you specify what node you want your bulk share to run on. Once selected the Bulk Share will only process on this specified node. You may want to use this option to distribute the Perspectium processing across several nodes.

You can select the node that you would like this Bulk Share to run on via a reference. This is a reference to the table “sys_cluster_state” on the ServiceNow instance. You will need “admin” access (or a custom ACL rule) in order to see and set this value.

Limit the Audit Data Shared

C Release
The “Enable Audit Limit” option allows you to specify the number of audit records that are shared per record in a Bulk Share when you have “Include Audit Log” checked.

After selecting this option, you can then specify the max number of audit records to share per record. The default value is 100 audit records shared per record. It will grab the most recent audit records first.

Note: This option is only available if you have “Include Audit Log” selected on the Additional Settings tab.

You may want to use this if you have a Bulk Share on Incident with Audit Log checked and you want to avoid sharing out the entire audit history of each record every interval.

Enable Debug Logging

D Release

An option available in both bulk and dynamic sharing that will enable debug logging for a particular share configuration. This will take precedence over the global debug logging property under Control and Configuration > Properties.


UI Actions

Create/Run Another Bulk Share like this

At the bottom of the bulk share form there are options to “Create a new bulk share like this one” and “Run another bulk share like this one”

The Create a new bulk share option creates but does NOT execute another bulk share with the same configuration.

The Run another bulk share creates and executes another bulk share with the same configuration.

Preview

On a bulk share that has yet to be executed, you will see the “Preview” related link option:



When selected, this option will take you to the Bulk Share Preview page to give you a preview of the bulk share before it is executed:

The preview page will display the following information:

  • The number of records that will be shared from the selected table.
  • If “Include attachments” is selected, an estimate on the number of attachment records that will be shared. This will be the combined total of both sys_attachment and sys_attachment_doc records since both tables are required to re-construct attachments when replicating.
  • If “Include journal fields” is selected, an estimate on the number of sys_journal_field records that will be shared.
  • If “Include audit log” is selected, an estimate on the number of sys_audit records that will be shared.
  • A trend chart showing a monthly breakdown of when records were created in the selected table in the last 12 months.

Estimates are based on sampling the 10 latest records from the selected table and seeing how many entries are in the related table (i.e. sys_journal_field, sys_audit, etc.) to calculate an estimated total for all

The trend chart requires the Report Charting v2 plugin which is included and enabled by default in all ServiceNow instances running Eureka and newer build versions.

Outbound Messages

On each dynamic and bulk share, a related list will display all messages in the Outbound queue for the share:

The “Delete this share's messages” list button is provided if you want to quickly delete all messages related to this share in the Outbound queue. Note that if the messages are in a “Sent” state, they will have already been sent to the Perspectium MBS server so deleting them here will only remove the entries from the Outbound queue.

This option requires access to the “parent” global variable. If the variable is undefined (changed by another business rule) then this option will not be displayed.

Records Processed

The “Records Processed” entry reflects the complete total number of records processed and bulk shared out, including the number of auxiliary records shared if options such as “Include attachments”, “Include journal field” are selected. To see a breakdown of records per table (i.e. how many sys_attachment, sys_journal_field, etc. records), the “Records Processed” related list at the bottom of the form is provided to show these details:

D Release

The entry will update as the bulk share runs to display progress. Its value will include current total number of records processed including auxiliary records from attachments, journal fields, etc.

replicator_snc_bulk_share.txt · Last modified: 2018/07/27 16:57 by paul