Omni-Gen™ Operation and Management Guide > Work Orders

Work Orders

Topics:

A work order processes data for a given subject through a series of work order items. A work order is associated with a transaction ID. Records marked with the work order transaction ID will be included in subsequent processing.

Work Order Types

The following list describes the types of work orders.

  • BULK. In a BULK work order, data is loaded into Omni-Gen from the ramp tables for a given subject. This work order is created when an os_ramp_control record is processed or when an Omni Interface Document (OID) is loaded from the File Input Location.
  • IMMEDIATE. In an IMMEDIATE work order, data is loaded into Omni-Gen from an Omni Interface Document using the REST API service.

    The following syntax shows the REST API service:

    /server/processInstance
  • SETUP_DELETE_STALE/DELETE_STALE. A SETUP_DELETE_STALE work order is created at the Purge Inactive Time (Runtime setting), an interval specified as a cron expression. It will only become ACTIVE if there are no other work orders in progress. When a SETUP_DELETE_STALE work order becomes ACTIVE, a DELETE_STALE work order will be created for each subject in the model that is populated with data.

    In a DELETE_STALE work order, all INACTIVE records of a given subject that were last updated before the Purge Inactive Age are deleted. This includes source, instance, and history reference records. The Purge Inactive Age is a Runtime setting to specify the number of days after which INACTIVE records will be deleted.

  • RESET_SUBJECT. In a RESET_SUBJECT work order, all data for a given subject is purged from the system. This work order is created using the Reset button on the Deployment screen in the console. It will only become ACTIVE if there are no other work orders in progress. All tables associated with the subject are truncated, including ramp, source, instance, and history tables. Other records such as overrides and code-field references are deleted.

Work Order States

The following list describes the work order states.

  • NEW. The work order is created, but not ready to process until all work order items are added. This state is managed in general code.
  • READY. The work order is ready to be scheduled. Generally, a subject-based work order will remain in READY state until prior work orders of this subject are complete. This state is managed in general code.
  • SCHEDULED. This is a temporary state, set by the scheduler, indicating the work order will be executed immediately.
  • ACTIVE. The work order is actively running. The first step of every work order is the START work order item, which moves the work order to ACTIVE state.
  • PAUSED. The Pause console menu item on the Work Orders screen can be used to pause an ACTIVE work order. Note that the work order will not go into PAUSED state until the currently executing work order item can safely come to a stop. A PAUSED work order must be resumed using the Resume console menu item. The work order will then resume processing from the last active work order item.
  • COMPLETE. The work order has finished executing. The result can be PASS or FAIL. A failed work order must be restarted or ignored to proceed. The Restart console menu item on the Work Orders screen can be used to restart work order processing beginning with the failed work order item. The Ignore console menu item will set the result of the work order to IGNORE and allow the next READY work order of this subject to be scheduled.

Work Order Items

Topics:

This section lists and describes the work order items.

START

In this step, the work order state moves from SCHEDULED to ACTIVE. This is the first item in every work order.

RAMP_TO_MODEL

The following processes describe the steps involved in the RAMP_TO_MODEL work order item, which is invoked for non-mastered, non-cleansed subjects.

  • Ramp Processing. In this step, data of a subject is copied from ramp tables to instance tables based on a ramp batch ID and optionally, a source name. Each subject collection is copied independently on its own thread. The instance record and/or its transaction ID and Omni-Gen modified date are updated if the ramp data differs from the source data (business fields only), if there is a change in the status of the record, or if the ChangeDetection IGNORE ramp load option is specified. The ramp load policy can cause a record status change. The default ramp load policy is UPSERT. The default record status is ACTIVE and it can become INACTIVE under REPLACE or DELETE ramp load policies.

    For more information on the Ramp Load policy, see the Omni-Gen Integration Edition Relational OnRamp User's Guide.

  • Code Processing. All unique source codes are collected in memory during ramp processing. checkForMissingCodes determines which of those codes are new. A ramp batch is created for the new codes and a SourceCodeSet BULK work order is submitted for processing.

    A list of codes and the fields which reference those codes are also collected in memory during ramp processing. checkForMissingCodeXRefs identifies the new code-field references and adds them to the os_source_code_xref table.

    Note: When a SourceInstanceID is trimmed of leading and trailing whitespace, a warning message is logged. If the trimmed SourceInstanceID contains a space or a colon, the record will not be moved to the source table and an error is logged.

    Parent records are not auto-generated for orphan records.

HISTORY

The following steps will be included if the captureHistory attribute in the instance is true.

  • HISTORY_INSTANCE. All instance records in the current transaction are copied to the corresponding history table. A snapshot of the entire record is recorded in history. Each entity is copied independently on its own thread.

CDC_RECORD

This step runs only if the Enable/Disable CDC Notification runtime setting is true.

The os_cdc_change table is populated with the XML representation (Omni Interface Document) of instance records in the current transaction for which a CDC subscription exists.

STOP

In this step, the status, result type, and end date of the work order are updated. This is the last item in every work order.

Work Order Automation

Topics:

The following is a high-level summary of how to automate a sequential set of batches to process:

  1. The integrator creates a series of batches to process with batch control records.
  2. Using the batch IDs, create a work order automation JSON document that includes entries for each batch to process.
  3. Copy and paste the JSON document into the work order Swagger API.
  4. Click Try it out to test the work order.
  5. View the automation executing on the console work order screen.

Executing the Automation

The automation job is a work order of its own. It is composed of a series of work order items that may also create other work orders. With this in mind, you are able to track the progress of the automation from the work order processing screen.

Each automation work order item will be attempted until one fails or the work is complete. This is no different than regular work order processing.

To support automation, the following functions were added:

  • START_BATCH. The start batch function takes batch the information and starts the batch as a BULK work order. In the work order processing screen, the batch will be visible as normal and the batch parameters are provided in the jsonContext aggregate.
  • WAIT_FOR_ALL_COMPLETE. Using the automation work order as a starting point, this function polls the work order table looking for any new work orders to complete. It will wait a configurable number of iterations with configurable n second pause between iterations. If any subsequent work order fails, then the automation work order will also fail. This function will also wait for an additional duration, making sure no other work comes into the system. This is also a configurable wait. This combination is intended to support not only the immediately created batch, but also any work orders it may create.

Creating an Automation

You are required to use a text editor and a Swagger interface for automation purposes.

Automation creates a work order and corresponding work order items from the provided JSON document. The tags match the role and purpose of the tags in their respective tables.

The following syntax shows a sample automation JSON document.

{
    "sourceType":"USER_DEFINED",  <-- Required.
    "subject":"dynamic", <-- Required, value up to user, should not be the name of a subject.
    "sourceName":"devops", <-- Required, value up to user.
    "omniSystemType":"SERVER", <-- Only SERVER supported at the moment.
    "workOrderItem":[
        {
            "workOrderItemName":"START_BATCH",  <-- Required as is.
            "workOrderItemNameExtension":"car-batch", <-- Used to differentiate different batches.
            "processorOrder":1, <-- Required and must be sequential across work order items.
            "jsonContext":{ <-- These are the parameters passed in to starting the batch.
                "subject":"Car",
                "batchId":"abb5bca6-0565-4344-b85a-fa975456ec11",
                "sourceSubType":"MERGE_PRESERVE_ON_NULL",
                "source" : "TestSource”
            } 
        },
        {
            "workOrderItemName":"WAIT_FOR_ALL_COMPLETE", <-- This task has an 1 min idle time, be patient.            
            "processorOrder":2 <-- Make sure to increment this.
        },
        {
            "workOrderItemName":"START_BATCH",
            "workOrderItemNameExtension":"pet-batch",
            "processorOrder":3,
            "jsonContext":{
                "subject":"Pet",
                "batchId":"e24f889a-bc4d-473b-85e3-3cd97d06dd42",
                "sourceSubType":"MERGE_PRESERVE_ON_NULL",
                "source" : "TestSource"
            } 
        },
        {
            "workOrderItemName":"WAIT_FOR_ALL_COMPLETE",            
            "processorOrder":4
        }
    ]
}

Starting the Automated Work Order (Submitting the Job)

To start the automated work order:

  1. Use Swagger on the controller. For example:
    https://<hostname>:9500/swagger-ui.html#!/WorkOrder/postWorkOrderUsingPOST
  2. Copy and paste the JSON document into the work order Swagger API.
  3. Click Try it out.

    View the automation executing on the console work order screen.