How to Poll for Large Response Payloads

If you're building a polling integration that has response payloads larger than 5MB then you'll need to save them as attachments and adapt your response processing.

This documentation is specific to handling large response payloads in a Poller integration.

Sometimes a polling integration that is fetching data from another system is required to handle response payloads larger than the 5MB limit imposed by ServiceNow. The normal setup for a Poll Processor includes the response payload being returned as a string in the Request script. Here we look at an alternative approach which avoids handling the response payload as a string and so avoids the 5MB string limit.

Alternative approach for response payloads > 5MB:

1. Make the request (using RESTMessageV2()).

2. Save the response as an attachment on a record (using saveResponseBodyAsAttachment()).

3. Pass the attachment sys_id through (using getResponseAttachmentSysid()).

4. Fetch it and do something with the newly generated attachment - passing the stream of the attachment on to be processed however required, e.g. text/xml processing.

Don't pass the response body from the request script to the response script. Use saveResponseBodyAsAttachment() and getResponseAttachmentSysid() instead.

Example Use Case

The following example is taken from our Incident Attachment Poller Guide.

Request Script

// Process the request e.g. by executing a web service and returning the response
(function (poll_request, poller, connection, params) {

    var request, response;

    request = new sn_ws.RESTMessageV2();
    request.setHttpMethod('GET');
    request.setEndpoint(poll_request.endpoint_url);
    request.setRequestHeader('Content-Type','application/json');
    request.setBasicAuth(connection.getBasicAuthUser(), connection.getBasicAuthPassword());

    // Save the response as a new attachment (on the poll request)
    request.saveResponseBodyAsAttachment(
      poll_request.getTableName(), poll_request.sys_id, params.attachment.file_name
    );

    response = request.execute();

    poll_request.response_code   = response.getStatusCode();
    poll_request.response_status = response.getErrorMessage();
    if (response.haveError()) {
      throw '\nResponse Code: ' + response.getErrorCode() + '\nResponse error: ' + response.getErrorMessage();
    }

    // Return the sys_id of the saved attachment
    answer = response.getResponseAttachmentSysid() + '';

})(poll_request, poller, connection, params);

Request script: This script uses the ServiceNow RESTMessageV2() web service to make a REST call to the endpoint url created in the Setup script. It returns the body of the request as an attachment who's sys_id it passes to the Response script

saveResponseBodyAsAttachment(): This method takes three parameters:

tableName - the table that contains the record you want to attach the saved file to.

recordSysId - the sys_id of the record you want to attach the saved file to.

fileName - the file name to give to the saved file.

request.execute(): The response object returned by request.execute() provides a method called getResponseAttachmentSysid().

getResponseAttachmentSysid(): This method returns the sys_id of the attachment generated by the REST call.

Response Script

// Process the response returned by the request script
// The 'answer' variable from the request script is passed in here as the 'response' parameter 
(function (poll_request, poller, response, params) {

    // Establish the environment
    var poll_helper = new x_snd_eb.PollHelper(poll_request);
    var info = [
        'Incident : ' + params.int_ref,
        '- Attachment file name: ' + params.attachment.file_name,
        '- Attachment id: ' + response
    ];

    // The response should be the sys_id of the created attachment
    params.attachment.data = '<x-attachment-data sys_id="' + response + '" />';

    // Build the payload for Unifi
    var payload = {
        sys_id : params.ext_ref,
        correlation_id : params.int_ref,
        attachment : params.attachment
    };

    // Submit the message into Unifi
    poll_helper.processInbound({
        message_name : 'AddAttachmentInbound',
        payload : JSON.stringify(payload)
    });

    poll_request.response_status = info.join('\n');

})(poll_request, poller, response, params);

Response script: This script sets up some objects to help us; this includes the essential PollHelper() function (which we initialise from the poll_request) along with the info [] array.

After that it sets params.attachment.data to the sys_id of the created attachment, setting up a payload object and submitting it to Unifi by calling the processInbound() method.

After processing the single result, it is logged to the Response status field of the Poll Request.

In our example we've taken the inbound attachment, built a payload and passed it to Unifi to process.