Attachment Handling

An overview of Attachment handling in Unifi.

Outbound

The mechanism for outbound attachment transmission has the following steps:

• Detect attachment creation (insert to the Attachments table) and generate Bonded Attachment (BA) records (in the Ready state)

• Identify any Message Type(s) which have attachment_added = true, and if found, initiate the message(s) immediately

  • If no messages have attachment_added = true, we take no further action here; the BA records in the Ready state will be picked up by subsequent outbound message(s)

• When a message is being sent, one or more BA records will be selected for inclusion in the message and these will be set to the Sending state.

• When the transaction enters the Sending state the outbound payload will be constructed and placeholders will be added for the selected attachments; the BA records will be set to the Sent state

• Just before the request is sent, the placeholders will be substituted with the real attachment data encoded in Base64 format

The following sections describe the steps above in more detail.

Attachment Detection

The Attachments table [sys_attachment] has a business rule [ws] Unifi attachments which detects newly inserted attachments (it does not run for x_snd_eb tables). This after insert rule creates Bonded Attachment (BA) records as required by calling AttachmentHandler.createBondAttachment(current).

For every bond that references the attachment's target record, a BA record is created in the Ready state.

If any BA records were created, Message.processOutbound is called against the target record, with an option that indicates that this was as a result of an attachment being added.

Message Selection

A standard scan of matching messages is initiated (Active, Create/Update, Outbound/Bidirectional, for the target table) but also limited to only messages with attachment_added = true. If any messages are found, they are immediately triggered to be sent.

If no messages have attachment_added = true then the standard message processing will find the 'Ready' bonded attachments the next time a suitable message is triggered.

Message Sending / Attachment Selection

When a message is being sent, a check is made for any BA records in the Ready or Rejected states. This is performed in AttachmentHandler#getAttachmentsToSend, called from Message#send

For each one found, the state will be set to Failed, Sending or left unchanged.

• Failed means we can never send the attachment (e.g. attachments inhibited, attachment too large, or other reason)

• Sending means we can include the attachment in this message; the BA will be linked to the current transaction

• No change means the attachment won't be sent in this message, but is still a candidate for inclusion in future messages

Transaction Sending

When a transaction enters the Sending state, the Transaction#send method will create a new request and call Message#generatePayload. This method processes the Stage to Request (Outbound) script, if present, and if that returns no payload then the XML template will be processed.

The XML Template of the Message can be set up to include a UI macro <snd_eb_attachments> which finds each Bonded Attachment in the Sending state, embeds an attachment placeholder for each one into the payload, and sets the BA to the Sent state (the UI Macro is part of "[ws] Unifi Standard API" application).

This UI Macro uses AttachmentSender to insert <x-attachment-data> elements. There will be one element per attachment returned and it will contain the sys_id of sys_attachment to be sent. AttachmentSender is essentially an iterator over all BA records for a specific transaction where the state is Sending. It marks each BA as Sent once it has been selected

• If no UI macro is included, the BA will be left in the Sending state against the current transaction but will never be included in the payload and never sent

HTTP Request Transmission

Currently all attachments are carried as embedded Base64 data within the body of a Message. For outbound attachments, the data is injected into the payload at HttpRequest send time. The position for inclusion of attachments into the message body is marked using XML element placeholders.

The conversion of these placeholder elements is done in HttpRequest#send which calls the static method AttachmentHandler.injectAttachments. This replaces the <x-attachment-data> elements with Base64 data. This behaviour is hard-coded.

The same substitution is performed regardless of whether the outbound message is SOAP or REST. While the substitution string looks like an XML element, the substitution is done using regex and so we could use these strings to indicate the position of Base64 attachment data to be embedded into an outbound REST message.

Multipart Form Data

Multipart form data is not naturally supported OOTB in native ServiceNow. However, we do have a way of supporting it in Unifi. You can read more about that on the following page:

Multipart Form Data

Inbound

Unifi processes attachments embedded in inbound messages as follows:

• The inbound request is identified as a specific Message and the extract_attachments script is called

• The script analyses the inbound payload, removes the attachment data, and creates attachments linked to the inbound HTTPRequest record

• We establish the transaction and stage and then create a Bonded Attachment record for each attachment extracted

• If processing of the inbound message succeeds (process_state is Accepted) the BA records are linked to the Bond and set to Complete, the attachments moved from the HTTPRequest record to the target record, and the bond updated to reflect the attachments in this transaction

• The newly added Bonded Attachments are copied to every other bond that the target record is associated with, to allow those bonds to send copies of the attachments to their bonded systems

The following sections describe some steps above in more detail.

Extract Attachment

The snd_eb_api_utils.extractAttachments script expects an XML document or string as input and searches for an element called eb:Attachments.

This is expected to contain a sequence of eb:Attachment elements which each have eb:FileName, eb:MimeCode and eb:Data child nodes. The text content of these are read and stored in a result object. The text content of the Data element is removed from the input XML (so when it is passed back we can log the inbound request_payload without the attachment data).

The extracted information is passed to AttachmentHandler.saveAttachment (with the raw Base64 data already decoded). This method creates the attachment and links it to the HTTPRequest record (note that this will not trigger the after insert rule on the Attachments table, since HTTPRequest is an x_snd_eb type table). The method is expected to return an x-attachment-data tag containing the sys_id of the attachment created as an attribute. This tag is inserted into the original XML.

The payload is returned with x-attachment-data placeholder tags substituted for the raw attachment data.

Create Inbound Attachments

The AttachmentHandler.createInboundAttachments script expects to process a request_payload which has x-attachment-data placeholders embedded within. The sys_ids of these placeholders are extracted using regex search, and for each sys_id we call AttachmentHandler.addToTransaction, which creates a Bonded Attachment in the Ready state (There will be no Bond specified on the BA at this point).

Processing of x-attachment-data placeholder tags is hard-coded into Unifi.