Saturday, March 26, 2016

Sending and Receiving MTOM-encoded binary data

PeopleSoft supports the MTOM protocol for sending and receiving binary data using service operations. While you can send and receive binary data using SOAP, doing so requires that you Base64-encode the data, which can increase message size by 33 percent or more. The MTOM protocol enables you to send and receive binary data in its original binary form, without any increase in size due to encoding.
For sending or receiving MTOM-encoded binary data, we have to use message segments to store the data. The SegmentContentType property of the message object is used to set or read the content type of each message segment.

Following is a test that I did to send a XML file as an attachment in a SOAP message and then read the attachment that is sent by the 3rd party system that I am interacting with. I am running PT 8.53.22.

Sending:

Request message is as shown below. For this test I am storing this in a html object called as MY_MESSAGE but this can be generated dynamically as needed using SOAPDoc or XMLDoc classes. The request message defined on the service operation is nonrowset based. 

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
<submitLargeDocument>        
    <Document>  
         <PsftXopInclude SegmentNumber='1'/> 
    </Document>      
</submitLargeDocument>
</soapenv:Body>
</soapenv:Envelope>

document.xml is my payload which I have created in a different routine and I am just using it here. Its just like creating any other XML file in PeopleSoft. Payload can be a binary file like a pdf or a image file.
Sender node is my default local node and receiving node is the delivered WSDL_NODE. You can always create a custom receiving node if needed. Routing is using local gateway and HTTPTARGET connector. Under routing, connector properties content-type is set to text/xml as my payload is a xml file, HTTPPROPERTY MTOM is set to Y, METHOD is POST, SOAPUpContent is set to N as I have already built the SOAP wrapper in my html object above. If you need IB to create the wrapper then set this property to Y. Provided PRIMARYURL to destination 3rd party application. Took all other defaults.

PeopleCode:

&str = GetHTMLText(HTML.MY_MESSAGE);
&requestXMLDoc = CreateXmlDoc();
&ret = &requestXMLDoc.ParseXmlString(&str);

&request = CreateMessage(Operation.SEND_OPERATION);
&request.SetXmlDoc(&requestXMLDoc);

&MTOMFile = GetFile("C:\temp\document.xml", "R", %FilePath_Absolute);
If &MTOMFile.IsOpen Then   
   &theBase64encodedString = &MTOMFile.GetBase64StringFromBinary();   
   &MTOMFile.Close();
End-If;

&request.CreateNextSegment();

If (&request.SetContentString(&theBase64encodedString)) Then   
    &request.SegmentContentType = "application/xml";  
    &request.SegmentContentTransfer = %ContentTransfer_Binary;
End-If;

&response = %IntBroker.SyncRequest(&request);

Receiving:

Response message defined on the service operation is a non-rowset based message. Sender node is the default local node and receiving node is WSDL_NODE. Using local gateway and HTTPTARGET connector. Setting HEADER properties Content-Type to text/xml as the response attachment that I am receiving is a xml file, sendUncompressed is Y, HTTPPROPERTY Method is POST and SOAPUpContent is Y and finally the PRIMARYURL to the 3rd party service.

On the weblogic webserver, in the integrationGateway.properties file enable the MTOM Listening Connectors. 
ig.MTOM.enablePeopleSoftServiceListeningConnector=true
ig.MTOM.enableHttpListeningConnector=true

Bounce the webserver after making this change.

PeopleCode:

This is pretty straightforward. Once the request is made, read the response and parse out the document.

&response = %IntBroker.SyncRequest(&request);
&responseXMLDoc = &response.GetXMLDoc();

If (&response.ResponseStatus = 0) Then
     &dataNode = &responseXMLDoc.DocumentElement.GetElementByTagName("data");
     &theData = &dataNode [1].GetCDataValues();
     &responsestr = &theData.Shift();
End-If;

The &responsestr string variable will have the response SOAP envelope as well as the attachment separated by message segments as shown below. Parsed it out using string functions.

<?xml version="1.0"?>
<data psnonxml="Yes">
  <![CDATA[
------=_Part_624_1792156364.1458048147094
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: 8bit
Content-ID: <soap.xml@xfire.codehaus.org>

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
<soap:Body>
.......
.......
.......
</soap:Body>
</soap:Envelope>
------=_Part_624_1792156364.1458048147094--

------=_Part_724_1792156364.1458048147094
Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"
Content-Transfer-Encoding: 8bit
Content-ID: <soap.xml@xfire.codehaus.org>

<Document>
........
........
........
</Document>

------=_Part_724_1792156364.1458048147094--

]]>
</data>


40 comments:

  1. Is this working using Synchronous?, thanks for posting anyway :)

    ReplyDelete
    Replies
    1. Yes it works, are you running into any issues?

      Delete
    2. Yes we have issues using both Sync and Async, we are sending binary from PeopleSfot > OSB > Sharepoint, we tried the message using SoapUI and works fine but in PS we are unable to send. is it necessary to use Service Operation? also oracle mention that MTOM will work only in Asynchronous, i hope you can guide.

      Delete
    3. I am using SyncRequest method so synchronous works. Here is the Peoplebooks link I used as a reference. I am integrating with Taleo.
      https://docs.oracle.com/cd/E57990_01/pt853pbh2/eng/pt/tibr/task_SendingandReceivingBinaryData-727e75.html#topofpage

      Delete
    4. Thank you, is there a way i could communicate with you?

      Delete
    5. This comment has been removed by the author.

      Delete
    6. Thank you so much! we will contact you.

      Delete
    7. Were you able to get Synchronous working?

      Delete
  2. This comment has been removed by the author.

    ReplyDelete
  3. We are running PT8.55.19 and trying what you wrote above using Sync and not Async but the IB doesn't appear to be sending the response to allow peoplecode to parse it out. Keeps saying the message is null.
    PeopleSoft/Oracle's official response is that "IB cannot handle MTOM messages Synchronously "
    Can you help in anyway?
    Thanks in advanced,
    Ed

    ReplyDelete
  4. Ed, Yes I have it working using %IntBroker.SyncRequest as mentioned in my post. I tried a quick test in PT 8.56.03 and it works in that version too. I agree that the latest version of PeopleBooks does state that it will work only for async, but in my case it works using Sync.

    ReplyDelete
  5. Not sure if I'm doing something wrong here but the IB msglog shows that I'm getting the MTOM data but when I try and read the data in via peoplecode "&xmlres = &msgResponse.GetXmlDoc();" it says that it's NULL.
    Any suggestions?
    Thanks,
    Ed

    ReplyDelete
  6. set user exception flag on the routing to get fault response.
    Then you can do the following to get more details.
    &Msg_Set = &msgResponse.IBException.MessageSetNumber;
    &Msg_Number = &msgResponse.IBException.MessageNumber;
    &Msg_Descr = &msgResponse.IBException.ToString();

    ReplyDelete
  7. PS_dips, I did what you said but I'm not getting anything.
    This is what we get:
    Response IBException.ToString - GetMessageText: No default message.

    Also I turned on tracing and this is what it show for GetXMLDoc:
    PSAPPSRV.3672 (3798) 1-394420 10.53.08 0.000000 25: &xmlres = &msgResponse.GetXmlDoc();
    PSAPPSRV.3672 (3798) 1-394421 10.53.08 0.000000 Message
    PSAPPSRV.3672 (3798) 1-394422 10.53.08 0.000000 Str[9]=GetXmlDoc
    PSAPPSRV.3672 (3798) 1-394423 10.53.08 0.000000 Message
    PSAPPSRV.3672 (3798) 1-394424 10.53.08 0.000000 Object=NULL

    Got anything else?
    Again Thanks in advanced.
    Ed

    ReplyDelete
    Replies
    1. Anything in the errorLog.html file under PSIGW?

      Delete
    2. Hi Ed, I have the same Null response for a syncronous service, could you post your Routing HTTP Connector Configuration ?

      If is there any way to contact you I would appreciate it.

      Thanks,

      Delete
  8. Hi may I know what are the variable types for the receiving peoplecode? Thanks in advance!

    ReplyDelete
    Replies
    1. not sure what you are looking for. Can you please elaborate?

      Delete
    2. For example the variable "&response" is it a local xmldoc?

      Delete
    3. &response is of type message and &requestXMLDoc is XMLDoc.

      Delete
  9. Hi,
    iam trying to send attachment to Peoplesoft through SOAP web service. After creating the necessary setup and config in poeplesoft, i tried to send data from SOAP UI. iam getting error as 'MTOMUtility:getInputStringForMtomEnabledConnector : unable to extract MimeMultipart'. Any Idea ?

    Thanks in advance.

    ReplyDelete
    Replies
    1. So in your case PeopleSoft is hosting the service, is it? What I have tried is consuming a service in PeopleSoft which is hosted by a 3rd party. If you don't mind post a sample of your request and let me see if I can somehow simulate it.

      Delete
  10. Dear,

    I followed the guidelines of your Blog that were very useful, however I am getting the following message of return, can you tell what may be occurring?

    PS: I followed the PeopleCode logic of receipt that you exemplified, and I'm also getting the IB return, and both return the same message:

    "MTOM: XML Processing Does Not Find PsftXopInclude Elements."

    ReplyDelete
    Replies
    1. I would just verify the setup/configuration and also write the request message (&requestXMLDoc.GenFormattedXmlString()) to file before sending it in order to check that it is working as desired.

      Delete

    2. Connector Properties:
      Header> Content Type> text / xml
      Header> SoapAction> ""
      Header> sendUncompressed> Y
      HTTPPROPERTY> MTOM> Y
      HTTPPROPERTY> Method> POST
      HTTPPROPERTY> SoapUpContent> Y
      Primary URL> URL> "Third Party URL."

      Delete
    3. XML REQUEST:





      2019-07-10T14:31:47.000Z
      2019-07-10T14:33:47.000Z


      329545
      z8CYxHoHJV4Ol0DUPkLoBVK7uTk=
      8As1r5reKEOCbl1eFDKg9w==
      2019-07-10T14:31:47.000Z







      195498
      61858
      187427

      329545
      4
      51016



      Delete
    4. XML REQUEST:

      < SOAP-ENV:Envelope xmlns:SOAP-ENC="http://schemas.xmlsoap.org/soap/encoding/" xmlns:SOAP-ENV="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ser="http://services.soc.age.com/">
      < SOAP-ENV:Header>
      < wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
      < wsu:Timestamp wsu:Id="TS-">
      < wsu:Created>2019-07-10T14:31:47.000Z< /wsu:Created>
      < wsu:Expires>2019-07-10T14:33:47.000Z< /wsu:Expires>
      < /wsu:Timestamp>
      < wsse:UsernameToken wsu:Id="UsernameToken-">
      < wsse:Username>329545< /wsse:Username>
      < wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordDigest">z8CYxHoHJV4Ol0DUPkLoBVK7uTk=< /wsse:Password>
      < wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">8As1r5reKEOCbl1eFDKg9w==< /wsse:Nonce>
      < wsu:Created>2019-07-10T14:31:47.000Z< /wsu:Created>
      < /wsse:UsernameToken>
      < /wsse:Security>
      < /SOAP-ENV:Header>
      < SOAP-ENV:Body>
      < ser:downloadArquivosGedPorLote>
      < downloadPorLote>
      < identificacaoWsVo>
      < codigoEmpresaPrincipal>195498< /codigoEmpresaPrincipal>
      < codigoResponsavel>61858< /codigoResponsavel>
      < codigoUsuario>187427< /codigoUsuario>
      < /identificacaoWsVo>
      < codigoEmpresa>329545< /codigoEmpresa>
      < codigosArquivosGed>4< /codigosArquivosGed>
      < codigoGed>51016< /codigoGed>
      < /downloadPorLote>
      < /ser:downloadArquivosGedPorLote>
      < /SOAP-ENV:Body>
      < /SOAP-ENV:Envelope>

      Delete
    5. In your request message doesn't look like you are sending a file or any binary data, so I don't think you need to use MTOM, or PsftXopInclude declaration. If this is not true send me a snippet of your request peoplecode, where you are trying to add binary data info to your request message.

      Delete
  11. I'm not sending binary data, I'm expecting a response on binary data, I trigger a request message to receive a response with a file.

    ReplyDelete
  12. Could you send your Connector Properties parameters to the Service that is using MTOM?
    You can delete PRIMARYURL for security.

    ReplyDelete
    Replies
    1. In your case following is all you need - I am doing the same for retrieving an attachment (XML file) in my response.
      Connector Properties:
      Header> Content Type> text/xml
      Header> sendUncompressed> Y
      HTTPPROPERTY> Method> POST
      HTTPPROPERTY> SoapUpContent> Y
      Primary URL> URL> "Third Party URL."
      My request and response messages were created when I imported the WSDL into PeopleSoft. These are standard messages non-rowset based. Part message option is not selected.

      The &responsestr string variable will have the response SOAP envelope as well as the attachment separated by message segments as detailed in the post above. Parsed it out using string functions.

      Delete

    2. I applied the parameterizations but I did not succeed.
      When I replicate the connector properties that reported the & MessageResponse.GetXmlDoc () function returns NULL, and when I add the connector properties to the Header option> MTOM> Y, it returns the message that brought me here.
      "MTOM: XML Processing Does Not Find PsftXopInclude Elements"

      I believe that the third system needs to parameterize so that the XML response is sent by a segmented message.

      Delete
  13. Another question, obligatorily the third, should we return the XML response in segments?
    Or Peoplesoft supports MTOM response in a single part (segment).

    ReplyDelete
  14. Are you aware if there a size limitation for receiving a binary? I was able to get this working with a teeny test pdf (~3kb), but as soon as I try requesting a slightly-larger pdf (~50kb), it seems like I don't get any response. Message monitoring shows the status as Done, and I see my request xml...but there is no XML response.

    If I try the Service Operation Tester, it works with the 3kb file, but I get a "First operand of . is NULL, so cannot access member GenFormattedXmlString" with the 50kb file.

    I also don't see any errors in the errorLog.html...SOAP UI works for all files. File size is the only thing that comes to mind, but 50kb still seems pretty small for a limit.

    ReplyDelete
    Replies
    1. Unfortunately in my case the files are lesser than 3kb so I don't know if that is a limitation. I have tested larger files via SOPAUI and Postman but I am not sure if this is a limitation in PeopleSoft. If I can somehow test it I will let you know.

      Delete
    2. I was able to run a quick test and download a 3MB file, no issues via service operation tester.

      Delete
  15. Is it possible to send a large zip files using MTOM for REST based synchronous message in JSON format in Peoplesoft ? we are ion tools 8.55.14

    ReplyDelete
    Replies
    1. I have not tried this but based on peoplebooks looks like MTOM is only applicable to XML based SOAP messages.

      Delete
  16. Those guidelines additionally worked to become a good way to recognize that other people online have the identical fervor like mine to grasp a great deal more around this condition. and I could assume you are an expert on this subject. Same as your blog i found another one Oracle Taleo .Actually I was looking for the same information on internet for Oracle Taleo and came across your blog. I am impressed by the information that you have on this blog. Thanks a million and please keep up the gratifying work.

    ReplyDelete
  17. I tried the above approach of PsftXOPInclude in SOAP message for synchronous integrations but was not able to complete the transaction. Please suggest anything need to be taken care while sending the attachment to 3rd party system using synchronous integration.

    ReplyDelete