PeopleSoft web service response namespace issue

Running PT 8.53.x, wrote a simple custom web service "Hello World". Service works fine but saw one issue where the namespace returned by the response message is not defined in the WSDL. So any other 3rd party solution like a .Net program when they consume and execute this service it fails to receive the response due to the unidentified namespace.

Request 

<soapenv:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsa="http://schemas.xmlsoap.org/ws/2003/03/addressing/" xmlns:xsd="http://www.w3.org/2001/XMLSchema/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance/">
  <soapenv:Header xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
  </soapenv:Header>
  <soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <HelloWorld xmlns="http://xmlns.oracle.com/Enterprise/Tools/schemas/HELLOWORLD.V1"></HelloWorld>
  </soapenv:Body>
</soapenv:Envelope>

Response via PeopleSoft SOAP message template

<soapenv:Envelope xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsa="http://schemas.xmlsoap.org/ws/2003/03/addressing/" xmlns:xsd="http://www.w3.org/2001/XMLSchema/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance/">
  <soapenv:Header xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
  </soapenv:Header>
  <soapenv:Body xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <HelloWorldResponse xmlns="http://xmlns.oracle.com/Enterprise/Tools/schemas/HELLOWORLDRESPONSE.V1">
      <HelloWorldResult>XYZ</HelloWorldResult>
    </HelloWorldResponse>
  </soapenv:Body>
</soapenv:Envelope>

Actual response via SOAPUI or Postman

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Body>
      <HelloWorldResponse xmlns="http://peoplesoft.com/HelloWorldResponseResponse">
         <HelloWorldResult>Success</HelloWorldResult>
      </HelloWorldResponse>
   </soapenv:Body>
</soapenv:Envelope>


As you see the the problem is the namespace value returned for the root tag "HelloWorldResponse".

<HelloWorldResponse xmlns="http://peoplesoft.com/HelloWorldResponseResponse">


PeopleSoft puts the default value (not sure where this is stored)
http://peoplesoft.com/HelloWorldResponseResponse
instead of what is defined in the WSDL which is 
http://xmlns.oracle.com/Enterprise/Tools/schemas/HELLOWORLDRESPONSE.V1

I have a very basic synchronous service operation, with a request and response message and the handler OnRequest peoplecode is doing all the work.

Following was my application package peoplecode initially.

&response = CreateMessage(Operation.HELLOWORLD_SVCOP, %IntBroker_Response);
&xmldata = "<?xml version='1.0'?><HelloWorldResponse/>";
&xmlresponsedoc = CreateXmlDoc(&xmldata);

&HWResponseNode = &xmlNode.AddElement("HelloWorldResult");
&HWResponseNode.NodeValue = "Success";
&response.SetXmlDoc(&xmlresponsedoc);
Return &response;


So when creating the XmlDoc I was not specifying any namespace and I was hoping PeopleSoft will add it based on the schema definition. 

Instead following is what I had to do to correct the issue. (only displaying the change, no change to rest of the code)

&xmlresponsedoc = CreateXmlDoc("");

&docTypeNode = &xmlresponsedoc.CreateDocumentType("", "", "");
&rootNode = &xmlresponsedoc.CreateDocumentElement("HelloWorldResponse", "http://xmlns.oracle.com/Enterprise/Tools/schemas/HELLOWORLDRESSPONSE.V1", &docTypeNode);

&xmlNode = &xmlresponsedoc.DocumentElement;

So I have to use the CreateDocumentElement method of CreateXmlDoc in order to define the namespace. In order to use this method I had to first initialize the &docTypeNode variable by using the CreateDocumentType method.

After making the above change the response comes back correctly.

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
   <soapenv:Body>
      <HelloWorldResponse xmlns="http://xmlns.oracle.com/Enterprise/Tools/schemas/HELLOWORLDRESSPONSE.V1">
         <HelloWorldResult>Success</HelloWorldResult>
      </HelloWorldResponse>
   </soapenv:Body>
</soapenv:Envelope>

Application Engine Parallel Processing (batch)

Parallel processing comes in picture when there is a requirement to process large amount of rows without compromising the performance which might otherwise get impacted greatly with non-parallel based processing. Sharing this concept with an actual example where this concept has been used.

I had written an application engine program which read data via a SQL object, performed some computations and finally wrote the result to an extract file which was sFTP'ed to the vendor. Pretty much run of the mill stuff. The AE was reading from PS_JOB, PS_PERSONAL_DATA, PS_EMAIL_ADDRESSES tables etc via a single single SQL object in a peoplecode step, performing the data manipulation as required and writing to a file. It was sequentially selecting an employee at a time, performing the data manipulations and writing to the file. To process around 3K employees it was taking a little over an hour, so not a lot of data but too much time to process this data-set.

So to solve this issue of performance, this is what I did. 

1. I created a single TAO or temporary table. The fields in this table are essentially the unique list of fields or values that the single SQL object was returning. Did not build the TAO table yet, will do it in a later step.
2. The TAO table has two keys, PROCESS_INSTANCE and EMPLID.
3. Then before the peoplecode step was called added a new step to select from the various HR tables and write to this newly created TAO table. 
4. SQL step looks something like this.

         INSERT INTO %Table(N_MY_TAO)          SELECT %Bind(PROCESS_INSTANCE)         , B.EMPLID         .....        .....

        .....

        FROM PS_JOB A, PS_PERSONAL_DATA B .....

        WHERE ....

    5. After this added a new step to update statistics on the newly populated TAO table.
        %UpdateStats(N_MY_TAO)
    6. In the Peoplecode step replaced the SQL fetch as follows.
        &EESQL = CreateSQL(FetchSQL(SQL.N_MY_SQL), N_MY_AET.PROCESS_INSTANCE);
    7. I had already defined a state record in my AE which has process_instance field in it, so did not have to do anything different with the state record. In the N_MY_SQL SQL object I am selecting all fields FROM %Table(N_MY_TAO) WHERE PROCESS_INSTANCE = :1 

ORDER BY EMPLID. I have some SQL case statements and formatting rules defined in the SQL itself.

8. Added the newly created TAO table under temp tables program properties and provided an instance count of 1. Instance count has to be 1 or more, if its set to 0 then you will see the following message in your AE log and there won't be much of performance improvement. 

WARNING: NO DEDICATED INSTANCES AVAILABLE FOR N_MY_TAO - USING BASE TABLE. (108,544)

9. Under PeopleTools > Utilities > Administration > PeopleTools Options, in my case the Temp Table Instances (Total) and Temp Table Instances (Online) is set to 3. So when I add 1 as the instance count in my AE and then build the TAO table, it will create PS_N_MY_TAO, PS_N_MY_TAO1, 2, 3 and 4 and when the process runs it will use PS_N_MY_TAO4 as the first 3 are used for online processing. 

10. You can add a step at the beginning or end to purge the TAO table like %TruncateTable (%Table(N_MY_TAO)) or better yet use the check-box under program properties "Use Delete for Truncate Table". With this option the TAO table is purged at the beginning of each run.

Addition of a single TAO table enabled the process to complete in 7 mins. Massive improvement.

Native sFTP transmission in PeopleSoft using RSA Keys

I have been using PeopleSoft's URL definition to setup sftp based transmission based on password authentication successfully. Recently was challenged to use sFTP but with RSA keys. Seemed simple at first but then as I dug deeper found out that there are a lot of nuances that one needs to be aware off. So here is what I learnt so far. I am using PT 8.53.26 for this exercise.

At a high level the steps are
1. Create a RSA public/private key pair. I am using PuTTY generator but there are a lot of other tools out there which do the same.
2. Create a URL definition in PeopleSoft and associate the RSA keys with the same.

So lets look at creating the RSA key pair first.

Launch the PuTTY Key Generator and select SSH-2 RSA (default) and then click on the Generate button. Once complete the public key, key fingerprint and key comment values are displayed. Key comment can be updated to some other value or can be left as-is. Put in a key passphrase.
Save the keys by clicking on the "save public key" and "save private key" button respectively.
Use .pub extension for the  public key and .ppk for the private key. Filename can be anything for your choice. So for this discussion we will call this putty-gen.pub and putty-gen.ppk.

The highlighted text in blue in the above picture is the public key in OpenSSH format so copy that and then paste it to a text file. We will name this file sftpuser. No extension just sftpuser. I will explain shortly the reason behind this naming convention.

Provide putty-gen.pub or sftpuser file to the vendor so that they can add it to their sftp server.
Once vendor sets an account for you they will provide the sftp address, port is generally 22 and an userid. This userid should match the filename, so we will consider that the userid created by the external vendor is called sftpuser. Hence the naming convention. When you send the public key to the vendor you wouldn't know the userid so at that point the file can be called anything, the filename is more critical to the PeopleSoft setup than the vendor's setup.

Via the PuTTY gen tool navigate to Conversions > export OpenSSH Key and export the private key to a text file. Name this file sftpuser.ppk

You can always go back in time load the privatekey "putty-gen.ppk" file in PuTTY Generator and update the private key password.

While the vendor is doing their setup and configuration we have following two options in PeopleSoft.
The first one and the preferred one is to import the keys in the Digital Certificates component as this options stores them in the database. The second option is to store the keys on the file server. We will discuss both the options here.

Digital Certificate Option (preferred option)

Navigate to PeopleTools > Security > Security Objects > Digital Certificates and add a new row. Select Type as SSH, provide some value for Alias and then a "Copy", hyperlink should show up. Click on that link and then paste the contents of sftpuser.ppk and sftpuser in the appropriate boxes and click on Save (which is to the bottom-right of the page).

Next navigate to PeopleTools > Utilities > Administration > URLs and create a new URL.

Add in the sftp URL provided by the vendor. It could be something like

sftp://vendorname.com

or  sftp://vendorname.com/~/drop-directory

or sftp://vendorname.com/drop-directory

The ~ indicates that a successful logon takes you to the home directory and from there you can navigate to lets say a sub-directory called as "drop-directory" where you have to drop the files.

Then click on the URL Properties link

Provide the following properties

AUTHTYPE = 1 - PUBLICKEY 

SSHKEYALIAS = alias set on the digital certificates component ( you can do a lookup here and pick the correct value)

USER = sftpuser (userid provided by the vendor)

PASSWORDKEY = private key passphrase (in plain text not encrypted) 

Option 2 – Keys stored on file system

We have to create some directories under the PS_SERVDIR location. So if the call to transmit the files is going to be made from the application server then in case of de-coupled PS_HOME it would be PS_CFG_HOME\appserv\{Domain} and process scheduler it would be PS_CFG_HOME\appserv\prcs\{Domain} and in case of a non de-coupled PS_HOME like the older way (traditional) it would be PS_HOME\appserv\{Domain} and PS_HOME\prcs\appserv\{Domain}

If you do not want to create sub-directories under here then there is some more information in Peoplebooks to create a new environment variable to point to a new location. 

So under PS_SERVDIR create a directory called sshkeys and 2 sub-directories under that called as private and public. The names of these 3 directories cannot change. Copy the sftpuser.ppk file to the private directory and sftpuser file to public directory. The filename naming convention is primarily enforced for this option. 

Now back to within PIA create a new URL definition. The URL properties is what is slightly different from the previous option. 

Provide the following properties

AUTHTYPE = 1 - PUBLICKEY 

PRIVATEKEY = sftpuser.ppk 

PUBLICKEY = sftpuser

USER = sftpuser (userid provided by the vendor)

PASSWORDKEY = private key passphrase (in plain text not encrypted) 

So that's essentially it. 

Following is the basic peoplecode call to transmit the file. 

&returncode = PutAttachment(URL.N_VENDOR, &filename, &fullpathtofile);

where N_VENDOR is the URL definition created above
&filename is the filename as it test.txt

and &fullpathtofile is the complete path as in C:\temp\test.txt

JMS Connectors and Queues

Goal of this exercise is to design a web service solution wherein we are hosting the service and it’s consumed by a SaaS solution. In this POC I am exposing/hosting a simple SOAP based XML web service. 

I am testing this out on PeopleSoft HCM 9.1 running PT 8.53.26.

Transaction data flow is as follows:

• Expose web service which takes in the request message
• Request XML message is written to the request queue on the JMS server
• A listener process reads the request queue and fetches the message
• A process/program processes the request message and populates the result or response XML message on the response queue.
• The web service reads the response queue and sends the response back to the caller

JMS Correlation ID is used as a key to fetch/write the appropriate message from/to the queue.

JMS Server and Queue Setup

Following are the steps to setup a JMS Server, Connection Factory and three queues, Request queue, Response queue and Error queue via the weblogic admin console. I am running Weblogic version 10.3.6.0.13.

Open the Weblogic Console and navigate to Services > Messaging > JMS Modules, Click Lock&Edit, then New, then enter a name for the JMS Module and click "Next”:

Select PIA as Target and click "Next": 
Uncheck "Would you like to add resources to this JMS system module?" and click "Finish"

Navigate to Services > Messaging > JMS Servers and Click "new", enter the name of the JMS Server and click "Next":

Navigate to Services > Messaging > JMS Modules and select the JMS Module created in previous steps.

Now create a queue as below. 

Similarly create two more queues, the ResponseQueue and ErrorQ

PeopleSoft setup

Nodes:
Create two nodes, of type external, one for listening on the request queue and another one to write back to the response queue. Setup is pretty much identical except for the queue name. Use the JMSTARGET connector ID for both the nodes and key in properties as follows:
JMSFactory = MyConnectionFactory (this is the JNDI name defined via the weblogic console)
JMSMessageType = Text
JMSProvider = Weblogic
JMSQueue = RequestQueue for one node and ResponseQueue for the other node
JMSUrl = Weblogic JMS Server in the form t3://server:port

Messages:

Create two non-rowset based messages, again one for request and the other for response.
Added a schema to each message (this is optional, you can manage it via peoplecode too). Request message for this POC has the following structure:

<?xml version='1.0'?>
<person>
    <emplid>string</emplid>
</person>

and response message would be as follows:

<?xml version='1.0'?>
<person>
    <name>string</name>

</person>

Service Operation:

Created a service and assigned two service operations to it. Assigned security to both the service operations. The listener service operation has handler application package onNotify peoplecode defined, nothing for the response service operation. 
So in case of the listening service operation the routing would be:
Sender Node = newly created JMS listening node (above)
Receiver Node = local default node 

In case of response service operation the routing would be:
Sender Node = local default node
Receiver Node =  newly created JMS response node (above)

Handler Peoplecode:

method onNotify

   /+ &MSG as Message +/

   /+ Extends/implements PS_PT:Integration:INotificationHandler.OnNotify +/

  

   Local XmlDoc &xmldoc = &MSG.GetXmlDoc();

   Local XmlNode &xmlNode = &xmldoc.DocumentElement;

   Local string &emplid = %This.FetchDataValue(&xmldoc, 1, "emplid");

   Local string &name;

  

   SQLExec("select ISNULL((SELECT NAME FROM PS_PERSONAL_DATA WHERE EMPLID = :1),'No Name')", &emplid, &name);

  

/* create the reponse message */

Local Message &ResponseMSG = CreateMessage(Operation.JMS_RESP_SO, %IntBroker_Request);

   Local string &xmldata = "<?xml version='1.0'?><person/>";

   Local XmlDoc &ResponseDOC = CreateXmlDoc(&xmldata);

   &xmlNode = &ResponseDOC.DocumentElement;

  

   Local XmlNode &NameNode = &xmlNode.AddElement("name");

   &NameNode.NodeValue = &name;

  

/* create the reponse message */

   &ResponseMSG.SetXmlDoc(&ResponseDOC);

   %IntBroker.Publish(&ResponseMSG);

  

end-method;

method FetchDataValue

   /+ &xmldoc as XmlDoc, +/

   /+ &row as Number, +/

   /+ &dataTag as String +/

   /+ Returns String +/

  

   Local string &dataValue;

   Local array of XmlNode &dataList;

  

   &dataList = &xmldoc.DocumentElement.FindNodes(&dataTag);

   If &dataList.Len = 0 Then

      &dataValue = "";

   Else

      &dataValue = &dataList [&row].NodeValue;

   End-If;

   Return &dataValue;

end-method;

IB changes:

Integrationgateway.properties file can be edited via the gateway setup properties link from the gateway component. Updated that as follows for the local gateway.

Showing only the pieces that I changed updated, under the JMS configuration section. need to only define the listening queue here. Error queue is optional but good to have. 

ig.jms.Queues=1

ig.jms.Queue1=RequestQueue
ig.jms.Queue1.Provider=Weblogic
ig.jms.Queue1.JMSFactory=MyConnectionFactory
ig.jms.Queue1.Url=t3://server:port
# following service operation details are required only for a listening operation.  
ig.jms.Queue1.OperationName=JMS_REQ_SO.v1
ig.jms.Queue1.OperationVersion=v1
ig.jms.Queue1.RequestingNode=JMS_LISTENER_NODE
ig.jms.Queue1.DestinationNode=PSFT_HR

ig.jms.ErrorQueue=ErrorQueue

ig.jms.ErrorQueue-Provider=Weblogic
ig.jms.ErrorQueue-JMSFactory=MyConnectionFactory
ig.jms.ErrorQueue-Url=t3://server:port 

Testing:

Create a message via the weblogic console in MyQueue1 or the RequestQueue and verify via integration broker monitor that the message is picked up processed and written to MyQueue2 or the ResponseQueue.

Any issues or errors will be written to errorLog.html and/or msgLog.html files on the webserver.

Sending and Receiving MTOM-encoded binary data

PeopleSoft supports the MTOM protocol for sending and receiving binary data using service operations. While you can send and receive binary data using SOAP, doing so requires that you Base64-encode the data, which can increase message size by 33 percent or more. The MTOM protocol enables you to send and receive binary data in its original binary form, without any increase in size due to encoding.

For sending or receiving MTOM-encoded binary data, we have to use message segments to store the data. The SegmentContentType property of the message object is used to set or read the content type of each message segment.

Following is a test that I did to send a XML file as an attachment in a SOAP message and then read the attachment that is sent by the 3rd party system that I am interacting with. I am running PT 8.53.22.

Sending:

Request message is as shown below. For this test I am storing this in a html object called as MY_MESSAGE but this can be generated dynamically as needed using SOAPDoc or XMLDoc classes. The request message defined on the service operation is nonrowset based. 

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">

<soapenv:Header/>

<soapenv:Body>

<submitLargeDocument>        

    <Document>  

         <PsftXopInclude SegmentNumber='1'/> 

    </Document>      

</submitLargeDocument>

</soapenv:Body>

</soapenv:Envelope>

document.xml is my payload which I have created in a different routine and I am just using it here. Its just like creating any other XML file in PeopleSoft. Payload can be a binary file like a pdf or a image file.

Sender node is my default local node and receiving node is the delivered WSDL_NODE. You can always create a custom receiving node if needed. Routing is using local gateway and HTTPTARGET connector. Under routing, connector properties content-type is set to text/xml as my payload is a xml file, HTTPPROPERTY MTOM is set to Y, METHOD is POST, SOAPUpContent is set to N as I have already built the SOAP wrapper in my html object above. If you need IB to create the wrapper then set this property to Y. Provided PRIMARYURL to destination 3rd party application. Took all other defaults.

PeopleCode:

&str = GetHTMLText(HTML.MY_MESSAGE);

&requestXMLDoc = CreateXmlDoc();

&ret = &requestXMLDoc.ParseXmlString(&str);

&request = CreateMessage(Operation.SEND_OPERATION);

&request.SetXmlDoc(&requestXMLDoc);

&MTOMFile = GetFile("C:\temp\document.xml", "R", %FilePath_Absolute);

If &MTOMFile.IsOpen Then   

   &theBase64encodedString = &MTOMFile.GetBase64StringFromBinary();   

   &MTOMFile.Close();

End-If;

&request.CreateNextSegment();

If (&request.SetContentString(&theBase64encodedString)) Then   

    &request.SegmentContentType = "application/xml";  

    &request.SegmentContentTransfer = %ContentTransfer_Binary;

End-If;

&response = %IntBroker.SyncRequest(&request);

Receiving:

Response message defined on the service operation is a non-rowset based message. Sender node is the default local node and receiving node is WSDL_NODE. Using local gateway and HTTPTARGET connector. Setting HEADER properties Content-Type to text/xml as the response attachment that I am receiving is a xml file, sendUncompressed is Y, HTTPPROPERTY Method is POST and SOAPUpContent is Y and finally the PRIMARYURL to the 3rd party service.

On the weblogic webserver, in the integrationGateway.properties file enable the MTOM Listening Connectors. 

ig.MTOM.enablePeopleSoftServiceListeningConnector=true

ig.MTOM.enableHttpListeningConnector=true

Bounce the webserver after making this change.

PeopleCode:

This is pretty straightforward. Once the request is made, read the response and parse out the document.

&response = %IntBroker.SyncRequest(&request);

&responseXMLDoc = &response.GetXMLDoc();

If (&response.ResponseStatus = 0) Then

     &dataNode = &responseXMLDoc.DocumentElement.GetElementByTagName("data");

     &theData = &dataNode [1].GetCDataValues();

     &responsestr = &theData.Shift();

End-If;

The &responsestr string variable will have the response SOAP envelope as well as the attachment separated by message segments as shown below. Parsed it out using string functions.

<?xml version="1.0"?>

<data psnonxml="Yes">

  <![CDATA[

------=_Part_624_1792156364.1458048147094

Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"

Content-Transfer-Encoding: 8bit

Content-ID: <soap.xml@xfire.codehaus.org>

<soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">

<soap:Body>

.......

.......

.......

</soap:Body>

</soap:Envelope>

------=_Part_624_1792156364.1458048147094--

------=_Part_724_1792156364.1458048147094

Content-Type: application/xop+xml; charset=UTF-8; type="text/xml"

Content-Transfer-Encoding: 8bit

Content-ID: <soap.xml@xfire.codehaus.org>

<Document>

........

........

........

</Document>

------=_Part_724_1792156364.1458048147094--

]]>

</data>