Showing posts with label tables. Show all posts
Showing posts with label tables. Show all posts

Friday, November 30, 2018

Query Transformations

I rarely do much work in PS Query, but while doing some other troubleshooting for PS Query related issues, I noticed the "Transformations" tab in Query Manager. I am running PT 8.56.x in HCM 9.2, but looks like this has been around since 8.53 or 8.54. 

So decided to do a quick test to see what this does and how it works. So the benefit here is that the output can be transformed to create reports right out off the output delivered by PS Query.

So as part of this POC used any existing query. Transforming the output to HTML and using a XSL transformation to highlight the row when a certain condition is matched.

On the Transformation tab within Query Manager, click on Add XSLT which opens a window to key in free-form XSLT text. It would have been nice if there is a wizard of some sort to build the XSLT. 

Gave it a Name and in the "Output File Type" lookup selected HTML. Following are the available choices - 001, A01, ABA, AET, APC, CSS, CSV, D01, DAT, DTD, DTL, ERR, GIF, GIR, HTM, HTML, IDX, JCD, JS, LIS, LOG, MDL, MT9, N01, NVT, NXT, OUT, PS, SQL, STDOUT, SYS, TMP, TRC, TRN, TXT, XLSM, XLSX, XML, XSD, ZIP

Not sure what is the difference between HTM and HTML, but I selected HTML. My sample XSLT is designed as follows.

<?xml version="1.0" encoding="UTF-8"?> <xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" version="1.0"> <xsl:template match="/"> <html>
<!-- defined a style for the row that is going to be highlighted -->
<style type="text/css">
tr.redRow td {
font-size: 100%;
font-family: calibri;
font-weight:bold;
background-color: #FF2626;
padding-right: 10px;
color:#000;
}
</style>
<body>
<!-- in the html body defined a html table --> <table border="1">
<!-- Added a heading --> <font color="blue"> <h2>Sample Transformation test</h2> </font>
<!-- defined column headers --> <tr> <th>Employee ID</th> <th>Last Name</th> <th>First Name</th>
<th>Location</th> </tr>
<!-- Here I am selecting the data from PS Query --> <xsl:for-each select="query/row"> <tr>
<!-- defined a if condition to check if -->
<!-- value is McLean then highlight the row --> <xsl:if test="L.DESCR='McLean'"> <xsl:attribute name="class">redRow</xsl:attribute> </xsl:if>
<!-- Output the PS Query results --> <td> <xsl:value-of select="A.EMPLID" /> </td> <td> <xsl:value-of select="C.LAST_NAME" /> </td> <td> <xsl:value-of select="C.FIRST_NAME" /> </td> <td> <xsl:value-of select="L.DESCR" /> </td> </tr> </xsl:for-each> </table> </body> </html> </xsl:template> </xsl:stylesheet>

So saved this and hit the preview button to preview the results. My query uses runtime prompts so those pop'ed up first, provided values for them and results are displayed in a new window (disable pop-up blocker or allow pop-up for this site).


Sample Transformation test


Employee IDLast NameFirst NameLocation
12345DoeJohnMadrid
13244DoeJaneEgypt
43423HuftonAbcdeLondon
42343FlemingJillMcLean
34423TiboldJozsefLondon
67554FarrellWilliamJohannesburg
43556MyersTonyIreland
75757KramerNicoleMcLean
27341WadaMarkNew York
27045RogerWaiMcLean
27098WayneJessicaNew York
27125GarciaGuillermoNew York
75688Deepak ShirguppiMaryland

If this query is scheduled using Query Scheduler option then a new format of XFORM is available and then I can pick any transformations (can define n number of transformations) that I have defined on the query.









Monday, March 12, 2018

Application Engine Parallel Processing (batch)

Parallel processing comes in picture when there is a requirement to process large amount of rows without compromising the performance which might otherwise get impacted greatly with non-parallel based processing. Sharing this concept with an actual example where this concept has been used.

I had written an application engine program which read data via a SQL object, performed some computations and finally wrote the result to an extract file which was sFTP'ed to the vendor. Pretty much run of the mill stuff. The AE was reading from PS_JOB, PS_PERSONAL_DATA, PS_EMAIL_ADDRESSES tables etc via a single single SQL object in a peoplecode step, performing the data manipulation as required and writing to a file. It was sequentially selecting an employee at a time, performing the data manipulations and writing to the file. To process around 3K employees it was taking a little over an hour, so not a lot of data but too much time to process this data-set.

So to solve this issue of performance, this is what I did. 
  1. I created a single TAO or temporary table. The fields in this table are essentially the unique list of fields or values that the single SQL object was returning. Did not build the TAO table yet, will do it in a later step.
  2. The TAO table has two keys, PROCESS_INSTANCE and EMPLID.
  3. Then before the peoplecode step was called added a new step to select from the various HR tables and write to this newly created TAO table. 
  4. SQL step looks something like this.
         INSERT INTO %Table(N_MY_TAO)          SELECT %Bind(PROCESS_INSTANCE)         , B.EMPLID         .....        .....
        .....
        FROM PS_JOB A, PS_PERSONAL_DATA B .....
        WHERE ....

    5. After this added a new step to update statistics on the newly populated TAO table.
        %UpdateStats(N_MY_TAO)
    6. In the Peoplecode step replaced the SQL fetch as follows.
        &EESQL = CreateSQL(FetchSQL(SQL.N_MY_SQL), N_MY_AET.PROCESS_INSTANCE);
    7. I had already defined a state record in my AE which has process_instance field in it, so did not have to do anything different with the state record. In the N_MY_SQL SQL object I am selecting all fields FROM %Table(N_MY_TAO) WHERE PROCESS_INSTANCE = :1 
ORDER BY EMPLID. I have some SQL case statements and formatting rules defined in the SQL itself.
8. Added the newly created TAO table under temp tables program properties and provided an instance count of 1. Instance count has to be 1 or more, if its set to 0 then you will see the following message in your AE log and there won't be much of performance improvement. 

WARNING: NO DEDICATED INSTANCES AVAILABLE FOR N_MY_TAO - USING BASE TABLE. (108,544)

9. Under PeopleTools > Utilities > Administration > PeopleTools Options, in my case the Temp Table Instances (Total) and Temp Table Instances (Online) is set to 3. So when I add 1 as the instance count in my AE and then build the TAO table, it will create PS_N_MY_TAO, PS_N_MY_TAO1, 2, 3 and 4 and when the process runs it will use PS_N_MY_TAO4 as the first 3 are used for online processing. 

10. You can add a step at the beginning or end to purge the TAO table like %TruncateTable (%Table(N_MY_TAO)) or better yet use the check-box under program properties "Use Delete for Truncate Table". With this option the TAO table is purged at the beginning of each run.

Addition of a single TAO table enabled the process to complete in 7 mins. Massive improvement.