ibmi-brunch-learn

Announcement

Collapse
No announcement yet.

How to compare 2 data structures field by field?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • JonBoy
    replied
    Not 100% completed - but here's the base code that I'll be referencing in the upcoming article. Thanks for the inspiration!

    Note that it handles and reports on both packed and zoned numerics. There are a lot of comments so hopefully it will be understandable even without the accompanying article.

    Code:
    **Free
    
    // This code is provided only as a teaching example of a concept
    // It is not complete and should not be used in production code
    
    ctl-opt  Option(*NoDebugIO : *SrcStmt )  DftActgrp(*No);
    ctl-opt CCSID(*GRAPH:*JOBRUN);
    
    // FieldData is the oufile from:
    //   DSPFFD FILE(PRODUCT) OUTPUT(*OUTFILE) OUTFILE(FIELDDATA)
    // Normally that command would be built in to this program via QCMDEXC
    //   or system() but we're "cheating" here to keep the example short.
    
    dcl-f  FieldData;
    
    // These are the DS images of the before and after state of the records
    // They could be pointer mapped to trigger buffer images, or mapped from
    //    journal records, or simply be before and after images from an
    //    operator's screen. Basically anything where you need to know what changed
    // For this proof of concept example we will simply hard-code some values
    //    in the "after" DS to ensure a difference between "before" and "after"
    
    dcl-ds  before  ExtName('PRODUCTEXT')  Qualified  Inz  end-ds;
    
    dcl-ds  after   ExtName('PRODUCTEXT')  Qualified  Inz  end-ds;
    
    // The following DS array contains the details of the fields in
    //   the DS. Their name, data type, position, etc.
    // It provides the data needed for comparing before and after images
    // In this case it was built from DSPFFD data but it could have
    //   been from (say) SYSCOLUMNS2 or the QUSLFLD API.
    
    dcl-ds  fieldDetail  Dim(99)  Qualified;
       startPosn  Like(WHIBO);   // Offset within the record
       length     like(WHFLDB);  // Length in bytes
       name       like(WHFLDI);
       dataType   like(WHFLDT);
       decimals   like(WHFLDP);  // Number of decimal places
    End-Ds;
    
    dcl-s  fieldBefore  varchar(256);  // This length assumes that no fields
    dcl-s  fieldAfter   varchar(256);  //   will exceed 256 bytes in length
                                       // Adjust size as needed
    
    // This structure is used to enable the bytes that represent a
    //   numeric field's value to be processed as numeric in
    //   this program. It basically re-maps bytes as packed or zoned numeric
    
    dcl-ds  mapNumeric;
       zonedValue   zoned(15);
       packedValue  packed(29) Overlay(zonedValue);
    end-Ds;
    
    dcl-s decimalValue  packed(29:5);
    
    dcl-s  i      int(5);
    dcl-s  count  int(5);
    dcl-s  start  int(5);
    
    // Data types for numeric fields - this code assumes that only
    //   character/packed/zoned fields are present in the DS
    dcl-c PACKED  'P';
    dcl-c ZONED   'S';
    
    // This loop simply reads the file produced by DSPFFD and stores
    //   the required data in the fieldData array for later reference
    // Normally this code would be in a Service Program and might use
    //   SQL or API calls as noted earlier.
    
    read FieldData;
    
    DoU  %Eof(FieldData);
       count += 1;
       fieldDetail(count).name      = WHFLDI;
       fieldDetail(count).dataType  = WHFLDT;
       fieldDetail(count).startPosn = WHIBO;
       fieldDetail(count).length    = WHFLDB;
       fieldDetail(count).dataType  = WHFLDT;
       fieldDetail(count).decimals  = WHFLDP;
    
       read FieldData;
    
    EndDo;
    
    // To show the logic works we'll change some values in DS after.
    //   The rest of the after DS & all of before DS has blanks/zeros
    
    after.STOH    = 99;
    after.SELLPR  = 123.45;
    after.MAXDISC = 9.999;
    
    // This loop will process each field in turn reporting on
    //   any that have changed.
    
    For i = 1 to count;
       // Copy relevant portions of before/after images to a
       //   temporary field to simplify subsequent logic
       fieldAfter  = %subst(after:fieldDetail(i).startPosn:fieldDetail(i).length);
       fieldBefore = %subst(before:fieldDetail(i).startPosn:fieldDetail(i).length);
    
       if fieldAfter <> fieldBefore;
          // Field has changed so report it
          // Replace with any logic needed to report before & after values
          Dsply ('Field ' + fieldDetail(i).name + ' has changed');
    
          // Determine where numeric field data starts in the
          //   mapNumeric DS.
          start = %Len(mapNumeric) - fieldDetail(i).length + 1;
    
          // To handle packed and numeric values we convert them
          //   and scale them so that we can display them. The logic
          //   here is intended to give you ideas - it is not a definitive
          //   solution for all data types and sizes.
    
          If fieldDetail(i).dataType = PACKED;
             packedValue = 0; // Initialize packed work field
             %subst(mapNumeric: start ) = fieldAfter;
             // Now scale the numeric value to account for decimals
             If fieldDetail(i).decimals > 0;
                decimalValue = packedValue / (10 ** fieldDetail(i).decimals);
             Else;
                decimalValue = packedValue;
             EndIf;
             Dsply ('New value ' + %Char(decimalValue) ) ;
    
          ElseIf fieldDetail(i).dataType = ZONED;
             zonedValue = 0; // Initialize all positions in zoned work field
             %subst(mapNumeric: start ) = fieldAfter;
             If fieldDetail(i).decimals > 0;
               decimalValue = zonedValue / (10 ** fieldDetail(i).decimals);
             Else;
                decimalValue =zonedValue;
             EndIf;
             Dsply ('New value ' + %Char(decimalValue) ) ;
    
          EndIf;
    
       EndIf;
    
    EndFor;
    
    *InLr = *On;
    Last edited by JonBoy; November 25, 2019, 12:10 PM.

    Leave a comment:


  • JonBoy
    replied
    The only thing I can think of off the top of my head is to do something like this:

    Code:
           dcl-ds remap;
             inpChar     char(10);
             outGraph    graph(5)  Overlay(inpChar);
           end-ds;
    Basically you take the character substring and place it in inpChar - you can then use the outGraph string.

    Considering that a graphic field has to include a shift-in/shift-out pair 5 characters seems awfully small. Whatever useful double-byte data is stored there?

    P.S. Look at the values in the de=bugger in hex form and see what the actual content is.


    Leave a comment:


  • sri8707
    replied
    Hi JonBoy, Can you provide me your expertise on one more query that I have?

    As you know from the code, Data structure BEFORE is based on an external file - FILE1. One of the field in this file is defined as GRAPHIC and of length 10.

    When the below condition is satisfied, I have to write this particular field into my LOG file which has all fields as CHARACTERS. How can I convert the graphics to character type? I tried %Graph %UCS2 and %Char functions but no luck.

    if %subst(after:1:10) <> %subst(before: 1:10); Write this particular field to log file EndIf;

    Leave a comment:


  • sri8707
    replied
    Originally posted by JonBoy View Post
    Adding Java just adds complexity with no benefits that I can think of. You could do stuff with PHP or Python (for example) since they can use dynamic field names to access data - but you'd have to use (say) DATA-GEN with the DS to generate json or XML so that the PHP/Python script would have something to work with. Be way more work than needed - not to mention that none of those languages really understand packed and zoned numbers in the way that RPG does.

    And there more I think about it the more reasons I find not to bother ...

    Anyway - thanks for providing article fodder. I'll let you know when it is published.
    I could have tried DATA-GEN but then, its not supported in V7R2 and below. Please keep me updated once you publish the article. Eager to learn

    Leave a comment:


  • JonBoy
    replied
    Adding Java just adds complexity with no benefits that I can think of. You could do stuff with PHP or Python (for example) since they can use dynamic field names to access data - but you'd have to use (say) DATA-GEN with the DS to generate json or XML so that the PHP/Python script would have something to work with. Be way more work than needed - not to mention that none of those languages really understand packed and zoned numbers in the way that RPG does.

    And there more I think about it the more reasons I find not to bother ...

    Anyway - thanks for providing article fodder. I'll let you know when it is published.

    Leave a comment:


  • sri8707
    replied
    For some reasons, we tend to avoid SQL programs. (Been the practice). So, i initially went ahead with QUSLFLD API. But, again I needed to capture the length offset/field names into a new array/DS and then proceed from there. I was also contemplating invoking Java procedures and let Java handle the DS comparisons. (We do use quite a lot of Java calls for our programs). However, the method you gave me is really the simplest solution of all, and works like a charm. Thanks again.

    Leave a comment:


  • JonBoy
    commented on 's reply
    My pleasure. I did a preliminary version with SQL and SYSCOLUMNS but since I didn't need any of the features of SQL it just added lines of code and the data was harder to use. Likewise QUSLFLD - just more complex to handle than I needed. Since this will end up as a teaching example I wanted to keep the logic as simple as possible so as to focus on the "how do I know which field changed" aspect - and hence the use of DSPFFD.

  • sri8707
    replied
    Hi JonBoy, I can't thank you enough for taking so much time to provide me with more details. I was contemplating with QUSLFLD API and SYSCOLUMNS to fetch the details.

    This really solves my issue. I will run through my program and get back to you if I still have any queries. Appreciate your help again. Thank you.

    Leave a comment:


  • JonBoy
    replied
    OK - here's a very simple program. In the end I used DSPFFD with an outfile for reasons that I won't get into here.

    Code:
           ctl-opt  Option(*NoDebugIO : *SrcStmt )  DftActgrp(*No);
    
           // FieldData is the oufile from:
           //   DSPFFD FILE(PRODUCT) OUTPUT(*OUTFILE) OUTFILE(FIELDDATA)
           // Normally that would be built in to this program via QCMDEXC or system()
    
           dcl-f  FieldData;
    
           dcl-ds  before  ExtName('PRODUCT')  Qualified  Inz  end-ds;
    
           dcl-ds  after   ExtName('PRODUCT')  Qualified  Inz  end-ds;
    
           dcl-ds  fieldDetail  Dim(99)  Qualified;
    
              startPosn  Like(WHIBO);
              length     like(WHFLDB);
              name       like(WHFLDI);
              dataType   like(WHFLDT);
    
           End-Ds;
    
           dcl-s  i      int(5);
           dcl-s  count  int(5);
    
           read FieldData;
    
           DoU  %Eof(FieldData);
    
              count += 1;
    
              fieldDetail(count).name      = WHFLDI;
              fieldDetail(count).dataType  = WHFLDT;
              fieldDetail(count).startPosn = WHIBO;
              fieldDetail(count).length    = WHFLDB;
    
              read FieldData;
    
           EndDo;
    
           // Just to show the logic works change a couple of values
    
           after.STOH = 99;
           after.SELLPR = 123.45;
    
           For i = 1 to count;
    
              if %subst(after:fieldDetail(i).startPosn:fieldDetail(i).length) <>
    
                 %subst(before: fieldDetail(i).startPosn:fieldDetail(i).length);
                 // Field has changed so report it
                 // Add whatever logicx needed to rep[ort the before and adfter values
                 Dsply ('Field ' + fieldDetail(i).name + ' has changed');
    
              EndIf;
    
           EndFor;
    
           *InLr = *On;
    If you have any questions just holler

    Leave a comment:


  • sri8707
    replied
    Originally posted by JonBoy View Post
    Not sure this is the best set of columns to use but I think this would work:

    select column_name, column_heading, storage, ordinal_position from SYSCOLUMNS
    where table_name = 'PRODUCT' and system_table_schema = 'PARTNER400' order by ordinal_position;

    Then loop through the result set.
    Thanks again. But, can you provide additional details of how to use this with the DS i have? So, currently I have 2 DS - one storing the before image of 20 fields and another storing after image of the same 20 fields. How should i compare the fields using the approach that you have mentioned? Should i write DS to a new file and then compare based on the offset positioning?

    Leave a comment:


  • JonBoy
    replied
    Not sure this is the best set of columns to use but I think this would work:

    select column_name, column_heading, storage, ordinal_position from SYSCOLUMNS
    where table_name = 'PRODUCT' and system_table_schema = 'PARTNER400' order by ordinal_position;

    Then loop through the result set.

    Leave a comment:


  • JonBoy
    commented on 's reply
    Yes I thought about suggesting that but I'm not deeply enough into DATA-GEN yet - whereas I knew how to do it with OA. In both cases the problem lies with there being no easy way for the generator/handler to differentiate between the calls for the two different DS. You could use the extra user parameter on the OA handler and specify via that whether this is a before or after image. In DATA-GEN I guess a naming convention could be used on the DS themselves.

    That said - I think the SQL and SYSCOLUMNS (or a similar view) is probably a better way to go if the DS are externally described.

  • sri8707
    replied
    Originally posted by JonBoy View Post
    No - it doesn't have such a compare. But use SQL and system tables not the QUSLFLD API. Much easier.

    I think I'm going to play with the technique a bit in a few minutes.

    As I said before though - try to get away from the renaming of fields. Use qualified DS.
    Hi JonBoy, thanks again. Ya will use qualified DS instead of prefixing the fields. Also, will check on system tables as you suggested. Meanwhile, if you come across any other alternative, let me know.

    Leave a comment:


  • sri8707
    replied
    Originally posted by Barbara Morris View Post
    Another possibility is DATA-GEN. http://ibm.biz/fall_2019_rpg_enhancements

    You could write a generator that outputs the names and values, and also puts the name-value pairs into an array in the user-area.

    This would be similar to the open access solution suggested by Jon, but I think it nught a bit easier to deal with the data in a data-gen generator than an open-access handler.

    The generator would be quite similar to the generators in the GENPROP source member in QOAR/SAMPLE2, at least as far as the callbacks to output the document. The code to put the names and values into the user-area would be "ordinary" RPG code, aside from a bit of pointer stuff.

    Code:
    dcl-ds ds1_array qualified; // this definition would also be used by the generator
    num int(10);
    dcl-ds subf dim(100); // max number of subfields
    name char(10);
    value varchar(2000); // longest possible value
    end-ds;
    end-ds;
    dcl-ds ds2_array likeds(ds1_array);
    clear ds1_array;
    clear ds2_array;
    DATA-GEN ds1 %data(output1) %gen('MYGENPGM' : ds1_array);
    DATA-GEN ds2 %data(output2) %gen('MYGENPGM' : ds2_array);
    if ds1 <> ds2;
    go through the arrays to see what's different
    endif;
    Hi Barbara Morris, thanks for your suggestion. I find that this DATA-GEN is a V7R3 enhancement. However, I am running V7R1 and hence would not be able to use this.

    Leave a comment:


  • Barbara Morris
    replied
    Another possibility is DATA-GEN. http://ibm.biz/fall_2019_rpg_enhancements

    You could write a generator that outputs the names and values, and also puts the name-value pairs into an array in the user-area.

    This would be similar to the open access solution suggested by Jon, but I think it nught a bit easier to deal with the data in a data-gen generator than an open-access handler.

    The generator would be quite similar to the generators in the GENPROP source member in QOAR/SAMPLE2, at least as far as the callbacks to output the document. The code to put the names and values into the user-area would be "ordinary" RPG code, aside from a bit of pointer stuff.

    Code:
    dcl-ds ds1_array qualified; // this definition would also be used by the generator
       num int(10);
       dcl-ds subf dim(100); // max number of subfields
          name char(10);
          value varchar(2000); // longest possible value
       end-ds;
    end-ds;
    dcl-ds ds2_array likeds(ds1_array);
    clear ds1_array;
    clear ds2_array;
    DATA-GEN ds1 %data(output1) %gen('MYGENPGM' : ds1_array);
    DATA-GEN ds2 %data(output2) %gen('MYGENPGM' : ds2_array);
    if ds1 <> ds2;
       go through the arrays to see what's different
    endif;

    Leave a comment:

Working...
X