ibmi-brunch-learn

Announcement

Collapse
No announcement yet.

How to compare 2 data structures field by field?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • How to compare 2 data structures field by field?

    I would like to compare 2 DS field by field and capture the changed fields into an audit file. Below is the scenario -

    User is provided with a display screen where he can change any of the 20 fields pertaining to a file. Say if the user changed 10 fields, I want to write 10 records into the audit log file with below details -

    Field changed
    Before value
    After value
    Date/Time, etc

    At the moment, I have written logic as below -


    <code>
    D File E DS EXTNAME(File1) // Contains latest image of the 20 fields
    *
    D P_File E DS EXTNAME(File1) Prefix(P_) // Contains previous image of the 20 fields

    C If Field1 <> P_Field1
    C Eval BeforeImage = P_Field1
    C Eval AfterImage = Field1
    C Eval Comment = 'Field1 changed'
    C Write Auditfile
    C EndIf

    C If Field2 <> P_Field2
    C Eval BeforeImage = P_Field2
    C Eval AfterImage = Field2
    C Eval Comment = 'Field2 changed'
    C Write Auditfile
    C EndIf
    </code>

    As you can see, I am writing 20 IF statements for manually checking changes one by one. But, is there a better approach to do this? (Something like looping through data structure to check all fields)

  • #2
    One point before talking solutions.

    If I was trying to do this generically I would not use prefixing. I would use Qualified DS and a template. Like so:

    Code:
    **free
    
    dcl-ds myDS_T extname('FILE1') Template end-ds;
    
    dcl-ds Ds1  LikeDS(myDS_T);
    
    dcl-ds Ds2  LikeDS(myDS_T);
    That said, there are a couple of options I can think of.

    1) Write a simple code generator to produce the IF statements you need. Generate the code into a copy file and incorporate it into the program where needed.

    2) A generics solution that could work really well if all of your cases involve externally described DS. Basically you could write a name/value Open Access handler and then "write" each DS to a file and have the handler store and compare the values. This is probably much easier to write than to describe! Shout if you want the think about this as an option.

    P.S. I'm fascinated as to why you don't just use journaling with before/after images to achieve your goals. Much simpler and requires no modifications to the user programs.

    Comment


    • #3
      Hi JonBoy, thanks for your reply. With regards to journaling, I do not want to add journals/triggers to the existing file to capture the audit log.

      Can you provide more details on your second point? I didn't get that.

      Another way I could think of is to have a CT array as below -

      Field1_New Field1_Old FieldName1
      Field2_New Field2_Old FieldName2
      ................
      ................
      Field20_New Field20_Old FieldName20

      Then program as,
      Read the array from 1 to 20
      If value1(i) <> value2(i) // If field1_new not equal to field1_old
      Eval change = value3(i) // Set changed field as fieldname1

      But, again, this doesnt look a good practice to me to manually write all the file-fieldnames into the array. I want to somehow avoid using all 20 field names in a file. Because at later instance, few additional fields might be added to the file, and i do not want to re-edit the program then.

      Comment


      • #4
        I don't see how your CVT array <shudder - how I hate those things> would help. RPG just doesn't support the use of a variable to supply the name of a field.

        Some years ago I described a "solution" to this kind of problem http://archive.ibmsystemsmag.com/ibm...calculations-/ and http://archive.ibmsystemsmag.com/ibm...-calculations/

        I don't think it really helps in your case but ...

        OK - the Open Access solution. Have you written an OA handler or studied them? How quickly do you need this? I need a topic for a magazine article that is dues this weekend and I'm tempted to use this as an example.

        Comment


        • JonBoy
          JonBoy commented
          Editing a comment
          I should add that an OA based approach, while affording maximum flexibility, will obviously not perform anywhere near as well as a series of IF statements.

      • #5
        The technique we use captures the image of the record before and after update (i.e. two records written) to an audit file which is similar to the actual file being maintained, additional fields contain date, time stamp, user etc. Checking so many fields for a change seems like a lot of work so using array would seem a good choice.

        Comment


        • #6
          Been think ing some more about this.

          As long as there is an external description to work with, another approach would be to use the system tables to determine the field names and their individual offsets. Then do the compare in a loop using the offsets to determine the substring of the DS to compare.

          Comment


          • #7
            Originally posted by JonBoy View Post
            Been think ing some more about this.

            As long as there is an external description to work with, another approach would be to use the system tables to determine the field names and their individual offsets. Then do the compare in a loop using the offsets to determine the substring of the DS to compare.
            Thanks JonBoy. I was thinking about using arrays but realised that wouldnt work either. (Because RPGLE is not a language designed for dynamically obtaining values for the fields from the arrays. I could define array as FIELD1 FIELD1_OLD, but then I cannot retrieve the value pertaining to FIELD_1 dynamically. Hence, that approach is ruled out).

            Yes, i have external description to work with. Somehow I need to offset it for my comparison. I will try with QUSLFLD API to see if it helps.

            I presume RPGLE do not have a method of comparing 2 DS field by field? If so, kindly let me know. Again, below is my DS image -

            DS1:
            Field1
            Field2
            Field3
            Field4
            Field5

            DS2:
            Field1_Old
            Field2_Old
            Field3_Old
            Field4_Old
            Field5_Old

            Comment


            • #8
              No - it doesn't have such a compare. But use SQL and system tables not the QUSLFLD API. Much easier.

              I think I'm going to play with the technique a bit in a few minutes.

              As I said before though - try to get away from the renaming of fields. Use qualified DS.

              Comment


              • #9
                Another possibility is DATA-GEN. http://ibm.biz/fall_2019_rpg_enhancements

                You could write a generator that outputs the names and values, and also puts the name-value pairs into an array in the user-area.

                This would be similar to the open access solution suggested by Jon, but I think it nught a bit easier to deal with the data in a data-gen generator than an open-access handler.

                The generator would be quite similar to the generators in the GENPROP source member in QOAR/SAMPLE2, at least as far as the callbacks to output the document. The code to put the names and values into the user-area would be "ordinary" RPG code, aside from a bit of pointer stuff.

                Code:
                dcl-ds ds1_array qualified; // this definition would also be used by the generator
                   num int(10);
                   dcl-ds subf dim(100); // max number of subfields
                      name char(10);
                      value varchar(2000); // longest possible value
                   end-ds;
                end-ds;
                dcl-ds ds2_array likeds(ds1_array);
                clear ds1_array;
                clear ds2_array;
                DATA-GEN ds1 %data(output1) %gen('MYGENPGM' : ds1_array);
                DATA-GEN ds2 %data(output2) %gen('MYGENPGM' : ds2_array);
                if ds1 <> ds2;
                   go through the arrays to see what's different
                endif;

                Comment


                • JonBoy
                  JonBoy commented
                  Editing a comment
                  Yes I thought about suggesting that but I'm not deeply enough into DATA-GEN yet - whereas I knew how to do it with OA. In both cases the problem lies with there being no easy way for the generator/handler to differentiate between the calls for the two different DS. You could use the extra user parameter on the OA handler and specify via that whether this is a before or after image. In DATA-GEN I guess a naming convention could be used on the DS themselves.

                  That said - I think the SQL and SYSCOLUMNS (or a similar view) is probably a better way to go if the DS are externally described.

              • #10
                Originally posted by Barbara Morris View Post
                Another possibility is DATA-GEN. http://ibm.biz/fall_2019_rpg_enhancements

                You could write a generator that outputs the names and values, and also puts the name-value pairs into an array in the user-area.

                This would be similar to the open access solution suggested by Jon, but I think it nught a bit easier to deal with the data in a data-gen generator than an open-access handler.

                The generator would be quite similar to the generators in the GENPROP source member in QOAR/SAMPLE2, at least as far as the callbacks to output the document. The code to put the names and values into the user-area would be "ordinary" RPG code, aside from a bit of pointer stuff.

                Code:
                dcl-ds ds1_array qualified; // this definition would also be used by the generator
                num int(10);
                dcl-ds subf dim(100); // max number of subfields
                name char(10);
                value varchar(2000); // longest possible value
                end-ds;
                end-ds;
                dcl-ds ds2_array likeds(ds1_array);
                clear ds1_array;
                clear ds2_array;
                DATA-GEN ds1 %data(output1) %gen('MYGENPGM' : ds1_array);
                DATA-GEN ds2 %data(output2) %gen('MYGENPGM' : ds2_array);
                if ds1 <> ds2;
                go through the arrays to see what's different
                endif;
                Hi Barbara Morris, thanks for your suggestion. I find that this DATA-GEN is a V7R3 enhancement. However, I am running V7R1 and hence would not be able to use this.

                Comment


                • #11
                  Originally posted by JonBoy View Post
                  No - it doesn't have such a compare. But use SQL and system tables not the QUSLFLD API. Much easier.

                  I think I'm going to play with the technique a bit in a few minutes.

                  As I said before though - try to get away from the renaming of fields. Use qualified DS.
                  Hi JonBoy, thanks again. Ya will use qualified DS instead of prefixing the fields. Also, will check on system tables as you suggested. Meanwhile, if you come across any other alternative, let me know.

                  Comment


                  • #12
                    Not sure this is the best set of columns to use but I think this would work:

                    select column_name, column_heading, storage, ordinal_position from SYSCOLUMNS
                    where table_name = 'PRODUCT' and system_table_schema = 'PARTNER400' order by ordinal_position;

                    Then loop through the result set.

                    Comment


                    • #13
                      Originally posted by JonBoy View Post
                      Not sure this is the best set of columns to use but I think this would work:

                      select column_name, column_heading, storage, ordinal_position from SYSCOLUMNS
                      where table_name = 'PRODUCT' and system_table_schema = 'PARTNER400' order by ordinal_position;

                      Then loop through the result set.
                      Thanks again. But, can you provide additional details of how to use this with the DS i have? So, currently I have 2 DS - one storing the before image of 20 fields and another storing after image of the same 20 fields. How should i compare the fields using the approach that you have mentioned? Should i write DS to a new file and then compare based on the offset positioning?

                      Comment


                      • #14
                        OK - here's a very simple program. In the end I used DSPFFD with an outfile for reasons that I won't get into here.

                        Code:
                               ctl-opt  Option(*NoDebugIO : *SrcStmt )  DftActgrp(*No);
                        
                               // FieldData is the oufile from:
                               //   DSPFFD FILE(PRODUCT) OUTPUT(*OUTFILE) OUTFILE(FIELDDATA)
                               // Normally that would be built in to this program via QCMDEXC or system()
                        
                               dcl-f  FieldData;
                        
                               dcl-ds  before  ExtName('PRODUCT')  Qualified  Inz  end-ds;
                        
                               dcl-ds  after   ExtName('PRODUCT')  Qualified  Inz  end-ds;
                        
                               dcl-ds  fieldDetail  Dim(99)  Qualified;
                        
                                  startPosn  Like(WHIBO);
                                  length     like(WHFLDB);
                                  name       like(WHFLDI);
                                  dataType   like(WHFLDT);
                        
                               End-Ds;
                        
                               dcl-s  i      int(5);
                               dcl-s  count  int(5);
                        
                               read FieldData;
                        
                               DoU  %Eof(FieldData);
                        
                                  count += 1;
                        
                                  fieldDetail(count).name      = WHFLDI;
                                  fieldDetail(count).dataType  = WHFLDT;
                                  fieldDetail(count).startPosn = WHIBO;
                                  fieldDetail(count).length    = WHFLDB;
                        
                                  read FieldData;
                        
                               EndDo;
                        
                               // Just to show the logic works change a couple of values
                        
                               after.STOH = 99;
                               after.SELLPR = 123.45;
                        
                               For i = 1 to count;
                        
                                  if %subst(after:fieldDetail(i).startPosn:fieldDetail(i).length) <>
                        
                                     %subst(before: fieldDetail(i).startPosn:fieldDetail(i).length);
                                     // Field has changed so report it
                                     // Add whatever logicx needed to rep[ort the before and adfter values
                                     Dsply ('Field ' + fieldDetail(i).name + ' has changed');
                        
                                  EndIf;
                        
                               EndFor;
                        
                               *InLr = *On;
                        If you have any questions just holler

                        Comment


                        • #15
                          Hi JonBoy, I can't thank you enough for taking so much time to provide me with more details. I was contemplating with QUSLFLD API and SYSCOLUMNS to fetch the details.

                          This really solves my issue. I will run through my program and get back to you if I still have any queries. Appreciate your help again. Thank you.

                          Comment


                          • JonBoy
                            JonBoy commented
                            Editing a comment
                            My pleasure. I did a preliminary version with SQL and SYSCOLUMNS but since I didn't need any of the features of SQL it just added lines of code and the data was harder to use. Likewise QUSLFLD - just more complex to handle than I needed. Since this will end up as a teaching example I wanted to keep the logic as simple as possible so as to focus on the "how do I know which field changed" aspect - and hence the use of DSPFFD.
                        Working...
                        X