Conundrum - Prefix vs. Eval-Corr

[Viking]

Hi,

I am reading a PF that gets populated by another system, and all the fields are defined as character fields, even those fields that contain numeric values. (the file originates as a text file)

I read the file and then write the records to a header and a detail file, manipulating the data a little along the way.

The header file has about 40 fields from the input file, and those fields are named the same in the header file as the header fields from the input file, even though some of the field types are different (for example the numeric fields). Similarly, the detail file has the same field names as the detail fields in the input file, even though again some of the field types are different.

An easy way to move fields with the same names from the input file to the header and detail files is using Eval-Corr, so I have defined a data structure for each of the three files. I do Eval-Corr dsHeader = dsInput and Eval-Corr dsDetail = dsInput, and it moves all the fields that are named the same and are compatible types and ignores the rest. Then for the fields where I need to convert the alpha fields to numeric, I do dsHeader.myField = %dec(dsInput.myField:7:2) and dsDetail.myOtherField = %dec(dsInput.myOtherField:7:2). So that approach takes care of all the header and detail fields that didn't get moved with the EVAL-CORR statements.

This all seems great EXCEPT: Since the same field names are defined in the input file and also the header file, and also in the input and detail file, but some of these have different attributes, I get the error:

RNF7406: Data-Format entry does not match the definition of the input field MYFIELD.

Normally, to deal with this I can just use PREFIX on the files in the F-specs to name the fields differently. But if I do that, my EVAL-CORR won't move any of the fields because now the subfields aren't named the same anymore.

How to deal with this? I'd like the convenience of using EVAL-CORR to move the majority of the fields yet need to rename the fields in order for the files to be defined properly. The only way I can think of is to use SQL for all the I/O instead of native I/O so that I wouldn't have to define the files in the F-specs. Any other ideas?

Thanks.

Current code:

     FFEC010    IF   E           K DISK
     FFEC011    UF A E           K DISK
     FFEC012    UF A E           K DISK

     D ds010           Ds                  LikeRec(IEC010:*input)
     D ds011           Ds                  LikeRec(IEC011:*output)
     D ds012           Ds                  LikeRec(IEC012:*output)

      //-- Variables
     D saveOrderID     s                   Like(ORDERID) Inz

      /Free

       //-- Read through raw order records in FEC010
       Setll *start FEC010;
       Dow not %eof(FEC010);
         Read FEC010 ds010;
         If not %eof(FEC010);

           //-- Write order to order header and detail files
           If ds010.orderID <> saveOrderID;
             Exsr WriteHeader;
             saveOrderID = ds010.orderID;
           Endif;
           Exsr WriteDetail;

         Endif;
       EndDo;

       *inLR = *on;
       Return;

       //------------------------------------------------------------
       // WriteHeader :  Write order header record
       //------------------------------------------------------------
       Begsr WriteHeader;

         Clear IEC011;

         //-- Move the majority of the header fields straight across
         Eval-Corr ds011 = ds010;

         //-- Handle the incompatible fields individually
         ds011.ORDERTS = %date(%subst(ds010.ORDERDATE:1:10))
                       + %time(%subst(ds010.ORDERDATE:11:8));
         ds011.ORDERSUBT  = pToDecimal(ds010.ORDERSUBT);
         ds011.ORDERTAX   = pToDecimal(ds010.ORDERTAX);
         ds011.ORDERSHIP  = pToDecimal(ds010.ORDERSHIP);
         ds011.ORDERDISC  = pToDecimal(ds010.ORDERDISC);
         ds011.ORDERTOTAL = pToDecimal(ds010.ORDERTOTAL);
         ds011.ORDERPAID  = pToDecimal(ds010.ORDERPAID);
         ds011.ORDERREF   = pToDecimal(ds010.ORDERREF);
         ds011.ORDERDUE   = pToDecimal(ds010.ORDERDUE);
         ds011.ORDERQTY   = pToDecimal(ds010.ORDERQTY);

         Write IEC011 ds011;

       Endsr;

       //------------------------------------------------------------
       // WriteDetail :  Write order detail records
       //------------------------------------------------------------
       Begsr WriteDetail;

         Clear IEC012;

         //-- Move the majority of the detail fields straight across
         Eval-Corr ds012 = ds010;

         //-- Handle the incompatible fields individually
         ds012.ITEMLINE = %int(pToDecimal(ds010.ITEMLINE));
         Monitor;
           ds012.ITEMSKU = %int(ds010.ITEMSKU);
         On-Error;
           ds012.ITEMSKU = 0;
         Endmon;
         ds012.ITEMPRICEO = pToDecimal(ds010.ITEMPRICEO);
         ds012.ITEMPRICE  = pToDecimal(ds010.ITEMPRICE);
         ds012.ITEMQTYORD = %int(pToDecimal(ds010.ITEMQTYORD));
         ds012.ITEMQTYINV = %int(pToDecimal(ds010.ITEMQTYINV));
         ds012.ITEMQTYSHP = %int(pToDecimal(ds010.ITEMQTYSHP));
         ds012.ITEMQTYCNC = %int(pToDecimal(ds010.ITEMQTYCNC));
         ds012.ITEMQTYREF = %int(pToDecimal(ds010.ITEMQTYREF));
         ds012.ITEMTAX    = pToDecimal(ds010.ITEMTAX);
         ds012.ITEMDISC   = pToDecimal(ds010.ITEMDISC);
         ds012.ITEMTOTAL  = pToDecimal(ds010.ITEMTOTAL);

         Write IEC012 ds012;

       Endsr;

      /End-Free

[Scott Klement]

Re: Conundrum - Prefix vs. Eval-Corr

One way (which is somewhat ugly, but...) is to use PREFIX, but specify the data structure instead of just modifying the field name. Note the dot at the end of the prefix:

     FFEC010    IF   E           K DISK     PREFIX('DS010.')
     FFEC011    UF A E           K DISK     PREFIX('DS011.')
     FFEC012    UF A E           K DISK     PREFIX('DS012.')

One problem with this approach is that the input/output specs that the compiler generates (under the covers) have a limit of 14 characters for a field name, and it's now using up 6 of those characters for 'DS011.' (etc.) So this will only work if the existing field names are 8 chars long or less... but that's one way to deal with it.

The other way, which is the 'modern' solution, is to use the QUALIFIED F-spec keyword. This prevents it from generating I/O specs entirely, so all I/O must be done through data structures (but you're doing that, anyway) and also record format names MUST be qualfied with the filename when used in the RPG code (which always feels a little weird to me... but it works)

     FFEC010    IF   E           K DISK    QUALIFIED
     FFEC011    UF A E           K DISK    QUALIFIED
     FFEC012    UF A E           K DISK    QUALIFIED

     D ds010           Ds                  LikeRec(FEC010.IEC010:*input)
     D ds011           Ds                  LikeRec(FEC011.IEC011:*output)
     D ds012           Ds                  LikeRec(FEC012.IEC012:*output)

    //  Also change 'IEC010' to 'FEC010.IEC010' everywhere that record format name is used,
    //  such as on your WRITE and UPDATE statements.
    //  Same with FEC011.IEC011 and FEC012.IEC012, etc.

The QUALIFIED F-spec keyword was added in V6R1.

[TedHolt]

Re: Conundrum - Prefix vs. Eval-Corr

EVAL-CORR only makes sense to me in situations where you've got compatible data in various places. For instance, customer number is 5 digits numeric in the customer master file and 5 digits numeric in the sales orders file and 5 digits numeric in the shipments file, and so forth. Well-designed information systems should be like that, but that's often not the case.

In your situation, the same data element (a datum, although we don't use that word) can be defined differently in different data structures. I would be inclined to read each file into (and write out of) its own qualified data structure and copy the fields one at a time between the data structures.

[Viking]

Re: Conundrum - Prefix vs. Eval-Corr

Scott, thank you very much for these suggestions and explanation, I really appreciate it! I went with the QUALIFIED F-spec solution, and I can see that this is going to be useful going forward as well.

The first option you presented reminded me that I had seen that trick a while back but didn't quite understand at the time why it would be useful to use prefix like that, but of course now I see what that accomplishes. I suppose when using that technique we can minimize the negative effects of the name length restriction by just naming the datastructures a, b, and c instead of ds010, ds011, and ds12, for example.

TedHolt,

For instance, customer number is 5 digits numeric in the customer master file and 5 digits numeric in the sales orders file and 5 digits numeric in the shipments file, and so forth. Well-designed information systems should be like that, but that's often not the case.

Yes, we do have the same field defined the same everywhere, and in fact that's what I'm doing with this program... bringing in the data from a temporary file that was populated by another system and then formatting the data properly into our files.

copy the fields one at a time between the data structures

Since Eval-Corr already only moves compatible fields and ignores the rest, I guess I don't see why I would want to take 40 lines of code to copy the compatible fields one at a time when I can do it with one line of code using Eval-Corr. Either way, I have to handle the incompatible fields individually.

I would be inclined to read each file into (and write out of) its own qualified data structure

I'm doing that, right? Maybe I'm not quite understanding what you're suggesting.

[Scott Klement]

Re: Conundrum - Prefix vs. Eval-Corr

I suppose when using that technique we can minimize the negative effects of the name length restriction by just naming the datastructures a, b, and c instead of ds010, ds011, and ds12, for example.

That's true, but maybe using short DS names like a,b,c might sacrifice code readability? It's important to understand that if you use the PREFIX('DS.') method, it's changing the I/O specs that the compiler generates to load into these fields, but... that does not necessarily mean that you have to use those I/O specs! Since your code is reading/write directly via data structures, you aren't using the I/O specs at all, but the compiler still generates them.

So I guess what I'm ultimately saying is that you could potentially do this:

     FFEC010    IF   E           K DISK     PREFIX('A.')
     FFEC011    UF A E           K DISK     PREFIX('B.')
     FFEC012    UF A E           K DISK     PREFIX('C.')

     D A               Ds                  LikeRec(IEC010:*input)
     D B               Ds                  LikeRec(IEC011:*output)
     D C               Ds                  LikeRec(IEC012:*output)
     D ds010           Ds                  LikeRec(IEC010:*input)
     D ds011           Ds                  LikeRec(IEC011:*output)
     D ds012           Ds                  LikeRec(IEC012:*output)

        .
        .
        Read FEC010 ds010;

So, while the compiler generates I/O specs prefixed by A, B or C, you aren't using those I/O specs. You're reading directly into the DS010 (etc) data structures, and therefore you still refer to the ds010.theFieldName fields. The A.TheFieldName fieldnames are generated and will prevent the naming conflict, but aren't otherwise used.

This method is a bit confusing, I think... but works, and has a minimal amount of problem with the length of fieldnames, since the I/O specs have only a 2-byte prefix (allowing the file's fieldnames to be up to 12 characters long... which shouldn't be a problem.) Field length is not an issue on the C-specs where fields can be up to 4096 characters long.... so you can still use long DS names there without issue.

Of course, the QUALIFIED option is even better since it completely eliminates the I/O specs, so you don't have the problem in the first place.

[Viking]

Re: Conundrum - Prefix vs. Eval-Corr

Thanks for the thorough explanation. Yes, I agree that it would be a bit confusing at first to come across source like that ("This clown is using PREFIX and has defined data structures and isn't even using them!"), so I would include some explanatory comments if I ever needed to do this. The QUALIFIED solution is simple and clean. As you point out, the I/O specs aren't there anymore, so other variable definitions based on database fields such as LIKE(MYFIELD) need to be changed to LIKE(MYDS.MYFIELD).

Thanks again!

[tomliotta]

Re: Conundrum - Prefix vs. Eval-Corr

...that's what I'm doing with this program... bringing in the data from a temporary file that was populated by another system and then formatting the data properly into our files.

Is the temporary file close enough that you could create a VIEW that presented the columns with the final attributes that you need? Then the program might not be needed at all. The data could be imported through the VIEW.

[tomliotta]

Re: Conundrum - Prefix vs. Eval-Corr

{Edit: Odd... I'm not sure how this got posted twice... It somehow was sent once and apparently still looked as if it hadn't done anything. A few moments later, I didn't remember sending, and I added a new thought and sent again.}

...that's what I'm doing with this program... bringing in the data from a temporary file that was populated by another system and then formatting the data properly into our files.

Is the temporary file close enough that you could create a VIEW that presented the columns with the final attributes that you need? (Then the program might not be needed at all. The data could possibly be imported through the VIEW.)

As long as you have a good solution, alternatives probably don't matter. It's almost always worth learning/understanding something new anyway.