ibmi-brunch-learn

Announcement

Collapse
No announcement yet.

How to change a LF to a CRLF

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • How to change a LF to a CRLF

    Hi All,

    I've been Googling all day and just can find an answer that works but can't find anything.

    I'm creating a file in the IFS from a CLLE program using CPYTOIMPF but for a blank fields the command leaves 1 blank between the delimiters which the receiving software unbelievably can't handle.

    I therefore then used Qshell to remove the blank but this changes the end of line from a Windows(CRLF) to a Unix(LF) which surprise surprise the receiving software can't handle.

    I tried all day using "sed" to change the LF back to a CRLF but have failed terrible. Has anyone managed to do this and can give me the code required ?

  • #2
    I gave up on CPYTOIMPF long ago and use Open Access handlers instead. Take a look at the one described here: https://authory.com/JonParisAndSusan...dler-Revisited

    I honestly don't recall if that one inserts blanks, I don't think so. But even if it does unlike CPYTOIMPF you control the code that does the IFS write so you can do whatever you want.

    Comment


    • #3
      It leave a blank even if you specify the RMVBLANK parameter? That sounds like a bug.

      Comment


      • #4
        Even if it is, unless it is a problem that IBM introduced recently, I can't see them changing it. It would break too many people.
        Last edited by JonBoy; March 17, 2021, 04:20 PM.

        Comment


        • #5
          If you want an easy utility to convert between LF and CRLF sequences in Qshell or PASE, install the dos2unix package from Yum. Once installed, you can do:

          Code:
          unix2dos myfile.csv
          And it'll convert it to the DOS CRLF end-of-line sequence.

          Having said that... depending on how your QShell script works, you may be able to convince it to keep the CRLF in the first place as described here: https://www.ibm.com/support/pages/ca...removed-qshell

          Or, of course, you could write a program (rather than shell script) to fix things up. It's easy to write code in RPG or C/C++ that converts LF to CRLF, or to strip the blanks in empty strings from the CSV file in a way that does not require it to switch to unix file endings... Heck, for that matter, its probably possible to write one in CL. A lot more awkward than RPG or C, but possible.

          Comment


          • #6
            There should be a way to add the CRLF with sed, but anyway the AWK method is a lot clearer. Try the below:
            awk '{printf "%s\r\n", $0}' input.csv >output.csv

            Comment


            • #7
              Hi Everyone and thanks for your input.

              JONBOY
              This may be the way forward but all programs are already written and the client is not going to pay to completely rewrite them.

              JTAYLOR
              Having Googled it seems it's a feature s re-importing the file back to the AS400 would fail without the blank.

              SCOTT KLEMENT
              a) Unfortunately it's a clients machine so I can't install new software and to be honest I couldn't work out how to either.
              b) My knowledge of Qshell is poor and I copy solutions from Google normally so I might have done this wrong but I entered commands "export QIBM_DESCRIPTOR_STDOUT=CRLN=N" etc and then ran my Sed command but the CR were still stripped.
              c) Same answer as for JonBoy as it already coded and I had no idea until UAT that the receiving package would object to a blank.

              DENNIS SEE
              a) This seem to change only the last line in the file rather than all, is there a tweak needed ?
              b) Any chance you could break the command down, I've tried Googling Awk but can't find any example that simply states what the "%s\r\n",$0 means. I can work out \r is CR and \n is LF and all that I've read seem to point $0 to all lines but I'm not 100% sure there.

              Comment


              • #8
                Originally posted by Chris Knott View Post
                Hi Everyone and thanks for your input.

                JONBOY
                This may be the way forward but all programs are already written and the client is not going to pay to completely rewrite them.
                Not going to attempt to change your mind but you are so mistaken that I want to set the record straight for the sake of others reading this in the future ...

                There is NO rewrite required to use an OA handler. That's the whole point. Unless you regard adding the Handler keyword to the file definition as a rewrite.

                You simply add the appropriate handler keyword to the definition of the output file that currently is the input to CPYTOIMPF and remove CPYTOIMPF from the job stream. Job done. Not exactly rewriting.

                Comment


                • #9
                  Chris,

                  Isn't this discussion about writing a CL program and/or shell script that solves a problem that you're having?! How can that be done if you're not allowed to write any code or place a program on the machine? I'm confused.

                  Comment


                  • #10
                    Hi Chris, my test results of a file with 2 lines, does show that both lines would have a CRLF. Maybe, it is something to do with the CCSID of the file. Mine is 1252, and notice that the hex codes for my test is in ASCII?
                    cat test.csv
                    abc
                    123

                    od -x test.csv
                    0000000 6162 630a 3132 330a

                    awk '{printf "%s\r\n", $0}' test.csv >test2.csv

                    od -x test2.csv
                    0000000 6162 630d 0a31 3233 0d0a

                    The printf syntax of awk takes a format argument, followed by the arguments for printing. %s is for a string variable, and \r\n is the CRLF that you already expected. $0 is the entire line of the input file, and $1, $2, etc, would be the 'words' of the line if you want to print certain words/fields instead of the whole line.

                    If it still doesn't work on your machine, it could be also due to a difference in OS version. But you can check the CCSID of the file first.

                    Comment


                    • #11
                      Hi Dennis,

                      In the end I found I could get rid of the blanks and get the line to end in a CRLF all in one command. As my initial File had CRLF as it was from CPYTOIMPF it was stripped slightly differently by 'awk' . Just in case someone is reading this with the same problem my command ended up as :-

                      export QIBM_CCSID=1252; awk ''{gsub(/ /,"", $0); printf "%s\n", $0}' FromFile.txt > ToFile.txt

                      Comment


                      • Dennis See
                        Dennis See commented
                        Editing a comment
                        Sounds great. As Scott pointed out, CRs are removed by C runtime functions, and they would be preserved if you don't use any utility that goes through these functions. The gsub command you used are removing all blanks, by the way, which may not be what you want. You should replace ", ," with ",," if I understand correctly, ie one blank space between commas to be replaced with nothing.
                    Working...
                    X