ibmi-brunch-learn

Announcement

Collapse
No announcement yet.

Using grep to search all files in the directory for a string

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Using grep to search all files in the directory for a string

    Hi All,

    I am trying to write a program that will search all the files in a directory(&PATH) for a string(&SEARCH). The program is supposed to return the name of all the files that contain the string. Here's what I have so far. I tried running the program but STDOUT turns out empty even when I know there is at least one file with the string I am looking for. Am I missing something?

    Code:
    PGM (&PATH &SEARCH &ERROR)                                     
    
    DCL &PATH *CHAR LEN(21)                                        
    DCL &SEARCH *CHAR LEN(24)                                      
    DCL &ERROR *CHAR LEN(10)                                       
    DCL &CMD *CHAR LEN(100)                                        
    DCL &QUOTE *CHAR LEN(1) VALUE('''')                            
    
    CHGVAR &ERROR VALUE(' ')                                       
    
    CD DIR(&PATH)                                                  
    MONMSG MSGID(CPFA09C) EXEC(DO)                                 
      CHGVAR &ERROR VALUE('NOT_AUTH')                              
      RETURN                                                       
    ENDDO                                                          
    
    CHGVAR &CMD VALUE('grep -l -i ' !! &QUOTE !! &SEARCH !! &QUOTE)
    
    CLRPFM QTEMP/STDOUT                  
    
    MONMSG MSGID(CPF0000)                                         
    
    OVRDBF FILE(STDOUT) TOFILE(QTEMP/STDOUT) OVRSCOPE(*CALLLVL)   
    
    QSH CMD(&CMD)                                                 
    
    DLTOVR FILE(STDOUT) LVL(*)                                    
    
    ENDPGM

  • #2
    In order to grep to search the whole directory, you need to specify the -R switch.

    Comment


    • #3
      It works now! Thanks! Is there a way to make it search faster? The directories that need to be search can contain up to 10,000 files

      I plan to have this program called by another batch job which processes a lot of records. I can't have my program slowing down the calling program.

      Here's the new code:
      Code:
       
       PGM (&PATH &SEARCH &ERROR)                                       DCL &PATH *CHAR LEN(21)                                         DCL &SEARCH *CHAR LEN(24)                                       DCL &ERROR *CHAR LEN(10)                                        DCL &CMD *CHAR LEN(100)                                         DCL &QUOTE *CHAR LEN(1) VALUE('''')                              CHGVAR &ERROR VALUE(' ')                                         CD DIR(&PATH)                                                   MONMSG MSGID(CPFA09C) EXEC(DO)                                    CHGVAR &ERROR VALUE('NOT_AUTH')                                 RETURN                                                        ENDDO                                                            CHGVAR &CMD VALUE('grep -F -l -R -i ' !! &QUOTE !! &SEARCH !! &QUOTE+                !! ' ' !! &PATH)  CLRPFM QTEMP/STDOUT                    MONMSG MSGID(CPF0000)                                           OVRDBF FILE(STDOUT) TOFILE(QTEMP/STDOUT) OVRSCOPE(*CALLLVL)     QSH CMD(&CMD)                                                   DLTOVR FILE(STDOUT) LVL(*)                                      ENDPGM
      grep: 001-2103 Error found getting information for file or directory -l. No such path or directory.
      grep: 001-2107 Error found processing pattern file -l.

      Comment


      • #4
        I don't know any way to make it faster. You could try using different grep tools (such as the one from Linux) to see if it works faster than the one IBM provides with QShell, maybe that'd help?

        Having said that, I would expect that a search of tens of thousands of files would indeed take a lot of time. When organizations want to search a large amount of data, they don't typically use a tool like grep, but instead put the data into an indexed database designed for searching. For example, Google does not download everyone's web pages and then use grep on them, this would be very slow. It instead uses complex algorithms to get the key details from each page, put them in a searchable database, updates the indexes periodically, and uses the databases for searching.

        Of course... there's a downside to that approach, too. It requires more expertise to set up, and there's a lag between the time the document is updated and the time the search indexes are updated. So searching is much faster, but at the expense of paying engineers to maintain it and being willing to wait until the updates occur in the database. And, of course, your data set size would never be nearly as large as Google's.

        Comment


        • #5
          Is there a way to code a program in RPG that will search all files in the IFS for the string? Would that be faster?

          Comment


          • #6
            Is it possible? Of course.

            I don't know if it'd be faster. It depends on how well its written vs. how well the grep program is written....

            Comment


            • #7
              Originally posted by JustinWithoutAnE View Post
              Is there a way to code a program in RPG that will search all files in the IFS for the string? Would that be faster?
              Working with the IFS in RPG IV

              https://www.scottklement.com/rpg/ifs_ebook/

              Comment

              Working...
              X