Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie

Bash Script Read Large File

Options
  • 30-03-2009 3:01pm
    #1
    ridonkulous
    Registered Users Posts: 528 ✭✭✭


    Hi All,

    Im back again with more bash problems. I have a script that reads in a file line by line using "while read line". The problem I have is the file is rather large ~5Gb. As the file is large the execution of the script takes forever. I need to know if there is a way of improving the speed of the execution. Basically what the script does is read in the file, places the lines where the first field matches into a seperate file and then performs some simple arithmetic with the fields in the new file. Any help would be very much appreciated. Thanks in advance.


Comments

  • Options
    #2
    Thomas_S_Hunterson
    Closed Accounts Posts: 6,151 ✭✭✭Thomas_S_Hunterson


    Could you just use grep to get the required lines and > them into the new file.


  • Options
    #3
    ridonkulous
    Registered Users Posts: 528 ✭✭✭ridonkulous


    Yes. Yes i could. So simple and yet I managed to miss it. Im too used to programming in C++. Thanks for that.


Advertisement