Advertisement
Help Keep Boards Alive. Support us by going ad free today. See here: https://subscriptions.boards.ie/.
If we do not hit our goal we will be forced to close the site.

Current status: https://keepboardsalive.com/

Annual subs are best for most impact. If you are still undecided on going Ad Free - you can also donate using the Paypal Donate option. All contribution helps. Thank you.
https://www.boards.ie/group/1878-subscribers-forum

Private Group for paid up members of Boards.ie. Join the club.

Bash Script Read Large File

  • 30-03-2009 03:01PM
    #1
    ridonkulous
    Registered Users, Registered Users 2 Posts: 528 ✭✭✭


    Hi All,

    Im back again with more bash problems. I have a script that reads in a file line by line using "while read line". The problem I have is the file is rather large ~5Gb. As the file is large the execution of the script takes forever. I need to know if there is a way of improving the speed of the execution. Basically what the script does is read in the file, places the lines where the first field matches into a seperate file and then performs some simple arithmetic with the fields in the new file. Any help would be very much appreciated. Thanks in advance.


Comments

  • #2
    Thomas_S_Hunterson
    Closed Accounts Posts: 6,151 ✭✭✭Thomas_S_Hunterson


    Could you just use grep to get the required lines and > them into the new file.


  • #3
    ridonkulous
    Registered Users, Registered Users 2 Posts: 528 ✭✭✭ridonkulous


    Yes. Yes i could. So simple and yet I managed to miss it. Im too used to programming in C++. Thanks for that.


Advertisement