Advertisement
If you have a new account but are having problems posting or verifying your account, please email us on hello@boards.ie for help. Thanks :)
Hello all! Please ensure that you are posting a new thread or question in the appropriate forum. The Feedback forum is overwhelmed with questions that are having to be moved elsewhere. If you need help to verify your account contact hello@boards.ie

Advanced Text editor recommend?

Options
  • 26-03-2014 5:24pm
    #1
    Registered Users Posts: 27


    hello,

    i need a advanced text editor recommended.
    it needs to be able open files of upto 10gb, 10's of millions of lines
    does not need to be ram restricted
    can edit and remove rows, edit, replace etc
    text to columns if possible

    initially some people told me Vim, but it crashes i found with files this big. opens a "swp" file nad crashes out at about 2gb.

    for example

    data line needs changed from
    2007.03.30 16:01:15.688,1.33755,1.3374,18.1,35.2
    to
    2007.03.30 16:01:15.688,1.33755,18.1,35.2
    this needs done 50 million times or so. notepad and excel are not capable of this!

    oh yes..and ideally a free editor (or low cost)

    thanks


Comments

  • Registered Users Posts: 2,031 ✭✭✭colm_c


    saj1011 wrote: »
    hello,

    i need a advanced text editor recommended.
    it needs to be able open files of upto 10gb, 10's of millions of lines
    does not need to be ram restricted
    can edit and remove rows, edit, replace etc
    text to columns if possible

    initially some people told me Vim, but it crashes i found with files this big. opens a "swp" file nad crashes out at about 2gb.

    for example

    data line needs changed from
    2007.03.30 16:01:15.688,1.33755,1.3374,18.1,35.2
    to
    2007.03.30 16:01:15.688,1.33755,18.1,35.2
    this needs done 50 million times or so. notepad and excel are not capable of this!

    oh yes..and ideally a free editor (or low cost)

    thanks

    Most editors are not designed for this type of use case as they load the file into memory and so bomb out if the file is that large.

    You don't really need an 'editor' to do this, any scripting language would do the job (bash, perl, ruby, php etc), which you could run against the files anytime you want.


  • Registered Users Posts: 1,819 ✭✭✭howamidifferent




  • Moderators, Technology & Internet Moderators Posts: 11,016 Mod ✭✭✭✭yoyo



    Notepad++ is useless for large files. I've used TextPad before to open large files. Not free but it's nagware i.e you can use it for free just put up with a nag screen on open

    Nick


  • Registered Users Posts: 400 ✭✭tagoona


    To be honest - at that size, you're gonna need some server system
    Acutually, a linux install might not be too bad an idea
    sed could be used to replace lines on the fly.
    split could be used to split the files into manageable sizes.


  • Closed Accounts Posts: 8,015 ✭✭✭CreepingDeath


    UltraEdit says it supports files in excess of 4Gb.

    ( Payware )


  • Advertisement
  • Registered Users Posts: 148 ✭✭goalscoringhero


    saj1011 wrote: »
    hello,

    i need a advanced text editor recommended.
    it needs to be able open files of upto 10gb, 10's of millions of lines
    does not need to be ram restricted
    can edit and remove rows, edit, replace etc
    text to columns if possible

    initially some people told me Vim, but it crashes i found with files this big. opens a "swp" file nad crashes out at about 2gb.

    for example

    data line needs changed from
    2007.03.30 16:01:15.688,1.33755,1.3374,18.1,35.2
    to
    2007.03.30 16:01:15.688,1.33755,18.1,35.2
    this needs done 50 million times or so. notepad and excel are not capable of this!

    oh yes..and ideally a free editor (or low cost)

    thanks
    cat file.csv | cut -f1,2,4,5 -d"," > newfile.csv
    


  • Registered Users Posts: 1,931 ✭✭✭PrzemoF


    What system you're using? What was the crash message with vim?
    There are plugins & tricks that can help with editing huge files in vim. (one of them is disabling swap file)


Advertisement