VB Out of string space error

mountainyman

I have an application which unzips files. As part of this process it loads the file contents into memory as a string.

I am now (with very very large zips) getting the error

Run Time error 14
Out of string Space

Anyone have any detailed suggestions?

Many Thanks

MM

Mutant_Fruit

don't load it into memory :P

Detailed suggestion would be to not load the actual data into memory all at once. Stream it to disk. If you need to perform work on it, then only have 1meg in memory at any one time (or whatever) and stream the rest of the file as needed.

mountainyman

How would one do that in VB6 any ideas?

MM

Mutant_Fruit

Use a FileStream, or StreamReader, or whatever the equivalent thing is in VB. I can't tell you exactly how to do it, as i don't know how you're application is working. It should be possible to read the zip index, and hold a list of files and the relevant details in memory. Using that list (which should use well under 50kB of memory) you can select what files you want to decompress/read etc.

Smoggy

i think a vb6 string can hold 2 billion chars ! can you be going over this value ?

Mutant_Fruit

Smoggy wrote:

i think a vb6 string can hold 2 billion chars ! can you be going over this value ?

He can't be going over it as unless he has a 64bit computer, he can't address enough ram to max out that string but doesn't it have to be 2 billion characters of continuous space? That's what he doesn't have.

mountainyman

The file in question has 500 million characters.
What does '2 billion characters of continus space' mean.

MM

Mutant_Fruit

The thing is when you allocate a 500 byte item, your computer will search for the first block of free memory that is 500bytes long and then take that for the string. If you try to allocation 500 million bytes of a string, your computer will search for the first block of free memory thats 500 million bytes long, which may not exist! If it doesn't exist, you'll get an out of memory error.

The moral of the story, don't try to allocate huge objects, it won't work. Assuming that you're only using one byte per character (but thats not actually what's happening) you'd need a block of free memory thats over 470megabytes in size. But i think you'd actually be looking at over a block of free memory that's a gigabyte in size

mountainyman

Mutant_Fruit wrote:

But i think you'd actually be looking at over a block of free memory that's a gigabyte in size

2 bytes per character. The relevant string is a block of un gzipped data which is set up to 'break' the code.

Thing is I can't figure out how to pass the ungzipped data back as a series of strings.

It's a tough one alright.

MM

Mutant_Fruit

Can i see your code for reading in the file?

mountainyman

Delighted to show you the code: The ungzip class references zlib and was taken from

[url]
http://tinyurl.com/qwgal
[/url]

credit is given at the top of the .cls file.:p

Private Function UnGZipFiles(ByRef strFileName As String) As String
' The procedure recives the name of the .gz file, unzips the file into memory and
' creates a new file to hold the contents

    Dim intInFile As Integer
    Dim intOutFile As Integer
    Dim strFileData As String
    Dim strOutFileName As String
    Dim objUnGZip As UnGZipFile

    ' Create the unzip object
    Set objUnGZip = New UnGZipFile

    ' Read the compressed file into a string
    intInFile = FreeFile
    Open strFileName For Binary As #intInFile
    strFileData = Space(LOF(intInFile))
    Get #intInFile, , strFileData
    Close #intInFile

    ' Decompress the string now
    Call objUnGZip.UncompressString(strFileData, Z_GZIP)

    ' Create the output filename
    strOutFileName = GetUnzipFileName(strFileName)
    strOutFileName = Left(strOutFileName, Len(strOutFileName) - 4) & ".ocp"

    ' Write the uncompressed file data to the output file
    intOutFile = FreeFile
    Open (strOutFileName) For Output As #intOutFile
    Print #intOutFile, Left(strFileData, Len(strFileData) - 2)
    Close #intOutFile

    ' Clear the memory
    Set objUnGZip = Nothing

    ' Return the filename to the parser for processing
    UnGZipFiles = strOutFileName

End Function