File Buffer Class

Lu[ifer

I am looking for a file buffer class. Something that has the same basic functionality of a STL queue but stores data on disk instead of in memory. Ideally this class would be a template class. Also it would be nice if it was thread-safe. Am I forgetting anything? It should be written in C++ and/or Win32. I could even live with it if it used MFC.

Has anyone come accross something meeting this criteria? I'd imaging it would be common enough. Having said that, I've looked and this is the closest I found
http://codeguru.earthweb.com/cpp_mfc/CCircularFile.shtml and it's rather poor.

Talliesin

What does the std::basic_fstream lack that you need here?

Lu[ifer

I would like to be able to write code like this...

FileBuffer<MyData> gFileBuffer;

MyData myData;

gFileBuffer.push_back(myData);

gFileBuffer.pop_front(myData);

bool isEmpty = gFileBuffer.empty();

int size = gFileBuffer.size();

Of course, the push and pop methods would write/read a MyData Object to a file (eg. an fstream).

Template<class T>
class FileBuffer {
public:
  FileBuffer(char* filename) {
    f.open(filename, ios::binary);
    s = 0;
  }
  void push_back(T& obj) {
    f.write((char*)&obj, sizeof(obj));
    s++;
  }
  void pop_front(T& obj) {
    if(!s) return;
    f.read((char*)&obj), sizeof(obj));
    s--;
  }
  int size() {return s;}
  bool empty() {return (s==0);}

private:
  fstream f;
  int s;
}


class MyData {
  MyData(int l = 0, char* d  = NULL) {
    length = l;
    data = new char[length];
    memcpy(data, d, length);
  }

private:
  int length;
  char* data;
}

Well I'm sure there's alot to be desired with the above. I'm quite new to this sort of I/O programming. In particualar I would like the FileBuffer class to be thread-safe (i.e. it would have some sort of lock as a private member, and methods call would have to asquire the lock first, etc).

Also, as the MyData class has a char* as a private member, will this get dereferenced and hence written to disk by the call f.write((char*)obj, sizeof(obj))? i doubt it somehow.

Anyway, thanks for any help. I was hoping a class for this sort of thing would already exist (i'm a big fan of NOT reinventing the wheel).

Talliesin

Originally posted by Lu[ifer
Also, as the MyData class has a char* as a private member, will this get dereferenced and hence written to disk by the call f.write((char*)obj, sizeof(obj))? i doubt it somehow.

Well for any type of class to handle MyData objects it is going to have to rely on members defined by MyData or functions that "know" about MyData in the same scope.

In the case of collection classes (including queue) it relies on the constructors, destructor an assignment operators.

A normal fstream would rely on (frequently, but not always inline) operators like:

istream& operator<<(istream& is, MyData& md){
	/* MyData specific code to input into md;
	Perferably the final assignment to md is done in
	one go after all relevant data is read, this makes
	it more exception-safe.*/

	return is;
}

ostream& operator<<(ostream& os, const MyData& md){
	/* MyData specific code to output to os*/

	return os;
}

Now two things are noteworthy at this stage:

1. Although you are currently only considering fstream for your use we define these operators in terms of ostream and istream so that you could use the same code with read-only and write-only streams, and streams other than file streams.
2. it may be possible to define these functions as templates so that they will work with wistream, streams of unusual character types, etc.

This would let your file buffer class use << and >> to implement the push_back and pop_front.

Note that the semantics of your pop_front is different to the STLs which uses a blind pop (that is it pops and returns nothing, if you want to use the object you have to peek before you pop). I don't like the STL style of popping and I'm guessing you don't either if your choice of pop mechanism differs, though in the majority of cases the STL's pop is more efficient in many important cases so it was a good choice I suppose.