Hello,

We are currently porting legacy software from DBISAM 3 to 4. We have a backup utility that stores whole tables into blob fields.

The following line always worked fine n V3 :

BKData.FieldByName('TBL_FIELDS').AsString := ZCompressStr(MimeEncodeString(BBuf.DataString), zcDefault);

BBuf is a TStringStream into which the content of the BLB file was copied.

Its size is roughly 120Mo, storing jpg data.

With DBISAM 4, we now get an "out of memory" exception in the Post method.

I have already tried various solutions along the lines of :


  bf := TMemoField(BKData.FieldByName('TBL_FIELDS'));
  bs := TDBISAMBlobStream.Create(bf, bmWrite);
  chn := ZCompressStr(MimeEncodeString(BBuf.DataString), zcDefault);
  bs.Write(chn, length(chn));
  bs.Seek(0, soFromBeginning);


And I still get an exception trying to post it.

Any suggestions ?
Thanks

Gaunet Patrick wrote:

> TDBISAMBlobStream

I've not used this myself, but some code I have access to includes the following, which might help?

 Field := TBlobField(Table.FieldByName('something'));

      if FSomething.Count = 0 then
        // if there are no somethings then just clear the field
        Field.Clear
      else begin
        // otherwise save the somethings to it
        BlobStream := TDBISAMBlobStream.Create(Field, bmWrite);
        try
          FSomething.SaveToStream(BlobStream);
        finally
          // this must be done before the record is posted
          BlobStream.Free;
        end;
      end;

--

Matthew Jones

Gaunet


Its a long time since I used DBISAM but I seem to remember some of these sort of things. If my memory is correct its not the size of the data that's the problem its the characters in there (possibly #0) which I presume was the reason for mime encoding and then compressing.

Try just stuffing in a a massive string of printable characters and see if its OK. If so its the data

Roy Lambert

Gaunet Patrick wrote:

Hello,

We are currently porting legacy software from DBISAM 3 to 4. We have a backup utility that stores whole tables into blob fields.

The following line always worked fine n V3 :

BKData.FieldByName('TBL_FIELDS').AsString := ZCompressStr(MimeEncodeString(BBuf.DataString), zcDefault);

BBuf is a TStringStream into which the content of the BLB file was copied.

Its size is roughly 120Mo, storing jpg data.

With DBISAM 4, we now get an "out of memory" exception in the Post method.

I have already tried various solutions along the lines of :


  bf := TMemoField(BKData.FieldByName('TBL_FIELDS'));
  bs := TDBISAMBlobStream.Create(bf, bmWrite);
  chn := ZCompressStr(MimeEncodeString(BBuf.DataString), zcDefault);
  bs.Write(chn, length(chn));
  bs.Seek(0, soFromBeginning);


And I still get an exception trying to post it.

Any suggestions ?
Thanks

--------------------------------
Thanks to both of you :
Matthew : I'd never have thought of freeing the blob before posting !
Roy : you were right : data is the culprit !

I tried stuffing a string up to 4Go, and it still worked fine :

  bf := TBlobField(BKData.FieldByName('TBL_FIELDS'));
  bs := TDBISAMBlobStream.Create(bf, bmWrite);
  //chn := StringOfChar('a', 4294967296);                                       //<-- This works like a charm! 4Gb string !
  chn := MimeEncodeString(BBuf.DataString);                               //<-- this raises an mem error ! (only 120Mb)
  ts := TStringStream.Create(chn);
  ts.Seek(0, soFrombeginning);
  try
    bs.CopyFrom(ts, length(ts.DataString));
  finally
    // this must be done before the record is posted
    bs.Free;
  end;
  BKData.Post ;

Any idea what I could do to get acceptable data ? As you can see, I tried to remove ZCompress, to no avail. Isn't MimeEncode supposed to yield a "clean" ASCII string ?

Gaunet


>Thanks to both of you :
>Matthew : I'd never have thought of freeing the blob before posting !
>Roy : you were right : data is the culprit !
>
>I tried stuffing a string up to 4Go, and it still worked fine :

That surprises me a bit - in my antiquated version of DBISAM the max blob size is ONLY 2GB

You are right mime encoding is supposed to convert everything to printable characters (or thereabouts) but without looking at the code (and your mime encoding implementation may vary from mine)  I can't remember what it does with #0

Next suggestion is to parse the output of ZCompressStr(MimeEncodeString(BBuf.DataString), zcDefault

1. is the length what is expected
2. are there any non printable characters (probably just need to look at < #32)

I don't know if it makes any difference but are you using a memo field or a blob field?

Roy Lambert

<< With DBISAM 4, we now get an "out of memory" exception in the Post method.

I have already tried various solutions along the lines of :


  bf := TMemoField(BKData.FieldByName('TBL_FIELDS'));
  bs := TDBISAMBlobStream.Create(bf, bmWrite);
  chn := ZCompressStr(MimeEncodeString(BBuf.DataString), zcDefault);
  bs.Write(chn, length(chn)); >>

If you stop this code in the debugger at the chn := line, what is the reported length of the chn string ?

Tim Young
Elevate Software
www.elevatesoft.com

Hello,

Unfortunately, we are migrating one step at a time...
So, the Delphi version we use is still Good Old D5 (yes, not XE5, but D5, back to 1999...)

So no unicode issue there.

I haven't had the oporturnity to run the code from the debugget today, but I hink I remember that the size is around 160Mb. More detailed information in a few days.

Thanks for your time
Yves

I have just run the code in the debugger, the exact size of the temporary string is : 167220568.

Hope this helps,

Yves

Patrick,

<< I have just run the code in the debugger, the exact size of the temporary string is : 167220568. >>

Please see the project that I emailed you that shows how to do this without any issues.  It doesn't use any 3rd party code, however, so that is the primary difference. I suspect that the issue is somewhere in the MIME encoding or compression code, but I didn't have a chance to debug it further.

Can you reproduce the problem in a cut-down D5 project ?  By "cut-down", I mean one without any 3rd party references.

Tim Young
Elevate Software
www.elevatesoft.com