In case it helps, here is an example of a query that hangs when the value returned by MarcFieldWithoutNonfiling() is longer than 524 characters. The query is created on-the-fly by the software based on search parameters and the in-memory temporary table has already been created (the Title field is 100 characters wide):


INSERT INTO "\Memory\TempResults-72EB5F32-2BDE-4F64-9EF4-610A32B90E2C" EXCLUSIVE
SELECT
 RUNSUM(1) AS RecNo,
 InvMarc.Number,
 InvCtrl.Call,
 MarcField(InvMarc.Contents,100,'',1,1) AS Author,
 MarcFieldWithoutNonfiling(InvMarc.Contents,245,'',1,1) AS Title,
 MarcFieldNonfilingOnly(InvMarc.Contents,245,'',1,1) AS Nonfile,
 InvCtrl.Category,
 CAST(MarcISBN13(InvMarc.Contents),VARCHAR(13)) AS ISBN,
 FormatLCForSort(InvCtrl.Call) AS CallSort,
 CAST(CAST(MarcCopyrightYear(InvMarc.Contents),VARCHAR(4))+'-01-01',TIMESTAMP) AS Date,
 0 AS Rank,
 TRUE AS Owned,
 CURRENT_TIMESTAMP AS AvailabilityTime,
 SUM(IF((Status='I') AND (UPPER(Location)=UPPER('MMCG')),1,0)) AS AvailableHere,
 MIN(IF((Status='O') AND (UPPER(Location)=UPPER('MMCG')),LastDue,NULL)) AS FirstDueHere,
 SUM(IF((Status='H') AND (UPPER(Location)=UPPER('MMCG')),1,0)) AS HoldingHere,
 SUM(IF((Status='I') AND (UPPER(Location)<>UPPER('MMCG')),1,0)) AS AvailableOther,
 SUM(IF((Status='H') AND (UPPER(Location)<>UPPER('MMCG')),1,0)) AS HoldingOther,
 MIN(IF((Status='O') AND (UPPER(Location)<>UPPER('MMCG')),LastDue,NULL)) AS FirstDueOther,
 MIN(Reserves.StartDate) AS FirstDesired
FROM
 BookbagDetail
JOIN InvCtrl ON (BookbagDetail.MarcNumber=InvCtrl.Number)
 AND ( UPPER(InvCtrl.Location)=UPPER('MMCG') )
 AND ( NOT InvCtrl.Category IN ('A/V','HP','KIT','NON','TV') )
 AND ( NOT InvCtrl.Status='W' )

JOIN InvMarc ON (BookbagDetail.MarcNumber=InvMarc.Number)
LEFT OUTER JOIN Reserves ON ( (InvCtrl.Status='O') AND (InvCtrl.Number=Reserves.TitleNum) AND (InvCtrl.Location=Reserves.Location) AND (Reserves.Status='E') )
WHERE
(BookbagDetail.Number=717)

GROUP BY Number, Call, Category
ORDER BY Rank DESCENDING, "Date" DESCENDING
TOP 10
;

Danny,

<< I'm still having the problem with returning large (525+ character)
strings from custom SQL functions causing a query lockup. >>

Have to distilled this down to just a simple custom function that returns
such a string as a constant to see if it still occurs ?  What is the
resulting data type of the custom function ?

--
Tim Young
Elevate Software
www.elevatesoft.com

Daniel

> The code as-is wouldn't do you much good because it has a lot of
dependencies that you don't have and a lot of extra un-related code that you
don't need. But, here is snip from it:
>
> FMarc := TMarc.Create(FUseANSEL);
>
> if (AnsiCompareText(FunctionName,'MarcField')=0) then begin
>       try
>         FMarc.Marc := FunctionParams[0].AsString;
>         Result := MarcField(FMarc,
>                             FunctionParams[1].AsInteger,
>                             lowercase(FunctionParams[2].AsString+' ')[1],
>                             FunctionParams[3].AsInteger,
>                             FunctionParams[4].AsInteger,
>                             False, False);
>         if Result='' then
>           Result := null;
>       except
>         Result := null;
>       end;
>     end

As the function I posted (which returns a long string) does not error I was
trying to see if there was another factor.  Do you still get the error with
a real simple insert and a cut down function?

John

I hard-coded the long string in my MarcFieldWithoutNonfiling() function and have the same problem: the query locks up. So, I have eliminated the Marc object from being the cause.

I tested by removing the INSERT INTO and the query ran just fine -- and quickly! It might have something to do with that.

However, when the application closes, I get an invalid pointer operation error raised by dbisamlb.DeAllocMem:

date/time         : 2010-09-23, 13:22:44, 221ms
computer name     : DANNY-LAPTOP
user name         : Danny Humphress
operating system  : Windows NT New x64 build 7600
system language   : English
system up time    : 3 days 2 hours
program up time   : 25 seconds
processors        : 2x Intel(R) Core(TM)2 Duo CPU     T9900  @ 3.06GHz
physical memory   : 3021/8123 MB (free/total)
free disk space   : (C206.74 GB
display mode      : 1920x1200, 32 bit
process id        : $6530
allocated memory  : 75.27 MB
command line      : "C:\Surpass\v6\Safari\Desktop\Safari.exe"
executable        : Safari.exe
exec. date/time   : 2010-09-23 13:22
version           : 6.3.2.0
madExcept version : 2.7k
exception class   : EInvalidPointer
exception message : Invalid pointer operation.

Main ($38e4):
00629aba +026 Safari.exe   dbisamlb        dbisamlb.DeAllocMem
00608d72 +05e Safari.exe   dbisamlb        dbisamlb.ResizeBuffer
00404e7a +002 Safari.exe   System    31 +0 System.@ClassDestroy
0059fc62 +026 Safari.exe   dbisamen        dbisamen.TBuffer.Destroy
005b14f5 +281 Safari.exe   dbisamen        dbisamen.TDataCursor.CloseCursor
00565212 +01a Safari.exe   dbisamtb        dbisamtb.TDBISAMDataSet.DestroyHandle
00569d0d +005 Safari.exe   dbisamtb        dbisamtb.TDBISAMTable.DestroyHandle
005651e2 +01e Safari.exe   dbisamtb        dbisamtb.TDBISAMDataSet.CloseCursor
00568c89 +005 Safari.exe   dbisamtb        dbisamtb.TDBISAMDBDataSet.CloseCursor
0054652f +0a7 Safari.exe   DB              DB.TDataSet.SetActive
005462e9 +009 Safari.exe   DB              DB.TDataSet.Close
00568cfc +000 Safari.exe   dbisamtb        dbisamtb.TDBISAMDBDataSet.Disconnect
00562fe0 +018 Safari.exe   dbisamtb        dbisamtb.TDBISAMDatabase.CloseDataSets
00562f8f +00b Safari.exe   dbisamtb        dbisamtb.TDBISAMDatabase.DoDisconnect
005399d6 +072 Safari.exe   DB              DB.TCustomConnection.SetConnected
00539960 +004 Safari.exe   DB              DB.TCustomConnection.Close
00561dee +3e6 Safari.exe   dbisamtb        dbisamtb.TDBISAMSession.StartSession
00561735 +021 Safari.exe   dbisamtb        dbisamtb.TDBISAMSession.SetActive
00560b9f +03f Safari.exe   dbisamtb        dbisamtb.TDBISAMEngine.CloseAllSessions
0055e232 +17a Safari.exe   dbisamtb        dbisamtb.TDBISAMEngine.SetActive
0055e02a +016 Safari.exe   dbisamtb        dbisamtb.TDBISAMEngine.FreeInstance
00404e7a +002 Safari.exe   System    31 +0 System.@ClassDestroy
00459348 +070 Safari.exe   Classes         Classes.TComponent.Destroy
00404ac4 +008 Safari.exe   System    31 +0 System.TObject.Free
00571bfe +01e Safari.exe   dbisamtb        dbisamtb.Finalization
004056c1 +039 Safari.exe   System    31 +0 System.FinalizeUnits
004300a8 +054 Safari.exe   madExcept       madExcept.InterceptFinalizeUnits
00405996 +056 Safari.exe   System    31 +0 System.@Halt0
0096d497 +05f Safari.exe   Safari    66 +6 Safari.Safari
76ba3675 +010 kernel32.dll                 BaseThreadInitThunk

Danny,

<< I hard-coded the long string in my MarcFieldWithoutNonfiling() function
and have the same problem: the query locks up. So, I have eliminated the
Marc object from being the cause. >>

Can you cut everything down to a sample project and still get the lock-up ?
If so, then please send this to me and I'll tell you what the issue is.

<< I tested by removing the INSERT INTO and the query ran just fine -- and
quickly! It might have something to do with that.

However, when the application closes, I get an invalid pointer operation
error raised by dbisamlb.DeAllocMem: >>

Do you still get the invalid pointer error if you remove any calls to the
custom function ?

--
Tim Young
Elevate Software
www.elevatesoft.com

Hi Tim,

It has been a long time since I visited this (I have managed to step around it in code) but it's back and this time I have more information. It looks like dbisamlb.EvaluateCustomFunction and dbisamlb.ResizeBuffer are both trying to release the same block of memory, leading to memory corruption.

I am attaching a sample report from FastMM.

Thanks,
Danny

Danny,

<< It has been a long time since I visited this (I have managed to step
around it in code) but it's back and this time I have more information. It
looks like dbisamlb.EvaluateCustomFunction and dbisamlb.ResizeBuffer are
both trying to release the same block of memory, leading to memory
corruption.  >>

What happens if you strip out your code in the custom function event handler
?  Does it happen then ?  If not, then the issue is with your code.  If you
still get the error after stripping out your code, then please let me know
and I'll look into this further.

--
Tim Young
Elevate Software
www.elevatesoft.com

Tim,

Yes, it still happens when I strip out my code (see earlier post). The problem seems to crop up when there is a large string (>500 or so characters) returned from BLOB custom function. Even a simple function that returns a static string will cause the problem. The memory manager debug report seems to indicate that DBISAM is freeing the same block of memory twice.

Thanks a bunch!
Danny

"Tim Young [Elevate Software]" wrote:

What happens if you strip out your code in the custom function event handler
?  Does it happen then ?  If not, then the issue is with your code.  If you
still get the error after stripping out your code, then please let me know
and I'll look into this further.

Just checking in since I haven't heard anything in a couple of weeks.

Was that FastMM trace helpful? It seems to show memory block corruption occurs within dbisamtb.TDBISAMEngine.CustomFunctionCallback and it looks like the same block of memory is released twice. I don't have the source, so I can't go much further than that.

Thanks!

Daniel

>
> Was that FastMM trace helpful? It seems to show memory block corruption occurs within
dbisamtb.TDBISAMEngine.CustomFunctionCallback and it looks like the same block of memory is released twice. I don't have
the source, so I can't go much further than that.

I just had a look at the source and the only calls to free memory are those allocated within the
evaluationcustomfunction procedure and the variant returned in the custom function.  I would suspect it is this call
(freeing the variant) causing the problem and to trace it will require an example custom function which causes the
error.

John