Login ProductsSalesSupportDownloadsAbout |
Home » Technical Support » DBISAM Technical Support » Support Forums » DBISAM General » View Thread |
Messages 1 to 10 of 15 total |
OT - Pointers, comparing memory and TxQuery |
Thu, Jun 13 2013 9:54 PM | Permanent Link |
Adam H. | Hi,
Sorry for the off topic post - but I know there are a few guru's around here that may be able to assist me with a problem I'm having with DBISam and TxQuery. I'll post another message with more detail - but the short of it is this. I have traced the problem down to comparing memory pointers. The code is: Result := Comparemem( ( Buffer1 + SizeOf( Integer ) ), ( Buffer2 + SizeOf( Integer ) ), FRecordBufferSize - SizeOf( Integer ) ) The problem is that I have no idea how to view the actual contents of the memory during debugging. (Since it's a pointer and not a variable). I'd like to be able to determine what the data is that it's seeing - and why it is different. Can anyone advise if it's possible to view the data in memory given the information above? Thanks & Regards Adam. |
Thu, Jun 13 2013 10:28 PM | Permanent Link |
Adam H. | Firstly, summarising - the problem only shows its head when:
- A certain range of records is in the source data. - It only occurs when using DBISAM. (v4.35 Build 2 at present) but not when using BDE - It will perform correctly if the result set has the same data in it - but also a few additional records. ------------- As promised, here's a more detailed explanation of my problem (for anyone who's interested and/or wants more information). I've been using txQuery for some time now, but recently have ran into a bit of a problem where it is not pivoting / grouping correctly. As an example of what's happening, if I have the following data: TICKET PRODUCT QA Value 114661 CBALES 114662 SORGHUM ADM 0.20 114662 SORGHUM MOI 14.9 114662 SORGHUM SCR 1.10 114663 SORGHUM ADM 0.30 114663 SORGHUM MOI 14.5 114663 SORGHUM SCR 1.00 and then run my TxQuery - I SHOULD get the following result: TICKET PRODUCT ADM MOI SCR 114661 CBALES 0.00 0.00 0.00 114662 SORGHUM 0.20 14.9 1.10 114663 SORGHUM 0.30 14.5 1.00 However, when I run my query - I am getting the following: TICKET PRODUCT ADM MOI SCR 114661 CBALES 0.00 0.00 0.00 114662 SORGHUM 0.20 0.00 0.00 114662 SORGHUM 0.00 14.9 0.00 114662 SORGHUM 0.00 0.00 1.10 114663 SORGHUM 0.30 0.00 0.00 114663 SORGHUM 0.00 14.5 0.00 114663 SORGHUM 0.00 0.00 1.00 But where it gets very strange is that if I change the result set and have one (or more) additional records in it (so same data, but include data for ticket 114664 - it performs exactly as expected for tickets 114662 and 114663 and condenses them to the one record). The info above is an extract from what the query really sorts through. Adding to the strange behaviour - I have a demonstration project of the error using the BDE instead of DBISAM (so I can submit a bug), using exactly the same data, and it performs correctly as expected. I have the demo project running both a TQuery, and a TDBISAMQuery. The TQuery works correctly, whereas when using the TDBISAMQuery component it misbehaves and shows the above problems. If anyone has TxQuery (freely available from the web now I believe) and would like me to send through a copy of my demo project with the offending data, please drop me an email and I'll send you a copy. At this stage though, I'd just be happy to view the data that it's comparing that's sitting in the memory to see if I can get a glimpse of why it's misbehaving. Cheers Adam. |
Fri, Jun 14 2013 4:17 AM | Permanent Link |
Roy Lambert NLH Associates Team Elevate | Adam
You don't mention which version of Delphi you're using and that might have some impact. As far as displaying memory is concerned you should be able to do that by using the ^ or @ operators (again I'm incompetent with them). A start point to look at is http://rvelthuis.de/articles/articles-pointers.html Another little thought. The test Result := Comparemem( ( Buffer1 + SizeOf( Integer ) ), ( Buffer2 +SizeOf( Integer ) ), FRecordBufferSize - SizeOf( Integer ) ) From my limited understanding of memory operations and all things pointers it looks as though you're trying to compare a full record less the record number. Buffer1 & Buffer2 are the records? If I'm right you should be able to walk along the "fields" until you find which one(s) fail the test eg if the first "field" after the record number was an integer if Comparemem( ( Buffer1 +SizeOf( Integer )+ SizeOf( Integer ) ), ( Buffer2 +SizeOf( Integer ) +SizeOf( Integer ) ), FRecordBufferSize -( SizeOf( Integer +SizeOf( Integer ) )) ) then ShowMessage('ok') else ShowMessage('found it'); The akward one might be string fields since the size of string in the post D2009 days insn't a simple column size but column size * SizeOf(Char) or somesuch. Not a lot of help but it may get you going in some direction right or wrong. Roy Lambert |
Fri, Jun 14 2013 2:23 PM | Permanent Link |
Tim Young [Elevate Software] Elevate Software, Inc. timyoung@elevatesoft.com | Adam,
<< If anyone has TxQuery (freely available from the web now I believe) and would like me to send through a copy of my demo project with the offending data, please drop me an email and I'll send you a copy. >> Fire one off to me - if I get a chance, I'll take a look. I suspect that if TxQuery is doing any straight-up memory comparisons, that it may run into issues since each database can use vastly different buffer arrangement schemes. Tim Young Elevate Software www.elevatesoft.com |
Sun, Jun 16 2013 10:38 PM | Permanent Link |
Adam H. | > Fire one off to me - if I get a chance, I'll take a look. I suspect
> that if TxQuery is doing any straight-up memory comparisons, that it may > run into issues since each database can use vastly different buffer > arrangement schemes. Thanks for that. I have sent through an email with TXquery (just incase you don't have it), along with the example project that contains data for both DBISAM and BDE and a demonstration of the conversion with both. Best Regards Adam. |
Sun, Jun 16 2013 10:56 PM | Permanent Link |
Adam H. | Hi Roy,
> You don't mention which version of Delphi you're using and that might have some impact. > > As far as displaying memory is concerned you should be able to do that by using the ^ or @ operators (again I'm incompetent with them). A start point to look at is http://rvelthuis.de/articles/articles-pointers.html Thanks for your reply.. Sorry - I'm using Delphi 2007. I've tried using the ^ operator, and only get the result of #2 each time. (Even when comparemem returns false). I believe you are right about comparing less the record number, but I'm not sure how to walk along the field numbers. (I'm not sure what part of that actually even refers to a field number). I fear I'm a little out of my depth when it comes to pointers and buffers. I was hoping that ^buffer1 would return some sort of result where I could physically see the actual content, but I think my expectations are incorrect. Cheers Adam. |
Mon, Jun 17 2013 3:52 AM | Permanent Link |
Roy Lambert NLH Associates Team Elevate | Adam
>Sorry - I'm using Delphi 2007. That takes one piece out of the puzzle. >I've tried using the ^ operator, and only get the result of #2 each >time. (Even when comparemem returns false). > >I believe you are right about comparing less the record number, but I'm >not sure how to walk along the field numbers. (I'm not sure what part of >that actually even refers to a field number). I think Tim could be right here (surprise, surprise) and that the two systems could be arranging things and storing then differently. I know you've sent the info to Tim but out of interest can you send it to me as well. I'd just like to have a look. >I fear I'm a little out of my depth when it comes to pointers and >buffers. I'm not a little out of my depth - I'm drowning, waving my arms and legs and taking in about a gallon of water a minute! >I was hoping that ^buffer1 would return some sort of result >where I could physically see the actual content, but I think my >expectations are incorrect. All right thinking languages refuse to allow programmers direct access to memory. A good thing in my mind. Roy Lambert [Team Elevate] |
Mon, Jun 17 2013 7:12 PM | Permanent Link |
Adam H. | Hi Roy,
>> I believe you are right about comparing less the record number, but I'm >> not sure how to walk along the field numbers. (I'm not sure what part of >> that actually even refers to a field number). > > I think Tim could be right here (surprise, surprise) and that the two systems could be arranging things and storing then differently. I know you've sent the info to Tim but out of interest can you send it to me as well. I'd just like to have a look. Too easy. I've sent a copy to your email address. Tim may be right. I don't even know what a buffer is in regards to databases, so that's a bit of a mystery to me. From what I can tell in the code, it loads data from two records into a component of it's own, and then does a comparison on that data. My thought would be if it loads the data into it's own component first - then maybe that should eliminate any differences between databases, so maybe the problem is elsewhere in the code. (Maybe there's a problem when it copies the data). The hurdle I face is that when I look at this code, it took me the good part of a day just to find the comparemem function due to my struggle with tracing pointers and a whole lot of other coding in there I have nothing about. >> I fear I'm a little out of my depth when it comes to pointers and >> buffers. > > I'm not a little out of my depth - I'm drowning, waving my arms and legs and taking in about a gallon of water a minute! Nice. I'm not even sure why people use pointers. To me, it seems easier just to pass the variable through the function. Maybe it is faster to use pointers, I'm not sure. >> I was hoping that ^buffer1 would return some sort of result >> where I could physically see the actual content, but I think my >> expectations are incorrect. > > All right thinking languages refuse to allow programmers direct access to memory. A good thing in my mind. Probably - except when you want to see what the results are. Cheers Adam. |
Tue, Jun 18 2013 4:23 AM | Permanent Link |
Roy Lambert NLH Associates Team Elevate | Adam
>Too easy. I've sent a copy to your email address. Its arrived - I'll have a look later on. >Tim may be right. I don't even know what a buffer is in regards to >databases, so that's a bit of a mystery to me. From what I can tell in >the code, it loads data from two records into a component of it's own, >and then does a comparison on that data. There's lots of different buffers involved with databases but this one is the record buffer (I think). Its generated when you read a record from disk or wherever and plonk it into memory so you can work on it. Its what's looked at when you do something like tablefield.asstring. >Nice. I'm not even sure why people use pointers. To me, it seems easier >just to pass the variable through the function. Maybe it is faster to >use pointers, I'm not sure. Speed was (probably still is) the main issue but also you can do things when accessing memory directly that you can't do otherwise, or not very easily eg if you know where screen memory is then altering it directly, especially in the early days of PCs, was a lot faster than calling an OS routine to do it for you. >>> I was hoping that ^buffer1 would return some sort of result >>> where I could physically see the actual content, but I think my >>> expectations are incorrect. >> >> All right thinking languages refuse to allow programmers direct access to memory. A good thing in my mind. > >Probably - except when you want to see what the results are. Yup, but you should never have to do that. I may be wrong but in the languages I used before Delphi (ie mainly pre PC days) I had no way to access memory directly. That was the domain of the systems programmers. You could recognise them by the open toed sandals, beards, scraggy hair do and bemused expressions. Roy |
Tue, Jun 18 2013 7:34 AM | Permanent Link |
Roy Lambert NLH Associates Team Elevate | Adam
I've started to have a look and I don't think its the comparemem routine. IsEqual (the function that calls comparemem) is called from DoGroupBy, DoTransform, and CreateResultSet. I added some debug code to IsEqual if result then sl.Add(IntToSTr(Recno1)+'x'+IntToStr(recno2)+' .T.') else sl.add(IntToStr(Recno1)+'x'+IntToStr(recno2)+'.F.'); and altered the parameters to pass in a string indicating where it came from and it seems to indicate that the problem comes from DoTransform. Only DoTransform and DoGroupBy get called so its got to be one of those. Looking through the code I'd guess its somewhere in between the two calls to IsEqual but it could also be in Transf_CreateSelectValues. Without spending a lot of time tracking through it its hard for me to say. If you look at the table below C and D are the first & second call to IsEqual. In all cases its comparing a record 1 and 2 but what seems to have happened is that the number 2 record isn't being changed properly. My advice is to forget the comparemem thing and concentrate on DoTransform. Could be totally wrong though. BDE DBISAM C 1x2.F. C 1x2.F. D 1x2.F. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2.F. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. C 1x2.F. C 1x2.F. D 1x2 .T. D 1x2.F. Roy Lambert |
Page 1 of 2 | Next Page » | |
Jump to Page: 1 2 |
This web page was last updated on Friday, March 29, 2024 at 03:30 AM | Privacy PolicySite Map © 2024 Elevate Software, Inc. All Rights Reserved Questions or comments ? E-mail us at info@elevatesoft.com |