Icon View Thread

The following is the text of the current message along with any replies.
Messages 1 to 10 of 16 total
Thread Temp Folder Location
Fri, Mar 31 2017 5:05 AMPermanent Link

David

Just a quick question which I think I already know the answer too but just wondered if anyone has any comments.

Recently my server has had a high disk I/O load which has slowed the server down, along with the database.  Once the I/O load drops it works perfectly.

I considering moving the temp folder location to a separate drive on the same server which has a faster hard drive.  I am hoping this will help speed up the database access a bit plus take the high load that happens occasionally from the other drive.  

The reason for the high I/O load is because I have a Dell H200 Perc card in Raid 1 and this, as I have just found out has no disk cache, so the disk writes are crap.  I am intending in the longer term to replace the server but just wondering if the above idea might help in the short term.

Cheers
David.
Fri, Mar 31 2017 7:09 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

David

I assume you're intending to relocate the server's temporary folder and not Windows? If that's the case, and the data is on the Windows drive you might want to consider moving the Windows temp folder and paging file as well. I bunged an SSD into my laptop and moved them to spinning rust because Windows insists on ignoring the fact that I have oudles of RAM and writes stuff to the paging file every so often and becomes unhappy if its not there or big enough. The temp folder got moved so the accumulation of webjunk doesn't bother my SSD. Naturally the ElevateDB private directories stay on the SSD.


Just remember that high disk I/O doesn't guarantee that's where the bottleneck is Smiley


Roy Lambert
Fri, Mar 31 2017 9:51 AMPermanent Link

David

Hi Roy.

Sorry I wasn't clear.  I am only thinking of changing where DBISam stores its temporary files, not Windows at the moment anyway.

Roy Lambert wrote:

David

I assume you're intending to relocate the server's
Fri, Mar 31 2017 10:30 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

David


>Sorry I wasn't clear. I am only thinking of changing where DBISam stores its temporary files, not Windows at the moment anyway.

You were clear enough which is why I assumed that was what you meant Smiley

A few questions:

What do you mean by a high I/O load? You seem to be blaming the PERC card for this which I don't understand.

What is the speed / cache difference between the two HDDs?

Are they on different controllers?

Is the server being used for anything else which could impact things?

Moving the DBISAM temporary directory to a faster drive should help to some level depending on what's going on. If the problem is caused by the creation / erasure of lots of temporary files (say caused by some whopping big transactions or lots of canned queries) then it probably will. If its caused by table restructuring or simple table updates then probably not.

The best bet is to try it and see there should be no downside even if there's no upside.

Roy
Fri, Mar 31 2017 11:10 AMPermanent Link

David

Hi Roy.

The Dell Perc H200 has no cache and in fact it disables the cache on the 2 Raid drives I am using so everything has to be I/O queued until the write op has been committed to disk.  I am seeing high I/O response times sometimes into the high hundreds of Ms for various temporary tables and other files.  This has come to light as our IT department has been rolling out updates with SCCM recently that caused a lot of disk writes and slowed everything right down to a grinding halt.

The other disk I am using is a non raid drive and the controller allows the cache be enabled so it is technically much faster than the one using Raid on the Perc H200 card.  

My theory is that if I could reduce the number of disk writes by separating the DBISam temporary files to the faster disk that it might help reduce any I/O wait caused by many temporary files that DBISam creates.  

I realise it will not fix everything, but just wondering if it will possibly help. Also is there any reason why having the DBISam tables on one drive and the temp files created by DBISam on a separate drive that would cause a problem or have a negative impact?

Roy Lambert wrote:

David


>Sorry I wasn't clear. I am only thinking of changing where DBISam stores its temporary files, not Windows at the moment anyway.

You were clear enough which is why I assumed that was what you meant Smiley

A few questions:

What do you mean by a high I/O load? You seem to be blaming the PERC card for this which I don't understand.

What is the speed / cache difference between the two HDDs?

Are they on different controllers?

Is the server being used for anything else which could impact things?

Moving the DBISAM temporary directory to a faster drive should help to some level depending on what's going on. If the problem is caused by the creation / erasure of lots of temporary files (say caused by some whopping big transactions or lots of canned queries) then it probably will. If its caused by table restructuring or simple table updates then probably not.

The best bet is to try it and see there should be no downside even if there's no upside.

Roy
Fri, Mar 31 2017 11:31 AMPermanent Link

Roy Lambert

NLH Associates

Team Elevate Team Elevate

David


DBISAM doesn't really care where your files are as long as it can find them and grab a handle for them - its down to Windows to sort it out once you've given the path.

Roy Lambert
Fri, Mar 31 2017 11:38 AMPermanent Link

David

Thanks Roy.

That is what I though, I was just looking to see if anyone had any experience of doing this and if it had any ill effect.

Cheers
David.

Roy Lambert wrote:

David


DBISAM doesn't really care where your files are as long as it can find them and grab a handle for them - its down to Windows to sort it out once you've given the path.

Roy Lambert
Mon, Apr 3 2017 11:56 AMPermanent Link

Tim Young [Elevate Software]

Elevate Software, Inc.

Avatar

Email timyoung@elevatesoft.com

David,

<< Recently my server has had a high disk I/O load which has slowed the server down, along with the database.  Once the I/O load drops it works perfectly. >>

The Windows file system should mitigate this normally via its caching.  However, this may not apply if you're committing transactions or flushing the file buffers to disk:

http://www.elevatesoft.com/manual?action=viewtopic&id=dbisam4&product=rsdelphi&version=XE&topic=Buffering_Caching

under "OS Buffering".

and

http://www.elevatesoft.com/manual?action=viewtopic&id=dbisam4&product=rsdelphi&version=XE&topic=Transactions

under "Flushing Data to Disk During a Commit".

Interesting about RAID and the disk controller.  I'll have to bookmark that one. Smile

Tim Young
Elevate Software
www.elevatesoft.com
Wed, Apr 5 2017 11:58 AMPermanent Link

David

Hi Tim.

Yes it is amazing the things you learn as you go along.  My issues is mainly to do with the speed of the disk, a Windows service SMS Agent was causing the disk to be written too constantly so its active time was always 100%.  Adding to this, the raid card not having any cache and the drives cache being disabled by Mr Dell has only added to the issue I have been having.  I am seeing a massive improvement when I stopped the service so this seems to be the issue in the main, although lots of requests at the same time as well as printing etc also has an impact, but it is only short lived.

So anyway can you see no issue with changing the path or the temp files to a separate drive?  I see lots of temporary files being generated from different clients, my thinking is simple, if I can improve the disk response time, this may improve the over all system speed until I can convince my employer to buy me a new shiny server with Raid Cahce this time Smile

One thing I have found to test the impact of the slow raid card was to setup a local DB and change the temp folder for DBISam to a Sandisk Cruizer USB stick.  When opening the database you do notice a significant drop in performance if you wish to try it.

I have one other question and I am a bit baffled. If I run a query, I see the temp files created, I note the size of one file in explorer 167kb but when I right click and check the properties it states 1.9mb on disk.  I assume this is something to do with compression but still baffled.  What I did was I ended the DBISam Server process and instantly the file size shown in explorer increases to match the size reported in the properties window.  Can you explain what is happening to me?.  I modified my app to have remote compression set to 0 but it still operating as described above.  Is the remote compression in C/S controlled by the server or by the session property?

Thanks Tim.
Regards
David.

Tim Young [Elevate Software] wrote:

Interesting about RAID and the disk controller.  I'll have to bookmark that one. Smile

Tim Young
Elevate Software
www.elevatesoft.com
Wed, Apr 5 2017 2:15 PMPermanent Link

Raul

Globestar Systems

Team Elevate Team Elevate

On 4/5/2017 11:58 AM, David wrote:
> I have one other question and I am a bit baffled. If I run a query, I see the temp files created, I note the size of one file in explorer 167kb but when I right click and check the properties it states 1.9mb on disk.  I assume this is something to do with compression but still baffled.  What I did was I ended the DBISam Server process and instantly the file size shown in explorer increases to match the size reported in the properties window.  Can you explain what is happening to me?.  I modified my app to have remote compression set to 0 but it still operating as described above.  Is the remote compression in C/S controlled by the server or by the session property?

Temp file is open by the DBISAM so windows explorer does not real-time
update the size(press F5 in explorer and it should update). it's not
related to (on the wire) compression at all.

Raul
Page 1 of 2Next Page
Jump to Page:  1 2
Image