Login ProductsSalesSupportDownloadsAbout |
Home » Technical Support » ElevateDB Technical Support » Support Forums » ElevateDB General » View Thread |
Messages 1 to 8 of 8 total |
Poor compression using BACKUP DATABASE |
Tue, Jul 14 2009 3:23 PM | Permanent Link |
Hedley Muscroft | I use the following SQL to allow my customers to backup their databases :-
BACKUP DATABASE "my_db" AS "backup_filename" TO STORE "Backup_Store" COMPRESSION 9 INCLUDE CATALOG; I tested this on a database with has 132 tables and the whole database folder weighs in at 404Mb. The above "BACKUP DATABASE" takes >4 minutes to produce a 150Mb backup file. I then tried using 7zip (www.7zip.com) to produce a ZIP file and the results were similar, although it only took about half the time. Next, I tried creating a .7z file on NORMAL compression and it took less than 60 seconds to produce a 17Mb file! I was so shocked that I extracted all the contents to a different and loaded up the database in EDBManager just to be sure! I also used WinRAR to create a .rar file (with NORMAL compression) and this also took less than 60 seconds but resulted in a 22Mb file. The point is that some of my customers databases are now getting on for a Gb in size and really need to offer them an integrated backup solution with better compression than the aging zlib compression which EDB uses internally. Is there a way to implement a different compression protocol in the "BACKUP DATABASE" routine in order to improve the situation? Many thanks! Hedley |
Tue, Jul 14 2009 4:51 PM | Permanent Link |
Tim Young [Elevate Software] Elevate Software, Inc. timyoung@elevatesoft.com | Hedley,
<< The above "BACKUP DATABASE" takes >4 minutes to produce a 150Mb backup file. I then tried using 7zip (www.7zip.com) to produce a ZIP file and the results were similar, although it only took about half the time. >> Are you executing the backup on the EDB Server or directly in the .NET Data Provider ? If the latter, then you may just be experiencing the performance implications of .NET managed code vs. native Win32 execution. Also, what sort of time do you get when you use the default compression setting of 6 instead of 9 ? You may be incurring a lot of extra time without much benefit in terms of size. << Is there a way to implement a different compression protocol in the "BACKUP DATABASE" routine in order to improve the situation? >> No. -- Tim Young Elevate Software www.elevatesoft.com |
Tue, Jul 14 2009 6:51 PM | Permanent Link |
Charalampos Michael | Dear Tim,
> << The above "BACKUP DATABASE" takes >4 minutes to produce a 150Mb backup > file. I then tried using 7zip (www.7zip.com) to produce a ZIP file and the > results were similar, although it only took about half the time. >> > > Are you executing the backup on the EDB Server or directly in the .NET Data > Provider ? If the latter, then you may just be experiencing the performance > implications of .NET managed code vs. native Win32 execution. > > Also, what sort of time do you get when you use the default compression > setting of 6 instead of 9 ? You may be incurring a lot of extra time > without much benefit in terms of size. > > << Is there a way to implement a different compression protocol in the > "BACKUP DATABASE" routine in order to improve the situation? >> > > No. There is a pascal implementation of LZMA library which you can take advantage some time ... (I'm telling you this before 1 year or so) Also the 7zip uses better zip algorithms than the current zlib library which EDB uses ... -- Charalampos Michael - [Creation Power] - http://www.creationpower.gr |
Thu, Jul 16 2009 2:16 PM | Permanent Link |
Tim Young [Elevate Software] Elevate Software, Inc. timyoung@elevatesoft.com | Michael,
<< There is a pascal implementation of LZMA library which you can take advantage some time ... (I'm telling you this before 1 year or so) >> Yes, and it's 100% Win32 native code, which we *cannot* use with .NET as-is, hence it is not an option for us unless we want to spend 2 months again porting a new compression algorithm over to managed code. -- Tim Young Elevate Software www.elevatesoft.com |
Sat, Jul 18 2009 4:44 AM | Permanent Link |
Hedley Muscroft | I'm running it straight from the .NET data provider. Normal compression is about the 1/2
the time and only results in a file that's about 1 Mb bigger so I guess that's the one to go for. I agree with you that Managed code is probably responsible for the slightly poorer performance compared to Win32. Regarding Michael's suggestion about LZMA compression, you said :- >> Yes, and it's 100% Win32 native code, which we *cannot* use with .NET I'm not sure that's accurate as there are many .NET components/projects/samples out there which use 7zip.LZMA compression :- http://www.7-zip.org/sdk.html http://dev.nomad-net.info/articles/sevenzipinterface http://www.google.co.uk/search?q=7zip+c%23 If you're not keen on implementing this yourself, is there anyway you could allow the developers to supply a compression algorithm for compressing/decompressing a byte[] ourselves - perhaps via an event/callback during the backup process? I believe you did something similar with DBISAM for encryption purposes (if memory serves?). Thanks, Hedley |
Sat, Jul 18 2009 4:53 AM | Permanent Link |
Hedley Muscroft | <edit to previous message>
Sorry - just realised that you said you were after a 100% managed code solution for 7zip. There are lots of .NET managed wrappers around 7zip.dll. The best I have found so far is :- http://sevenzipsharp.codeplex.com Is there any reason you couldn't use this in EDB? |
Mon, Jul 20 2009 11:27 AM | Permanent Link |
Tim Young [Elevate Software] Elevate Software, Inc. timyoung@elevatesoft.com | Hedley,
<< Is there any reason you couldn't use this in EDB? >> Well, we try to keep distribution as simple as possible, meaning that the default situation should be that no additional DLLs, etc. are required. The other problem with calling external DLLs is the call overhead, especially if there are a lot of calls. The backup/restore compress/decompress in 64k blocks. I'll look into implementing additional compression options, but I can't make any promises as to when something like that could happen. -- Tim Young Elevate Software www.elevatesoft.com |
Mon, Jul 20 2009 6:24 PM | Permanent Link |
"Hedley Muscroft" | OK - thanks Tim. It's not a high priority as I can probably implement it
myself by just 7zipping the database folder. EDB 2.03 is definitely far more important - especially the ability to RENAME tables and columns. Any news on that by the way? <grin> |
This web page was last updated on Saturday, May 4, 2024 at 12:54 AM | Privacy PolicySite Map © 2024 Elevate Software, Inc. All Rights Reserved Questions or comments ? E-mail us at info@elevatesoft.com |