Anthony wrote:

> Server 2019 comes with Windows Defender installed

And I only tried disabling a third party AV because I thought windows AV is disabled whenever there is another AV on the system. Which isn't the case.

Thank you very much Anthony! That wasn't everything, I'm still not satisfied with the performance, but it was a big step.

Does the EDB server benefit from more CPU cores?

Heiko Knuettel wrote:

>> I'm still not satisfied with the performance, but it was a big step. Does the EDB server benefit from more CPU cores?

Did you try the client application on the server itself to remove any network speed issues?

Here is a link to a previous thread about multi cores https://www.elevatesoft.com/forums?action=view&category=edb&id=edb_suggestions&page=1&msg=578#578

OK, so more cores won't do much. Tried it out by giving the machine twice as much (16) cores as it had before, but no difference at all.

Unfortunately, disabling Windows Defender AV didn't do the trick - I was misleaded by trying it out in the evening with no user load. The situation hasn't improved.

Running the application on the server showed no difference between running it on the clients. And I could post the autoruns here, but I haven't seen anything extraordinary.

Today, a little desperate, I moved the server and database to a Windows 10 machine, disabled the same Antivirus that runs on the server and...everything is three times faster.

Now I have a 5000 Euro server with VMWare ESXi and 24 cores at 2700Mhz...and a 500 Euro Windows 10 workstation with 8 cores at 3700Mhz that beats the crap out of it.

Something is still wrong...I'm open to all ideas.

On 3/12/2020 5:11 PM, Heiko Knuettel wrote:
> OK, so more cores won't do much. Tried it out by giving the machine twice as much (16) cores as it had before, but no difference at all.

It should help if you have lot of active concurrent session. I believe
every incoming connection/sessopm gets it's own thread so IF you have
lots of threads they should get distributed across multiple cores.
However any single session/connection would not benefit from more cores
and even with many the benefit would only be visible if they are at high
utilization.

> Today, a little desperate, I moved the server and database to a Windows 10 machine, disabled the same Antivirus that runs on the server and...everything is three times faster.
> Now I have a 5000 Euro server with VMWare ESXi and 24 cores at 2700Mhz...and a 500 Euro Windows 10 workstation with 8 cores at 3700Mhz that beats the crap out of it.

Technically that desktop CPU is 30+% faster but i don't think that's it.

> Something is still wrong...I'm open to all ideas.

Did you just disable or actually uninstall windows defender - might be
worthwhile to actually remove it

Otherwise there are some potential network issues on 2019 - might be
worth a try if you have not already tried:

RSC  :
https://serverfault.com/questions/976324/very-poor-network-performance-with-server-2019
and
https://docs.microsoft.com/en-us/windows-server/networking/technologies/hpn/rsc-in-the-vswitch

VMQ might be applicable  :
https://www.dell.com/support/article/en-ca/sln132131/windows-server-slow-network-performance-on-hyper-v-virtual-machines-with-virtual-machine-queue-vmq-enabled?lang=en

LRO and TSO : https://kb.vmware.com/s/article/2055140


SMB signing (likely not but should not hurt :
https://community.spiceworks.com/topic/2225528-slow-network-performance-windows-2019-to-windows-10-read


Raul

Heiko Knuettel wrote:
> Now I have a 5000 Euro server with VMWare ESXi and 24 cores at 2700Mhz...and a 500 Euro Windows 10 workstation with 8 cores at 3700Mhz that beats the crap out of it.

Wow, that is a lot of server is it just running the database or are other VM's running other workloads on the sever at the same time. You mentioned that you did tests outside of hours which biased the results, which may indicate server load.

How much RAM does the server and VM have?

What storage configuration is in place, RAID?

Also what is the size of your database and how many users (max and simultaneous)?

A 500 Euro PC, does that have a SSD drive providing faster disk performance and more dedicated RAM?

If the application was slow when executed directly on the server then maybe more server load and not networking.

Anthony

Raul,

thank you very much, I will try everything out and report, will take a while. Since people can work normally again, they have other wishes...*sigh*



Anthony,

the server (128GB RAM) runs 4 VMs at the moment, the Domain Controller/File Server where the EDB server resides, an additional Server 2019 with Exchange 2019, and two appliances: MailCleaner and VMWare vSphere Server. Overall, CPU load is never more than 20-30%, it was designed for the growth of the next 10 years.

The database is about 20GB in size. Normal workday load is about 20-30 users, can get up to 45, and we'll see what happens in the future. Server has fast (3000 MB/s) NVME SSDs for everything that needs speed (EDB) and big HDDs for file storage. Everything is NTFS software-RAID 1.

>>A 500 Euro PC, does that have a SSD drive providing faster disk performance and more dedicated RAM?<<

It IS faster - performance per core is higher, and drive access isn't slowed down by a virtual layer, but that shouldn't make THAT of a difference. The old server had almost the same configuration (VMWare ESXi, Server 2008), in comparison poor CPU, RAM and SSD, and was about as fast as the workstation now.

So something is definitely slowing that thing down.

Heiko Knuettel wrote:


>>the server (128GB RAM) runs 4 VMs at the moment ... The database is about 20GB in size. ... Server has fast (3000 MB/s) NVME SSDs for everything that needs speed (EDB) ... Everything is NTFS software-RAID 1.

>> The old server had almost the same configuration (VMWare ESXi, Server 2008), in comparison poor CPU, RAM and SSD, and was about as fast as the workstation now.

The server specification certainly should accommodate the workloads. A few observations and real time monitoring may be the key. I use Windows Resource Monitor to monitor performance of Windows but VMWare has monitoring of the host. Assuming RAM is allocated appropriately for the VMs then check disk activity and % utilisation, if you spare RAM in VMWare it may be worth adding more to the EDB VM or checking its % used.

You also mention NTFS software RAID, this would be within the WIndows VMs. What is the VMware storage configuration, are these just a disks and you have created paired VHDs and mirrored them within Windows as I can't image this being good for performance, does the server hardware have a RAID controller which could be configured or read/write cache which may be disabled for safety?

It may be worth performing some disk I/O performance tests on the old and new servers and the PC for some comparisons. I've used Crystal Disk Mark but there are others.

From the description i'm thinking disk performance rather than CPU or RAM maybe storage configuration, I/O contention with other VMs or application or even a driver. I'm also assuming that all other applications are working at an acceptable speed, although Exchange would likely be cached via Outlook on the desktops, but what about the backup times? The file servers may not be immediately obvious unless you perform test copies to the server with various file sizes e.g. 1000+ small files then a large 5GB ISO or something and monitor network and disk performance, maybe both during and outside of working hours.