REALTHINCLIENT SDK TEST RESULTS
RealThinClient SDK v6.05
Results of batch tests with 1.100, 1.200, 1.300, 1.400, 1.500, 1.600, 1.700, 1.800, 1.900 and 2.000 active connections:
Results of batch tests with 2.100, 2.200, 2.300, 2.400, 2.500, 2.600, 2.700, 2.800, 2.900 and 3.000 active connections:
Results of batch tests with 3.100, 3.200, 3.300, 3.400, 3.500, 3.600, 3.700, 3.800, 3.900 and 4.000 active connections:
Results of batch tests with 4.100, 4.200, 4.300, 4.400, 4.500, 4.600, 4.700, 4.800, 4.900 and 5.000 active connections:
For comparison reasons (to see how much the CPU on the Server can affect performance), here are the results of the same stress-test
when the Windows Resource Manager (perfmon.exe) is RUNNING on the Server and draining the CPU by 25% – 45% …
RealThinClient SDK v3.76
RealThinClient SDK v3.76 (available to developers with a RealThinClient Subscription ) has been extensively stress-tested using 9.900 concurrent connections over HTTPS with a stress-test running one full week without a break (24/7). There were NO errors throughout the whole stress-test, there were NO problems closing any of the RTC Clients nor the RTC Server and there were absolutely NO memory leaks on either of the Clients nor the Server (FastMM 4.94 used as Memory Manager with memory leak reporting and StreamSec Tools 2.1.9.240 used for SSL encryption).
In this stress-test, the RTC Server has responded to more than 394 million requests from RTC Clients, which have opened and closed more than 2,7 million connections to the Server. I have been changing test parameters on all Clients once every 24 hours, so that most RTC SDK properties and settings have been tested for cca 24 hours each.
Here is one Screenshot of a RTC Server showing how the RealThinClient SDK v3.76 is being stress-tested by simulating 9.900 concurrent clients, where each Client is sending a remote function call to the Server, waiting for the result and then sending the next call in a continuous loop, flooding the Server with requests (image shrinked here to reduce file size):
What you see above are 11 Client PCs, each running 3 instances of the AppClient demo, each AppClient instance using 64 threads in MultiThreaded mode to send remote function calls through 300 concurrent connections to the Server (11 x 3 x 300 = 9.900 connections). The Server is running in MultiThreaded mode and using 64 threads to handle all connections simultaneously.
Each vertical line on each AppClient instance represents a connection to the Server, where the red color shows the percentage of calls to be made in the current loop and the green/cyan color shows the percentage of results received in the current loop. AppClients were all working in multi-threaded mode and were set up to use different settings, while every connection is set up to make 50 remote calls to the Server per loop, and each loop is repeated one second after the last one was finished, which results in an endless loop of remote calls (until stopped).
RealThinClient SDK v2.47 –Extensive Stress-Test
Connections in use: 2.000 (10 x 200)
Server: Dual-Core AMD X2 at 4,4 GHz (2 x 2,2 GHz) with 2 GB RAM
Average processing speed: 518,5 req/sec
Average data transfer rate: 33,1 MBit/sec
Total requests processed: 653.960.000
10 Client PCs (200 connections each):
7 x Pentium III at 667 MHz with 256 MB RAM
3 x Pentium III at 650 MHz with 384 MB RAM
– connected to one 24-port 100 MBit switch
The purpose of this test was to see how stable the Clients and the Server are
when generating Extreme Load for a very long time (350 Hours), without a pause.
There were no problems, the test would still be running if I hadn’t stopped it.
For performance tests, please check other test results (below).
Test running time: 1.261.336 seconds (14 days, 14:22 Hours)
|
||||||||||
CPU
[MHz] |
Data
Structure |
Data
Format |
Auto-
Encrypt |
Auto-
Compress |
Requests
processed |
Bytes/
Request |
Bytes/
Response |
Req/
sec |
Bytes per
second |
Memory
in use |
Clients with 200 connections active (all at the same time)
|
||||||||||
667
|
Simple
|
RTC
|
No
|
No
|
81.120.000
|
180
|
54
|
64,3
|
15.046
|
43 MB
|
667
|
Simple
|
RTC
|
Yes
|
No
|
90.000.000
|
192
|
65
|
71,4
|
18.350
|
44 MB
|
667
|
Simple
|
RTC
|
No
|
Yes
|
90.160.000
|
180
|
54
|
71,5
|
16.731
|
43 MB
|
667
|
Simple
|
RTC
|
Yes
|
Yes
|
83.880.000
|
192
|
65
|
66,5
|
17.091
|
43 MB
|
650
|
Simple
|
XML
|
No
|
No
|
86.760.000
|
855
|
176
|
68,8
|
70.933
|
42 MB
|
667
|
Complex
|
RTC
|
No
|
No
|
45.280.000
|
13.975
|
2.321
|
35,9
|
585.026
|
47 MB
|
667
|
Complex
|
RTC
|
Yes
|
No
|
54.600.000
|
13.961
|
2.330
|
43,3
|
705.400
|
47 MB
|
667
|
Complex
|
RTC
|
No
|
Yes
|
46.240.000
|
2.432
|
2.321
|
36,7
|
174.435
|
49 MB
|
650
|
Complex
|
RTC
|
Yes
|
Yes
|
45.280.000
|
2.443
|
2.332
|
35,9
|
171.423
|
46 MB
|
650
|
Complex
|
XML
|
No
|
No
|
30.640.000
|
53.268
|
52.428
|
24,3
|
2.568.413
|
57 MB
|
Server with 2.000 connections active
|
||||||||||
4.400
|
Both
|
Both
|
Yes
|
Yes
|
653.960.000
|
518,5
|
4.342.848 |
206 MB
|
Since all Clients were connected to the same Server during this test, all parameter combinations have been in use at the same time, sending simple and complex data structures to the server, receiving simple and complex results. Because of this and very high network load during the test, this was a good stress-test in a flooded environment, but it should NOT be seen as a performance test, since it is not.
Information provided in the table above is only to show you how load was distributed between clients when they are sending different-size requests. All results should be evaluated as a whole, showing you what would happen if all these clients would be hitting the Server continually for 14 days, without a pause.
Since the RealThinClient SDK supports 12 simple data types (boolean, integer, largeint, float, currency, datetime,string, text, widestring, variable, exteption, byte stream) and 3 complex data types (records, arrays and datasets),
where every element inside a complex data type can be of ANY other data type, including complex data types,the resulting structure used in this stress-test was quite big (14-53 KB). And to test how sending and receiving of all those data types works, the same data was packed again on the Server and sent back to the Client.
For performance tests, please check the other test results (below).
RealThinClient SDK v2.16 – using RTC and XML-RPC formats
Test running time: ~ 7 days, 23 hours Connections in use: 600 (12 x 50) Total processed: 2.252.950.000 requests |
Test running time: ~ 4 days, 16 hours Connections in use: 3.600 (12 x 300) Total processed: 859.170.000 requests |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Server: Dual-Core AMD X2 at 4,4 GHz (2 x 2,2 GHz) with 2 GB RAM Average processing speed: 3278 req/sec 12 Client PCs: 9 x Pentium III at 667 MHz with 256 MB RAM 3 x Pentium III at 650 MHz with 384 MB RAM – connected to one 24-port 100 MBit switch |
Server: Dual-Core AMD X2 at 4,4 GHz (2 x 2,2 GHz) with 2 GB RAM Average processing speed: 2137 req/sec 12 Client PCs: 9 x Pentium III at 667 MHz with 256 MB RAM 3 x Pentium III at 650 MHz with 384 MB RAM – connected to one 24-port 100 MBit switch |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
RealThinClient SDK v1.9j – using the RTC format
Test started on Sunday, August 6th 2006 – 19:47 CET Test closed on Friday, August 11th 2006 – 19:27 CET Test running time: ~ 5 days Connections in use: 600 (12 x 50) Total processed: 2.479.536.000 requests |
Test started on Tuesday, July 4th 2006 – 16:01 CET Test closed on Friday, July 7th 2006 – 17:01 CET Test running time: 3 days, 1 hour Connections in use: 3000 Requests processed: 725,4 million |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Server: Dual-Core AMD X2 at 4,4 GHz (2 x 2,2 GHz) with 2 GB RAM Average processing speed: 5297 req/sec Total connections in use: 60012 Client PCs: 9 x Pentium III at 667 MHz with 256 MB RAM 3 x Pentium III at 650 MHz with 384 MB RAM – connected to one 24-port 100 MBit switch |
Server: Dual-processor AMD X2 at 4,4 GHz with 2 GB RAM Average processing speed: 2760 req/sec Total connections in use: 300012 Client PCs: 3 x Pentium III at 650 MHz with 384 MB RAM 9 x Pentium III at 667 MHz with 256 MB RAM – connected to one 24-port 100 MBit switch |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Detailed Test Results can be download here: SDK19j_16T_3000con.zip ( 1,8 MB log file ) |
RealThinClient SDK v1.8 – different Memory Managers – 512 Threads
Server: Dual-processor AMD X2 at 4,4 GHz with 2 GB RAM 13 Client PCs: 3 x Pentium III at 650 MHz with 384 MB RAM 10 x Pentium III at 667 MHz with 256 MB RAM – connected to one 24-port 100 MBit switch |
Thread Pool Size: 512 1) FastMM 4.56 (optimized) 2) BucketMem_ASM (optimized) 3) Nexus DB MM V3 4) RTL (standard in Delphi 7) |
All 13 Client PCs were running the RTC SDK AppClient Demo with the Multiple Connection Flood test,using 300 connections in Multithreaded mode (pool size set to 16 Threads) with automatic compression and encryption to constantly flood the Server with remote function calls (total 3.900 connections).Server was running the RTC SDK AppServer Demo in multithreaded mode (using 512 Threads) with automatic encryption and compression. | |
AppServer Performance: Different MMs, 512 Threads, 3.900 Connections: |
AppServer Memory Usage: Different MMs, 512 Threads, 3.900 connections: |
Complete Test Results can be downloaded here: AppServerTest512T.zip |
RealThinClient SDK v1.8 –different Memory Managers – 16 Threads
Server: Dual-processor AMD X2 at 4,4 GHz with 2 GB RAM 13 Client PCs: 3 x Pentium III at 650 MHz with 384 MB RAM 10 x Pentium III at 667 MHz with 256 MB RAM – connected to one 24-port 100 MBit switch |
Thread Pool Size: 16 1) FastMM 4.56 (optimized) 2) BucketMem_ASM (optimized) 3) Nexus DB MM V3 4) RTL (standard in Delphi 7) |
All 13 Client PCs were running the RTC SDK AppClient Demo with the Multiple Connection Flood test,using 300 connections in Multithreaded mode (pool size set to 16 Threads) with automatic compression and encryption to constantly flood the Server with remote function calls (total 3.900 connections). Server was running the RTC SDK AppServer Demo in multithreaded mode(using 16 Threads) with automatic encryption and compression. Winner in those RTC SDK Tests is FastMM 4.56 (best performance and lowest memory usage) and the worst possible results were achieved using the standard Delphi Runtime Library (RTL – awful performance with highest memory usage). |
|
AppServer Performance: Different MMs, 16 Threads, 3.900 Connections: ![]() |
AppServer Memory Usage: Different MMs, 16 Threads, 3.900 connections: ![]() |
Complete Test Results can be downloaded here: AppServerTest16T.zip |
RealThinClient SDK v1.8 –tests with BucketMem and FastMM4
Server: Dual-processor AMD X2 at 4,4 GHz (2 x 2,2 GHz) with 2 GB RAM 14 Client PCs: 1 x AMD at 1,2 GHz with 768 MB RAM 3 x Pentium III at 650 MHz with 384 MB RAM 10 x Pentium III at 667 MHz with 256 MB RAM – connected to one 24-port 100 MBit switch |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
RealThinClient SDK v1.7 – RTLMM using Encryption
Server: Dual-processor AMD X2 at 4,4 GHz (2 x 2,2 GHz) with 2 GB RAM
14 Client PCs: |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
RealThinClient SDK v1.7 –Encryption and manual connect + disconnect
Server: Dual-processor AMD X2 at 4,4 GHz (2 x 2,2 GHz) with 2 GB RAM 14 Client PCs: 1 x AMD at 1,2 GHz with 768 MB RAM 3 x Pentium III at 650 MHz with 384 MB RAM 10 x Pentium III at 667 MHz with 256 MB RAM – connected to one 24-port 100 MBit switch |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
RealThinClient SDK v1.7 –Encryption and manual connect
Server: Dual-processor AMD X2 at 4,4 GHz (2 x 2,2 GHz) with 2 GB RAM 13 Client PCs: 3 x Pentium III at 650 MHz with 384 MB RAM 10 x Pentium III at 667 MHz with 256 MB RAM |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
RealThinClient SDK v1.7 –Encryption and manual connect
Server: Pentium III at 667 MHz with 256 MB RAM 12 Client PCs 3 x Pentium III at 650 MHz with 384 MB RAM 9 x Pentium III at 667 MHz with 256 MB RAM |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
RealThinClient SDK v1.7 –using 14 PCs
Average processing speed: 822 req/sec Total connections in use: 2.400
13 Client PCs:
3 x Pentium III at 650 MHz with 384 MB RAM
10 x Pentium III at 667 MHz with 256 MB RAM
– connected to one 24-port 100 MBit switchTest started on Friday, October 7th 2005, 17:00 CET, Closed on Monday, October 10th 2005, 14:00 CET
Test running time: 2 days, 21 hours
Requests processed: 204,2 million
RealThinClient SDK v1.7 –Tests using 2 PCs
Here are the results of some tests done using one client PC and one server PC over a Wireless LAN with a low signal strength. All tests have passed, there were no problems.
– 10,7 million requests processed = 248 req/s
– 31,8 GBits sent, avg upload speed = 736 kbit/s
– 16,5 GBits received, avg download speed = 381 kbit/s> 1.000 connections flooding RTC Server for 12 hours:
– 22 million requests processed = 509 req/s
– 63,6 GBits sent, avg upload speed = 1472 kbit/s
– 41,4 GBits received, avg download speed = 958 kbit/s
> 500 connections flooding RTC Server for 84 hours (3 days and 12 hours):
– Server CPU usage (1,25 GHz with 768 MB RAM) = 60-80%
– Client CPU usage (1 GHz with 1024 MB RAM) = 45-65%
– 180,2 million requests processed = 596 req/s
– 477,3 GBits sent, avg upload speed = 1578 kbit/s
– 295,1 GBits received, avg download speed = 976 kbit/s
Note: If you shold be asking yourself why there’s a difference between the amount of data being sent and received, the explanation is rather simple: request data size is larger than response data size, since requests from the client in this test send information needed to call a few remote functions (function names and all parameters), while response from the server only has to return a single value as a result from those functions.
RealThinClient SDK v1.6 –Tests done by our Beta-Testers and Partners
One hour of such flood-testing is equivalent to one full day of maximum application usage in a very productive environment.
* Client 1 with 600 connections: P4 CPU 1.50GHz 512RAM Win XP proff = 1.283.480 requests completed
* Client 2 with 600 connections: PIII CPU 800 512 RAM Win 2k = 1.851.520 requests completed
* Client 3 with 600 connections: Celeron 1,00 GHz 512 RAM Win 2k = 1.227.880 requests completed
* Client 4 with 600 connections: P4 CPU 2,4 GHz 512 RAM Win XP proff = 1.792.770 requests completed
* SERVER processing 2.400 connections: P4 2,4 GHz 1 GB RAM Win 2k SERVER = 6.155.650 TOTAL requests processed
>>> “ESTRA” d.o.o. Croatia
> 3.000 connections from 3 PCs flooding the AppServer for 13:30 hours:
– “3 clients over 10MBit LAN; 4,9 million requests processed; no problems!”
>>>“TEKOM” d.o.o. Croatia
>4.101 connections from 7 PCs flooding one AppServer for 11:30 hours:
– “1 PC from location 1; 6 PCs from location 2; all through Internet; no problems.”
* PC 1 (Internet line 1) with 541 conn; 41.490 completed
* PC 2 (Internet line 2) with 600 conn; 25.590 completed
* PC 3 (Internet line 2) with 600 conn; 12.060 completed
* PC 4 (Internet line 2) with 600 conn; 14.960 completed
* PC 5 (Internet line 2) with 600 conn; 19.040 completed
* PC 6 (Internet line 2) with 560 conn; 12.056 completed
* PC 7 (Internet line 2) with 600 conn; 20.400 completed
= Total 145.596 requests completed
– Anthoni
>600 connections from 2 PCs flooding one AppServer for 5 hours:- “1 client over the Internet, 1 client on a Wireless LAN, everything ok.”
* Client 1 with 100 connections over the Internet (ADSL)= 964.120 requests; 158 MB sent; 56,7 MB received
* Client 2 with 500 connections over a Wireless-b LAN = 10.522.631 requests; 1,7 GB sent; 619 MB received
– Mario
> 19clients from 4 PCs flooding one AppServer for 4-5 hours: – “1 client over WLAN + 18 clients over a 100MBit LAN; no glitches!” – Malcolm
MORE …