[gdal-dev] Memory utilization and GDAL_HTTP_MULTIRANGE
I have been experimenting with the GDAL_HTTP_MULTIRANGE config option added in gdal-2.3. I have noticed that setting this to YES requires fewer curl requests, but utilizes a more memory compared to SERIAL. From a very simple test (details below):
1. gdal-2.2.4: Maximum resident set size (kbytes): 45544
2. gdal-2.3.0 with GDAL_HTTP_MULTIRANGE=SERIAL Maximum resident set size (kbytes): 46120
3. gdal-2.3.0 with GDAL_HTTP_MULTIRANGE=YES
Maximum resident set size (kbytes): 83828
The biggest thing I noticed from the log is that multiple HTTP connections are left intact when GDAL_HTTP_MULTIRANGE=YES. I'm guessing this is where the additional memory utilization comes from. Is there any way to reduce the memory footprint when GDAL_HTTP_MULTIRANGE=YES?