Allocate large memory space with CPLMalloc(). Bug or not?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Allocate large memory space with CPLMalloc(). Bug or not?

Alexander Zhdanov
Hello!

Take a look on the following code:

...
size_t w = 10000, h = 5000, d = 150;
size_t sz = w * h * d * sizeof(float); // ~27.94 Gb

float *volume = (float*)CPLMalloc(sz);
...


In this case GDAL throws error in runtime: "ERROR 1: CPLMalloc(-664771072): Silly size requested."
But I think that it's not silly, because such amount of free RAM is available on my system. For example I can successfully allocate it calling standart malloc().

I have opened CPLMalloc() source code and seen the following lines:

void *CPLMalloc( size_t nSize )
...
    if( long(nSize) < 0 )
    {
        CPLError( CE_Failure, CPLE_AppDefined,
                  "CPLMalloc(%ld): Silly size requested.\n",
                  (long) nSize );
        return NULL;
    }
...


In my case long( size_t nSize ) returns negative value and it leads to error.
As far as I understand converting one integer type to another smaller integer type (e.g. long long/size_t to long) is implementation-defined, so it's not safe to use CPLMalloc() with 64-bit integers.
Reply | Threaded
Open this post in threaded view
|

Re: Allocate large memory space with CPLMalloc(). Bug or not?

Even Rouault-2
Le jeudi 01 octobre 2015 21:18:33, fazotron a écrit :

> Hello!
>
> Take a look on the following code:
>
> *...
> size_t w = 10000, h = 5000, d = 150;
> size_t sz = w * h * d * sizeof(float); // ~27.94 Gb
>
> float *volume = (float*)CPLMalloc(sz);
> ...*
>
> In this case GDAL throws error in runtime: "ERROR 1: CPLMalloc(-664771072):
> Silly size requested."
> But I think that it's not silly, because such amount of free RAM is
> available on my system. For example I can successfully allocate it calling
> standart malloc().
>
> I have opened CPLMalloc() source code and seen the following lines:
>
> *void *CPLMalloc( size_t nSize )
> ...
>     if( long(nSize) < 0 )
>     {
>         CPLError( CE_Failure, CPLE_AppDefined,
>                   "CPLMalloc(%ld): Silly size requested.\n",
>                   (long) nSize );
>         return NULL;
>     }
> ...*
>
> In my case long( size_t nSize ) returns negative value and it leads to
> error.
> As far as I understand converting one integer type to another smaller
> integer type (e.g. long long/size_t to long) is implementation-defined, so
> it's not safe to use CPLMalloc() with 64-bit integers.

CPLMalloc() is not meant at being used for big allocations since it aborts if
the allocation fails. Use VSIMalloc() instead that will return NULL in case of
failed alloc

>
>
>
> --
> View this message in context:
> http://osgeo-org.1560.x6.nabble.com/Allocate-large-memory-space-with-CPLMa
> lloc-Bug-or-not-tp5227443.html Sent from the GDAL - Dev mailing list
> archive at Nabble.com.
> _______________________________________________
> gdal-dev mailing list
> [hidden email]
> http://lists.osgeo.org/mailman/listinfo/gdal-dev

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Allocate large memory space with CPLMalloc(). Bug or not?

Alexander Zhdanov
Thanks for reply, I'll try VSIMalloc().
But can you explain the goal of check if( long(nSize) < 0 ) {...} in CPLMalloc()? Cast size_t to long leads to implicit result.