[gdal-dev] Is writing/reading a VRTs an atomic operation?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

[gdal-dev] Is writing/reading a VRTs an atomic operation?

Pablo Sanfilippo
Hello!

I'm building an application in which I will have multiple concurrent process reading and writing to a VRT file. The writers are just multiple instances of the same process that create a .vrt off a list of tiffs in a folder. The writer process implement file locking, so I'm sure no two processes will try to write the .vrt at the same time. But the readers (which are gdal_translate calls that use the .vrt as input) don't lock the .vrt so they can be ran at the same time. I can see a race condition happening if gdalbuildvrt overwrites the file when some gdal_translate is opening it. So, if I'm understanding this correctly, the question boils down to: Is the read call that gdal_translate makes when reading a vrt file atomic? And is the write call that gdalbuildvrt makes when writing the file atomic?

Hope this makes sense.

Pablo

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Is writing/reading a VRTs an atomic operation?

Even Rouault-2
On jeudi 6 décembre 2018 13:59:07 CET Pablo Sanfilippo wrote:

> Hello!
>
> I'm building an application in which I will have multiple concurrent
> process reading and writing to a VRT file. The writers are just multiple
> instances of the same process that create a .vrt off a list of tiffs in a
> folder. The writer process implement file locking, so I'm sure no two
> processes will try to write the .vrt at the same time. But the readers
> (which are gdal_translate calls that use the .vrt as input) don't lock the
> .vrt so they can be ran at the same time. I can see a race condition
> happening if gdalbuildvrt overwrites the file when some gdal_translate is
> opening it. So, if I'm understanding this correctly, the question boils
> down to: Is the read call that gdal_translate makes when reading a vrt file
> atomic? And is the write call that gdalbuildvrt makes when writing the file
> atomic?

Much likely not. At least there's no effort in the code to make read/write
atomic.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com
_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|

Re: Is writing/reading a VRTs an atomic operation?

Pablo Sanfilippo
Thanks! 

On Thu, Dec 6, 2018 at 2:25 PM Even Rouault <[hidden email]> wrote:
On jeudi 6 décembre 2018 13:59:07 CET Pablo Sanfilippo wrote:
> Hello!
>
> I'm building an application in which I will have multiple concurrent
> process reading and writing to a VRT file. The writers are just multiple
> instances of the same process that create a .vrt off a list of tiffs in a
> folder. The writer process implement file locking, so I'm sure no two
> processes will try to write the .vrt at the same time. But the readers
> (which are gdal_translate calls that use the .vrt as input) don't lock the
> .vrt so they can be ran at the same time. I can see a race condition
> happening if gdalbuildvrt overwrites the file when some gdal_translate is
> opening it. So, if I'm understanding this correctly, the question boils
> down to: Is the read call that gdal_translate makes when reading a vrt file
> atomic? And is the write call that gdalbuildvrt makes when writing the file
> atomic?

Much likely not. At least there's no effort in the code to make read/write
atomic.

Even

--
Spatialys - Geospatial professional services
http://www.spatialys.com

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev