[gdal-dev] FileGDB locks don't produce error during indexing

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

[gdal-dev] FileGDB locks don't produce error during indexing

bespin
I have 2 concurrent processes creating indexes on FileGDB tables.  I expected to receive some sort of error or warning due to locking.  Unfortunately, both processes complete without any message or error. The end result is that one of the indexes gets created and the other does not.

Run in 2 shells concurrently
ogrinfo -sql "create index ndx_one on foo_table (field1)" C:\temp\my.gdb
ogrinfo -sql "create index ndx_two on foo_table (field2)" C:\temp\my.gdb

Both return the following message but only one index is actually created.
INFO: Open of `C:\temp\my.gdb'
      using driver `FileGDB' successful.


I actually found this during some processing using python/ogr like this.  Nothing gets raised here either.
ogr.UseExceptions()
driver = ogr.GetDriverByName('FileGDB')
ds = driver.Open(self.gdb, 0)
layer = ds.GetLayer(self.fc)
sqlcmd = "create index {0} on {1}({2})".format(self.ndx_name, self.fc, self.field)
ds.ExecuteSQL(sqlcmd)
ds = None

Is there anything I can do to catch this at runtime?  At the very least is there a pattern to look for the existence of an index?  I went through different dialects and couldn't figure anything out.

Thanks,
bespin

_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FileGDB locks don't produce error during indexing

Even Rouault-2

On mardi 20 juin 2017 19:35:47 CEST Travis Featherston wrote:

> I have 2 concurrent processes creating indexes on FileGDB tables. I

> expected to receive some sort of error or warning due to locking.

> Unfortunately, both processes complete without any message or error. The

> end result is that one of the indexes gets created and the other does not.

>

> Run in 2 shells concurrently

> ogrinfo -sql "create index ndx_one on foo_table (field1)" C:\temp\my.gdb

> ogrinfo -sql "create index ndx_two on foo_table (field2)" C:\temp\my.gdb

>

> Both return the following message but only one index is actually created.

> INFO: Open of `C:\temp\my.gdb'

> using driver `FileGDB' successful.

>

>

> I actually found this during some processing using python/ogr like this.

> Nothing gets raised here either.

> ogr.UseExceptions()

> driver = ogr.GetDriverByName('FileGDB')

> ds = driver.Open(self.gdb, 0)

> layer = ds.GetLayer(self.fc)

> sqlcmd = "create index {0} on {1}({2})".format(self.ndx_name, self.fc,

> self.field)

> ds.ExecuteSQL(sqlcmd)

> ds = None

>

> Is there anything I can do to catch this at runtime? At the very least is

> there a pattern to look for the existence of an index? I went through

> different dialects and couldn't figure anything out.

 

Travis,

 

Do you use the latest version of the FileGDB SDK (1.5) ? just in case.

 

I believe that the SDK should be responsible for setting locks here (it does for some operations, but I haven't checked if that works properly). If that doesn't work, then you should probably raise the issue directly to ESRI. This could perhaps be done at the OGR level by explicitly calling SetWriteLock() / FreeWriteLock(), although the documentation of those functions only suggest they should be used for bulk loading (and that's what is actually used when doing bulk loading by OGR)

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: FileGDB locks don't produce error during indexing

bespin
Yes, I just confirmed this same behavior happens in the 1.5 SDK through the latest OSGeo4W install.

Is the special GetLayerDefinition the best way to check for successful indexes?

On Wed, Jun 21, 2017 at 1:07 AM, Even Rouault <[hidden email]> wrote:

On mardi 20 juin 2017 19:35:47 CEST Travis Featherston wrote:

> I have 2 concurrent processes creating indexes on FileGDB tables. I

> expected to receive some sort of error or warning due to locking.

> Unfortunately, both processes complete without any message or error. The

> end result is that one of the indexes gets created and the other does not.

>

> Run in 2 shells concurrently

> ogrinfo -sql "create index ndx_one on foo_table (field1)" C:\temp\my.gdb

> ogrinfo -sql "create index ndx_two on foo_table (field2)" C:\temp\my.gdb

>

> Both return the following message but only one index is actually created.

> INFO: Open of `C:\temp\my.gdb'

> using driver `FileGDB' successful.

>

>

> I actually found this during some processing using python/ogr like this.

> Nothing gets raised here either.

> ogr.UseExceptions()

> driver = ogr.GetDriverByName('FileGDB')

> ds = driver.Open(self.gdb, 0)

> layer = ds.GetLayer(self.fc)

> sqlcmd = "create index {0} on {1}({2})".format(self.ndx_name, self.fc,

> self.field)

> ds.ExecuteSQL(sqlcmd)

> ds = None

>

> Is there anything I can do to catch this at runtime? At the very least is

> there a pattern to look for the existence of an index? I went through

> different dialects and couldn't figure anything out.

 

Travis,

 

Do you use the latest version of the FileGDB SDK (1.5) ? just in case.

 

I believe that the SDK should be responsible for setting locks here (it does for some operations, but I haven't checked if that works properly). If that doesn't work, then you should probably raise the issue directly to ESRI. This could perhaps be done at the OGR level by explicitly calling SetWriteLock() / FreeWriteLock(), although the documentation of those functions only suggest they should be used for bulk loading (and that's what is actually used when doing bulk loading by OGR)

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com



_______________________________________________
gdal-dev mailing list
[hidden email]
https://lists.osgeo.org/mailman/listinfo/gdal-dev
Loading...