There I describe that I am wrapping a function from library(raster), which uses rgdal, with a foreach construct and parallelizing using multicore/doMC. I suspect that this and other errors have something to do with file I/O but I'm not sure how to interpret them or track them down. Thanks for any suggestions.
Le samedi 29 janvier 2011 15:16:01, Neil Best a écrit :
> How should I interpret this error?
> GDAL Error 1: TIFFFetchDirectory:Sanity check on directory count failed,
> this is probably not a valid IFD offset
> I saw this in an R session. I descibe the context on Stackoverflow:
> http://stackoverflow.com/questions/4831882/mysterious-error-from-foreach >
> There I describe that I am wrapping a function from library(raster), which
> uses rgdal, with a foreach construct and parallelizing using
> multicore/doMC. I suspect that this and other errors have something to do
> with file I/O but I'm not sure how to interpret them or track them down.
> Thanks for any suggestions.
First, does the opening of the TIFF file with a utility such gdalinfo works
without emitting this warning ? If it works, then your code is the likely
culprit, otherwise the TIFF file is corrupted.
I don't know how you use the multi threading, but this could certainly be the
responsible for the error : you cannot use the same GDAL dataset object from
multiple threads, even if you just use it for reading. If you want to do
parallelized I/O, you need to open as many dataset objects as you have threads
(you can open several datasets pointing to the same file). See
Le samedi 29 janvier 2011 15:53:30, Neil Best a écrit :
> Thanks, Even. That makes sense. If I understand your suggestion I can
> create a trivial VRT pointing to the input data for each worker, correct?
Thank you, Even. I think that did it. Instead of passing around raster() objects I need to pass the file names and re-instantiate the objects on the workers within the foreach closure. Are you on Stackoverflow? If not, please consider signing up. I think it's a great concept and I would like to give you credit for this solution.