Unmanaged Data APIs RFC

classic Classic list List threaded Threaded
19 messages Options
Reply | Threaded
Open this post in threaded view
|

Unmanaged Data APIs RFC

Tony Fang

I have just posted a draft of the RFC for “Unmanaged Data APIs” in MGOS to the following location:

 

http://wiki.osgeo.org/index.php/MapGuide_RFC_2_-_Unmanaged_Data_APIs

 

Please take a look, and post any feedback to this group.

 

Thanks,

Tony Fang

 

Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Andy Morsell
I really think this is badly needed piece of functionality so I am fully supportive of it.  Storing the paths in serverconfig.ini sounds like a reasonable approach.  I'm wondering if aliases of some sort will be used to discern directories of files? 
 
What I'm afraid of happening is much like how MapGuide 5 and below worked with SDF files (and how 6.5 still works with raster): when you are choosing a feature source to author from a GUI, all of the paths are enumerated and the results listed together.  This can get very cumbersome for people who work on many simultaneous projects on the same server (like me).  I would personally like to see this taken one step further where you have separate lines in an Unmanaged Data Paths section, with one alias per directory.  Then, when choosing the feature source in Studio, Web Studio or whatever, it would initially display the individual aliases, the user would choose the pertinent alias, and then that single directory would be enumerated for the feature sources.  I realize this is an extra piece of functionality, but I would rather see it done this way from the start.
 
Also, will the user be able to configure these folders through any GUI?  How about a new section in Server Admin that allows the users to set these aliases up and browse to the folder?  The resultant information could then be written into serverconfig.ini.
 
Thanks. 
 

Andy Morsell, P.E.
Spatial Integrators, Inc.
http://www.SpatialGIS.com


From: Tony Fang [mailto:[hidden email]]
Sent: Thursday, November 02, 2006 4:35 PM
To: [hidden email]
Subject: [mapguide-dev] Unmanaged Data APIs RFC

I have just posted a draft of the RFC for “Unmanaged Data APIs” in MGOS to the following location:

 

http://wiki.osgeo.org/index.php/MapGuide_RFC_2_-_Unmanaged_Data_APIs

 

Please take a look, and post any feedback to this group.

 

Thanks,

Tony Fang

 

Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Jason Birch
In reply to this post by Tony Fang
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Jason Birch
In reply to this post by Andy Morsell
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Unmanaged Data APIs RFC

Paul Spencer-2
In reply to this post by Jason Birch
Jason, I believe the spec explicitly states that network drives will  
be accessible:

"For windows, specified directories can be physical drives or network  
paths. For linux, it can be any accessible path."

Cheers

Paul

On 2-Nov-06, at 8:57 PM, Jason Birch wrote:

> I am a little worried that the spec refers to paths on the server.
>
> Does this mean that network shares (NFS, SMB, whatever) will not be  
> supported?
>
> Most of my unmanaged file-based data (orthos) are on a SAN, and  
> only accessible by UNC or mapped drive/directory.
>
> Jason
>
> ________________________________
>
> From: Tony Fang
> Sent: Thu 2006-11-02 4:35 PM
> To: [hidden email]
> Subject: [mapguide-dev] Unmanaged Data APIs RFC
>
>
>
> I have just posted a draft of the RFC for "Unmanaged Data APIs" in  
> MGOS to the following location:
>
>
>
> http://wiki.osgeo.org/index.php/MapGuide_RFC_2_-_Unmanaged_Data_APIs
>
>
>
> Please take a look, and post any feedback to this group.
>
>
>
> Thanks,
>
> Tony Fang
>
>
>
> <winmail.dat>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]

+-----------------------------------------------------------------+
|Paul Spencer                          [hidden email]    |
+-----------------------------------------------------------------+
|Chief Technology Officer                                         |
|DM Solutions Group Inc                http://www.dmsolutions.ca/ |
+-----------------------------------------------------------------+





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Jason Birch
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Unmanaged Data APIs RFC

Robert Bray-2
I too am concerned this will make data management hard for people with lots of data / multiple projects. It might be better to dedicate a section in serverconfig.ini to define path mappings as follows:

[Path Mappings]
SomeSdfFiles = <some path>
SomeShpFiles = <some other path>
BigArseImages = <path to SAN disk>
...

In Studio / Web Studio the user would be presented the list of path mappings. Once they select a path mapping they can browse all files and folders below that path.

My 2 cents...

Bob

Jason Birch wrote:
For crying out loud... I must have tuned out by the last sentence in Motivation.
 
Earlier references that got me wound up:
 
"certain directories on the server machine will be specified as unmanaged data directories"
"Certain directories on the server machine will be specified to be unmanaged data directories"
 
hmm, almost the same :)
 
Jason

________________________________

From: Paul Spencer
Sent: Thu 2006-11-02 6:03 PM
To: [hidden email]
Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC



Jason, I believe the spec explicitly states that network drives will 
be accessible:

"For windows, specified directories can be physical drives or network 
paths. For linux, it can be any accessible path."

Cheers

Paul

On 2-Nov-06, at 8:57 PM, Jason Birch wrote:

  
I am a little worried that the spec refers to paths on the server.

Does this mean that network shares (NFS, SMB, whatever) will not be 
supported?

Most of my unmanaged file-based data (orthos) are on a SAN, and 
only accessible by UNC or mapped drive/directory.

Jason

________________________________

From: Tony Fang
Sent: Thu 2006-11-02 4:35 PM
To: [hidden email]
Subject: [mapguide-dev] Unmanaged Data APIs RFC



I have just posted a draft of the RFC for "Unmanaged Data APIs" in 
MGOS to the following location:



http://wiki.osgeo.org/index.php/MapGuide_RFC_2_-_Unmanaged_Data_APIs



Please take a look, and post any feedback to this group.



Thanks,

Tony Fang



<winmail.dat>
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]
    

+-----------------------------------------------------------------+
|Paul Spencer                          [hidden email]    |
+-----------------------------------------------------------------+
|Chief Technology Officer                                         |
|DM Solutions Group Inc                http://www.dmsolutions.ca/ |
+-----------------------------------------------------------------+





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



  

--------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email]

--------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Jason Birch
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Unmanaged Data APIs RFC

Robert Bray-2
Jason,

Good questions. In addition I think it would make sense / be highly desirable for all RFCs that add or modify APIs to include more detail with respect to those changes. It would be nice to see the actual API function signatures for example.

Bob


Jason Birch wrote:
That would be cool.  That way we can have a theme, and a nested hierarchy within that theme if we need the complexity.  Are we restricted to single-word mappings, or will spaces be supported?  ASCII characters only or full internationalisation?
 
What about schema changes?  Will there be a way of updating the repository's knowledge of unmanaged data schemae?  Or are these maybe cached on a session basis like the RDBMS schemae?
 
There would also need to be a method that allows the author UIs to validate resources that reference unmanaged data sources, to make sure that any dependancies (properties, maptips, themes, etc) are still valid, and give a graphical indication when the world slips.
 
Jason

________________________________

From: Robert Bray [[hidden email]]
Sent: Thu 2006-11-02 10:46 PM
To: [hidden email]
Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC


I too am concerned this will make data management hard for people with lots of data / multiple projects. It might be better to dedicate a section in serverconfig.ini to define path mappings as follows:

[Path Mappings]
SomeSdfFiles = <some path>
SomeShpFiles = <some other path>
BigArseImages = <path to SAN disk>
...

In Studio / Web Studio the user would be presented the list of path mappings. Once they select a path mapping they can browse all files and folders below that path.

My 2 cents...

Bob

--------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] 
  

--------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email]

--------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Tony Fang
In reply to this post by Tony Fang
Re: [mapguide-dev] Unmanaged Data APIs RFC

Hi Jason,

 

Paul is correct.

Network drives are supported.

 

I didn’t intend to cause confusion with those two sentences. Thanks for pointing it out.

 

 

Regards,

Tony Fang

 


From: Jason Birch [mailto:[hidden email]]
Sent: Thursday, November 02, 2006 7:31 PM
To: [hidden email]
Subject: RE: [mapguide-dev] Unmanaged Data APIs RFC

 

For crying out loud... I must have tuned out by the last sentence in Motivation.

 

Earlier references that got me wound up:

 

"certain directories on the server machine will be specified as unmanaged data directories"

"Certain directories on the server machine will be specified to be unmanaged data directories"

 

hmm, almost the same :)

 

Jason


From: Paul Spencer

Sent: Thu 2006-11-02 6:03 PM
To: [hidden email]
Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC

Jason, I believe the spec explicitly states that network drives will 
be accessible:

"For windows, specified directories can be physical drives or network 
paths. For linux, it can be any accessible path."

Cheers

Paul

On 2-Nov-06, at 8:57 PM, Jason Birch wrote:

> I am a little worried that the spec refers to paths on the server.
>
> Does this mean that network shares (NFS, SMB, whatever) will not be 
> supported?
>
> Most of my unmanaged file-based data (orthos) are on a SAN, and 
> only accessible by UNC or mapped drive/directory.
>
> Jason
>
> ________________________________
>
> From: Tony Fang
> Sent: Thu 2006-11-02 4:35 PM
> To: [hidden email]
> Subject: [mapguide-dev] Unmanaged Data APIs RFC
>
>
>
> I have just posted a draft of the RFC for "Unmanaged Data APIs" in 
> MGOS to the following location:
>
>
>
> http://wiki.osgeo.org/index.php/MapGuide_RFC_2_-_Unmanaged_Data_APIs
>
>
>
> Please take a look, and post any feedback to this group.
>
>
>
> Thanks,
>
> Tony Fang
>
>
>
> <winmail.dat>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]

+-----------------------------------------------------------------+
|Paul Spencer                          [hidden email]    |
+-----------------------------------------------------------------+
|Chief Technology Officer                                         |
|DM Solutions Group Inc                http://www.dmsolutions.ca/ |
+-----------------------------------------------------------------+





---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Tony Fang
In reply to this post by Tony Fang

I’ve updated the RFC with the drive mappings suggestion. Any feedback on the returned list of drive mappings (using square brackets on the drive mapping name) would be appreciated.

 

 

The server's unmanaged data directories will be specified using the serverconfig.ini. A new section defining directory mappings will be used to specify the unmanaged data directories.

For windows, it may look like this:

[Unmanaged Data Mappings]
SomeSdfFiles = c:\mydata\sdf
Some Shp Files = d:\otherdata\shp
BigArseImages = \\some_other_machine\data\images
Some
很大中文 DwfFiles = c:\mydata\很大中文Dwf
大きsdf = c:\mydata\bigsdf

For linux, it may look like this:

[Unmanaged Data Mappings]
SomeSdfFiles = /usr/mydata/sdf
Some Shp Files = /usr/otherdata/shp
BigArseImages = /mnt/some_other_machine/data/images
Some
很大中文 DwfFiles = /usr/mydata/很大中文Dwf
大きsdf = /usr/mydata/bigsdf

Unicode characters are supported in the mapping names. Spaces in the mappings are also supported. Square brackets [ and ] are not allowed. They are reserved characters for the serverconfig.ini section titles.

We will add API methods to return the unmanaged data mappings, set the mappings, and also verify that the mappings are valid and accessible.

We will add an API to enumerate the unmanaged data files available on the server machine. The files will be prefaced with their mapping names. You may enumerate all files from all the available drive mappings, or you may select a single drive mapping.

A returned list from all available drive mappings may look like this:

[SomeSdfFiles]world.sdf
[Some Shp Files]ecuador.shp
[Some
很大中文 DwfFiles]large.dwf
[Some
很大中文 DwfFiles]subdir/Big.dwf
[
大きsdf]reallybig.sdf

 

 

Thanks,

Tony

 


From: Robert Bray [mailto:[hidden email]]
Sent: Friday, November 03, 2006 9:05 AM
To: [hidden email]
Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC

 

Jason,

Good questions. In addition I think it would make sense / be highly desirable for all RFCs that add or modify APIs to include more detail with respect to those changes. It would be nice to see the actual API function signatures for example.

Bob


Jason Birch wrote:

That would be cool.  That way we can have a theme, and a nested hierarchy within that theme if we need the complexity.  Are we restricted to single-word mappings, or will spaces be supported?  ASCII characters only or full internationalisation?
 
What about schema changes?  Will there be a way of updating the repository's knowledge of unmanaged data schemae?  Or are these maybe cached on a session basis like the RDBMS schemae?
 
There would also need to be a method that allows the author UIs to validate resources that reference unmanaged data sources, to make sure that any dependancies (properties, maptips, themes, etc) are still valid, and give a graphical indication when the world slips.
 
Jason
 
________________________________
 
From: Robert Bray [[hidden email]]
Sent: Thu 2006-11-02 10:46 PM
To: [hidden email]
Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
 
 
I too am concerned this will make data management hard for people with lots of data / multiple projects. It might be better to dedicate a section in serverconfig.ini to define path mappings as follows:
 
[Path Mappings]
SomeSdfFiles = <some path>
SomeShpFiles = <some other path>
BigArseImages = <path to SAN disk>
...
 
In Studio / Web Studio the user would be presented the list of path mappings. Once they select a path mapping they can browse all files and folders below that path.
 
My 2 cents...
 
Bob
 
--------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] 
  
 



 
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

 

--------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email]
Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Jason Birch
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

Re: Unmanaged Data APIs RFC

Zak James-2
This is a minor compatibility issue, but the current version of  
webstudio allows the user to specify an unmanaged feature source with  
an arbitrary server path and then enumerates files in that path.  
Obviously, there are some security issues with this, but is that  
functionality going to disappear with the changes proposed in the RFC?

If so, I think that needs to be stated in the Proposed Changes section.

zak
--
Zak James
Applications and Software Development
DM Solutions Group Inc.
http://www.dmsolutions.ca


On 3 Nov 2006, at 20:57, Jason Birch wrote:

> Looks good.
>
> I would personally prefer to see a single delimiter in the returned  
> list, as it's slightly easier to deal with in code.
>
> I'm probably overly sensitive to this, but I'm wondering if the  
> resource list will be cached on the server?  I know that the 6.5  
> MapGuide author caches the responses internally.  I have to hit  
> "update" to get a new list if something (data store, feature  
> source, feature source schema) changes in the current session.  Can  
> we count on the clients doing this to prevent unnecessary network  
> traffic re-enumerating the resources?
>
> Jason
>
>
> ________________________________
>
> From: Tony Fang [mailto:[hidden email]]
> Sent: Fri 2006-11-03 4:41 PM
> To: [hidden email]
> Subject: RE: [mapguide-dev] Unmanaged Data APIs RFC
>
>
>
> I've updated the RFC with the drive mappings suggestion. Any  
> feedback on the returned list of drive mappings (using square  
> brackets on the drive mapping name) would be appreciated.
>
>
>
>
>
> The server's unmanaged data directories will be specified using the  
> serverconfig.ini. A new section defining directory mappings will be  
> used to specify the unmanaged data directories.
>
> For windows, it may look like this:
>
> [Unmanaged Data Mappings]
> SomeSdfFiles = c:\mydata\sdf
> Some Shp Files = d:\otherdata\shp
> BigArseImages = \\some_other_machine\data\images
> Some ???? DwfFiles = c:\mydata\????Dwf
> ??sdf = c:\mydata\bigsdf
>
> For linux, it may look like this:
>
> [Unmanaged Data Mappings]
> SomeSdfFiles = /usr/mydata/sdf
> Some Shp Files = /usr/otherdata/shp
> BigArseImages = /mnt/some_other_machine/data/images
> Some ???? DwfFiles = /usr/mydata/????Dwf
> ??sdf = /usr/mydata/bigsdf
>
> Unicode characters are supported in the mapping names. Spaces in  
> the mappings are also supported. Square brackets [ and ] are not  
> allowed. They are reserved characters for the serverconfig.ini  
> section titles.
>
> We will add API methods to return the unmanaged data mappings, set  
> the mappings, and also verify that the mappings are valid and  
> accessible.
>
> We will add an API to enumerate the unmanaged data files available  
> on the server machine. The files will be prefaced with their  
> mapping names. You may enumerate all files from all the available  
> drive mappings, or you may select a single drive mapping.
>
> A returned list from all available drive mappings may look like this:
>
> [SomeSdfFiles]world.sdf
> [Some Shp Files]ecuador.shp
> [Some ???? DwfFiles]large.dwf
> [Some ???? DwfFiles]subdir/Big.dwf
> [??sdf]reallybig.sdf
>
>
>
>
>
> Thanks,
>
> Tony
>
>
>
> ________________________________
>
> From: Robert Bray [mailto:[hidden email]]
> Sent: Friday, November 03, 2006 9:05 AM
> To: [hidden email]
> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
>
>
>
> Jason,
>
> Good questions. In addition I think it would make sense / be highly  
> desirable for all RFCs that add or modify APIs to include more  
> detail with respect to those changes. It would be nice to see the  
> actual API function signatures for example.
>
> Bob
>
>
> Jason Birch wrote:
>
> That would be cool.  That way we can have a theme, and a nested  
> hierarchy within that theme if we need the complexity.  Are we  
> restricted to single-word mappings, or will spaces be supported?  
> ASCII characters only or full internationalisation?
>
> What about schema changes?  Will there be a way of updating the  
> repository's knowledge of unmanaged data schemae?  Or are these  
> maybe cached on a session basis like the RDBMS schemae?
>
> There would also need to be a method that allows the author UIs to  
> validate resources that reference unmanaged data sources, to make  
> sure that any dependancies (properties, maptips, themes, etc) are  
> still valid, and give a graphical indication when the world slips.
>
> Jason
>
> ________________________________
>
> From: Robert Bray [mailto:[hidden email]]
> Sent: Thu 2006-11-02 10:46 PM
> To: [hidden email]
> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
>
>
> I too am concerned this will make data management hard for people  
> with lots of data / multiple projects. It might be better to  
> dedicate a section in serverconfig.ini to define path mappings as  
> follows:
>
> [Path Mappings]
> SomeSdfFiles = <some path>
> SomeShpFiles = <some other path>
> BigArseImages = <path to SAN disk>
> ...
>
> In Studio / Web Studio the user would be presented the list of path  
> mappings. Once they select a path mapping they can browse all files  
> and folders below that path.
>
> My 2 cents...
>
> Bob
> ---------------------------------------------------------------------  
> To unsubscribe, e-mail: [hidden email] For  
> additional commands, e-mail: [hidden email]
> <winmail.dat>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Tom Fukushima
In reply to this post by Tony Fang
Hi Zak,

I don't want to lock down the server so that it is restricted to those
files in the Unmanaged Data Mappings. In other words, this means that a
feature source can still reference a file that is not in one of the
subdirectories specified by the mappings. The reason we don't have to
restrict to those subdirectories is because we can use the operating
system's or network's built-in security to limit access to files
(hopefully they've allowed access to the files in the Unmanaged Data,
otherwise they won't be very useful :) ).

Will WebStudio use the new APIs instead of allowing the enumeration of
arbitrary directories?

Tom

-----Original Message-----
From: Zak James [mailto:[hidden email]]
Sent: Friday, November 03, 2006 9:15 PM
To: [hidden email]
Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC

This is a minor compatibility issue, but the current version of
webstudio allows the user to specify an unmanaged feature source with an
arbitrary server path and then enumerates files in that path.  
Obviously, there are some security issues with this, but is that
functionality going to disappear with the changes proposed in the RFC?

If so, I think that needs to be stated in the Proposed Changes section.

zak
--
Zak James
Applications and Software Development
DM Solutions Group Inc.
http://www.dmsolutions.ca


On 3 Nov 2006, at 20:57, Jason Birch wrote:

> Looks good.
>
> I would personally prefer to see a single delimiter in the returned
> list, as it's slightly easier to deal with in code.
>
> I'm probably overly sensitive to this, but I'm wondering if the
> resource list will be cached on the server?  I know that the 6.5
> MapGuide author caches the responses internally.  I have to hit
> "update" to get a new list if something (data store, feature source,
> feature source schema) changes in the current session.  Can we count
> on the clients doing this to prevent unnecessary network traffic
> re-enumerating the resources?
>
> Jason
>
>
> ________________________________
>
> From: Tony Fang [mailto:[hidden email]]
> Sent: Fri 2006-11-03 4:41 PM
> To: [hidden email]
> Subject: RE: [mapguide-dev] Unmanaged Data APIs RFC
>
>
>
> I've updated the RFC with the drive mappings suggestion. Any feedback
> on the returned list of drive mappings (using square brackets on the
> drive mapping name) would be appreciated.
>
>
>
>
>
> The server's unmanaged data directories will be specified using the  
> serverconfig.ini. A new section defining directory mappings will be  
> used to specify the unmanaged data directories.
>
> For windows, it may look like this:
>
> [Unmanaged Data Mappings]
> SomeSdfFiles = c:\mydata\sdf
> Some Shp Files = d:\otherdata\shp
> BigArseImages = \\some_other_machine\data\images
> Some ???? DwfFiles = c:\mydata\????Dwf
> ??sdf = c:\mydata\bigsdf
>
> For linux, it may look like this:
>
> [Unmanaged Data Mappings]
> SomeSdfFiles = /usr/mydata/sdf
> Some Shp Files = /usr/otherdata/shp
> BigArseImages = /mnt/some_other_machine/data/images
> Some ???? DwfFiles = /usr/mydata/????Dwf
> ??sdf = /usr/mydata/bigsdf
>
> Unicode characters are supported in the mapping names. Spaces in  
> the mappings are also supported. Square brackets [ and ] are not  
> allowed. They are reserved characters for the serverconfig.ini  
> section titles.
>
> We will add API methods to return the unmanaged data mappings, set  
> the mappings, and also verify that the mappings are valid and  
> accessible.
>
> We will add an API to enumerate the unmanaged data files available  
> on the server machine. The files will be prefaced with their  
> mapping names. You may enumerate all files from all the available  
> drive mappings, or you may select a single drive mapping.
>
> A returned list from all available drive mappings may look like this:
>
> [SomeSdfFiles]world.sdf
> [Some Shp Files]ecuador.shp
> [Some ???? DwfFiles]large.dwf
> [Some ???? DwfFiles]subdir/Big.dwf
> [??sdf]reallybig.sdf
>
>
>
>
>
> Thanks,
>
> Tony
>
>
>
> ________________________________
>
> From: Robert Bray [mailto:[hidden email]]
> Sent: Friday, November 03, 2006 9:05 AM
> To: [hidden email]
> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
>
>
>
> Jason,
>
> Good questions. In addition I think it would make sense / be highly  
> desirable for all RFCs that add or modify APIs to include more  
> detail with respect to those changes. It would be nice to see the  
> actual API function signatures for example.
>
> Bob
>
>
> Jason Birch wrote:
>
> That would be cool.  That way we can have a theme, and a nested  
> hierarchy within that theme if we need the complexity.  Are we  
> restricted to single-word mappings, or will spaces be supported?  
> ASCII characters only or full internationalisation?
>
> What about schema changes?  Will there be a way of updating the  
> repository's knowledge of unmanaged data schemae?  Or are these  
> maybe cached on a session basis like the RDBMS schemae?
>
> There would also need to be a method that allows the author UIs to  
> validate resources that reference unmanaged data sources, to make  
> sure that any dependancies (properties, maptips, themes, etc) are  
> still valid, and give a graphical indication when the world slips.
>
> Jason
>
> ________________________________
>
> From: Robert Bray [mailto:[hidden email]]
> Sent: Thu 2006-11-02 10:46 PM
> To: [hidden email]
> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
>
>
> I too am concerned this will make data management hard for people  
> with lots of data / multiple projects. It might be better to  
> dedicate a section in serverconfig.ini to define path mappings as  
> follows:
>
> [Path Mappings]
> SomeSdfFiles = <some path>
> SomeShpFiles = <some other path>
> BigArseImages = <path to SAN disk>
> ...
>
> In Studio / Web Studio the user would be presented the list of path  
> mappings. Once they select a path mapping they can browse all files  
> and folders below that path.
>
> My 2 cents...
>
> Bob
> ---------------------------------------------------------------------

> To unsubscribe, e-mail: [hidden email] For  
> additional commands, e-mail: [hidden email]
> <winmail.dat>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]



---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: Unmanaged Data APIs RFC

Zak James-2
Tom,

I think we will definitely be supporting the new API once it's  
available, but in light of what you've said, the current approach is  
still valid and I don't see a reason to stop users from creating  
feature sources from unmapped data sources.

Thanks for the clarification.

zak
--
Zak James
Applications and Software Development
DM Solutions Group Inc.
http://www.dmsolutions.ca


On 8 Nov 2006, at 23:03, Tom Fukushima wrote:

> Hi Zak,
>
> I don't want to lock down the server so that it is restricted to those
> files in the Unmanaged Data Mappings. In other words, this means  
> that a
> feature source can still reference a file that is not in one of the
> subdirectories specified by the mappings. The reason we don't have to
> restrict to those subdirectories is because we can use the operating
> system's or network's built-in security to limit access to files
> (hopefully they've allowed access to the files in the Unmanaged Data,
> otherwise they won't be very useful :) ).
>
> Will WebStudio use the new APIs instead of allowing the enumeration of
> arbitrary directories?
>
> Tom
>
> -----Original Message-----
> From: Zak James [mailto:[hidden email]]
> Sent: Friday, November 03, 2006 9:15 PM
> To: [hidden email]
> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
>
> This is a minor compatibility issue, but the current version of
> webstudio allows the user to specify an unmanaged feature source  
> with an
> arbitrary server path and then enumerates files in that path.
> Obviously, there are some security issues with this, but is that
> functionality going to disappear with the changes proposed in the RFC?
>
> If so, I think that needs to be stated in the Proposed Changes  
> section.
>
> zak
> --
> Zak James
> Applications and Software Development
> DM Solutions Group Inc.
> http://www.dmsolutions.ca
>
>
> On 3 Nov 2006, at 20:57, Jason Birch wrote:
>
>> Looks good.
>>
>> I would personally prefer to see a single delimiter in the returned
>> list, as it's slightly easier to deal with in code.
>>
>> I'm probably overly sensitive to this, but I'm wondering if the
>> resource list will be cached on the server?  I know that the 6.5
>> MapGuide author caches the responses internally.  I have to hit
>> "update" to get a new list if something (data store, feature source,
>> feature source schema) changes in the current session.  Can we count
>> on the clients doing this to prevent unnecessary network traffic
>> re-enumerating the resources?
>>
>> Jason
>>
>>
>> ________________________________
>>
>> From: Tony Fang [mailto:[hidden email]]
>> Sent: Fri 2006-11-03 4:41 PM
>> To: [hidden email]
>> Subject: RE: [mapguide-dev] Unmanaged Data APIs RFC
>>
>>
>>
>> I've updated the RFC with the drive mappings suggestion. Any feedback
>> on the returned list of drive mappings (using square brackets on the
>> drive mapping name) would be appreciated.
>>
>>
>>
>>
>>
>> The server's unmanaged data directories will be specified using the
>> serverconfig.ini. A new section defining directory mappings will be
>> used to specify the unmanaged data directories.
>>
>> For windows, it may look like this:
>>
>> [Unmanaged Data Mappings]
>> SomeSdfFiles = c:\mydata\sdf
>> Some Shp Files = d:\otherdata\shp
>> BigArseImages = \\some_other_machine\data\images
>> Some ???? DwfFiles = c:\mydata\????Dwf
>> ??sdf = c:\mydata\bigsdf
>>
>> For linux, it may look like this:
>>
>> [Unmanaged Data Mappings]
>> SomeSdfFiles = /usr/mydata/sdf
>> Some Shp Files = /usr/otherdata/shp
>> BigArseImages = /mnt/some_other_machine/data/images
>> Some ???? DwfFiles = /usr/mydata/????Dwf
>> ??sdf = /usr/mydata/bigsdf
>>
>> Unicode characters are supported in the mapping names. Spaces in
>> the mappings are also supported. Square brackets [ and ] are not
>> allowed. They are reserved characters for the serverconfig.ini
>> section titles.
>>
>> We will add API methods to return the unmanaged data mappings, set
>> the mappings, and also verify that the mappings are valid and
>> accessible.
>>
>> We will add an API to enumerate the unmanaged data files available
>> on the server machine. The files will be prefaced with their
>> mapping names. You may enumerate all files from all the available
>> drive mappings, or you may select a single drive mapping.
>>
>> A returned list from all available drive mappings may look like this:
>>
>> [SomeSdfFiles]world.sdf
>> [Some Shp Files]ecuador.shp
>> [Some ???? DwfFiles]large.dwf
>> [Some ???? DwfFiles]subdir/Big.dwf
>> [??sdf]reallybig.sdf
>>
>>
>>
>>
>>
>> Thanks,
>>
>> Tony
>>
>>
>>
>> ________________________________
>>
>> From: Robert Bray [mailto:[hidden email]]
>> Sent: Friday, November 03, 2006 9:05 AM
>> To: [hidden email]
>> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
>>
>>
>>
>> Jason,
>>
>> Good questions. In addition I think it would make sense / be highly
>> desirable for all RFCs that add or modify APIs to include more
>> detail with respect to those changes. It would be nice to see the
>> actual API function signatures for example.
>>
>> Bob
>>
>>
>> Jason Birch wrote:
>>
>> That would be cool.  That way we can have a theme, and a nested
>> hierarchy within that theme if we need the complexity.  Are we
>> restricted to single-word mappings, or will spaces be supported?
>> ASCII characters only or full internationalisation?
>>
>> What about schema changes?  Will there be a way of updating the
>> repository's knowledge of unmanaged data schemae?  Or are these
>> maybe cached on a session basis like the RDBMS schemae?
>>
>> There would also need to be a method that allows the author UIs to
>> validate resources that reference unmanaged data sources, to make
>> sure that any dependancies (properties, maptips, themes, etc) are
>> still valid, and give a graphical indication when the world slips.
>>
>> Jason
>>
>> ________________________________
>>
>> From: Robert Bray [mailto:[hidden email]]
>> Sent: Thu 2006-11-02 10:46 PM
>> To: [hidden email]
>> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
>>
>>
>> I too am concerned this will make data management hard for people
>> with lots of data / multiple projects. It might be better to
>> dedicate a section in serverconfig.ini to define path mappings as
>> follows:
>>
>> [Path Mappings]
>> SomeSdfFiles = <some path>
>> SomeShpFiles = <some other path>
>> BigArseImages = <path to SAN disk>
>> ...
>>
>> In Studio / Web Studio the user would be presented the list of path
>> mappings. Once they select a path mapping they can browse all files
>> and folders below that path.
>>
>> My 2 cents...
>>
>> Bob
>> ---------------------------------------------------------------------
>
>> To unsubscribe, e-mail: [hidden email] For
>> additional commands, e-mail: [hidden email]
>> <winmail.dat>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email]
>> For additional commands, e-mail: [hidden email]
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Tom Fukushima
Hi Zak,
 
Sorry, I didn't mean for my suggestion to sound like an edict.  I only meant to propose that as the way things should work.  If no one finds a problem with that suggestion, I'll get it put into the RFC.
 
Cheers
Tom

        -----Original Message-----
        From: Zak James [mailto:[hidden email]]
        Sent: Wed 2006/11/08 21:10
        To: [hidden email]
        Cc:
        Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
       
       

        Tom,
       
        I think we will definitely be supporting the new API once it's
        available, but in light of what you've said, the current approach is
        still valid and I don't see a reason to stop users from creating
        feature sources from unmapped data sources.
       
        Thanks for the clarification.
       
        zak
        --
        Zak James
        Applications and Software Development
        DM Solutions Group Inc.
        http://www.dmsolutions.ca
       
       
        On 8 Nov 2006, at 23:03, Tom Fukushima wrote:
       
        > Hi Zak,
        >
        > I don't want to lock down the server so that it is restricted to those
        > files in the Unmanaged Data Mappings. In other words, this means
        > that a
        > feature source can still reference a file that is not in one of the
        > subdirectories specified by the mappings. The reason we don't have to
        > restrict to those subdirectories is because we can use the operating
        > system's or network's built-in security to limit access to files
        > (hopefully they've allowed access to the files in the Unmanaged Data,
        > otherwise they won't be very useful :) ).
        >
        > Will WebStudio use the new APIs instead of allowing the enumeration of
        > arbitrary directories?
        >
        > Tom
        >
        > -----Original Message-----
        > From: Zak James [mailto:[hidden email]]
        > Sent: Friday, November 03, 2006 9:15 PM
        > To: [hidden email]
        > Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
        >
        > This is a minor compatibility issue, but the current version of
        > webstudio allows the user to specify an unmanaged feature source
        > with an
        > arbitrary server path and then enumerates files in that path.
        > Obviously, there are some security issues with this, but is that
        > functionality going to disappear with the changes proposed in the RFC?
        >
        > If so, I think that needs to be stated in the Proposed Changes
        > section.
        >
        > zak
        > --
        > Zak James
        > Applications and Software Development
        > DM Solutions Group Inc.
        > http://www.dmsolutions.ca
        >
        >
        > On 3 Nov 2006, at 20:57, Jason Birch wrote:
        >
        >> Looks good.
        >>
        >> I would personally prefer to see a single delimiter in the returned
        >> list, as it's slightly easier to deal with in code.
        >>
        >> I'm probably overly sensitive to this, but I'm wondering if the
        >> resource list will be cached on the server?  I know that the 6.5
        >> MapGuide author caches the responses internally.  I have to hit
        >> "update" to get a new list if something (data store, feature source,
        >> feature source schema) changes in the current session.  Can we count
        >> on the clients doing this to prevent unnecessary network traffic
        >> re-enumerating the resources?
        >>
        >> Jason
        >>
        >>
        >> ________________________________
        >>
        >> From: Tony Fang [mailto:[hidden email]]
        >> Sent: Fri 2006-11-03 4:41 PM
        >> To: [hidden email]
        >> Subject: RE: [mapguide-dev] Unmanaged Data APIs RFC
        >>
        >>
        >>
        >> I've updated the RFC with the drive mappings suggestion. Any feedback
        >> on the returned list of drive mappings (using square brackets on the
        >> drive mapping name) would be appreciated.
        >>
        >>
        >>
        >>
        >>
        >> The server's unmanaged data directories will be specified using the
        >> serverconfig.ini. A new section defining directory mappings will be
        >> used to specify the unmanaged data directories.
        >>
        >> For windows, it may look like this:
        >>
        >> [Unmanaged Data Mappings]
        >> SomeSdfFiles = c:\mydata\sdf
        >> Some Shp Files = d:\otherdata\shp
        >> BigArseImages = \\some_other_machine\data\images
        >> Some ???? DwfFiles = c:\mydata\????Dwf
        >> ??sdf = c:\mydata\bigsdf
        >>
        >> For linux, it may look like this:
        >>
        >> [Unmanaged Data Mappings]
        >> SomeSdfFiles = /usr/mydata/sdf
        >> Some Shp Files = /usr/otherdata/shp
        >> BigArseImages = /mnt/some_other_machine/data/images
        >> Some ???? DwfFiles = /usr/mydata/????Dwf
        >> ??sdf = /usr/mydata/bigsdf
        >>
        >> Unicode characters are supported in the mapping names. Spaces in
        >> the mappings are also supported. Square brackets [ and ] are not
        >> allowed. They are reserved characters for the serverconfig.ini
        >> section titles.
        >>
        >> We will add API methods to return the unmanaged data mappings, set
        >> the mappings, and also verify that the mappings are valid and
        >> accessible.
        >>
        >> We will add an API to enumerate the unmanaged data files available
        >> on the server machine. The files will be prefaced with their
        >> mapping names. You may enumerate all files from all the available
        >> drive mappings, or you may select a single drive mapping.
        >>
        >> A returned list from all available drive mappings may look like this:
        >>
        >> [SomeSdfFiles]world.sdf
        >> [Some Shp Files]ecuador.shp
        >> [Some ???? DwfFiles]large.dwf
        >> [Some ???? DwfFiles]subdir/Big.dwf
        >> [??sdf]reallybig.sdf
        >>
        >>
        >>
        >>
        >>
        >> Thanks,
        >>
        >> Tony
        >>
        >>
        >>
        >> ________________________________
        >>
        >> From: Robert Bray [mailto:[hidden email]]
        >> Sent: Friday, November 03, 2006 9:05 AM
        >> To: [hidden email]
        >> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
        >>
        >>
        >>
        >> Jason,
        >>
        >> Good questions. In addition I think it would make sense / be highly
        >> desirable for all RFCs that add or modify APIs to include more
        >> detail with respect to those changes. It would be nice to see the
        >> actual API function signatures for example.
        >>
        >> Bob
        >>
        >>
        >> Jason Birch wrote:
        >>
        >> That would be cool.  That way we can have a theme, and a nested
        >> hierarchy within that theme if we need the complexity.  Are we
        >> restricted to single-word mappings, or will spaces be supported?
        >> ASCII characters only or full internationalisation?
        >>
        >> What about schema changes?  Will there be a way of updating the
        >> repository's knowledge of unmanaged data schemae?  Or are these
        >> maybe cached on a session basis like the RDBMS schemae?
        >>
        >> There would also need to be a method that allows the author UIs to
        >> validate resources that reference unmanaged data sources, to make
        >> sure that any dependancies (properties, maptips, themes, etc) are
        >> still valid, and give a graphical indication when the world slips.
        >>
        >> Jason
        >>
        >> ________________________________
        >>
        >> From: Robert Bray [mailto:[hidden email]]
        >> Sent: Thu 2006-11-02 10:46 PM
        >> To: [hidden email]
        >> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
        >>
        >>
        >> I too am concerned this will make data management hard for people
        >> with lots of data / multiple projects. It might be better to
        >> dedicate a section in serverconfig.ini to define path mappings as
        >> follows:
        >>
        >> [Path Mappings]
        >> SomeSdfFiles = <some path>
        >> SomeShpFiles = <some other path>
        >> BigArseImages = <path to SAN disk>
        >> ...
        >>
        >> In Studio / Web Studio the user would be presented the list of path
        >> mappings. Once they select a path mapping they can browse all files
        >> and folders below that path.
        >>
        >> My 2 cents...
        >>
        >> Bob
        >> ---------------------------------------------------------------------
        >
        >> To unsubscribe, e-mail: [hidden email] For
        >> additional commands, e-mail: [hidden email]
        >> <winmail.dat>
        >> ---------------------------------------------------------------------
        >> To unsubscribe, e-mail: [hidden email]
        >> For additional commands, e-mail: [hidden email]
        >
        >
        > ---------------------------------------------------------------------
        > To unsubscribe, e-mail: [hidden email]
        > For additional commands, e-mail: [hidden email]
        >
        >
        >
        > ---------------------------------------------------------------------
        > To unsubscribe, e-mail: [hidden email]
        > For additional commands, e-mail: [hidden email]
        >
       
       
        ---------------------------------------------------------------------
        To unsubscribe, e-mail: [hidden email]
        For additional commands, e-mail: [hidden email]
       
       


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

winmail.dat (17K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Unmanaged Data APIs RFC

Zak James-2
Tom,

It didn't come across that way... I was just seeking clarification on  
what would happen to the existing capability and I have that, so I'm  
happy.

zak
--
Zak James
Applications and Software Development
DM Solutions Group Inc.
http://www.dmsolutions.ca


On 9 Nov 2006, at 00:01, Tom Fukushima wrote:

> Hi Zak,
>
> Sorry, I didn't mean for my suggestion to sound like an edict.  I  
> only meant to propose that as the way things should work.  If no  
> one finds a problem with that suggestion, I'll get it put into the  
> RFC.
>
> Cheers
> Tom
>
> -----Original Message-----
> From: Zak James [mailto:[hidden email]]
> Sent: Wed 2006/11/08 21:10
> To: [hidden email]
> Cc:
> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
>
>
>
> Tom,
>
> I think we will definitely be supporting the new API once it's
> available, but in light of what you've said, the current approach is
> still valid and I don't see a reason to stop users from creating
> feature sources from unmapped data sources.
>
> Thanks for the clarification.
>
> zak
> --
> Zak James
> Applications and Software Development
> DM Solutions Group Inc.
> http://www.dmsolutions.ca
>
>
> On 8 Nov 2006, at 23:03, Tom Fukushima wrote:
>
> > Hi Zak,
> >
> > I don't want to lock down the server so that it is restricted to  
> those
> > files in the Unmanaged Data Mappings. In other words, this means
> > that a
> > feature source can still reference a file that is not in one of the
> > subdirectories specified by the mappings. The reason we don't  
> have to
> > restrict to those subdirectories is because we can use the  
> operating
> > system's or network's built-in security to limit access to files
> > (hopefully they've allowed access to the files in the Unmanaged  
> Data,
> > otherwise they won't be very useful :) ).
> >
> > Will WebStudio use the new APIs instead of allowing the  
> enumeration of
> > arbitrary directories?
> >
> > Tom
> >
> > -----Original Message-----
> > From: Zak James [mailto:[hidden email]]
> > Sent: Friday, November 03, 2006 9:15 PM
> > To: [hidden email]
> > Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
> >
> > This is a minor compatibility issue, but the current version of
> > webstudio allows the user to specify an unmanaged feature source
> > with an
> > arbitrary server path and then enumerates files in that path.
> > Obviously, there are some security issues with this, but is that
> > functionality going to disappear with the changes proposed in  
> the RFC?
> >
> > If so, I think that needs to be stated in the Proposed Changes
> > section.
> >
> > zak
> > --
> > Zak James
> > Applications and Software Development
> > DM Solutions Group Inc.
> > http://www.dmsolutions.ca
> >
> >
> > On 3 Nov 2006, at 20:57, Jason Birch wrote:
> >
> >> Looks good.
> >>
> >> I would personally prefer to see a single delimiter in the  
> returned
> >> list, as it's slightly easier to deal with in code.
> >>
> >> I'm probably overly sensitive to this, but I'm wondering if the
> >> resource list will be cached on the server?  I know that the 6.5
> >> MapGuide author caches the responses internally.  I have to hit
> >> "update" to get a new list if something (data store, feature  
> source,
> >> feature source schema) changes in the current session.  Can we  
> count
> >> on the clients doing this to prevent unnecessary network traffic
> >> re-enumerating the resources?
> >>
> >> Jason
> >>
> >>
> >> ________________________________
> >>
> >> From: Tony Fang [mailto:[hidden email]]
> >> Sent: Fri 2006-11-03 4:41 PM
> >> To: [hidden email]
> >> Subject: RE: [mapguide-dev] Unmanaged Data APIs RFC
> >>
> >>
> >>
> >> I've updated the RFC with the drive mappings suggestion. Any  
> feedback
> >> on the returned list of drive mappings (using square brackets  
> on the
> >> drive mapping name) would be appreciated.
> >>
> >>
> >>
> >>
> >>
> >> The server's unmanaged data directories will be specified using  
> the
> >> serverconfig.ini. A new section defining directory mappings  
> will be
> >> used to specify the unmanaged data directories.
> >>
> >> For windows, it may look like this:
> >>
> >> [Unmanaged Data Mappings]
> >> SomeSdfFiles = c:\mydata\sdf
> >> Some Shp Files = d:\otherdata\shp
> >> BigArseImages = \\some_other_machine\data\images
> >> Some ???? DwfFiles = c:\mydata\????Dwf
> >> ??sdf = c:\mydata\bigsdf
> >>
> >> For linux, it may look like this:
> >>
> >> [Unmanaged Data Mappings]
> >> SomeSdfFiles = /usr/mydata/sdf
> >> Some Shp Files = /usr/otherdata/shp
> >> BigArseImages = /mnt/some_other_machine/data/images
> >> Some ???? DwfFiles = /usr/mydata/????Dwf
> >> ??sdf = /usr/mydata/bigsdf
> >>
> >> Unicode characters are supported in the mapping names. Spaces in
> >> the mappings are also supported. Square brackets [ and ] are not
> >> allowed. They are reserved characters for the serverconfig.ini
> >> section titles.
> >>
> >> We will add API methods to return the unmanaged data mappings, set
> >> the mappings, and also verify that the mappings are valid and
> >> accessible.
> >>
> >> We will add an API to enumerate the unmanaged data files available
> >> on the server machine. The files will be prefaced with their
> >> mapping names. You may enumerate all files from all the available
> >> drive mappings, or you may select a single drive mapping.
> >>
> >> A returned list from all available drive mappings may look like  
> this:
> >>
> >> [SomeSdfFiles]world.sdf
> >> [Some Shp Files]ecuador.shp
> >> [Some ???? DwfFiles]large.dwf
> >> [Some ???? DwfFiles]subdir/Big.dwf
> >> [??sdf]reallybig.sdf
> >>
> >>
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Tony
> >>
> >>
> >>
> >> ________________________________
> >>
> >> From: Robert Bray [mailto:[hidden email]]
> >> Sent: Friday, November 03, 2006 9:05 AM
> >> To: [hidden email]
> >> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
> >>
> >>
> >>
> >> Jason,
> >>
> >> Good questions. In addition I think it would make sense / be  
> highly
> >> desirable for all RFCs that add or modify APIs to include more
> >> detail with respect to those changes. It would be nice to see the
> >> actual API function signatures for example.
> >>
> >> Bob
> >>
> >>
> >> Jason Birch wrote:
> >>
> >> That would be cool.  That way we can have a theme, and a nested
> >> hierarchy within that theme if we need the complexity.  Are we
> >> restricted to single-word mappings, or will spaces be supported?
> >> ASCII characters only or full internationalisation?
> >>
> >> What about schema changes?  Will there be a way of updating the
> >> repository's knowledge of unmanaged data schemae?  Or are these
> >> maybe cached on a session basis like the RDBMS schemae?
> >>
> >> There would also need to be a method that allows the author UIs to
> >> validate resources that reference unmanaged data sources, to make
> >> sure that any dependancies (properties, maptips, themes, etc) are
> >> still valid, and give a graphical indication when the world slips.
> >>
> >> Jason
> >>
> >> ________________________________
> >>
> >> From: Robert Bray [mailto:[hidden email]]
> >> Sent: Thu 2006-11-02 10:46 PM
> >> To: [hidden email]
> >> Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
> >>
> >>
> >> I too am concerned this will make data management hard for people
> >> with lots of data / multiple projects. It might be better to
> >> dedicate a section in serverconfig.ini to define path mappings as
> >> follows:
> >>
> >> [Path Mappings]
> >> SomeSdfFiles = <some path>
> >> SomeShpFiles = <some other path>
> >> BigArseImages = <path to SAN disk>
> >> ...
> >>
> >> In Studio / Web Studio the user would be presented the list of  
> path
> >> mappings. Once they select a path mapping they can browse all  
> files
> >> and folders below that path.
> >>
> >> My 2 cents...
> >>
> >> Bob
> >>  
> ---------------------------------------------------------------------
> >
> >> To unsubscribe, e-mail: [hidden email] For
> >> additional commands, e-mail: [hidden email]
> >> <winmail.dat>
> >>  
> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [hidden email]
> >> For additional commands, e-mail: [hidden email]
> >
> >
> >  
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
> >
> >
> >  
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [hidden email]
> > For additional commands, e-mail: [hidden email]
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>
>
> <winmail.dat>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Jason Birch
In reply to this post by Tony Fang
CONTENTS DELETED
The author has deleted this message.
Reply | Threaded
Open this post in threaded view
|

RE: Unmanaged Data APIs RFC

Bill Dhimitri
In reply to this post by Tony Fang

Hi Tony,

 

I think this looks good so far.  One thing that might be worth clarifying is the API to enumerate the unmanaged data files

 

1)       Should the API provide a way to filter what is returned by file type?  For example, maybe the caller wants just a list of SHP files.  Or just a list of ECW files. Etc. I am assuming that the API would allow filtering based on a specific mapping (your example shows a list from “all” mappings).

2)       Should the API provide control over the depth of the search into the directory sub-structure?  For example, the caller might want only those files in the top-most folders.  The concern here is that if I have a very large directory substructure with thousands of files, there could be a performance hit…

3)       Should the API provide a way to get a list of sub-folders?  Here, I am thinking about FDO which allows a folder (as opposed to a file) to be treated as a data source. FDO does this for raster and SHP/DBF/PRJ files.  Notice that today, MapGuide Studio does not directly support the notion of a data source that consists of a folder containing SHP/DBF/PRJ files, but FDO and the MapGuide server handle this scenario just fine.  Another common scenario that would be helped by this is a folder that contains say 500 geo-contiguous raster files.  It’s much more convenient to treat the folder as a single data source, rather than deal with each individual file.

 

Bill

 

 

 


From: Tony Fang
Sent: Friday, November 03, 2006 7:41 PM
To: [hidden email]
Subject: RE: [mapguide-dev] Unmanaged Data APIs RFC

 

I’ve updated the RFC with the drive mappings suggestion. Any feedback on the returned list of drive mappings (using square brackets on the drive mapping name) would be appreciated.

 

 

The server's unmanaged data directories will be specified using the serverconfig.ini. A new section defining directory mappings will be used to specify the unmanaged data directories.

For windows, it may look like this:

[Unmanaged Data Mappings]
SomeSdfFiles = c:\mydata\sdf
Some Shp Files = d:\otherdata\shp
BigArseImages = \\some_other_machine\data\images
Some 很大中文 DwfFiles = c:\mydata\很大中文Dwf
大きsdf = c:\mydata\bigsdf

For linux, it may look like this:

[Unmanaged Data Mappings]
SomeSdfFiles = /usr/mydata/sdf
Some Shp Files = /usr/otherdata/shp
BigArseImages = /mnt/some_other_machine/data/images
Some 很大中文 DwfFiles = /usr/mydata/很大中文Dwf
大きsdf = /usr/mydata/bigsdf

Unicode characters are supported in the mapping names. Spaces in the mappings are also supported. Square brackets [ and ] are not allowed. They are reserved characters for the serverconfig.ini section titles.

We will add API methods to return the unmanaged data mappings, set the mappings, and also verify that the mappings are valid and accessible.

We will add an API to enumerate the unmanaged data files available on the server machine. The files will be prefaced with their mapping names. You may enumerate all files from all the available drive mappings, or you may select a single drive mapping.

A returned list from all available drive mappings may look like this:

[SomeSdfFiles]world.sdf
[Some Shp Files]ecuador.shp
[Some 很大中文 DwfFiles]large.dwf
[Some 很大中文 DwfFiles]subdir/Big.dwf
[大きsdf]reallybig.sdf

 

 

Thanks,

Tony

 


From: Robert Bray [mailto:[hidden email]]
Sent: Friday, November 03, 2006 9:05 AM
To: [hidden email]
Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC

 

Jason,

Good questions. In addition I think it would make sense / be highly desirable for all RFCs that add or modify APIs to include more detail with respect to those changes. It would be nice to see the actual API function signatures for example.

Bob


Jason Birch wrote:

That would be cool.  That way we can have a theme, and a nested hierarchy within that theme if we need the complexity.  Are we restricted to single-word mappings, or will spaces be supported?  ASCII characters only or full internationalisation?
 
What about schema changes?  Will there be a way of updating the repository's knowledge of unmanaged data schemae?  Or are these maybe cached on a session basis like the RDBMS schemae?
 
There would also need to be a method that allows the author UIs to validate resources that reference unmanaged data sources, to make sure that any dependancies (properties, maptips, themes, etc) are still valid, and give a graphical indication when the world slips.
 
Jason
 
________________________________
 
From: Robert Bray [[hidden email]]
Sent: Thu 2006-11-02 10:46 PM
To: [hidden email]
Subject: Re: [mapguide-dev] Unmanaged Data APIs RFC
 
 
I too am concerned this will make data management hard for people with lots of data / multiple projects. It might be better to dedicate a section in serverconfig.ini to define path mappings as follows:
 
[Path Mappings]
SomeSdfFiles = <some path>
SomeShpFiles = <some other path>
BigArseImages = <path to SAN disk>
...
 
In Studio / Web Studio the user would be presented the list of path mappings. Once they select a path mapping they can browse all files and folders below that path.
 
My 2 cents...
 
Bob
 
--------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email] 
  
 
 
 



 
 
 
---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

 

--------------------------------------------------------------------- To unsubscribe, e-mail: [hidden email] For additional commands, e-mail: [hidden email]