Current master branch of postgis causing clone issues ?

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Current master branch of postgis causing clone issues ?

John Harvey
Hello all,

I see something weird in PostGIS at the moment.
When I do:
mkdir ~/git
cd ~/git
git clone [hidden email]:postgis/postgis.git
cd postgis
git status

I notice the following:
modified:   extras/tiger_geocoder/create_geocode.bat
modified:   extras/tiger_geocoder/upgrade_geocoder.bat

Although, I didn't modify these files.  I just did a regular clone.  

I suspect the .gitattributes file has something to do with it.
The 2 files impacted are Windows files, so probably have the wrong carriage-return type.
And, in the .gitattributes file, I see this rule:
*.bat       eol=crlf
So, I think this forces a change after clone.

I suspect this problem was introduced in the latest commit from 8/17 (it seems the endline changes were committed there).

Does anyone else see this problem?  Is this something that can be fixed?

Thanks!
  -John Harvey

_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: Current master branch of postgis causing clone issues ?

John Harvey
Hello all,

Just to make the issue easier to track/understand, I submitted a pull request:

This contains the 2 changes that keep getting modified on my branch whenever I clone.
If I remember right, this was how we fixed a similar situation on another project.

If I'm wrong about it though, feel free to correct me (I'd be glad to understand the situation a little better).

Regards,
  -John

On Thu, Aug 25, 2016 at 10:19 AM, John Harvey <[hidden email]> wrote:
Hello all,

I see something weird in PostGIS at the moment.
When I do:
mkdir ~/git
cd ~/git
git clone [hidden email]:postgis/postgis.git
cd postgis
git status

I notice the following:
modified:   extras/tiger_geocoder/create_geocode.bat
modified:   extras/tiger_geocoder/upgrade_geocoder.bat

Although, I didn't modify these files.  I just did a regular clone.  

I suspect the .gitattributes file has something to do with it.
The 2 files impacted are Windows files, so probably have the wrong carriage-return type.
And, in the .gitattributes file, I see this rule:
*.bat       eol=crlf
So, I think this forces a change after clone.

I suspect this problem was introduced in the latest commit from 8/17 (it seems the endline changes were committed there).

Does anyone else see this problem?  Is this something that can be fixed?

Thanks!
  -John Harvey


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: Current master branch of postgis causing clone issues ?

Regina Obe
In reply to this post by John Harvey

John,

The .gitattributes ignore has been there for some time I think.  So that isn't new.

 

I did make commit to those two last batch files on 8/17.  Looking at the carriage return type they look like crlf to me. 

Then again I'm on windows so things usually come down that way anyway unless explicitly set in gitattributes.

 

When I do a git clone and git status I get:

 

On branch svn-trunk

Your branch is up-to-date with 'origin/svn-trunk'.

nothing to commit, working directory clean

 

Perhaps someone not on windows can check and see if they are seeing the same thing.

It might be something introduced by the mirroring, because github is not our official repo.  We still use SVN so when I commit I commit on the svn repo and it gets mirrored to github.

 

Thanks,

Regina

 

 

 

 

From: postgis-devel [mailto:[hidden email]] On Behalf Of John Harvey
Sent: Thursday, August 25, 2016 10:19 AM
To: PostGIS Development Discussion <[hidden email]>
Subject: [postgis-devel] Current master branch of postgis causing clone issues ?

 

Hello all,

 

I see something weird in PostGIS at the moment.

When I do:

mkdir ~/git

cd ~/git

git clone [hidden email]

cd postgis

git status

 

I notice the following:

            modified:   extras/tiger_geocoder/create_geocode.bat

            modified:   extras/tiger_geocoder/upgrade_geocoder.bat

 

Although, I didn't modify these files.  I just did a regular clone.  

 

I suspect the .gitattributes file has something to do with it.

The 2 files impacted are Windows files, so probably have the wrong carriage-return type.

And, in the .gitattributes file, I see this rule:

*.bat       eol=crlf

So, I think this forces a change after clone.

 

I suspect this problem was introduced in the latest commit from 8/17 (it seems the endline changes were committed there).

 

Does anyone else see this problem?  Is this something that can be fixed?

 

Thanks!

  -John Harvey


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: Current master branch of postgis causing clone issues ?

John Harvey
Hi Regina,

Thanks for your reply.

Ok, I think I know the issue and can explain it.
I *believe* that the 2 batch files were actually committed with LF-only.

Here's how I'll try to prove it:
First, let's grab the raw file from *before* the Tiger 2016 commit:
git show 2a35bea:extras/tiger_geocoder/create_geocode.bat > ~/before_create_geocode.bat

That command may look weird, but I'm doing it this way to prevent any gitattribute changes or git config changes.

Second, let's grab the raw file *from* the Tiger 2016 commit
git show 598fea:extras/tiger_geocoder/create_geocode.bat > ~/after_create_geocode.bat

When I open ~/before_create_geocode.bat in a hex editor, I see that each line ends in "$0D0A", i.e. CRLF 
However, when I open ~/after_create_geocode.bat, I see that each line now only ends in "$0A", i.e LF.  So, the CR was dropped in the last commit.  That's why in my environment (a mac, using git), the .gitattributes file starts "fixing" things to be "$0D0A".

I've also verified that my pull-request I've sent in does contain the right raw $0D0A endlines.
I know that it's an SVN project, so that may not be helpful-- I can try to produce a patch-file if possible (or whatever you need).

Regards,
  -John

On Thu, Aug 25, 2016 at 11:00 AM, Regina Obe <[hidden email]> wrote:

John,

The .gitattributes ignore has been there for some time I think.  So that isn't new.

 

I did make commit to those two last batch files on 8/17.  Looking at the carriage return type they look like crlf to me. 

Then again I'm on windows so things usually come down that way anyway unless explicitly set in gitattributes.

 

When I do a git clone and git status I get:

 

On branch svn-trunk

Your branch is up-to-date with 'origin/svn-trunk'.

nothing to commit, working directory clean

 

Perhaps someone not on windows can check and see if they are seeing the same thing.

It might be something introduced by the mirroring, because github is not our official repo.  We still use SVN so when I commit I commit on the svn repo and it gets mirrored to github.

 

Thanks,

Regina

 

 

 

 

From: postgis-devel [mailto:[hidden email]] On Behalf Of John Harvey
Sent: Thursday, August 25, 2016 10:19 AM
To: PostGIS Development Discussion <[hidden email]>
Subject: [postgis-devel] Current master branch of postgis causing clone issues ?

 

Hello all,

 

I see something weird in PostGIS at the moment.

When I do:

mkdir ~/git

cd ~/git

git clone [hidden email]

cd postgis

git status

 

I notice the following:

            modified:   extras/tiger_geocoder/create_geocode.bat

            modified:   extras/tiger_geocoder/upgrade_geocoder.bat

 

Although, I didn't modify these files.  I just did a regular clone.  

 

I suspect the .gitattributes file has something to do with it.

The 2 files impacted are Windows files, so probably have the wrong carriage-return type.

And, in the .gitattributes file, I see this rule:

*.bat       eol=crlf

So, I think this forces a change after clone.

 

I suspect this problem was introduced in the latest commit from 8/17 (it seems the endline changes were committed there).

 

Does anyone else see this problem?  Is this something that can be fixed?

 

Thanks!

  -John Harvey


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: Current master branch of postgis causing clone issues ?

Regina Obe

I'm thinking it's still the git mirroring that converted  CRLF to LF.  So might be something strk needs to look into since he's in charge of the mirroring to github.

 

Looking at the file locally that I committed to SVN, it has CRLFs and we have a similar svn property to force CRLF for bat files for svn.

 

The only thing I can think of why this is suddenly an issue is I think this might be the first commit I have made to these files since we added the git attributes for bat files.

So perhaps before it took the file literally and now (it does something weird like store it with LF for push it down with CRLF)

 

Any non-windows person seeing the same issue?

 

I thought internally that Git always stores files with some linebreak specific to server and it's when the user pulls it down, that it may change line breaks for select files based on rules, so should be ignoring these for cloning.  Like I always get files down with CRLF (even .sh files, .sql, and config files unless specific gitattribute rules are in place) which drove me nuts cause then they'd cause all sorts of compile errors and false regression errors when running under a Unix shell in windows.

 

Thanks,

Regina

 

 

 

 

 

 

 

 

 

From: postgis-devel [mailto:[hidden email]] On Behalf Of John Harvey
Sent: Thursday, August 25, 2016 1:32 PM
To: PostGIS Development Discussion <[hidden email]>
Subject: Re: [postgis-devel] Current master branch of postgis causing clone issues ?

 

Hi Regina,

 

Thanks for your reply.

 

Ok, I think I know the issue and can explain it.

I *believe* that the 2 batch files were actually committed with LF-only.

 

Here's how I'll try to prove it:

First, let's grab the raw file from *before* the Tiger 2016 commit:

git show 2a35bea:extras/tiger_geocoder/create_geocode.bat > ~/before_create_geocode.bat

 

That command may look weird, but I'm doing it this way to prevent any gitattribute changes or git config changes.

 

Second, let's grab the raw file *from* the Tiger 2016 commit

git show 598fea:extras/tiger_geocoder/create_geocode.bat > ~/after_create_geocode.bat

 

When I open ~/before_create_geocode.bat in a hex editor, I see that each line ends in "$0D0A", i.e. CRLF 

However, when I open ~/after_create_geocode.bat, I see that each line now only ends in "$0A", i.e LF.  So, the CR was dropped in the last commit.  That's why in my environment (a mac, using git), the .gitattributes file starts "fixing" things to be "$0D0A".

 

I've also verified that my pull-request I've sent in does contain the right raw $0D0A endlines.

I know that it's an SVN project, so that may not be helpful-- I can try to produce a patch-file if possible (or whatever you need).

 

Regards,

  -John

 

On Thu, Aug 25, 2016 at 11:00 AM, Regina Obe <[hidden email]> wrote:

John,

The .gitattributes ignore has been there for some time I think.  So that isn't new.

 

I did make commit to those two last batch files on 8/17.  Looking at the carriage return type they look like crlf to me. 

Then again I'm on windows so things usually come down that way anyway unless explicitly set in gitattributes.

 

When I do a git clone and git status I get:

 

On branch svn-trunk

Your branch is up-to-date with 'origin/svn-trunk'.

nothing to commit, working directory clean

 

Perhaps someone not on windows can check and see if they are seeing the same thing.

It might be something introduced by the mirroring, because github is not our official repo.  We still use SVN so when I commit I commit on the svn repo and it gets mirrored to github.

 

Thanks,

Regina

 

 

 

 

From: postgis-devel [mailto:[hidden email]] On Behalf Of John Harvey
Sent: Thursday, August 25, 2016 10:19 AM
To: PostGIS Development Discussion <[hidden email]>
Subject: [postgis-devel] Current master branch of postgis causing clone issues ?

 

Hello all,

 

I see something weird in PostGIS at the moment.

When I do:

mkdir ~/git

cd ~/git

git clone [hidden email]

cd postgis

git status

 

I notice the following:

            modified:   extras/tiger_geocoder/create_geocode.bat

            modified:   extras/tiger_geocoder/upgrade_geocoder.bat

 

Although, I didn't modify these files.  I just did a regular clone.  

 

I suspect the .gitattributes file has something to do with it.

The 2 files impacted are Windows files, so probably have the wrong carriage-return type.

And, in the .gitattributes file, I see this rule:

*.bat       eol=crlf

So, I think this forces a change after clone.

 

I suspect this problem was introduced in the latest commit from 8/17 (it seems the endline changes were committed there).

 

Does anyone else see this problem?  Is this something that can be fixed?

 

Thanks!

  -John Harvey


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel

 


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: Current master branch of postgis causing clone issues ?

John Harvey
Hello Regina,

I thought about what you said, and decided to do some validation.
I am now confident that you are correct-- that it's an "svn sync to git" issue.

Your commit for Tiger Encoding 2016 in svn definitely is CRLF in subversion, but LF on the git side only.
So, it looks like something gets lost on the sync.

I had a thought, and looked back in the history-- this issue seems to have happened once before.
The last Tiger Encoder updates (from 2015) were committed about last year, under svn commit 13906.
Afterward, strk made the follow-up svn commit 13909 that had comment "Commit the file with crlf as requested by .gitattribute (I think)".

When I inspect both of these commits, it's confusing-- I see that they both are using CRLF on the subversion side.
So, it looks like strk did some magic that I cannot understand.  I'm thinking that we'll need that magic once more.

Regards,
  -John


On Thu, Aug 25, 2016 at 3:33 PM, Regina Obe <[hidden email]> wrote:

I'm thinking it's still the git mirroring that converted  CRLF to LF.  So might be something strk needs to look into since he's in charge of the mirroring to github.

 

Looking at the file locally that I committed to SVN, it has CRLFs and we have a similar svn property to force CRLF for bat files for svn.

 

The only thing I can think of why this is suddenly an issue is I think this might be the first commit I have made to these files since we added the git attributes for bat files.

So perhaps before it took the file literally and now (it does something weird like store it with LF for push it down with CRLF)

 

Any non-windows person seeing the same issue?

 

I thought internally that Git always stores files with some linebreak specific to server and it's when the user pulls it down, that it may change line breaks for select files based on rules, so should be ignoring these for cloning.  Like I always get files down with CRLF (even .sh files, .sql, and config files unless specific gitattribute rules are in place) which drove me nuts cause then they'd cause all sorts of compile errors and false regression errors when running under a Unix shell in windows.

 

Thanks,

Regina

 

 

 

 

 

 

 

 

 

From: postgis-devel [mailto:[hidden email]] On Behalf Of John Harvey
Sent: Thursday, August 25, 2016 1:32 PM
To: PostGIS Development Discussion <[hidden email]>
Subject: Re: [postgis-devel] Current master branch of postgis causing clone issues ?

 

Hi Regina,

 

Thanks for your reply.

 

Ok, I think I know the issue and can explain it.

I *believe* that the 2 batch files were actually committed with LF-only.

 

Here's how I'll try to prove it:

First, let's grab the raw file from *before* the Tiger 2016 commit:

git show 2a35bea:extras/tiger_geocoder/create_geocode.bat > ~/before_create_geocode.bat

 

That command may look weird, but I'm doing it this way to prevent any gitattribute changes or git config changes.

 

Second, let's grab the raw file *from* the Tiger 2016 commit

git show 598fea:extras/tiger_geocoder/create_geocode.bat > ~/after_create_geocode.bat

 

When I open ~/before_create_geocode.bat in a hex editor, I see that each line ends in "$0D0A", i.e. CRLF 

However, when I open ~/after_create_geocode.bat, I see that each line now only ends in "$0A", i.e LF.  So, the CR was dropped in the last commit.  That's why in my environment (a mac, using git), the .gitattributes file starts "fixing" things to be "$0D0A".

 

I've also verified that my pull-request I've sent in does contain the right raw $0D0A endlines.

I know that it's an SVN project, so that may not be helpful-- I can try to produce a patch-file if possible (or whatever you need).

 

Regards,

  -John

 

On Thu, Aug 25, 2016 at 11:00 AM, Regina Obe <[hidden email]> wrote:

John,

The .gitattributes ignore has been there for some time I think.  So that isn't new.

 

I did make commit to those two last batch files on 8/17.  Looking at the carriage return type they look like crlf to me. 

Then again I'm on windows so things usually come down that way anyway unless explicitly set in gitattributes.

 

When I do a git clone and git status I get:

 

On branch svn-trunk

Your branch is up-to-date with 'origin/svn-trunk'.

nothing to commit, working directory clean

 

Perhaps someone not on windows can check and see if they are seeing the same thing.

It might be something introduced by the mirroring, because github is not our official repo.  We still use SVN so when I commit I commit on the svn repo and it gets mirrored to github.

 

Thanks,

Regina

 

 

 

 

From: postgis-devel [mailto:[hidden email]] On Behalf Of John Harvey
Sent: Thursday, August 25, 2016 10:19 AM
To: PostGIS Development Discussion <[hidden email]>
Subject: [postgis-devel] Current master branch of postgis causing clone issues ?

 

Hello all,

 

I see something weird in PostGIS at the moment.

When I do:

mkdir ~/git

cd ~/git

git clone [hidden email]

cd postgis

git status

 

I notice the following:

            modified:   extras/tiger_geocoder/create_geocode.bat

            modified:   extras/tiger_geocoder/upgrade_geocoder.bat

 

Although, I didn't modify these files.  I just did a regular clone.  

 

I suspect the .gitattributes file has something to do with it.

The 2 files impacted are Windows files, so probably have the wrong carriage-return type.

And, in the .gitattributes file, I see this rule:

*.bat       eol=crlf

So, I think this forces a change after clone.

 

I suspect this problem was introduced in the latest commit from 8/17 (it seems the endline changes were committed there).

 

Does anyone else see this problem?  Is this something that can be fixed?

 

Thanks!

  -John Harvey


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel

 


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: Current master branch of postgis causing clone issues ?

Sandro Santilli-4
On Fri, Aug 26, 2016 at 10:55:09AM -0400, John Harvey wrote:

> So, it looks like strk did some magic that I cannot understand.  I'm
> thinking that we'll need that magic once more.

I think all I did was committing the change proposed by git
on updating. Did the same now with 15046, is everyone happy now ?

--strk;
_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel
Reply | Threaded
Open this post in threaded view
|

Re: Current master branch of postgis causing clone issues ?

John Harvey
Looks good to me.

Thank you!
  -John

On Sat, Aug 27, 2016 at 6:51 PM, Sandro Santilli <[hidden email]> wrote:
On Fri, Aug 26, 2016 at 10:55:09AM -0400, John Harvey wrote:

> So, it looks like strk did some magic that I cannot understand.  I'm
> thinking that we'll need that magic once more.

I think all I did was committing the change proposed by git
on updating. Did the same now with 15046, is everyone happy now ?

--strk;
_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel


_______________________________________________
postgis-devel mailing list
[hidden email]
http://lists.osgeo.org/mailman/listinfo/postgis-devel