Submitting proj.4 to Google OSS Fuzz ?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
18 messages Options
Reply | Threaded
Open this post in threaded view
|

Submitting proj.4 to Google OSS Fuzz ?

Even Rouault-2

Hi,

 

OSS-Fuzz is Continuous Fuzzing for Open Source Software :

https://github.com/google/oss-fuzz/ (it has a good intro on what it consists of)

 

Basically OSS Fuzz checkouts the source code repo every day, builds it, runs fuzzing tools

on test program you create, files bugs when it finds some and notify developers,

and close them automatically once it has verified that a fix has been pushed to the repo

(within one or two days)

 

I've experimented in integrating proj.4 with it (after having used it successfully

in GDAL since more than one week)

 

If you have Docker installed, you can test it locally with :

 

git clone --branch=add_proj [hidden email]:rouault/oss-fuzz.git

cd oss-fuzz

export PROJECT_NAME=proj4

python infra/helper.py build_image $PROJECT_NAME

# or --sanitizer undefined

python infra/helper.py build_fuzzers --sanitizer address $PROJECT_NAME

python infra/helper.py run_fuzzer $PROJECT_NAME standard_fuzzer

 

See https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md for more details.

 

In a few seconds, it has found 2 issues for which I have a PR ready;

https://github.com/OSGeo/proj.4/pull/516

It is likely that more are pending...

 

The integration in OSS Fuzz is in 2 parts :

- a few new files to Google OSS Fuzz repository, mostly to mention the

proj.4 code source repo and bootstrap the build with fuzzers

https://github.com/google/oss-fuzz/compare/master...rouault:add_proj

 

- a few new files to proj.4 repository with the code to run under the fuzzer:

https://github.com/OSGeo/proj.4/compare/master...rouault:ossfuzz

I've create a simple fuzzer, fuzzers/standard_fuzzer.cpp, that checks that there

are 3 lines in the random (*) input provided by the fuzzer code to our code ,

takes the first one as a potential source proj.4 string, the second one as a

potential target proj.4 string, the third one as a potential pair of coordinates and

runs pj_transform() on it.

And that's it (we don't really care about the return of pj_transform() itself). If none of the above

crashes, raises undefined behaviour, leaks memory, allocates tons of memory or takes forever

to complete, things are good. Otherwise oss fuzz will raise a bug.

It would be easy to add fuzzer targets similar to the above to test other parts of the API.

 

QUESTION 1:

Are people happy if we submit

https://github.com/google/oss-fuzz/compare/master...rouault:add_proj?expand=1

to Google - if they accept it since they are still in beta for now -, so they run it on

their clusters ? (actually the projects/proj4/Dockerfile will be modified to point to

proj.4 master instead of my clone, once I've merged my proj.4 ossfuzz branch to master)

 

If they don't accept it yet, we can also merge my proj.4 ossfuzz branch to master and

people interested can follow the above procedure to run it locally on their machine.

 

I've put Howard and Kristian in the CC list of bug notifications that will be privately accessible

in the first 90 days of their discovery.

 

QUESTION 2 to Howard and Kristian :

Please confirm you are interested in being CC'ed of bugs, and

tell me if the email I put is associated with a Google email account (if not, you

will not be able to access the bug details / bug list) :

https://github.com/google/oss-fuzz/compare/master...rouault:add_proj?expand=1#diff-76deaed2c7f4f80693f34903d9f7ae34

(actually I had an issue when I did the GDAL integration: it seems the email must be

a Google email, not just associated with a Google account)

 

If other proj.4 developers are interested, tell me and give me your Google email.

 

Even

 

 

(*) not so random input since the fuzzers are quite smart to build a relevant dictionnary, but

it is also possible to feed it with a relevant initial dictionnary too. For example we could

put some grid names, proj parameter names, etc...

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Kristian Evers-2

Good stuff, Even! Glad you took the time to set it up.

 

Q1: Yes. I am happy with integrating PROJ.4 with OSS-Fuzz.

 

Q2: I'd like to be on the CC list. I dont' think I have my work email attached to my google account, so please change my address to [hidden email] instead.

 

It is a quite impressive thing google have created here. It is also quite elaborate, so it will probably take a while before I fully understand how it works. 

 

/Kristian


Fra: Even Rouault [[hidden email]]
Sendt: 20. maj 2017 23:06
Til: [hidden email]
Emne: Submitting proj.4 to Google OSS Fuzz ?

Hi,

 

OSS-Fuzz is Continuous Fuzzing for Open Source Software :

https://github.com/google/oss-fuzz/ (it has a good intro on what it consists of)

 

Basically OSS Fuzz checkouts the source code repo every day, builds it, runs fuzzing tools

on test program you create, files bugs when it finds some and notify developers,

and close them automatically once it has verified that a fix has been pushed to the repo

(within one or two days)

 

I've experimented in integrating proj.4 with it (after having used it successfully

in GDAL since more than one week)

 

If you have Docker installed, you can test it locally with :

 

git clone --branch=add_proj [hidden email]

cd oss-fuzz

export PROJECT_NAME=proj4

python infra/helper.py build_image $PROJECT_NAME

# or --sanitizer undefined

python infra/helper.py build_fuzzers --sanitizer address $PROJECT_NAME

python infra/helper.py run_fuzzer $PROJECT_NAME standard_fuzzer

 

See https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md for more details.

 

In a few seconds, it has found 2 issues for which I have a PR ready;

https://github.com/OSGeo/proj.4/pull/516

It is likely that more are pending...

 

The integration in OSS Fuzz is in 2 parts :

- a few new files to Google OSS Fuzz repository, mostly to mention the

proj.4 code source repo and bootstrap the build with fuzzers

https://github.com/google/oss-fuzz/compare/master...rouault:add_proj

 

- a few new files to proj.4 repository with the code to run under the fuzzer:

https://github.com/OSGeo/proj.4/compare/master...rouault:ossfuzz

I've create a simple fuzzer, fuzzers/standard_fuzzer.cpp, that checks that there

are 3 lines in the random (*) input provided by the fuzzer code to our code ,

takes the first one as a potential source proj.4 string, the second one as a

potential target proj.4 string, the third one as a potential pair of coordinates and

runs pj_transform() on it.

And that's it (we don't really care about the return of pj_transform() itself). If none of the above

crashes, raises undefined behaviour, leaks memory, allocates tons of memory or takes forever

to complete, things are good. Otherwise oss fuzz will raise a bug.

It would be easy to add fuzzer targets similar to the above to test other parts of the API.

 

QUESTION 1:

Are people happy if we submit

https://github.com/google/oss-fuzz/compare/master...rouault:add_proj?expand=1

to Google - if they accept it since they are still in beta for now -, so they run it on

their clusters ? (actually the projects/proj4/Dockerfile will be modified to point to

proj.4 master instead of my clone, once I've merged my proj.4 ossfuzz branch to master)

 

If they don't accept it yet, we can also merge my proj.4 ossfuzz branch to master and

people interested can follow the above procedure to run it locally on their machine.

 

I've put Howard and Kristian in the CC list of bug notifications that will be privately accessible

in the first 90 days of their discovery.

 

QUESTION 2 to Howard and Kristian :

Please confirm you are interested in being CC'ed of bugs, and

tell me if the email I put is associated with a Google email account (if not, you

will not be able to access the bug details / bug list) :

https://github.com/google/oss-fuzz/compare/master...rouault:add_proj?expand=1#diff-76deaed2c7f4f80693f34903d9f7ae34

(actually I had an issue when I did the GDAL integration: it seems the email must be

a Google email, not just associated with a Google account)

 

If other proj.4 developers are interested, tell me and give me your Google email.

 

Even

 

 

(*) not so random input since the fuzzers are quite smart to build a relevant dictionnary, but

it is also possible to feed it with a relevant initial dictionnary too. For example we could

put some grid names, proj parameter names, etc...

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Even Rouault-2
In reply to this post by Even Rouault-2

Hi,

>

> OSS-Fuzz is Continuous Fuzzing for Open Source Software :

> https://github.com/google/oss-fuzz/ (it has a good intro on what it

> consists of)

 

Good news: proj.4 has just been accepted into OSS Fuzz !

 

https://bugs.chromium.org/p/oss-fuzz/issues/list?q=proj4 should be populated in a few hours...

 

For those wanting to tackle bugs, have a look at

https://github.com/OSGeo/proj.4/blob/master/test/fuzzers/README.TXT

for the procedure I suggest to follow.

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Kristian Evers-2

Even,

 

Good news, indeed. And a bunch of bugs has already been found!

 

I am trying to reproduce them on my own system and struggling a bit on how. If I understand correctly I am supposed to compile the fuzzing target like so:

 

> g++ -g -std=c++11 standard_fuzzer.cpp -o standard_fuzzer -DSTANDALONE ../../src/.libs/libproj.a –lpthread

 

And then run the executable with the reproducer testcase file from OSS-Fuzz. After a bit of modification I got the standard_fuzzer working on my system (win7+mingw), but I don’t know how to interpret the output when I run the program against the testcase. Everything seem to exit gracefully with return code 0. Is this normal or should I expect the program to crash in a noisy way?

 

/Kristian

 

Fra: [hidden email] [mailto:[hidden email].org] På vegne af Even Rouault
Sendt: 22. maj 2017 17:44
Til: [hidden email]
Emne: Re: [Proj] Submitting proj.4 to Google OSS Fuzz ?

 

Hi,

>

> OSS-Fuzz is Continuous Fuzzing for Open Source Software :

> https://github.com/google/oss-fuzz/ (it has a good intro on what it

> consists of)

 

Good news: proj.4 has just been accepted into OSS Fuzz !

 

https://bugs.chromium.org/p/oss-fuzz/issues/list?q=proj4 should be populated in a few hours...

 

For those wanting to tackle bugs, have a look at

https://github.com/OSGeo/proj.4/blob/master/test/fuzzers/README.TXT

for the procedure I suggest to follow.

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Even Rouault-2

On mardi 23 mai 2017 08:49:29 CEST Kristian Evers wrote:

> Even,

>

> Good news, indeed. And a bunch of bugs has already been found!

>

> I am trying to reproduce them on my own system and struggling a bit on how. If I understand correctly I am supposed to compile the fuzzing target like so:

> > g++ -g -std=c++11 standard_fuzzer.cpp -o standard_fuzzer -DSTANDALONE

> > ../../src/.libs/libproj.a -lpthread

> And then run the executable with the reproducer testcase file from OSS-Fuzz.

> After a bit of modification I got the standard_fuzzer working on my system

> (win7+mingw), but I don't know how to interpret the output when I run the

> program against the testcase. Everything seem to exit gracefully with

> return code 0. Is this normal or should I expect the program to crash in a

> noisy way?

 

Kristian,

 

You may get obvious crashes in some cases, but some errors are memory leaks or more subtle memory misuses that will generally not result in a crash. I wouldn't use Windows to debug that (or perhaps with DrMemory ?) , but rather Linux + Valgrind

Or try building with -fsanitize=address,undefined in CFLAGS and LDFLAGS (that's what OSS Fuzz uses to detect issues) if they are supported on mingw

 

Even

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Even Rouault-2

On mardi 23 mai 2017 11:29:01 CEST Even Rouault wrote:

> On mardi 23 mai 2017 08:49:29 CEST Kristian Evers wrote:

> > Even,

> >

> > Good news, indeed. And a bunch of bugs has already been found!

> >

> > I am trying to reproduce them on my own system and struggling a bit on

> > how. If I

> understand correctly I am supposed to compile the fuzzing target like so:

> > > g++ -g -std=c++11 standard_fuzzer.cpp -o standard_fuzzer -DSTANDALONE

> > > ../../src/.libs/libproj.a -lpthread

> >

> > And then run the executable with the reproducer testcase file from

> > OSS-Fuzz. After a bit of modification I got the standard_fuzzer working

> > on my system (win7+mingw), but I don't know how to interpret the output

> > when I run the program against the testcase. Everything seem to exit

> > gracefully with return code 0. Is this normal or should I expect the

> > program to crash in a noisy way?

>

> Kristian,

>

> You may get obvious crashes in some cases, but some errors are memory leaks

> or more subtle memory misuses that will generally not result in a crash. I

> wouldn't use Windows to debug that (or perhaps with DrMemory ?) , but

> rather Linux + Valgrind Or try building with -fsanitize=address,undefined

> in CFLAGS and LDFLAGS (that's what OSS Fuzz uses to detect issues) if they

> are supported on mingw

 

I see a number of division by zero issues are reported. From my experience now with GDAL and OSS Fuzz, a number of them will not cause runtime crash. For example, when it is a floating point division by zero (contrary to integer division by zero which lead to crash). Apparently -fsanitize=undefined considers this as undefined behaviour.

I haven't looked at the details of the issues but perhaps we lack some validation of parameters and should probably refuse crazy values at pj_init() time (although I can see some validation done in pj_ell_set). More generally we should try to validate what we can at initialization time rather than in the forward or reverse methods of projections.

 

More generally I'd expect we have a lack of rubustness regarding those rather pedantic undefined behaviour warnings when feeding infinity, NaN and other such inputs either in proj.4 string or in coordinates (but given the way the fuzzer likely works, I don't think it is likely to try feeding "nan" or "inf" strings in the fuzzed input since nothing in the code compares against those strings. Perhas they should be added in a dictionary to make them more likely if we want to test for that)

It is also possible to whitelist / blacklist the type of sanitizers we want to use.

See the 'sanitizers' attribute of the project.yaml file: https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md

 

Even

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Kristian Evers-2

Even,

 

Windows is what I have on hand at work, so there’s that… I am sure everything is a lot smoother on Linux. I’ll try a bit more on windows and see if it is usable or not. I might modify the code slightly if my effort turn out successful.

 

I was specifically looking at one of the Division by zero errors (1801) and expected a proper crash, but as you have experienced with GDAL, nothing really happened. Better testing and rejection of input values are definitely a good place to start.

 

How is the fuzzer generating the input values? Completely at random, or does it somehow get help with setting up the proj-strings?

 

/Kristian

 

 

Fra: [hidden email] [mailto:[hidden email]] På vegne af Even Rouault
Sendt: 23. maj 2017 11:56
Til: [hidden email]
Emne: Re: [Proj] Submitting proj.4 to Google OSS Fuzz ?

 

On mardi 23 mai 2017 11:29:01 CEST Even Rouault wrote:

> On mardi 23 mai 2017 08:49:29 CEST Kristian Evers wrote:

> > Even,

> >

> > Good news, indeed. And a bunch of bugs has already been found!

> >

> > I am trying to reproduce them on my own system and struggling a bit on

> > how. If I

> understand correctly I am supposed to compile the fuzzing target like so:

> > > g++ -g -std=c++11 standard_fuzzer.cpp -o standard_fuzzer -DSTANDALONE

> > > ../../src/.libs/libproj.a -lpthread

> >

> > And then run the executable with the reproducer testcase file from

> > OSS-Fuzz. After a bit of modification I got the standard_fuzzer working

> > on my system (win7+mingw), but I don't know how to interpret the output

> > when I run the program against the testcase. Everything seem to exit

> > gracefully with return code 0. Is this normal or should I expect the

> > program to crash in a noisy way?

>

> Kristian,

>

> You may get obvious crashes in some cases, but some errors are memory leaks

> or more subtle memory misuses that will generally not result in a crash. I

> wouldn't use Windows to debug that (or perhaps with DrMemory ?) , but

> rather Linux + Valgrind Or try building with -fsanitize=address,undefined

> in CFLAGS and LDFLAGS (that's what OSS Fuzz uses to detect issues) if they

> are supported on mingw

 

I see a number of division by zero issues are reported. From my experience now with GDAL and OSS Fuzz, a number of them will not cause runtime crash. For example, when it is a floating point division by zero (contrary to integer division by zero which lead to crash). Apparently -fsanitize=undefined considers this as undefined behaviour.

I haven't looked at the details of the issues but perhaps we lack some validation of parameters and should probably refuse crazy values at pj_init() time (although I can see some validation done in pj_ell_set). More generally we should try to validate what we can at initialization time rather than in the forward or reverse methods of projections.

 

More generally I'd expect we have a lack of rubustness regarding those rather pedantic undefined behaviour warnings when feeding infinity, NaN and other such inputs either in proj.4 string or in coordinates (but given the way the fuzzer likely works, I don't think it is likely to try feeding "nan" or "inf" strings in the fuzzed input since nothing in the code compares against those strings. Perhas they should be added in a dictionary to make them more likely if we want to test for that)

It is also possible to whitelist / blacklist the type of sanitizers we want to use.

See the 'sanitizers' attribute of the project.yaml file: https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md

 

Even

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Kurt Schwehr-2
Kristian,

Thanks for joining OSS Fuzz.  I'm not on that team at Google, but I have been doing a lot of fuzzing at Google in the last year.  Specifically GDAL and Kakadu with a touch of GEOS.  https://github.com/schwehr/gdal-autotest2/tree/master/cpp   I live in a space where code gets a ton of craziness sent at it at high volumns, so I really appreciate the work you all are doing.

The very definition of undefined behavior means that the compiler is allowed to do literally whatever it wants.  Common compilers usually try to give you some value that will keep things moving forward... that makes them especially difficult to find :(

If you are allowed to use docker (or anything else that will host a virtual os) on your work machine, that can give you a linux environment where it will be easier to find some of these.   I know that's often explicitly not allowed at many places.

It definitely helps to be able to add extra logic and rerun with the same ASAN setup.

You might also consider the free tier with Google Compute Engine or any of the other cloud providers like Amazon/RedHat/etc.  I only really know anything about the Google offerings since that's where I work, but the f1 micro instance is enough to checkout a copy of proj.4 and do some debugging.  Just know that builds won't be particularly fast.

-Kurt
Engineer at Google

On Tue, May 23, 2017 at 3:50 AM, Kristian Evers <[hidden email]> wrote:

Even,

 

Windows is what I have on hand at work, so there’s that… I am sure everything is a lot smoother on Linux. I’ll try a bit more on windows and see if it is usable or not. I might modify the code slightly if my effort turn out successful.

 

I was specifically looking at one of the Division by zero errors (1801) and expected a proper crash, but as you have experienced with GDAL, nothing really happened. Better testing and rejection of input values are definitely a good place to start.

 

How is the fuzzer generating the input values? Completely at random, or does it somehow get help with setting up the proj-strings?

 

/Kristian

 

 

Fra: [hidden email] [mailto:[hidden email]] På vegne af Even Rouault
Sendt: 23. maj 2017 11:56
Til: [hidden email]
Emne: Re: [Proj] Submitting proj.4 to Google OSS Fuzz ?

 

On mardi 23 mai 2017 11:29:01 CEST Even Rouault wrote:

> On mardi 23 mai 2017 08:49:29 CEST Kristian Evers wrote:

> > Even,

> >

> > Good news, indeed. And a bunch of bugs has already been found!

> >

> > I am trying to reproduce them on my own system and struggling a bit on

> > how. If I

> understand correctly I am supposed to compile the fuzzing target like so:

> > > g++ -g -std=c++11 standard_fuzzer.cpp -o standard_fuzzer -DSTANDALONE

> > > ../../src/.libs/libproj.a -lpthread

> >

> > And then run the executable with the reproducer testcase file from

> > OSS-Fuzz. After a bit of modification I got the standard_fuzzer working

> > on my system (win7+mingw), but I don't know how to interpret the output

> > when I run the program against the testcase. Everything seem to exit

> > gracefully with return code 0. Is this normal or should I expect the

> > program to crash in a noisy way?

>

> Kristian,

>

> You may get obvious crashes in some cases, but some errors are memory leaks

> or more subtle memory misuses that will generally not result in a crash. I

> wouldn't use Windows to debug that (or perhaps with DrMemory ?) , but

> rather Linux + Valgrind Or try building with -fsanitize=address,undefined

> in CFLAGS and LDFLAGS (that's what OSS Fuzz uses to detect issues) if they

> are supported on mingw

 

I see a number of division by zero issues are reported. From my experience now with GDAL and OSS Fuzz, a number of them will not cause runtime crash. For example, when it is a floating point division by zero (contrary to integer division by zero which lead to crash). Apparently -fsanitize=undefined considers this as undefined behaviour.

I haven't looked at the details of the issues but perhaps we lack some validation of parameters and should probably refuse crazy values at pj_init() time (although I can see some validation done in pj_ell_set). More generally we should try to validate what we can at initialization time rather than in the forward or reverse methods of projections.

 

More generally I'd expect we have a lack of rubustness regarding those rather pedantic undefined behaviour warnings when feeding infinity, NaN and other such inputs either in proj.4 string or in coordinates (but given the way the fuzzer likely works, I don't think it is likely to try feeding "nan" or "inf" strings in the fuzzed input since nothing in the code compares against those strings. Perhas they should be added in a dictionary to make them more likely if we want to test for that)

It is also possible to whitelist / blacklist the type of sanitizers we want to use.

See the 'sanitizers' attribute of the project.yaml file: https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md

 

Even

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj



--

_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Thomas Knudsen
In reply to this post by Kristian Evers-2
In IEEE754 arithmetic, division by zero results in +/- infinity, not by a crash.

Hence, crashes are only expected if dividing by integer zero, as demonstrated below:

$ cat IEEE754_division_by_zero.c

#include <stdio.h>

int main (void) {
  double dresult, dzero = 0, dten = 10;
    int    iresult, izero = 0, iten = 10;
  puts ("Dividing by double zero");
  dresult = dten / dzero;
  printf ("dresult = %g\n", dresult);
  puts ("Dividing by integer zero");
  iresult = iten / izero;
  printf ("iresult = %d\n", iresult);
}

$ gcc ieee754division_by_zero.c 
$ a                             
Dividing by double zero         
dresult = 1.#INF                
Dividing by integer zero        

$

2017-05-23 12:50 GMT+02:00 Kristian Evers <[hidden email]>:

Even,

 

Windows is what I have on hand at work, so there’s that… I am sure everything is a lot smoother on Linux. I’ll try a bit more on windows and see if it is usable or not. I might modify the code slightly if my effort turn out successful.

 

I was specifically looking at one of the Division by zero errors (1801) and expected a proper crash, but as you have experienced with GDAL, nothing really happened. Better testing and rejection of input values are definitely a good place to start.

 

How is the fuzzer generating the input values? Completely at random, or does it somehow get help with setting up the proj-strings?

 

/Kristian

 

 

Fra: [hidden email] [mailto:[hidden email]] På vegne af Even Rouault
Sendt: 23. maj 2017 11:56
Til: [hidden email]
Emne: Re: [Proj] Submitting proj.4 to Google OSS Fuzz ?

 

On mardi 23 mai 2017 11:29:01 CEST Even Rouault wrote:

> On mardi 23 mai 2017 08:49:29 CEST Kristian Evers wrote:

> > Even,

> >

> > Good news, indeed. And a bunch of bugs has already been found!

> >

> > I am trying to reproduce them on my own system and struggling a bit on

> > how. If I

> understand correctly I am supposed to compile the fuzzing target like so:

> > > g++ -g -std=c++11 standard_fuzzer.cpp -o standard_fuzzer -DSTANDALONE

> > > ../../src/.libs/libproj.a -lpthread

> >

> > And then run the executable with the reproducer testcase file from

> > OSS-Fuzz. After a bit of modification I got the standard_fuzzer working

> > on my system (win7+mingw), but I don't know how to interpret the output

> > when I run the program against the testcase. Everything seem to exit

> > gracefully with return code 0. Is this normal or should I expect the

> > program to crash in a noisy way?

>

> Kristian,

>

> You may get obvious crashes in some cases, but some errors are memory leaks

> or more subtle memory misuses that will generally not result in a crash. I

> wouldn't use Windows to debug that (or perhaps with DrMemory ?) , but

> rather Linux + Valgrind Or try building with -fsanitize=address,undefined

> in CFLAGS and LDFLAGS (that's what OSS Fuzz uses to detect issues) if they

> are supported on mingw

 

I see a number of division by zero issues are reported. From my experience now with GDAL and OSS Fuzz, a number of them will not cause runtime crash. For example, when it is a floating point division by zero (contrary to integer division by zero which lead to crash). Apparently -fsanitize=undefined considers this as undefined behaviour.

I haven't looked at the details of the issues but perhaps we lack some validation of parameters and should probably refuse crazy values at pj_init() time (although I can see some validation done in pj_ell_set). More generally we should try to validate what we can at initialization time rather than in the forward or reverse methods of projections.

 

More generally I'd expect we have a lack of rubustness regarding those rather pedantic undefined behaviour warnings when feeding infinity, NaN and other such inputs either in proj.4 string or in coordinates (but given the way the fuzzer likely works, I don't think it is likely to try feeding "nan" or "inf" strings in the fuzzed input since nothing in the code compares against those strings. Perhas they should be added in a dictionary to make them more likely if we want to test for that)

It is also possible to whitelist / blacklist the type of sanitizers we want to use.

See the 'sanitizers' attribute of the project.yaml file: https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md

 

Even

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Even Rouault-2
In reply to this post by Kristian Evers-2

On mardi 23 mai 2017 10:50:52 CEST Kristian Evers wrote:

> Even,

>

> Windows is what I have on hand at work, so there's that... I am sure

> everything is a lot smoother on Linux. I'll try a bit more on windows and

> see if it is usable or not. I might modify the code slightly if my effort

> turn out successful.

 

Perhaps you can install VirtualBox and a Ubuntu/whatever distro image in it ? But for some of the errors the log is in the ticket is already a good start.

 

>

> I was specifically looking at one of the Division by zero errors (1801) and

> expected a proper crash, but as you have experienced with GDAL, nothing

> really happened.

 

Yes this is a floating point division. On Intel-like CPUs, floating point divisions by zero don't raise exceptions by default.

 

> Better testing and rejection of input values are

> definitely a good place to start.

>

> How is the fuzzer generating the input values? Completely at random, or does

> it somehow get help with setting up the proj-strings?

 

No, not completely at random, that's all the interest ! It instruments the code during compilation and at runtime observes which branches are taken or not given the input. So it favors new inputs that increase code coverage. I suspect also there's some instrumentation of strcmp(x, "SOME_LITERAL") calls to add SOME_LITERAL in the dictionary. When you locally run oss-fuzz and you look at the logs, it is obvious it detects the basic elements of a proj.4 strings like "init", parameter names and projection names.

 

>

> /Kristian

>

>

> Fra: [hidden email]

> [mailto:[hidden email]] På vegne af Even Rouault Sendt:

> 23. maj 2017 11:56

> Til: [hidden email]

> Emne: Re: [Proj] Submitting proj.4 to Google OSS Fuzz ?

>

> On mardi 23 mai 2017 11:29:01 CEST Even Rouault wrote:

> > On mardi 23 mai 2017 08:49:29 CEST Kristian Evers wrote:

> > > Even,

> > >

> > >

> > >

> > > Good news, indeed. And a bunch of bugs has already been found!

> > >

> > >

> > >

> > > I am trying to reproduce them on my own system and struggling a bit on

> > >

> > > how. If I

> >

> > understand correctly I am supposed to compile the fuzzing target like so:

> > > > g++ -g -std=c++11 standard_fuzzer.cpp -o standard_fuzzer -DSTANDALONE

> > > >

> > > > ../../src/.libs/libproj.a -lpthread

> > >

> > > And then run the executable with the reproducer testcase file from

> > >

> > > OSS-Fuzz. After a bit of modification I got the standard_fuzzer working

> > >

> > > on my system (win7+mingw), but I don't know how to interpret the output

> > >

> > > when I run the program against the testcase. Everything seem to exit

> > >

> > > gracefully with return code 0. Is this normal or should I expect the

> > >

> > > program to crash in a noisy way?

> >

> > Kristian,

> >

> >

> >

> > You may get obvious crashes in some cases, but some errors are memory

> > leaks

> >

> > or more subtle memory misuses that will generally not result in a crash. I

> >

> > wouldn't use Windows to debug that (or perhaps with DrMemory ?) , but

> >

> > rather Linux + Valgrind Or try building with -fsanitize=address,undefined

> >

> > in CFLAGS and LDFLAGS (that's what OSS Fuzz uses to detect issues) if they

> >

> > are supported on mingw

>

> I see a number of division by zero issues are reported. From my experience

> now with GDAL and OSS Fuzz, a number of them will not cause runtime crash.

> For example, when it is a floating point division by zero (contrary to

> integer division by zero which lead to crash). Apparently

> -fsanitize=undefined considers this as undefined behaviour.

>

> I haven't looked at the details of the issues but perhaps we lack some

> validation of parameters and should probably refuse crazy values at

> pj_init() time (although I can see some validation done in pj_ell_set).

> More generally we should try to validate what we can at initialization time

> rather than in the forward or reverse methods of projections.

>

>

>

> More generally I'd expect we have a lack of rubustness regarding those

> rather pedantic undefined behaviour warnings when feeding infinity, NaN and

> other such inputs either in proj.4 string or in coordinates (but given the

> way the fuzzer likely works, I don't think it is likely to try feeding

> "nan" or "inf" strings in the fuzzed input since nothing in the code

> compares against those strings. Perhas they should be added in a dictionary

> to make them more likely if we want to test for that)

>

> It is also possible to whitelist / blacklist the type of sanitizers we want

> to use.

>

> See the 'sanitizers' attribute of the project.yaml file:

> https://github.com/google/oss-fuzz/blob/master/docs/new_project_guide.md

>

>

>

> Even

>

>

>

>

>

> --

>

> Spatialys - Geospatial professional services

>

> http://www.spatialys.com

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Even Rouault-2
In reply to this post by Thomas Knudsen

On mardi 23 mai 2017 14:16:41 CEST Thomas Knudsen wrote:

> In IEEE754 arithmetic, division by zero results in +/- infinity, not by a

> crash.

>

> Hence, crashes are only expected if dividing by integer zero, as

> demonstrated below:

>

> $ cat IEEE754_division_by_zero.c

>

> #include <stdio.h>

>

> int main (void) {

> double dresult, dzero = 0, dten = 10;

> int iresult, izero = 0, iten = 10;

> puts ("Dividing by double zero");

> dresult = dten / dzero;

> printf ("dresult = %g\n", dresult);

> puts ("Dividing by integer zero");

> iresult = iten / izero;

> printf ("iresult = %d\n", iresult);

> }

>

> $ gcc ieee754division_by_zero.c

> $ a

> Dividing by double zero

> dresult = 1.#INF

> Dividing by integer zero

 

https://stackoverflow.com/questions/3004095/division-by-zero-undefined-behavior-or-implementation-defined-in-c-and-or-c

has some interesting references.

 

So it would seem that according to the C standard, division by zero, either in integer

or floating-point case is undefined behaviour. And that's what -fsanitize=undefined must

check.

 

In practice on Intel-like CPUs, the behaviour is what you mention above,

but could potentially be different on other architectures, or with a compiler that would

implement the C standard in an extreme way (like "crash everytime we encounter

a situation that is 'unspecified bheaviour' in the standard")

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Kristian Evers-2
Kurt, Even,

Thanks for your suggestions. Very helpful. I do have access to a Linux server
and if necessary I can work on that. It is just slightly inconvenient when on the move etc.


I must say, it is a very impressive piece software google has created!
Although it is a bit hard to grasp the finer details :-) I think I can fix some
of the discovered issues just from looking at the report, as you suggest, Even.


The only way to triggers OSS-Fuzz to test code is to commit on the master-branch, correct?

/Kristian
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Even Rouault-2

On mardi 23 mai 2017 13:41:50 CEST Kristian Evers wrote:

> Kurt, Even,

>

> Thanks for your suggestions. Very helpful. I do have access to a Linux

> server and if necessary I can work on that. It is just slightly

> inconvenient when on the move etc.

>

>

> I must say, it is a very impressive piece software google has created!

> Although it is a bit hard to grasp the finer details :-) I think I can fix

> some of the discovered issues just from looking at the report, as you

> suggest, Even.

>

>

> The only way to triggers OSS-Fuzz to test code is to commit on the

> master-branch, correct?

 

That's the most convenient. You can run OSS-Fuzz locally as instructed in proj.4 tests/fuzzers/README.TXT (but you need Linux and Docker for that), and there you could point the oss-fuzz/projects/proj4/Dockerfile to a branch of yours (that's how I tested it before submitting it)

 

>

> /Kristian

> _______________________________________________

> Proj mailing list

> [hidden email]

> http://lists.maptools.org/mailman/listinfo/proj

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Kristian Evers-2

> That's the most convenient. You can run OSS-Fuzz locally as instructed in proj.4 tests/fuzzers/README.TXT (but you need Linux and Docker for that), and there you could point the oss-fuzz/projects/proj4/Dockerfile to a branch of yours (that's how I tested it before submitting it)

 

Yeah, I figured. It would be super cool if you could trigger OSS-Fuzz targeting a specific bug via a pull request. Maybe that will be possible in the future…

 

Thanks for the help. I will try to run OSS-Fuzz locally to confirm fixes before I commit them to master.

 

Kristian

 

Fra: Even Rouault [mailto:[hidden email]]
Sendt: 23. maj 2017 16:09
Til: [hidden email]
Cc: Kristian Evers
Emne: Re: [Proj] Submitting proj.4 to Google OSS Fuzz ?

 

On mardi 23 mai 2017 13:41:50 CEST Kristian Evers wrote:

> Kurt, Even,

>

> Thanks for your suggestions. Very helpful. I do have access to a Linux

> server and if necessary I can work on that. It is just slightly

> inconvenient when on the move etc.

>

>

> I must say, it is a very impressive piece software google has created!

> Although it is a bit hard to grasp the finer details :-) I think I can fix

> some of the discovered issues just from looking at the report, as you

> suggest, Even.

>

>

> The only way to triggers OSS-Fuzz to test code is to commit on the

> master-branch, correct?

 

That's the most convenient. You can run OSS-Fuzz locally as instructed in proj.4 tests/fuzzers/README.TXT (but you need Linux and Docker for that), and there you could point the oss-fuzz/projects/proj4/Dockerfile to a branch of yours (that's how I tested it before submitting it)

 

>

> /Kristian

> _______________________________________________

> Proj mailing list

> [hidden email]

> http://lists.maptools.org/mailman/listinfo/proj

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Even Rouault-2

On mardi 23 mai 2017 14:24:11 CEST Kristian Evers wrote:

> > That's the most convenient. You can run OSS-Fuzz locally as instructed in

> > proj.4 tests/fuzzers/README.TXT (but you need Linux and Docker for that),

> > and there you could point the oss-fuzz/projects/proj4/Dockerfile to a

> > branch of yours (that's how I tested it before submitting it)

> Yeah, I figured. It would be super cool if you could trigger OSS-Fuzz

> targeting a specific bug via a pull request. Maybe that will be possible in

> the future...

>

> Thanks for the help. I will try to run OSS-Fuzz locally to confirm fixes

> before I commit them to master.

 

Well, in that case that means you're running Linux, so build proj.4 with CFLAGS="-fsanitize=undefined,address" and running the standalone standard_fuzzer should be sufficient (and much faster)

 

>

> Kristian

>

> Fra: Even Rouault [mailto:[hidden email]]

> Sendt: 23. maj 2017 16:09

> Til: [hidden email]

> Cc: Kristian Evers

> Emne: Re: [Proj] Submitting proj.4 to Google OSS Fuzz ?

>

> On mardi 23 mai 2017 13:41:50 CEST Kristian Evers wrote:

> > Kurt, Even,

> >

> >

> >

> > Thanks for your suggestions. Very helpful. I do have access to a Linux

> >

> > server and if necessary I can work on that. It is just slightly

> >

> > inconvenient when on the move etc.

> >

> >

> >

> >

> >

> > I must say, it is a very impressive piece software google has created!

> >

> > Although it is a bit hard to grasp the finer details :-) I think I can fix

> >

> > some of the discovered issues just from looking at the report, as you

> >

> > suggest, Even.

> >

> >

> >

> >

> >

> > The only way to triggers OSS-Fuzz to test code is to commit on the

> >

> > master-branch, correct?

>

> That's the most convenient. You can run OSS-Fuzz locally as instructed in

> proj.4 tests/fuzzers/README.TXT (but you need Linux and Docker for that),

> and there you could point the oss-fuzz/projects/proj4/Dockerfile to a

> branch of yours (that's how I tested it before submitting it)

> > /Kristian

> >

> > _______________________________________________

> >

> > Proj mailing list

> >

> > [hidden email]<mailto:[hidden email]>

> >

> > http://lists.maptools.org/mailman/listinfo/proj

>

> --

>

> Spatialys - Geospatial professional services

>

> http://www.spatialys.com

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Even Rouault-2

On mardi 23 mai 2017 17:23:32 CEST Even Rouault wrote:

> On mardi 23 mai 2017 14:24:11 CEST Kristian Evers wrote:

> > > That's the most convenient. You can run OSS-Fuzz locally as instructed

> > > in

> > > proj.4 tests/fuzzers/README.TXT (but you need Linux and Docker for

> > > that),

> > > and there you could point the oss-fuzz/projects/proj4/Dockerfile to a

> > > branch of yours (that's how I tested it before submitting it)

> >

> > Yeah, I figured. It would be super cool if you could trigger OSS-Fuzz

> > targeting a specific bug via a pull request. Maybe that will be possible

> > in

> > the future...

> >

> > Thanks for the help. I will try to run OSS-Fuzz locally to confirm fixes

> > before I commit them to master.

>

> Well, in that case that means you're running Linux, so build proj.4 with

> CFLAGS="- fsanitize=undefined,address" and running the standalone

> standard_fuzzer should be sufficient (and much faster)

 

Note: make sure the proj lib is built with a PROJ_LIB define that points to something valid, or define PROJ_LIB to point to the in-source nad directory when running standard_fuzzer as it can make a difference (in particular since the fuzzer might generate strings without +no_defs and ellipsoid values, and in that case they are valid due to nad/proj_def.dat containing ellps=WGS84. Whereas if you run standard_fuzzer with no valid PROJ_LIB you'll get an early exit due to proj_init() having failed. I've updated the example in test/fuzzers/README.TXT with that)

 

>

> > Kristian

> >

> > Fra: Even Rouault [mailto:[hidden email]]

> > Sendt: 23. maj 2017 16:09

> > Til: [hidden email]

> > Cc: Kristian Evers

> > Emne: Re: [Proj] Submitting proj.4 to Google OSS Fuzz ?

> >

> > On mardi 23 mai 2017 13:41:50 CEST Kristian Evers wrote:

> > > Kurt, Even,

> > >

> > >

> > >

> > > Thanks for your suggestions. Very helpful. I do have access to a Linux

> > >

> > > server and if necessary I can work on that. It is just slightly

> > >

> > > inconvenient when on the move etc.

> > >

> > >

> > >

> > >

> > >

> > > I must say, it is a very impressive piece software google has created!

> > >

> > > Although it is a bit hard to grasp the finer details :-) I think I can

> > > fix

> > >

> > > some of the discovered issues just from looking at the report, as you

> > >

> > > suggest, Even.

> > >

> > >

> > >

> > >

> > >

> > > The only way to triggers OSS-Fuzz to test code is to commit on the

> > >

> > > master-branch, correct?

> >

> > That's the most convenient. You can run OSS-Fuzz locally as instructed in

> > proj.4 tests/fuzzers/README.TXT (but you need Linux and Docker for that),

> > and there you could point the oss-fuzz/projects/proj4/Dockerfile to a

> > branch of yours (that's how I tested it before submitting it)

> >

> > > /Kristian

> > >

> > > _______________________________________________

> > >

> > > Proj mailing list

> > >

> > > [hidden email]<mailto:[hidden email]>

> > >

> > > http://lists.maptools.org/mailman/listinfo/proj

> >

> > --

> >

> > Spatialys - Geospatial professional services

> >

> > http://www.spatialys.com

 

 

--

Spatialys - Geospatial professional services

http://www.spatialys.com


_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

support-2
In reply to this post by Even Rouault-2

Hello,

QUESTION 1:

Are people happy if we submit

 

basically NO, since Google is what we know! Proj.4 will be used to spy citizens of different countries without asking them or anybody about it. But since Proj.4 is free source code there is not much to be done .. but I would NOT support any submitting by this list! - If they use it .. what can we do .. but we should not support anything they do since we know the amount of spying is connected to everything Google does (most likely without most of the software writers knowing it, since there is a different department (outside) that then uses those codes and adds the spying codes)..


Submit nothing but keep it open! (y)


Janne.






_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj
Reply | Threaded
Open this post in threaded view
|

Re: Submitting proj.4 to Google OSS Fuzz ?

Kristian Thy
On Mon, May 29, [hidden email] wrote:

> Hello,
>
> > QUESTION 1:
> >
> > Are people happy if we submit
>
> basically NO, since Google is what we know! Proj.4 will be used to spy
> citizens of different countries without asking them or anybody about it.
> But since Proj.4 is free source code there is not much to be done .. but
> I would NOT support any submitting by this list! - If they use it ..
> what can we do .. but we should not support anything they do since we
> know the amount of spying is connected to everything Google does (most
> likely without most of the software writers knowing it, since there is a
> different department (outside) that then uses those codes and adds the
> spying codes)..
>
> Submit nothing but keep it open! (y)
>
> Janne.

Could we cut down on the hyperbole and paranoia here?

I am no fan of Google's general business practices but that does not
mean that their technical contributions and services are without merit.
The OSS Fuzzer does not do commits to Proj.4; code changes are still
under the project's full control.

Cheers,
Kristian
_______________________________________________
Proj mailing list
[hidden email]
http://lists.maptools.org/mailman/listinfo/proj