missing search tab in 3.8.x statistics?

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

missing search tab in 3.8.x statistics?

Jo Cook
Hi All,

I have elasticsearch (7.6.0) and kibana (7.2.1) configured and running
in geonetwork 3.8.x. I have followed the instructions here:
https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
and here https://github.com/geonetwork/core-geonetwork/tree/master/es
and can see the content statistics dashboard correctly in geonetwork.

What I can't see is the search statistics dashboard! It's simply not
there. In kibana/elasticsearch/indexmanagement I can see that the
gn-searchlogs index has been loaded, but when I look for index
patterns there isn't one for it, and when I try creating one, kibana
doesn't recognise the gn-searchlogs name.

In geonetwork itself I have checked that the option to store search
statistics in the database is ticked, and I have run the tool to index
the catalog in a remote index and done some searches.

I feel like I'm missing something very simple, but I can't see what it is!

Any ideas would be more than welcome!

Jo

--
Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For
urgent responses at that time, please visit
support.astuntechnology.com or phone our office on 01372 744009

--
-- 
*Sign up to our mailing list
<https://astuntechnology.com/company/#email-updates> for updates on news,
products, conferences, events and training*
*
*

Astun Technology Ltd,
Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK 

t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/
twitter:@astuntech <https://twitter.com/astuntech>



iShare - enterprise
geographic intelligence platform <https://astuntechnology.com/ishare/>

GeoServer, PostGIS and QGIS training
<https://astuntechnology.com/training-courses/>
Helpdesk and customer
portal
<https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal>





Company registration no. 5410695. Registered in England and Wales.
Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
864201149.


_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: missing search tab in 3.8.x statistics?

Jose Garcia
Hi Jo

I notice that also last week, with a customer. Seem the search statistics
were removed in 3.8 as requires additional work for the Kibana panels due
to an upgrade of ElasticSearch version seem. I think should be created an
issue to handle this, as most people are not aware of this change.

I restored the search statistics locally from 3.6 version, where were
enabled, but indeed look pretty bad and useless, requiring a rework. Also
something to think about is about the feature itself. In my opinion,
processing the access logs from Apache can be much more valuable. At least
what seem stored currently in the ElasticSearch index for searches is too
low level and makes it quite hard to use get any useful statistics.

Regards,
Jose García

On Tue, Feb 25, 2020 at 7:43 PM Jo Cook <[hidden email]> wrote:

> Hi All,
>
> I have elasticsearch (7.6.0) and kibana (7.2.1) configured and running
> in geonetwork 3.8.x. I have followed the instructions here:
>
> https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
> and here https://github.com/geonetwork/core-geonetwork/tree/master/es
> and can see the content statistics dashboard correctly in geonetwork.
>
> What I can't see is the search statistics dashboard! It's simply not
> there. In kibana/elasticsearch/indexmanagement I can see that the
> gn-searchlogs index has been loaded, but when I look for index
> patterns there isn't one for it, and when I try creating one, kibana
> doesn't recognise the gn-searchlogs name.
>
> In geonetwork itself I have checked that the option to store search
> statistics in the database is ticked, and I have run the tool to index
> the catalog in a remote index and done some searches.
>
> I feel like I'm missing something very simple, but I can't see what it is!
>
> Any ideas would be more than welcome!
>
> Jo
>
> --
> Jo Cook
> t:+44 7930 524 155/twitter:@archaeogeek
> Please note that currently I do not work on Friday afternoons. For
> urgent responses at that time, please visit
> support.astuntechnology.com or phone our office on 01372 744009
>
> --
> --
> *Sign up to our mailing list
> <https://astuntechnology.com/company/#email-updates> for updates on news,
> products, conferences, events and training*
> *
> *
>
> Astun Technology Ltd,
> Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
>
> t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/>
> twitter:@astuntech <https://twitter.com/astuntech>
>
>
>
> iShare - enterprise
> geographic intelligence platform <https://astuntechnology.com/ishare/>
>
> GeoServer, PostGIS and QGIS training
> <https://astuntechnology.com/training-courses/>
> Helpdesk and customer
> portal
> <
> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
> >
>
>
>
>
>
> Company registration no. 5410695. Registered in England and Wales.
> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
> 864201149.
>
>
> _______________________________________________
> GeoNetwork-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
> GeoNetwork OpenSource is maintained at
> http://sourceforge.net/projects/geonetwork
>


--











*Vriendelijke groeten / Kind regards,Jose García
<http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
(0)318 416664 <+31318416664>Please consider the environment before printing
this email.*

_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: missing search tab in 3.8.x statistics?

Jo Cook
Hi Jose,

Ah yes that would explain it. To restore the statistics from 3.6, what is
involved? It's pretty important for me.

Thanks

Jo

On Wed, Feb 26, 2020 at 6:23 AM Jose Garcia <[hidden email]> wrote:

> Hi Jo
>
> I notice that also last week, with a customer. Seem the search statistics
> were removed in 3.8 as requires additional work for the Kibana panels due
> to an upgrade of ElasticSearch version seem. I think should be created an
> issue to handle this, as most people are not aware of this change.
>
> I restored the search statistics locally from 3.6 version, where were
> enabled, but indeed look pretty bad and useless, requiring a rework. Also
> something to think about is about the feature itself. In my opinion,
> processing the access logs from Apache can be much more valuable. At least
> what seem stored currently in the ElasticSearch index for searches is too
> low level and makes it quite hard to use get any useful statistics.
>
> Regards,
> Jose García
>
> On Tue, Feb 25, 2020 at 7:43 PM Jo Cook <[hidden email]>
> wrote:
>
>> Hi All,
>>
>> I have elasticsearch (7.6.0) and kibana (7.2.1) configured and running
>> in geonetwork 3.8.x. I have followed the instructions here:
>>
>> https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
>> and here https://github.com/geonetwork/core-geonetwork/tree/master/es
>> and can see the content statistics dashboard correctly in geonetwork.
>>
>> What I can't see is the search statistics dashboard! It's simply not
>> there. In kibana/elasticsearch/indexmanagement I can see that the
>> gn-searchlogs index has been loaded, but when I look for index
>> patterns there isn't one for it, and when I try creating one, kibana
>> doesn't recognise the gn-searchlogs name.
>>
>> In geonetwork itself I have checked that the option to store search
>> statistics in the database is ticked, and I have run the tool to index
>> the catalog in a remote index and done some searches.
>>
>> I feel like I'm missing something very simple, but I can't see what it is!
>>
>> Any ideas would be more than welcome!
>>
>> Jo
>>
>> --
>> Jo Cook
>> t:+44 7930 524 155/twitter:@archaeogeek
>> Please note that currently I do not work on Friday afternoons. For
>> urgent responses at that time, please visit
>> support.astuntechnology.com or phone our office on 01372 744009
>>
>> --
>> --
>> *Sign up to our mailing list
>> <https://astuntechnology.com/company/#email-updates> for updates on
>> news,
>> products, conferences, events and training*
>> *
>> *
>>
>> Astun Technology Ltd,
>> Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
>>
>> t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/>
>> twitter:@astuntech <https://twitter.com/astuntech>
>>
>>
>>
>> iShare - enterprise
>> geographic intelligence platform <https://astuntechnology.com/ishare/>
>>
>> GeoServer, PostGIS and QGIS training
>> <https://astuntechnology.com/training-courses/>
>> Helpdesk and customer
>> portal
>> <
>> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>> >
>>
>>
>>
>>
>>
>> Company registration no. 5410695. Registered in England and Wales.
>> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
>> 864201149.
>>
>>
>> _______________________________________________
>> GeoNetwork-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>> GeoNetwork OpenSource is maintained at
>> http://sourceforge.net/projects/geonetwork
>>
>
>
> --
>
>
>
>
>
>
>
>
>
>
>
> *Vriendelijke groeten / Kind regards,Jose García
> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
> (0)318 416664 <+31318416664>Please consider the environment before printing
> this email.*
>


--
*Jo Cook*
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent
responses at that time, please visit support.astuntechnology.com or phone
our office on 01372 744009

--
-- 
*Sign up to our mailing list
<https://astuntechnology.com/company/#email-updates> for updates on news,
products, conferences, events and training*
*
*

Astun Technology Ltd,
Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK 

t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/
twitter:@astuntech <https://twitter.com/astuntech>



iShare - enterprise
geographic intelligence platform <https://astuntechnology.com/ishare/>

GeoServer, PostGIS and QGIS training
<https://astuntechnology.com/training-courses/>
Helpdesk and customer
portal
<https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal>





Company registration no. 5410695. Registered in England and Wales.
Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
864201149.

_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: missing search tab in 3.8.x statistics?

Jose Garcia
Hi Jo

I'll check to list the steps and provide the files I used.

Regards,
Jose García

On Wed, Feb 26, 2020 at 8:31 AM Jo Cook <[hidden email]> wrote:

> Hi Jose,
>
> Ah yes that would explain it. To restore the statistics from 3.6, what is
> involved? It's pretty important for me.
>
> Thanks
>
> Jo
>
> On Wed, Feb 26, 2020 at 6:23 AM Jose Garcia <[hidden email]>
> wrote:
>
>> Hi Jo
>>
>> I notice that also last week, with a customer. Seem the search statistics
>> were removed in 3.8 as requires additional work for the Kibana panels due
>> to an upgrade of ElasticSearch version seem. I think should be created an
>> issue to handle this, as most people are not aware of this change.
>>
>> I restored the search statistics locally from 3.6 version, where were
>> enabled, but indeed look pretty bad and useless, requiring a rework. Also
>> something to think about is about the feature itself. In my opinion,
>> processing the access logs from Apache can be much more valuable. At least
>> what seem stored currently in the ElasticSearch index for searches is too
>> low level and makes it quite hard to use get any useful statistics.
>>
>> Regards,
>> Jose García
>>
>> On Tue, Feb 25, 2020 at 7:43 PM Jo Cook <[hidden email]>
>> wrote:
>>
>>> Hi All,
>>>
>>> I have elasticsearch (7.6.0) and kibana (7.2.1) configured and running
>>> in geonetwork 3.8.x. I have followed the instructions here:
>>>
>>> https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
>>> and here https://github.com/geonetwork/core-geonetwork/tree/master/es
>>> and can see the content statistics dashboard correctly in geonetwork.
>>>
>>> What I can't see is the search statistics dashboard! It's simply not
>>> there. In kibana/elasticsearch/indexmanagement I can see that the
>>> gn-searchlogs index has been loaded, but when I look for index
>>> patterns there isn't one for it, and when I try creating one, kibana
>>> doesn't recognise the gn-searchlogs name.
>>>
>>> In geonetwork itself I have checked that the option to store search
>>> statistics in the database is ticked, and I have run the tool to index
>>> the catalog in a remote index and done some searches.
>>>
>>> I feel like I'm missing something very simple, but I can't see what it
>>> is!
>>>
>>> Any ideas would be more than welcome!
>>>
>>> Jo
>>>
>>> --
>>> Jo Cook
>>> t:+44 7930 524 155/twitter:@archaeogeek
>>> Please note that currently I do not work on Friday afternoons. For
>>> urgent responses at that time, please visit
>>> support.astuntechnology.com or phone our office on 01372 744009
>>>
>>> --
>>> --
>>> *Sign up to our mailing list
>>> <https://astuntechnology.com/company/#email-updates> for updates on
>>> news,
>>> products, conferences, events and training*
>>> *
>>> *
>>>
>>> Astun Technology Ltd,
>>> Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
>>>
>>> t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/>
>>> twitter:@astuntech <https://twitter.com/astuntech>
>>>
>>>
>>>
>>> iShare - enterprise
>>> geographic intelligence platform <https://astuntechnology.com/ishare/>
>>>
>>> GeoServer, PostGIS and QGIS training
>>> <https://astuntechnology.com/training-courses/>
>>> Helpdesk and customer
>>> portal
>>> <
>>> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>>> >
>>>
>>>
>>>
>>>
>>>
>>> Company registration no. 5410695. Registered in England and Wales.
>>> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
>>> 864201149.
>>>
>>>
>>> _______________________________________________
>>> GeoNetwork-users mailing list
>>> [hidden email]
>>> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>>> GeoNetwork OpenSource is maintained at
>>> http://sourceforge.net/projects/geonetwork
>>>
>>
>>
>> --
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Vriendelijke groeten / Kind regards,Jose García
>> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
>> (0)318 416664 <+31318416664>Please consider the environment before printing
>> this email.*
>>
>
>
> --
> *Jo Cook*
> t:+44 7930 524 155/twitter:@archaeogeek
> Please note that currently I do not work on Friday afternoons. For urgent
> responses at that time, please visit support.astuntechnology.com or phone
> our office on 01372 744009
>
>
> --
> *Sign up to our mailing list
> <https://astuntechnology.com/company/#email-updates> for updates on news,
> products, conferences, events and training*
>
> Astun Technology Ltd, Epsom Square Centre, 6-7 The Derby Square, Epsom,
> Surrey, KT19 8AG, UK
> t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech
> <https://twitter.com/astuntech>
>
> iShare - enterprise geographic intelligence platform
> <https://astuntechnology.com/ishare/>
> GeoServer, PostGIS and QGIS training
> <https://astuntechnology.com/training-courses/>
> Helpdesk and customer portal
> <https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal>
>
> Company registration no. 5410695. Registered in England and Wales.
> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
> 864201149.
>


--











*Vriendelijke groeten / Kind regards,Jose García
<http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
(0)318 416664 <+31318416664>Please consider the environment before printing
this email.*

_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: missing search tab in 3.8.x statistics?

Francois Prunayre
In reply to this post by Jose Garcia
Hi Jose, in some projects it looks like sometimes webserver logs are
ingested in Elasticsearch and based on the GN API, it can be used to make
global stats about number of editing session per months, ... which is
indeed not focused only on search.
It may make sense to provide instructions on how to setup this kind of
reports.

Cheers.

Francois


Le mer. 26 févr. 2020 à 07:48, Jose Garcia <[hidden email]> a
écrit :

> Hi Jo
>
> I notice that also last week, with a customer. Seem the search statistics
> were removed in 3.8 as requires additional work for the Kibana panels due
> to an upgrade of ElasticSearch version seem. I think should be created an
> issue to handle this, as most people are not aware of this change.
>
> I restored the search statistics locally from 3.6 version, where were
> enabled, but indeed look pretty bad and useless, requiring a rework. Also
> something to think about is about the feature itself. In my opinion,
> processing the access logs from Apache can be much more valuable. At least
> what seem stored currently in the ElasticSearch index for searches is too
> low level and makes it quite hard to use get any useful statistics.
>
> Regards,
> Jose García
>
> On Tue, Feb 25, 2020 at 7:43 PM Jo Cook <[hidden email]>
> wrote:
>
> > Hi All,
> >
> > I have elasticsearch (7.6.0) and kibana (7.2.1) configured and running
> > in geonetwork 3.8.x. I have followed the instructions here:
> >
> >
> https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
> > and here https://github.com/geonetwork/core-geonetwork/tree/master/es
> > and can see the content statistics dashboard correctly in geonetwork.
> >
> > What I can't see is the search statistics dashboard! It's simply not
> > there. In kibana/elasticsearch/indexmanagement I can see that the
> > gn-searchlogs index has been loaded, but when I look for index
> > patterns there isn't one for it, and when I try creating one, kibana
> > doesn't recognise the gn-searchlogs name.
> >
> > In geonetwork itself I have checked that the option to store search
> > statistics in the database is ticked, and I have run the tool to index
> > the catalog in a remote index and done some searches.
> >
> > I feel like I'm missing something very simple, but I can't see what it
> is!
> >
> > Any ideas would be more than welcome!
> >
> > Jo
> >
> > --
> > Jo Cook
> > t:+44 7930 524 155/twitter:@archaeogeek
> > Please note that currently I do not work on Friday afternoons. For
> > urgent responses at that time, please visit
> > support.astuntechnology.com or phone our office on 01372 744009
> >
> > --
> > --
> > *Sign up to our mailing list
> > <https://astuntechnology.com/company/#email-updates> for updates on
> news,
> > products, conferences, events and training*
> > *
> > *
> >
> > Astun Technology Ltd,
> > Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
> >
> > t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/>
> > twitter:@astuntech <https://twitter.com/astuntech>
> >
> >
> >
> > iShare - enterprise
> > geographic intelligence platform <https://astuntechnology.com/ishare/>
> >
> > GeoServer, PostGIS and QGIS training
> > <https://astuntechnology.com/training-courses/>
> > Helpdesk and customer
> > portal
> > <
> >
> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
> > >
> >
> >
> >
> >
> >
> > Company registration no. 5410695. Registered in England and Wales.
> > Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
> > 864201149.
> >
> >
> > _______________________________________________
> > GeoNetwork-users mailing list
> > [hidden email]
> > https://lists.sourceforge.net/lists/listinfo/geonetwork-users
> > GeoNetwork OpenSource is maintained at
> > http://sourceforge.net/projects/geonetwork
> >
>
>
> --
>
>
>
>
>
>
>
>
>
>
>
> *Vriendelijke groeten / Kind regards,Jose García
> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
> (0)318 416664 <+31318416664>Please consider the environment before printing
> this email.*
>
> _______________________________________________
> GeoNetwork-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
> GeoNetwork OpenSource is maintained at
> http://sourceforge.net/projects/geonetwork
>

_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: missing search tab in 3.8.x statistics?

Jo Cook
Hi Francois/Jose,

For now I'd really like to get the original search statistics back
rather than ingesting the server log files, even if they don't look
that good in the dashboard. Can I follow the instructions for 3.6
(https://github.com/geonetwork/core-geonetwork/tree/3.6.x/es/es-dashboards)
to load the dashboard-mapping in or is there something else I need to
do?

Thanks

Jo

On Wed, Feb 26, 2020 at 8:38 AM Francois Prunayre <[hidden email]> wrote:

>
> Hi Jose, in some projects it looks like sometimes webserver logs are ingested in Elasticsearch and based on the GN API, it can be used to make global stats about number of editing session per months, ... which is indeed not focused only on search.
> It may make sense to provide instructions on how to setup this kind of reports.
>
> Cheers.
>
> Francois
>
>
> Le mer. 26 févr. 2020 à 07:48, Jose Garcia <[hidden email]> a écrit :
>>
>> Hi Jo
>>
>> I notice that also last week, with a customer. Seem the search statistics
>> were removed in 3.8 as requires additional work for the Kibana panels due
>> to an upgrade of ElasticSearch version seem. I think should be created an
>> issue to handle this, as most people are not aware of this change.
>>
>> I restored the search statistics locally from 3.6 version, where were
>> enabled, but indeed look pretty bad and useless, requiring a rework. Also
>> something to think about is about the feature itself. In my opinion,
>> processing the access logs from Apache can be much more valuable. At least
>> what seem stored currently in the ElasticSearch index for searches is too
>> low level and makes it quite hard to use get any useful statistics.
>>
>> Regards,
>> Jose García
>>
>> On Tue, Feb 25, 2020 at 7:43 PM Jo Cook <[hidden email]> wrote:
>>
>> > Hi All,
>> >
>> > I have elasticsearch (7.6.0) and kibana (7.2.1) configured and running
>> > in geonetwork 3.8.x. I have followed the instructions here:
>> >
>> > https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
>> > and here https://github.com/geonetwork/core-geonetwork/tree/master/es
>> > and can see the content statistics dashboard correctly in geonetwork.
>> >
>> > What I can't see is the search statistics dashboard! It's simply not
>> > there. In kibana/elasticsearch/indexmanagement I can see that the
>> > gn-searchlogs index has been loaded, but when I look for index
>> > patterns there isn't one for it, and when I try creating one, kibana
>> > doesn't recognise the gn-searchlogs name.
>> >
>> > In geonetwork itself I have checked that the option to store search
>> > statistics in the database is ticked, and I have run the tool to index
>> > the catalog in a remote index and done some searches.
>> >
>> > I feel like I'm missing something very simple, but I can't see what it is!
>> >
>> > Any ideas would be more than welcome!
>> >
>> > Jo
>> >
>> > --
>> > Jo Cook
>> > t:+44 7930 524 155/twitter:@archaeogeek
>> > Please note that currently I do not work on Friday afternoons. For
>> > urgent responses at that time, please visit
>> > support.astuntechnology.com or phone our office on 01372 744009
>> >
>> > --
>> > --
>> > *Sign up to our mailing list
>> > <https://astuntechnology.com/company/#email-updates> for updates on news,
>> > products, conferences, events and training*
>> > *
>> > *
>> >
>> > Astun Technology Ltd,
>> > Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
>> >
>> > t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/>
>> > twitter:@astuntech <https://twitter.com/astuntech>
>> >
>> >
>> >
>> > iShare - enterprise
>> > geographic intelligence platform <https://astuntechnology.com/ishare/>
>> >
>> > GeoServer, PostGIS and QGIS training
>> > <https://astuntechnology.com/training-courses/>
>> > Helpdesk and customer
>> > portal
>> > <
>> > https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>> > >
>> >
>> >
>> >
>> >
>> >
>> > Company registration no. 5410695. Registered in England and Wales.
>> > Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
>> > 864201149.
>> >
>> >
>> > _______________________________________________
>> > GeoNetwork-users mailing list
>> > [hidden email]
>> > https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>> > GeoNetwork OpenSource is maintained at
>> > http://sourceforge.net/projects/geonetwork
>> >
>>
>>
>> --
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Vriendelijke groeten / Kind regards,Jose García
>> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
>> (0)318 416664 <+31318416664>Please consider the environment before printing
>> this email.*
>>
>> _______________________________________________
>> GeoNetwork-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>> GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork



--
Jo Cook
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For
urgent responses at that time, please visit
support.astuntechnology.com or phone our office on 01372 744009

--
-- 
*Sign up to our mailing list
<https://astuntechnology.com/company/#email-updates> for updates on news,
products, conferences, events and training*
*
*

Astun Technology Ltd,
Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK 

t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/
twitter:@astuntech <https://twitter.com/astuntech>



iShare - enterprise
geographic intelligence platform <https://astuntechnology.com/ishare/>

GeoServer, PostGIS and QGIS training
<https://astuntechnology.com/training-courses/>
Helpdesk and customer
portal
<https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal>





Company registration no. 5410695. Registered in England and Wales.
Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
864201149.


_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: missing search tab in 3.8.x statistics?

Jose Garcia
Hi Jo

Not sure if importing the dashboards from 3.6 will break anything in 3.8,
but you can give a try instead of step 3 below (steps 1 and 2 are
required). Apologies as indicated to provide the files I created, but I
lost them accidentally, adding here the steps I did to enable the
statistics and add the Kibana dashoards:

1) Uncomment the following code (also previous lines if require the
metadata validation statistics)

https://github.com/geonetwork/core-geonetwork/blob/cd47db86fb04a7c88d252b50c792a5f60ca5160d/web-ui/src/main/resources/catalog/js/admin/DashboardController.js#L87-L93

2) I loaded the 3.10 Kibana dashboards as described in
https://github.com/geonetwork/core-geonetwork/tree/3.10.x/es/es-dashboards

Visit Kibana in a browser using one of the above links and go to 'Saved
Objects'. Import export.json from
https://github.com/geonetwork/core-geonetwork/blob/master/es/es-dashboards/data/export.json


3) Start up GeoNetwork and access the search statistics, all panels show
with an error related to the dashboard identifier. I checked the
identifiers failing in
https://github.com/geonetwork/core-geonetwork/blob/3.6.x/es/es-dashboards/data/index-dashboards.json
and
created a similar file as the export.json from previous step and loaded as
described there.

But as afaik 3.6 only had search statistics, no content statistics probably
is fine just to load the full file, please give it a try.

Regards,
Jose García

On Fri, Feb 28, 2020 at 10:18 AM Jo Cook <[hidden email]> wrote:

> Hi Francois/Jose,
>
> For now I'd really like to get the original search statistics back
> rather than ingesting the server log files, even if they don't look
> that good in the dashboard. Can I follow the instructions for 3.6
> (https://github.com/geonetwork/core-geonetwork/tree/3.6.x/es/es-dashboards
> )
> to load the dashboard-mapping in or is there something else I need to
> do?
>
> Thanks
>
> Jo
>
> On Wed, Feb 26, 2020 at 8:38 AM Francois Prunayre <[hidden email]>
> wrote:
> >
> > Hi Jose, in some projects it looks like sometimes webserver logs are
> ingested in Elasticsearch and based on the GN API, it can be used to make
> global stats about number of editing session per months, ... which is
> indeed not focused only on search.
> > It may make sense to provide instructions on how to setup this kind of
> reports.
> >
> > Cheers.
> >
> > Francois
> >
> >
> > Le mer. 26 févr. 2020 à 07:48, Jose Garcia <[hidden email]> a
> écrit :
> >>
> >> Hi Jo
> >>
> >> I notice that also last week, with a customer. Seem the search
> statistics
> >> were removed in 3.8 as requires additional work for the Kibana panels
> due
> >> to an upgrade of ElasticSearch version seem. I think should be created
> an
> >> issue to handle this, as most people are not aware of this change.
> >>
> >> I restored the search statistics locally from 3.6 version, where were
> >> enabled, but indeed look pretty bad and useless, requiring a rework.
> Also
> >> something to think about is about the feature itself. In my opinion,
> >> processing the access logs from Apache can be much more valuable. At
> least
> >> what seem stored currently in the ElasticSearch index for searches is
> too
> >> low level and makes it quite hard to use get any useful statistics.
> >>
> >> Regards,
> >> Jose García
> >>
> >> On Tue, Feb 25, 2020 at 7:43 PM Jo Cook <[hidden email]>
> wrote:
> >>
> >> > Hi All,
> >> >
> >> > I have elasticsearch (7.6.0) and kibana (7.2.1) configured and running
> >> > in geonetwork 3.8.x. I have followed the instructions here:
> >> >
> >> >
> https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
> >> > and here https://github.com/geonetwork/core-geonetwork/tree/master/es
> >> > and can see the content statistics dashboard correctly in geonetwork.
> >> >
> >> > What I can't see is the search statistics dashboard! It's simply not
> >> > there. In kibana/elasticsearch/indexmanagement I can see that the
> >> > gn-searchlogs index has been loaded, but when I look for index
> >> > patterns there isn't one for it, and when I try creating one, kibana
> >> > doesn't recognise the gn-searchlogs name.
> >> >
> >> > In geonetwork itself I have checked that the option to store search
> >> > statistics in the database is ticked, and I have run the tool to index
> >> > the catalog in a remote index and done some searches.
> >> >
> >> > I feel like I'm missing something very simple, but I can't see what
> it is!
> >> >
> >> > Any ideas would be more than welcome!
> >> >
> >> > Jo
> >> >
> >> > --
> >> > Jo Cook
> >> > t:+44 7930 524 155/twitter:@archaeogeek
> >> > Please note that currently I do not work on Friday afternoons. For
> >> > urgent responses at that time, please visit
> >> > support.astuntechnology.com or phone our office on 01372 744009
> >> >
> >> > --
> >> > --
> >> > *Sign up to our mailing list
> >> > <https://astuntechnology.com/company/#email-updates> for updates on
> news,
> >> > products, conferences, events and training*
> >> > *
> >> > *
> >> >
> >> > Astun Technology Ltd,
> >> > Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
> >> >
> >> > t:+44 1372 744 009 w: astuntechnology.com <
> http://astuntechnology.com/>
> >> > twitter:@astuntech <https://twitter.com/astuntech>
> >> >
> >> >
> >> >
> >> > iShare - enterprise
> >> > geographic intelligence platform <https://astuntechnology.com/ishare/
> >
> >> >
> >> > GeoServer, PostGIS and QGIS training
> >> > <https://astuntechnology.com/training-courses/>
> >> > Helpdesk and customer
> >> > portal
> >> > <
> >> >
> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
> >> > >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > Company registration no. 5410695. Registered in England and Wales.
> >> > Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT
> no.
> >> > 864201149.
> >> >
> >> >
> >> > _______________________________________________
> >> > GeoNetwork-users mailing list
> >> > [hidden email]
> >> > https://lists.sourceforge.net/lists/listinfo/geonetwork-users
> >> > GeoNetwork OpenSource is maintained at
> >> > http://sourceforge.net/projects/geonetwork
> >> >
> >>
> >>
> >> --
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> *Vriendelijke groeten / Kind regards,Jose García
> >> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT:
> +31
> >> (0)318 416664 <+31318416664>Please consider the environment before
> printing
> >> this email.*
> >>
> >> _______________________________________________
> >> GeoNetwork-users mailing list
> >> [hidden email]
> >> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
> >> GeoNetwork OpenSource is maintained at
> http://sourceforge.net/projects/geonetwork
>
>
>
> --
> Jo Cook
> t:+44 7930 524 155/twitter:@archaeogeek
> Please note that currently I do not work on Friday afternoons. For
> urgent responses at that time, please visit
> support.astuntechnology.com or phone our office on 01372 744009
>
> --
> --
> *Sign up to our mailing list
> <https://astuntechnology.com/company/#email-updates> for updates on news,
> products, conferences, events and training*
> *
> *
>
> Astun Technology Ltd,
> Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
>
> t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/>
> twitter:@astuntech <https://twitter.com/astuntech>
>
>
>
> iShare - enterprise
> geographic intelligence platform <https://astuntechnology.com/ishare/>
>
> GeoServer, PostGIS and QGIS training
> <https://astuntechnology.com/training-courses/>
> Helpdesk and customer
> portal
> <
> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
> >
>
>
>
>
>
> Company registration no. 5410695. Registered in England and Wales.
> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
> 864201149.
>


--











*Vriendelijke groeten / Kind regards,Jose García
<http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
(0)318 416664 <+31318416664>Please consider the environment before printing
this email.*

_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: missing search tab in 3.8.x statistics?

Jo Cook
Hi Jose and Francois,

I have managed to get this working, but in a slightly messy way- I
downloaded the saved objects from a working install (GeoNetwork 3.4), and
then uploaded that export.json into my new install (GeoNetwork 3.8). That
failed part of the way through due to the missing index pattern for
gn-searchlogs, but I could create that, and then when I re-ran the import
it worked. Not ideal, but I can use this approach moving forward.

If I can replicate this more cleanly, should I save export.json and commit
it somewhere?

Thanks

Jo

On Fri, Feb 28, 2020 at 9:35 AM Jose Garcia <[hidden email]> wrote:

> Hi Jo
>
> Not sure if importing the dashboards from 3.6 will break anything in 3.8,
> but you can give a try instead of step 3 below (steps 1 and 2 are
> required). Apologies as indicated to provide the files I created, but I
> lost them accidentally, adding here the steps I did to enable the
> statistics and add the Kibana dashoards:
>
> 1) Uncomment the following code (also previous lines if require the
> metadata validation statistics)
>
>
> https://github.com/geonetwork/core-geonetwork/blob/cd47db86fb04a7c88d252b50c792a5f60ca5160d/web-ui/src/main/resources/catalog/js/admin/DashboardController.js#L87-L93
>
> 2) I loaded the 3.10 Kibana dashboards as described in
> https://github.com/geonetwork/core-geonetwork/tree/3.10.x/es/es-dashboards
>
> Visit Kibana in a browser using one of the above links and go to 'Saved
> Objects'. Import export.json from
> https://github.com/geonetwork/core-geonetwork/blob/master/es/es-dashboards/data/export.json
>
>
> 3) Start up GeoNetwork and access the search statistics, all panels show
> with an error related to the dashboard identifier. I checked the
> identifiers failing in
> https://github.com/geonetwork/core-geonetwork/blob/3.6.x/es/es-dashboards/data/index-dashboards.json and
> created a similar file as the export.json from previous step and loaded as
> described there.
>
> But as afaik 3.6 only had search statistics, no content statistics
> probably is fine just to load the full file, please give it a try.
>
> Regards,
> Jose García
>
> On Fri, Feb 28, 2020 at 10:18 AM Jo Cook <[hidden email]>
> wrote:
>
>> Hi Francois/Jose,
>>
>> For now I'd really like to get the original search statistics back
>> rather than ingesting the server log files, even if they don't look
>> that good in the dashboard. Can I follow the instructions for 3.6
>> (
>> https://github.com/geonetwork/core-geonetwork/tree/3.6.x/es/es-dashboards
>> )
>> to load the dashboard-mapping in or is there something else I need to
>> do?
>>
>> Thanks
>>
>> Jo
>>
>> On Wed, Feb 26, 2020 at 8:38 AM Francois Prunayre <[hidden email]>
>> wrote:
>> >
>> > Hi Jose, in some projects it looks like sometimes webserver logs are
>> ingested in Elasticsearch and based on the GN API, it can be used to make
>> global stats about number of editing session per months, ... which is
>> indeed not focused only on search.
>> > It may make sense to provide instructions on how to setup this kind of
>> reports.
>> >
>> > Cheers.
>> >
>> > Francois
>> >
>> >
>> > Le mer. 26 févr. 2020 à 07:48, Jose Garcia <[hidden email]> a
>> écrit :
>> >>
>> >> Hi Jo
>> >>
>> >> I notice that also last week, with a customer. Seem the search
>> statistics
>> >> were removed in 3.8 as requires additional work for the Kibana panels
>> due
>> >> to an upgrade of ElasticSearch version seem. I think should be created
>> an
>> >> issue to handle this, as most people are not aware of this change.
>> >>
>> >> I restored the search statistics locally from 3.6 version, where were
>> >> enabled, but indeed look pretty bad and useless, requiring a rework.
>> Also
>> >> something to think about is about the feature itself. In my opinion,
>> >> processing the access logs from Apache can be much more valuable. At
>> least
>> >> what seem stored currently in the ElasticSearch index for searches is
>> too
>> >> low level and makes it quite hard to use get any useful statistics.
>> >>
>> >> Regards,
>> >> Jose García
>> >>
>> >> On Tue, Feb 25, 2020 at 7:43 PM Jo Cook <[hidden email]>
>> wrote:
>> >>
>> >> > Hi All,
>> >> >
>> >> > I have elasticsearch (7.6.0) and kibana (7.2.1) configured and
>> running
>> >> > in geonetwork 3.8.x. I have followed the instructions here:
>> >> >
>> >> >
>> https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
>> >> > and here
>> https://github.com/geonetwork/core-geonetwork/tree/master/es
>> >> > and can see the content statistics dashboard correctly in geonetwork.
>> >> >
>> >> > What I can't see is the search statistics dashboard! It's simply not
>> >> > there. In kibana/elasticsearch/indexmanagement I can see that the
>> >> > gn-searchlogs index has been loaded, but when I look for index
>> >> > patterns there isn't one for it, and when I try creating one, kibana
>> >> > doesn't recognise the gn-searchlogs name.
>> >> >
>> >> > In geonetwork itself I have checked that the option to store search
>> >> > statistics in the database is ticked, and I have run the tool to
>> index
>> >> > the catalog in a remote index and done some searches.
>> >> >
>> >> > I feel like I'm missing something very simple, but I can't see what
>> it is!
>> >> >
>> >> > Any ideas would be more than welcome!
>> >> >
>> >> > Jo
>> >> >
>> >> > --
>> >> > Jo Cook
>> >> > t:+44 7930 524 155/twitter:@archaeogeek
>> >> > Please note that currently I do not work on Friday afternoons. For
>> >> > urgent responses at that time, please visit
>> >> > support.astuntechnology.com or phone our office on 01372 744009
>> >> >
>> >> > --
>> >> > --
>> >> > *Sign up to our mailing list
>> >> > <https://astuntechnology.com/company/#email-updates> for updates on
>> news,
>> >> > products, conferences, events and training*
>> >> > *
>> >> > *
>> >> >
>> >> > Astun Technology Ltd,
>> >> > Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG,
>> UK
>> >> >
>> >> > t:+44 1372 744 009 w: astuntechnology.com <
>> http://astuntechnology.com/>
>> >> > twitter:@astuntech <https://twitter.com/astuntech>
>> >> >
>> >> >
>> >> >
>> >> > iShare - enterprise
>> >> > geographic intelligence platform <
>> https://astuntechnology.com/ishare/>
>> >> >
>> >> > GeoServer, PostGIS and QGIS training
>> >> > <https://astuntechnology.com/training-courses/>
>> >> > Helpdesk and customer
>> >> > portal
>> >> > <
>> >> >
>> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>> >> > >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > Company registration no. 5410695. Registered in England and Wales.
>> >> > Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT
>> no.
>> >> > 864201149.
>> >> >
>> >> >
>> >> > _______________________________________________
>> >> > GeoNetwork-users mailing list
>> >> > [hidden email]
>> >> > https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>> >> > GeoNetwork OpenSource is maintained at
>> >> > http://sourceforge.net/projects/geonetwork
>> >> >
>> >>
>> >>
>> >> --
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> *Vriendelijke groeten / Kind regards,Jose García
>> >> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe
>> NetherlandsT: +31
>> >> (0)318 416664 <+31318416664>Please consider the environment before
>> printing
>> >> this email.*
>> >>
>> >> _______________________________________________
>> >> GeoNetwork-users mailing list
>> >> [hidden email]
>> >> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>> >> GeoNetwork OpenSource is maintained at
>> http://sourceforge.net/projects/geonetwork
>>
>>
>>
>> --
>> Jo Cook
>> t:+44 7930 524 155/twitter:@archaeogeek
>> Please note that currently I do not work on Friday afternoons. For
>> urgent responses at that time, please visit
>> support.astuntechnology.com or phone our office on 01372 744009
>>
>> --
>> --
>> *Sign up to our mailing list
>> <https://astuntechnology.com/company/#email-updates> for updates on
>> news,
>> products, conferences, events and training*
>> *
>> *
>>
>> Astun Technology Ltd,
>> Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
>>
>> t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/>
>> twitter:@astuntech <https://twitter.com/astuntech>
>>
>>
>>
>> iShare - enterprise
>> geographic intelligence platform <https://astuntechnology.com/ishare/>
>>
>> GeoServer, PostGIS and QGIS training
>> <https://astuntechnology.com/training-courses/>
>> Helpdesk and customer
>> portal
>> <
>> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>> >
>>
>>
>>
>>
>>
>> Company registration no. 5410695. Registered in England and Wales.
>> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
>> 864201149.
>>
>
>
> --
>
>
>
>
>
>
>
>
>
>
>
> *Vriendelijke groeten / Kind regards,Jose García
> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
> (0)318 416664 <+31318416664>Please consider the environment before printing
> this email.*
>


--
*Jo Cook*
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent
responses at that time, please visit support.astuntechnology.com or phone
our office on 01372 744009

--
-- 
*Sign up to our mailing list
<https://astuntechnology.com/company/#email-updates> for updates on news,
products, conferences, events and training*
*
*

Astun Technology Ltd,
Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK 

t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/
twitter:@astuntech <https://twitter.com/astuntech>



iShare - enterprise
geographic intelligence platform <https://astuntechnology.com/ishare/>

GeoServer, PostGIS and QGIS training
<https://astuntechnology.com/training-courses/>
Helpdesk and customer
portal
<https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal>





Company registration no. 5410695. Registered in England and Wales.
Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
864201149.

_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: missing search tab in 3.8.x statistics?

Jose Garcia
Hi Jo

That's great, thank you. If you can do a pull request and set it to draft
if requires some additional work. I think that would be the best option as
other developers can try it and provide feedback or improvements if
required.

Regards,
Jose García

On Tue, Mar 17, 2020 at 5:38 PM Jo Cook <[hidden email]> wrote:

> Hi Jose and Francois,
>
> I have managed to get this working, but in a slightly messy way- I
> downloaded the saved objects from a working install (GeoNetwork 3.4), and
> then uploaded that export.json into my new install (GeoNetwork 3.8). That
> failed part of the way through due to the missing index pattern for
> gn-searchlogs, but I could create that, and then when I re-ran the import
> it worked. Not ideal, but I can use this approach moving forward.
>
> If I can replicate this more cleanly, should I save export.json and commit
> it somewhere?
>
> Thanks
>
> Jo
>
> On Fri, Feb 28, 2020 at 9:35 AM Jose Garcia <[hidden email]>
> wrote:
>
>> Hi Jo
>>
>> Not sure if importing the dashboards from 3.6 will break anything in 3.8,
>> but you can give a try instead of step 3 below (steps 1 and 2 are
>> required). Apologies as indicated to provide the files I created, but I
>> lost them accidentally, adding here the steps I did to enable the
>> statistics and add the Kibana dashoards:
>>
>> 1) Uncomment the following code (also previous lines if require the
>> metadata validation statistics)
>>
>>
>> https://github.com/geonetwork/core-geonetwork/blob/cd47db86fb04a7c88d252b50c792a5f60ca5160d/web-ui/src/main/resources/catalog/js/admin/DashboardController.js#L87-L93
>>
>> 2) I loaded the 3.10 Kibana dashboards as described in
>> https://github.com/geonetwork/core-geonetwork/tree/3.10.x/es/es-dashboards
>>
>> Visit Kibana in a browser using one of the above links and go to 'Saved
>> Objects'. Import export.json from
>> https://github.com/geonetwork/core-geonetwork/blob/master/es/es-dashboards/data/export.json
>>
>>
>> 3) Start up GeoNetwork and access the search statistics, all panels show
>> with an error related to the dashboard identifier. I checked the
>> identifiers failing in
>> https://github.com/geonetwork/core-geonetwork/blob/3.6.x/es/es-dashboards/data/index-dashboards.json and
>> created a similar file as the export.json from previous step and loaded as
>> described there.
>>
>> But as afaik 3.6 only had search statistics, no content statistics
>> probably is fine just to load the full file, please give it a try.
>>
>> Regards,
>> Jose García
>>
>> On Fri, Feb 28, 2020 at 10:18 AM Jo Cook <[hidden email]>
>> wrote:
>>
>>> Hi Francois/Jose,
>>>
>>> For now I'd really like to get the original search statistics back
>>> rather than ingesting the server log files, even if they don't look
>>> that good in the dashboard. Can I follow the instructions for 3.6
>>> (
>>> https://github.com/geonetwork/core-geonetwork/tree/3.6.x/es/es-dashboards
>>> )
>>> to load the dashboard-mapping in or is there something else I need to
>>> do?
>>>
>>> Thanks
>>>
>>> Jo
>>>
>>> On Wed, Feb 26, 2020 at 8:38 AM Francois Prunayre <[hidden email]>
>>> wrote:
>>> >
>>> > Hi Jose, in some projects it looks like sometimes webserver logs are
>>> ingested in Elasticsearch and based on the GN API, it can be used to make
>>> global stats about number of editing session per months, ... which is
>>> indeed not focused only on search.
>>> > It may make sense to provide instructions on how to setup this kind of
>>> reports.
>>> >
>>> > Cheers.
>>> >
>>> > Francois
>>> >
>>> >
>>> > Le mer. 26 févr. 2020 à 07:48, Jose Garcia <[hidden email]> a
>>> écrit :
>>> >>
>>> >> Hi Jo
>>> >>
>>> >> I notice that also last week, with a customer. Seem the search
>>> statistics
>>> >> were removed in 3.8 as requires additional work for the Kibana panels
>>> due
>>> >> to an upgrade of ElasticSearch version seem. I think should be
>>> created an
>>> >> issue to handle this, as most people are not aware of this change.
>>> >>
>>> >> I restored the search statistics locally from 3.6 version, where were
>>> >> enabled, but indeed look pretty bad and useless, requiring a rework.
>>> Also
>>> >> something to think about is about the feature itself. In my opinion,
>>> >> processing the access logs from Apache can be much more valuable. At
>>> least
>>> >> what seem stored currently in the ElasticSearch index for searches is
>>> too
>>> >> low level and makes it quite hard to use get any useful statistics.
>>> >>
>>> >> Regards,
>>> >> Jose García
>>> >>
>>> >> On Tue, Feb 25, 2020 at 7:43 PM Jo Cook <[hidden email]>
>>> wrote:
>>> >>
>>> >> > Hi All,
>>> >> >
>>> >> > I have elasticsearch (7.6.0) and kibana (7.2.1) configured and
>>> running
>>> >> > in geonetwork 3.8.x. I have followed the instructions here:
>>> >> >
>>> >> >
>>> https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
>>> >> > and here
>>> https://github.com/geonetwork/core-geonetwork/tree/master/es
>>> >> > and can see the content statistics dashboard correctly in
>>> geonetwork.
>>> >> >
>>> >> > What I can't see is the search statistics dashboard! It's simply not
>>> >> > there. In kibana/elasticsearch/indexmanagement I can see that the
>>> >> > gn-searchlogs index has been loaded, but when I look for index
>>> >> > patterns there isn't one for it, and when I try creating one, kibana
>>> >> > doesn't recognise the gn-searchlogs name.
>>> >> >
>>> >> > In geonetwork itself I have checked that the option to store search
>>> >> > statistics in the database is ticked, and I have run the tool to
>>> index
>>> >> > the catalog in a remote index and done some searches.
>>> >> >
>>> >> > I feel like I'm missing something very simple, but I can't see what
>>> it is!
>>> >> >
>>> >> > Any ideas would be more than welcome!
>>> >> >
>>> >> > Jo
>>> >> >
>>> >> > --
>>> >> > Jo Cook
>>> >> > t:+44 7930 524 155/twitter:@archaeogeek
>>> >> > Please note that currently I do not work on Friday afternoons. For
>>> >> > urgent responses at that time, please visit
>>> >> > support.astuntechnology.com or phone our office on 01372 744009
>>> >> >
>>> >> > --
>>> >> > --
>>> >> > *Sign up to our mailing list
>>> >> > <https://astuntechnology.com/company/#email-updates> for updates
>>> on news,
>>> >> > products, conferences, events and training*
>>> >> > *
>>> >> > *
>>> >> >
>>> >> > Astun Technology Ltd,
>>> >> > Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG,
>>> UK
>>> >> >
>>> >> > t:+44 1372 744 009 w: astuntechnology.com <
>>> http://astuntechnology.com/>
>>> >> > twitter:@astuntech <https://twitter.com/astuntech>
>>> >> >
>>> >> >
>>> >> >
>>> >> > iShare - enterprise
>>> >> > geographic intelligence platform <
>>> https://astuntechnology.com/ishare/>
>>> >> >
>>> >> > GeoServer, PostGIS and QGIS training
>>> >> > <https://astuntechnology.com/training-courses/>
>>> >> > Helpdesk and customer
>>> >> > portal
>>> >> > <
>>> >> >
>>> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>>> >> > >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > Company registration no. 5410695. Registered in England and Wales.
>>> >> > Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN
>>> VAT no.
>>> >> > 864201149.
>>> >> >
>>> >> >
>>> >> > _______________________________________________
>>> >> > GeoNetwork-users mailing list
>>> >> > [hidden email]
>>> >> > https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>>> >> > GeoNetwork OpenSource is maintained at
>>> >> > http://sourceforge.net/projects/geonetwork
>>> >> >
>>> >>
>>> >>
>>> >> --
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> *Vriendelijke groeten / Kind regards,Jose García
>>> >> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe
>>> NetherlandsT: +31
>>> >> (0)318 416664 <+31318416664>Please consider the environment before
>>> printing
>>> >> this email.*
>>> >>
>>> >> _______________________________________________
>>> >> GeoNetwork-users mailing list
>>> >> [hidden email]
>>> >> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>>> >> GeoNetwork OpenSource is maintained at
>>> http://sourceforge.net/projects/geonetwork
>>>
>>>
>>>
>>> --
>>> Jo Cook
>>> t:+44 7930 524 155/twitter:@archaeogeek
>>> Please note that currently I do not work on Friday afternoons. For
>>> urgent responses at that time, please visit
>>> support.astuntechnology.com or phone our office on 01372 744009
>>>
>>> --
>>> --
>>> *Sign up to our mailing list
>>> <https://astuntechnology.com/company/#email-updates> for updates on
>>> news,
>>> products, conferences, events and training*
>>> *
>>> *
>>>
>>> Astun Technology Ltd,
>>> Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
>>>
>>> t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/>
>>> twitter:@astuntech <https://twitter.com/astuntech>
>>>
>>>
>>>
>>> iShare - enterprise
>>> geographic intelligence platform <https://astuntechnology.com/ishare/>
>>>
>>> GeoServer, PostGIS and QGIS training
>>> <https://astuntechnology.com/training-courses/>
>>> Helpdesk and customer
>>> portal
>>> <
>>> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>>> >
>>>
>>>
>>>
>>>
>>>
>>> Company registration no. 5410695. Registered in England and Wales.
>>> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
>>> 864201149.
>>>
>>
>>
>> --
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Vriendelijke groeten / Kind regards,Jose García
>> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
>> (0)318 416664 <+31318416664>Please consider the environment before printing
>> this email.*
>>
>
>
> --
> *Jo Cook*
> t:+44 7930 524 155/twitter:@archaeogeek
> Please note that currently I do not work on Friday afternoons. For urgent
> responses at that time, please visit support.astuntechnology.com or phone
> our office on 01372 744009
>
>
> --
> *Sign up to our mailing list
> <https://astuntechnology.com/company/#email-updates> for updates on news,
> products, conferences, events and training*
>
> Astun Technology Ltd, Epsom Square Centre, 6-7 The Derby Square, Epsom,
> Surrey, KT19 8AG, UK
> t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech
> <https://twitter.com/astuntech>
>
> iShare - enterprise geographic intelligence platform
> <https://astuntechnology.com/ishare/>
> GeoServer, PostGIS and QGIS training
> <https://astuntechnology.com/training-courses/>
> Helpdesk and customer portal
> <https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal>
>
> Company registration no. 5410695. Registered in England and Wales.
> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
> 864201149.
>


--











*Vriendelijke groeten / Kind regards,Jose García
<http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
(0)318 416664 <+31318416664>Please consider the environment before printing
this email.*

_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: missing search tab in 3.8.x statistics?

Jo Cook
Hi Jose,

I think I have something working- but I'm not sure which branch to submit
it as a PR against- would it be 3.8.x or Master?

Thanks

Jo

On Wed, Mar 18, 2020 at 8:26 AM Jose Garcia <[hidden email]> wrote:

> Hi Jo
>
> That's great, thank you. If you can do a pull request and set it to draft
> if requires some additional work. I think that would be the best option as
> other developers can try it and provide feedback or improvements if
> required.
>
> Regards,
> Jose García
>
> On Tue, Mar 17, 2020 at 5:38 PM Jo Cook <[hidden email]>
> wrote:
>
>> Hi Jose and Francois,
>>
>> I have managed to get this working, but in a slightly messy way- I
>> downloaded the saved objects from a working install (GeoNetwork 3.4), and
>> then uploaded that export.json into my new install (GeoNetwork 3.8). That
>> failed part of the way through due to the missing index pattern for
>> gn-searchlogs, but I could create that, and then when I re-ran the import
>> it worked. Not ideal, but I can use this approach moving forward.
>>
>> If I can replicate this more cleanly, should I save export.json and
>> commit it somewhere?
>>
>> Thanks
>>
>> Jo
>>
>> On Fri, Feb 28, 2020 at 9:35 AM Jose Garcia <[hidden email]>
>> wrote:
>>
>>> Hi Jo
>>>
>>> Not sure if importing the dashboards from 3.6 will break anything in
>>> 3.8, but you can give a try instead of step 3 below (steps 1 and 2 are
>>> required). Apologies as indicated to provide the files I created, but I
>>> lost them accidentally, adding here the steps I did to enable the
>>> statistics and add the Kibana dashoards:
>>>
>>> 1) Uncomment the following code (also previous lines if require the
>>> metadata validation statistics)
>>>
>>>
>>> https://github.com/geonetwork/core-geonetwork/blob/cd47db86fb04a7c88d252b50c792a5f60ca5160d/web-ui/src/main/resources/catalog/js/admin/DashboardController.js#L87-L93
>>>
>>> 2) I loaded the 3.10 Kibana dashboards as described in
>>> https://github.com/geonetwork/core-geonetwork/tree/3.10.x/es/es-dashboards
>>>
>>> Visit Kibana in a browser using one of the above links and go to 'Saved
>>> Objects'. Import export.json from
>>> https://github.com/geonetwork/core-geonetwork/blob/master/es/es-dashboards/data/export.json
>>>
>>>
>>> 3) Start up GeoNetwork and access the search statistics, all panels show
>>> with an error related to the dashboard identifier. I checked the
>>> identifiers failing in
>>> https://github.com/geonetwork/core-geonetwork/blob/3.6.x/es/es-dashboards/data/index-dashboards.json and
>>> created a similar file as the export.json from previous step and loaded as
>>> described there.
>>>
>>> But as afaik 3.6 only had search statistics, no content statistics
>>> probably is fine just to load the full file, please give it a try.
>>>
>>> Regards,
>>> Jose García
>>>
>>> On Fri, Feb 28, 2020 at 10:18 AM Jo Cook <[hidden email]>
>>> wrote:
>>>
>>>> Hi Francois/Jose,
>>>>
>>>> For now I'd really like to get the original search statistics back
>>>> rather than ingesting the server log files, even if they don't look
>>>> that good in the dashboard. Can I follow the instructions for 3.6
>>>> (
>>>> https://github.com/geonetwork/core-geonetwork/tree/3.6.x/es/es-dashboards
>>>> )
>>>> to load the dashboard-mapping in or is there something else I need to
>>>> do?
>>>>
>>>> Thanks
>>>>
>>>> Jo
>>>>
>>>> On Wed, Feb 26, 2020 at 8:38 AM Francois Prunayre <
>>>> [hidden email]> wrote:
>>>> >
>>>> > Hi Jose, in some projects it looks like sometimes webserver logs are
>>>> ingested in Elasticsearch and based on the GN API, it can be used to make
>>>> global stats about number of editing session per months, ... which is
>>>> indeed not focused only on search.
>>>> > It may make sense to provide instructions on how to setup this kind
>>>> of reports.
>>>> >
>>>> > Cheers.
>>>> >
>>>> > Francois
>>>> >
>>>> >
>>>> > Le mer. 26 févr. 2020 à 07:48, Jose Garcia <[hidden email]>
>>>> a écrit :
>>>> >>
>>>> >> Hi Jo
>>>> >>
>>>> >> I notice that also last week, with a customer. Seem the search
>>>> statistics
>>>> >> were removed in 3.8 as requires additional work for the Kibana
>>>> panels due
>>>> >> to an upgrade of ElasticSearch version seem. I think should be
>>>> created an
>>>> >> issue to handle this, as most people are not aware of this change.
>>>> >>
>>>> >> I restored the search statistics locally from 3.6 version, where were
>>>> >> enabled, but indeed look pretty bad and useless, requiring a rework.
>>>> Also
>>>> >> something to think about is about the feature itself. In my opinion,
>>>> >> processing the access logs from Apache can be much more valuable. At
>>>> least
>>>> >> what seem stored currently in the ElasticSearch index for searches
>>>> is too
>>>> >> low level and makes it quite hard to use get any useful statistics.
>>>> >>
>>>> >> Regards,
>>>> >> Jose García
>>>> >>
>>>> >> On Tue, Feb 25, 2020 at 7:43 PM Jo Cook <[hidden email]>
>>>> wrote:
>>>> >>
>>>> >> > Hi All,
>>>> >> >
>>>> >> > I have elasticsearch (7.6.0) and kibana (7.2.1) configured and
>>>> running
>>>> >> > in geonetwork 3.8.x. I have followed the instructions here:
>>>> >> >
>>>> >> >
>>>> https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
>>>> >> > and here
>>>> https://github.com/geonetwork/core-geonetwork/tree/master/es
>>>> >> > and can see the content statistics dashboard correctly in
>>>> geonetwork.
>>>> >> >
>>>> >> > What I can't see is the search statistics dashboard! It's simply
>>>> not
>>>> >> > there. In kibana/elasticsearch/indexmanagement I can see that the
>>>> >> > gn-searchlogs index has been loaded, but when I look for index
>>>> >> > patterns there isn't one for it, and when I try creating one,
>>>> kibana
>>>> >> > doesn't recognise the gn-searchlogs name.
>>>> >> >
>>>> >> > In geonetwork itself I have checked that the option to store search
>>>> >> > statistics in the database is ticked, and I have run the tool to
>>>> index
>>>> >> > the catalog in a remote index and done some searches.
>>>> >> >
>>>> >> > I feel like I'm missing something very simple, but I can't see
>>>> what it is!
>>>> >> >
>>>> >> > Any ideas would be more than welcome!
>>>> >> >
>>>> >> > Jo
>>>> >> >
>>>> >> > --
>>>> >> > Jo Cook
>>>> >> > t:+44 7930 524 155/twitter:@archaeogeek
>>>> >> > Please note that currently I do not work on Friday afternoons. For
>>>> >> > urgent responses at that time, please visit
>>>> >> > support.astuntechnology.com or phone our office on 01372 744009
>>>> >> >
>>>> >> > --
>>>> >> > --
>>>> >> > *Sign up to our mailing list
>>>> >> > <https://astuntechnology.com/company/#email-updates> for updates
>>>> on news,
>>>> >> > products, conferences, events and training*
>>>> >> > *
>>>> >> > *
>>>> >> >
>>>> >> > Astun Technology Ltd,
>>>> >> > Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19
>>>> 8AG, UK
>>>> >> >
>>>> >> > t:+44 1372 744 009 w: astuntechnology.com <
>>>> http://astuntechnology.com/>
>>>> >> > twitter:@astuntech <https://twitter.com/astuntech>
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > iShare - enterprise
>>>> >> > geographic intelligence platform <
>>>> https://astuntechnology.com/ishare/>
>>>> >> >
>>>> >> > GeoServer, PostGIS and QGIS training
>>>> >> > <https://astuntechnology.com/training-courses/>
>>>> >> > Helpdesk and customer
>>>> >> > portal
>>>> >> > <
>>>> >> >
>>>> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>>>> >> > >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> >
>>>> >> > Company registration no. 5410695. Registered in England and Wales.
>>>> >> > Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN
>>>> VAT no.
>>>> >> > 864201149.
>>>> >> >
>>>> >> >
>>>> >> > _______________________________________________
>>>> >> > GeoNetwork-users mailing list
>>>> >> > [hidden email]
>>>> >> > https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>>>> >> > GeoNetwork OpenSource is maintained at
>>>> >> > http://sourceforge.net/projects/geonetwork
>>>> >> >
>>>> >>
>>>> >>
>>>> >> --
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> *Vriendelijke groeten / Kind regards,Jose García
>>>> >> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe
>>>> NetherlandsT: +31
>>>> >> (0)318 416664 <+31318416664>Please consider the environment before
>>>> printing
>>>> >> this email.*
>>>> >>
>>>> >> _______________________________________________
>>>> >> GeoNetwork-users mailing list
>>>> >> [hidden email]
>>>> >> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>>>> >> GeoNetwork OpenSource is maintained at
>>>> http://sourceforge.net/projects/geonetwork
>>>>
>>>>
>>>>
>>>> --
>>>> Jo Cook
>>>> t:+44 7930 524 155/twitter:@archaeogeek
>>>> Please note that currently I do not work on Friday afternoons. For
>>>> urgent responses at that time, please visit
>>>> support.astuntechnology.com or phone our office on 01372 744009
>>>>
>>>> --
>>>> --
>>>> *Sign up to our mailing list
>>>> <https://astuntechnology.com/company/#email-updates> for updates on
>>>> news,
>>>> products, conferences, events and training*
>>>> *
>>>> *
>>>>
>>>> Astun Technology Ltd,
>>>> Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
>>>>
>>>> t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/
>>>> >
>>>> twitter:@astuntech <https://twitter.com/astuntech>
>>>>
>>>>
>>>>
>>>> iShare - enterprise
>>>> geographic intelligence platform <https://astuntechnology.com/ishare/>
>>>>
>>>> GeoServer, PostGIS and QGIS training
>>>> <https://astuntechnology.com/training-courses/>
>>>> Helpdesk and customer
>>>> portal
>>>> <
>>>> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>>>> >
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Company registration no. 5410695. Registered in England and Wales.
>>>> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT
>>>> no.
>>>> 864201149.
>>>>
>>>
>>>
>>> --
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> *Vriendelijke groeten / Kind regards,Jose García
>>> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
>>> (0)318 416664 <+31318416664>Please consider the environment before printing
>>> this email.*
>>>
>>
>>
>> --
>> *Jo Cook*
>> t:+44 7930 524 155/twitter:@archaeogeek
>> Please note that currently I do not work on Friday afternoons. For urgent
>> responses at that time, please visit support.astuntechnology.com or
>> phone our office on 01372 744009
>>
>>
>> --
>> *Sign up to our mailing list
>> <https://astuntechnology.com/company/#email-updates> for updates on news,
>> products, conferences, events and training*
>>
>> Astun Technology Ltd, Epsom Square Centre, 6-7 The Derby Square, Epsom,
>> Surrey, KT19 8AG, UK
>> t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech
>> <https://twitter.com/astuntech>
>>
>> iShare - enterprise geographic intelligence platform
>> <https://astuntechnology.com/ishare/>
>> GeoServer, PostGIS and QGIS training
>> <https://astuntechnology.com/training-courses/>
>> Helpdesk and customer portal
>> <https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal>
>>
>> Company registration no. 5410695. Registered in England and Wales.
>> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
>> 864201149.
>>
>
>
> --
>
>
>
>
>
>
>
>
>
>
>
> *Vriendelijke groeten / Kind regards,Jose García
> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
> (0)318 416664 <+31318416664>Please consider the environment before printing
> this email.*
>


--
*Jo Cook*
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent
responses at that time, please visit support.astuntechnology.com or phone
our office on 01372 744009

--
-- 
*Sign up to our mailing list
<https://astuntechnology.com/company/#email-updates> for updates on news,
products, conferences, events and training*
*
*

Astun Technology Ltd,
Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK 

t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/
twitter:@astuntech <https://twitter.com/astuntech>



iShare - enterprise
geographic intelligence platform <https://astuntechnology.com/ishare/>

GeoServer, PostGIS and QGIS training
<https://astuntechnology.com/training-courses/>
Helpdesk and customer
portal
<https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal>





Company registration no. 5410695. Registered in England and Wales.
Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
864201149.

_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork
Reply | Threaded
Open this post in threaded view
|

Re: missing search tab in 3.8.x statistics?

Jo Cook
Here's my PR https://github.com/geonetwork/core-geonetwork/pull/4561

Jo

On Mon, Mar 30, 2020 at 6:18 PM Jo Cook <[hidden email]> wrote:

> Hi Jose,
>
> I think I have something working- but I'm not sure which branch to submit
> it as a PR against- would it be 3.8.x or Master?
>
> Thanks
>
> Jo
>
> On Wed, Mar 18, 2020 at 8:26 AM Jose Garcia <[hidden email]>
> wrote:
>
>> Hi Jo
>>
>> That's great, thank you. If you can do a pull request and set it to draft
>> if requires some additional work. I think that would be the best option as
>> other developers can try it and provide feedback or improvements if
>> required.
>>
>> Regards,
>> Jose García
>>
>> On Tue, Mar 17, 2020 at 5:38 PM Jo Cook <[hidden email]>
>> wrote:
>>
>>> Hi Jose and Francois,
>>>
>>> I have managed to get this working, but in a slightly messy way- I
>>> downloaded the saved objects from a working install (GeoNetwork 3.4), and
>>> then uploaded that export.json into my new install (GeoNetwork 3.8). That
>>> failed part of the way through due to the missing index pattern for
>>> gn-searchlogs, but I could create that, and then when I re-ran the import
>>> it worked. Not ideal, but I can use this approach moving forward.
>>>
>>> If I can replicate this more cleanly, should I save export.json and
>>> commit it somewhere?
>>>
>>> Thanks
>>>
>>> Jo
>>>
>>> On Fri, Feb 28, 2020 at 9:35 AM Jose Garcia <[hidden email]>
>>> wrote:
>>>
>>>> Hi Jo
>>>>
>>>> Not sure if importing the dashboards from 3.6 will break anything in
>>>> 3.8, but you can give a try instead of step 3 below (steps 1 and 2 are
>>>> required). Apologies as indicated to provide the files I created, but I
>>>> lost them accidentally, adding here the steps I did to enable the
>>>> statistics and add the Kibana dashoards:
>>>>
>>>> 1) Uncomment the following code (also previous lines if require the
>>>> metadata validation statistics)
>>>>
>>>>
>>>> https://github.com/geonetwork/core-geonetwork/blob/cd47db86fb04a7c88d252b50c792a5f60ca5160d/web-ui/src/main/resources/catalog/js/admin/DashboardController.js#L87-L93
>>>>
>>>> 2) I loaded the 3.10 Kibana dashboards as described in
>>>> https://github.com/geonetwork/core-geonetwork/tree/3.10.x/es/es-dashboards
>>>>
>>>> Visit Kibana in a browser using one of the above links and go to 'Saved
>>>> Objects'. Import export.json from
>>>> https://github.com/geonetwork/core-geonetwork/blob/master/es/es-dashboards/data/export.json
>>>>
>>>>
>>>> 3) Start up GeoNetwork and access the search statistics, all panels
>>>> show with an error related to the dashboard identifier. I checked the
>>>> identifiers failing in
>>>> https://github.com/geonetwork/core-geonetwork/blob/3.6.x/es/es-dashboards/data/index-dashboards.json and
>>>> created a similar file as the export.json from previous step and loaded as
>>>> described there.
>>>>
>>>> But as afaik 3.6 only had search statistics, no content statistics
>>>> probably is fine just to load the full file, please give it a try.
>>>>
>>>> Regards,
>>>> Jose García
>>>>
>>>> On Fri, Feb 28, 2020 at 10:18 AM Jo Cook <[hidden email]>
>>>> wrote:
>>>>
>>>>> Hi Francois/Jose,
>>>>>
>>>>> For now I'd really like to get the original search statistics back
>>>>> rather than ingesting the server log files, even if they don't look
>>>>> that good in the dashboard. Can I follow the instructions for 3.6
>>>>> (
>>>>> https://github.com/geonetwork/core-geonetwork/tree/3.6.x/es/es-dashboards
>>>>> )
>>>>> to load the dashboard-mapping in or is there something else I need to
>>>>> do?
>>>>>
>>>>> Thanks
>>>>>
>>>>> Jo
>>>>>
>>>>> On Wed, Feb 26, 2020 at 8:38 AM Francois Prunayre <
>>>>> [hidden email]> wrote:
>>>>> >
>>>>> > Hi Jose, in some projects it looks like sometimes webserver logs are
>>>>> ingested in Elasticsearch and based on the GN API, it can be used to make
>>>>> global stats about number of editing session per months, ... which is
>>>>> indeed not focused only on search.
>>>>> > It may make sense to provide instructions on how to setup this kind
>>>>> of reports.
>>>>> >
>>>>> > Cheers.
>>>>> >
>>>>> > Francois
>>>>> >
>>>>> >
>>>>> > Le mer. 26 févr. 2020 à 07:48, Jose Garcia <[hidden email]>
>>>>> a écrit :
>>>>> >>
>>>>> >> Hi Jo
>>>>> >>
>>>>> >> I notice that also last week, with a customer. Seem the search
>>>>> statistics
>>>>> >> were removed in 3.8 as requires additional work for the Kibana
>>>>> panels due
>>>>> >> to an upgrade of ElasticSearch version seem. I think should be
>>>>> created an
>>>>> >> issue to handle this, as most people are not aware of this change.
>>>>> >>
>>>>> >> I restored the search statistics locally from 3.6 version, where
>>>>> were
>>>>> >> enabled, but indeed look pretty bad and useless, requiring a
>>>>> rework. Also
>>>>> >> something to think about is about the feature itself. In my opinion,
>>>>> >> processing the access logs from Apache can be much more valuable.
>>>>> At least
>>>>> >> what seem stored currently in the ElasticSearch index for searches
>>>>> is too
>>>>> >> low level and makes it quite hard to use get any useful statistics.
>>>>> >>
>>>>> >> Regards,
>>>>> >> Jose García
>>>>> >>
>>>>> >> On Tue, Feb 25, 2020 at 7:43 PM Jo Cook <[hidden email]>
>>>>> wrote:
>>>>> >>
>>>>> >> > Hi All,
>>>>> >> >
>>>>> >> > I have elasticsearch (7.6.0) and kibana (7.2.1) configured and
>>>>> running
>>>>> >> > in geonetwork 3.8.x. I have followed the instructions here:
>>>>> >> >
>>>>> >> >
>>>>> https://geonetwork-opensource.org/manuals/trunk/en/maintainer-guide/statistics/index.html
>>>>> >> > and here
>>>>> https://github.com/geonetwork/core-geonetwork/tree/master/es
>>>>> >> > and can see the content statistics dashboard correctly in
>>>>> geonetwork.
>>>>> >> >
>>>>> >> > What I can't see is the search statistics dashboard! It's simply
>>>>> not
>>>>> >> > there. In kibana/elasticsearch/indexmanagement I can see that the
>>>>> >> > gn-searchlogs index has been loaded, but when I look for index
>>>>> >> > patterns there isn't one for it, and when I try creating one,
>>>>> kibana
>>>>> >> > doesn't recognise the gn-searchlogs name.
>>>>> >> >
>>>>> >> > In geonetwork itself I have checked that the option to store
>>>>> search
>>>>> >> > statistics in the database is ticked, and I have run the tool to
>>>>> index
>>>>> >> > the catalog in a remote index and done some searches.
>>>>> >> >
>>>>> >> > I feel like I'm missing something very simple, but I can't see
>>>>> what it is!
>>>>> >> >
>>>>> >> > Any ideas would be more than welcome!
>>>>> >> >
>>>>> >> > Jo
>>>>> >> >
>>>>> >> > --
>>>>> >> > Jo Cook
>>>>> >> > t:+44 7930 524 155/twitter:@archaeogeek
>>>>> >> > Please note that currently I do not work on Friday afternoons. For
>>>>> >> > urgent responses at that time, please visit
>>>>> >> > support.astuntechnology.com or phone our office on 01372 744009
>>>>> >> >
>>>>> >> > --
>>>>> >> > --
>>>>> >> > *Sign up to our mailing list
>>>>> >> > <https://astuntechnology.com/company/#email-updates> for updates
>>>>> on news,
>>>>> >> > products, conferences, events and training*
>>>>> >> > *
>>>>> >> > *
>>>>> >> >
>>>>> >> > Astun Technology Ltd,
>>>>> >> > Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19
>>>>> 8AG, UK
>>>>> >> >
>>>>> >> > t:+44 1372 744 009 w: astuntechnology.com <
>>>>> http://astuntechnology.com/>
>>>>> >> > twitter:@astuntech <https://twitter.com/astuntech>
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> > iShare - enterprise
>>>>> >> > geographic intelligence platform <
>>>>> https://astuntechnology.com/ishare/>
>>>>> >> >
>>>>> >> > GeoServer, PostGIS and QGIS training
>>>>> >> > <https://astuntechnology.com/training-courses/>
>>>>> >> > Helpdesk and customer
>>>>> >> > portal
>>>>> >> > <
>>>>> >> >
>>>>> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>>>>> >> > >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> >
>>>>> >> > Company registration no. 5410695. Registered in England and Wales.
>>>>> >> > Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN
>>>>> VAT no.
>>>>> >> > 864201149.
>>>>> >> >
>>>>> >> >
>>>>> >> > _______________________________________________
>>>>> >> > GeoNetwork-users mailing list
>>>>> >> > [hidden email]
>>>>> >> > https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>>>>> >> > GeoNetwork OpenSource is maintained at
>>>>> >> > http://sourceforge.net/projects/geonetwork
>>>>> >> >
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> *Vriendelijke groeten / Kind regards,Jose García
>>>>> >> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe
>>>>> NetherlandsT: +31
>>>>> >> (0)318 416664 <+31318416664>Please consider the environment before
>>>>> printing
>>>>> >> this email.*
>>>>> >>
>>>>> >> _______________________________________________
>>>>> >> GeoNetwork-users mailing list
>>>>> >> [hidden email]
>>>>> >> https://lists.sourceforge.net/lists/listinfo/geonetwork-users
>>>>> >> GeoNetwork OpenSource is maintained at
>>>>> http://sourceforge.net/projects/geonetwork
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Jo Cook
>>>>> t:+44 7930 524 155/twitter:@archaeogeek
>>>>> Please note that currently I do not work on Friday afternoons. For
>>>>> urgent responses at that time, please visit
>>>>> support.astuntechnology.com or phone our office on 01372 744009
>>>>>
>>>>> --
>>>>> --
>>>>> *Sign up to our mailing list
>>>>> <https://astuntechnology.com/company/#email-updates> for updates on
>>>>> news,
>>>>> products, conferences, events and training*
>>>>> *
>>>>> *
>>>>>
>>>>> Astun Technology Ltd,
>>>>> Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK
>>>>>
>>>>> t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/
>>>>> >
>>>>> twitter:@astuntech <https://twitter.com/astuntech>
>>>>>
>>>>>
>>>>>
>>>>> iShare - enterprise
>>>>> geographic intelligence platform <https://astuntechnology.com/ishare/>
>>>>>
>>>>> GeoServer, PostGIS and QGIS training
>>>>> <https://astuntechnology.com/training-courses/>
>>>>> Helpdesk and customer
>>>>> portal
>>>>> <
>>>>> https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> Company registration no. 5410695. Registered in England and Wales.
>>>>> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT
>>>>> no.
>>>>> 864201149.
>>>>>
>>>>
>>>>
>>>> --
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *Vriendelijke groeten / Kind regards,Jose García
>>>> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
>>>> (0)318 416664 <+31318416664>Please consider the environment before printing
>>>> this email.*
>>>>
>>>
>>>
>>> --
>>> *Jo Cook*
>>> t:+44 7930 524 155/twitter:@archaeogeek
>>> Please note that currently I do not work on Friday afternoons. For
>>> urgent responses at that time, please visit support.astuntechnology.com or
>>> phone our office on 01372 744009
>>>
>>>
>>> --
>>> *Sign up to our mailing list
>>> <https://astuntechnology.com/company/#email-updates> for updates on news,
>>> products, conferences, events and training*
>>>
>>> Astun Technology Ltd, Epsom Square Centre, 6-7 The Derby Square, Epsom,
>>> Surrey, KT19 8AG, UK
>>> t:+44 1372 744 009 w: astuntechnology.com twitter:@astuntech
>>> <https://twitter.com/astuntech>
>>>
>>> iShare - enterprise geographic intelligence platform
>>> <https://astuntechnology.com/ishare/>
>>> GeoServer, PostGIS and QGIS training
>>> <https://astuntechnology.com/training-courses/>
>>> Helpdesk and customer portal
>>> <https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal>
>>>
>>> Company registration no. 5410695. Registered in England and Wales.
>>> Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
>>> 864201149.
>>>
>>
>>
>> --
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> *Vriendelijke groeten / Kind regards,Jose García
>> <http://www.geocat.net/>Veenderweg 136721 WD BennekomThe NetherlandsT: +31
>> (0)318 416664 <+31318416664>Please consider the environment before printing
>> this email.*
>>
>
>
> --
> *Jo Cook*
> t:+44 7930 524 155/twitter:@archaeogeek
> Please note that currently I do not work on Friday afternoons. For urgent
> responses at that time, please visit support.astuntechnology.com or phone
> our office on 01372 744009
>
>

--
*Jo Cook*
t:+44 7930 524 155/twitter:@archaeogeek
Please note that currently I do not work on Friday afternoons. For urgent
responses at that time, please visit support.astuntechnology.com or phone
our office on 01372 744009

--
-- 
*Sign up to our mailing list
<https://astuntechnology.com/company/#email-updates> for updates on news,
products, conferences, events and training*
*
*

Astun Technology Ltd,
Epsom Square Centre, 6-7 The Derby Square, Epsom, Surrey, KT19 8AG, UK 

t:+44 1372 744 009 w: astuntechnology.com <http://astuntechnology.com/
twitter:@astuntech <https://twitter.com/astuntech>



iShare - enterprise
geographic intelligence platform <https://astuntechnology.com/ishare/>

GeoServer, PostGIS and QGIS training
<https://astuntechnology.com/training-courses/>
Helpdesk and customer
portal
<https://astuntech.atlassian.net/wiki/spaces/ISHAREHELP/pages/364970043/Astun+Technology+Support+Portal>





Company registration no. 5410695. Registered in England and Wales.
Registered office: 120 Manor Green Road, Epsom, Surrey, KT19 8LN VAT no.
864201149.

_______________________________________________
GeoNetwork-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/geonetwork-users
GeoNetwork OpenSource is maintained at http://sourceforge.net/projects/geonetwork