Missing cities in index (Berlin)

[macolu]

Hi

I have setup a Photon instance for a city autocomplete feature.
I need more languages than default list, so I also have setup nominatim import process.

It works pretty well, but Berlin is missing from results lists.

I can find Berlin on photon.komoot.io instance, with osm_id 240109189: https://photon.komoot.io/reverse?limit=1&lat=52&lon=13&radius=100&query_string_filter=osm_id:240109189

If I start a Photon instance with index available on GraphHopper, I also see the same document.

However, when building Photon index from my own Nominatim instance, this document is not in Elastic index (confirmed by direct ES query on port 9200).

I looked at Photon import code, it basically performs this query to grab list of documents to index:

SELECT ...
FROM placex
WHERE
    linked_place_id IS null
    AND centroid IS NOT null

The thing is, in my Nominatim DB, osm_id 240109189 is actually linked to another place:

place_id|osm_type|osm_id   |class   |type          |linked_place_id|centroid                     |
--------+--------+---------+--------+--------------+---------------+-----------------------------+
1778169 |R       |62422    |boundary|administrative|               |POINT (13.3996024 52.5186925)
101233  |N       |240109189|place   |city          |1778169        |POINT (13.3996024 52.5186925)

So now, I know why Berlin is missing (linked_place_id not NULL).
But since it's present on GraphHopper and photon.komoot.io, I assume there is a way to get rid of this link in Nominatim DB. Am I right?

Here is what I tried on nominatim import side, without success:

• nominatim version 3.7.2 and 4.0.1
• import_style admin and full
• with and without flatnodes
• with and without updates (nominatim replication)

Example of command I ran:

docker run -it --rm \                                                                                                        
  -e PBF_URL=https://download.geofabrik.de/europe/germany/brandenburg-latest.osm.pbf \
  -e REPLICATION_URL=http://download.geofabrik.de/europe/germany/brandenburg-updates/ \
  -e IMPORT_STYLE=admin \
  -e NOMINATIM_PASSWORD=xxxxx \
  -e NOMINATIM_FLATNODE_FILE=/nominatim/flatnode.file \
  -p 5432:5432 \
  --name nominatim \
  mediagis/nominatim:4.0

The only test that worked was to import an old PBF file from January 2021 (available on Geofabrik). I assume that link between these places has been established in OSM somewhere during 2021.
But sticking to an old dataset doesn't seem to be a good idea :)

Any hints on how I could sort this out?
Thanks

[lonvia]

Linking of boundaries and place nodes that describe the same place is a basic functionality and cannot be turned off. It is necessary for address computation and avoids result duplication. The reason that you sometimes get the node not linked is that the mapping is rather volatile.

The important question is: why do you need to get specifically the node? Why is the combined boundary/node result not sufficient?

[macolu]

When I query Photon, I add a filter on tags, so that I only have cities in return :

Something like: https://photon.komoot.io/api/?q=berlin&osm_tag=place:town&osm_tag=place:city&osm_tag=place:village

I need osm_id 240109189 because of its type/tag:

      "properties": {
        "osm_id": 240109189,
        "osm_type": "N",
        "osm_key": "place",
        "osm_value": "city",
        "name": "Berlin",
        "type": "city"
        ...
      }

The linked one has another type/tag, so it's not returned:

      "properties": {
        "osm_id": 62422,
        "osm_type": "R",
        "osm_key": "place",
        "osm_value": "state",
        "name": "Berlin",
        "type": "state"
      }

And I don't want to add &osm_tag=place:state in my Photon query, because it would return me irrelevant results for my usecase.

Thanks

[lonvia]

The linking isn't really the problem here. It's rather the filtering which is insufficient. The current filters are meant to filter for POI types not administrative entities. You would need to have osm_tag=place:town&osm_tag=place:city&osm_tag=place:village&osm_tag=boundary:administrative to get the expected result. But that also gets you more than you like.

What you probably need is rather to filter by what we call address type. This would you get all city-like results (cities, towns, independent villages etc.) or all street-like results (including footways etc.) or... All independent of the actual OSM tag. I can see how that might be useful. The address type is not indexed at the moment, so needs a database upgrade to implement.

[macolu]

That's really interesting. Thank you for pointing out this notion of address type derived from rank, I didn't see that.

Well, maybe I can try to work on a PR to add this filter in Photon API.

About database upgrade, as far as I can see field is already there, but indeed mapping has to be updated to enable indexing.
Is it a blocking point (memory impact, BC break, ...)?

[lonvia]

Yes, feel free to work on a PR. Shouldn't be too complex. We might have to think a bit more carefully about how the API parameter works but that shouldn't change much in the technical implementation. Just go ahead and make a first proposal.

Can I kindly request that you do not forget about unit tests a la https://github.com/komoot/photon/blob/master/src/test/java/de/komoot/photon/query/QueryFilterTagValueTest.java. I've cleaned up the tests recently, so hopefully it's not too much of a pain anymore.

The database upgrade is no showstopper. It just means that we can't sneak the change into a minor version update. People will need to download a new dump. It also means that you need to increase the database version in a PR implementing the new filter.