curl_reap / Guides
Geocoding
Turn scraped names and areas into coordinates with curl_reap's cascading, cached and rate-limited Geocoder over OpenStreetMap Nominatim.
On this page
Scraped listings often carry a name and an area but no coordinates. A single geocoding query fails often: the full marketing name (“Chelsea Cloisters Serviced Apartments”) is not in the gazetteer, and a bare district query puts every listing on the same centroid. Geocoder tries a cascade of queries and tells you how precise the hit is.
Usage
from curl_reap import Geocoder
geo = Geocoder()
hit = geo.geocode(name="Chelsea Cloisters Serviced Apartments",
area="Kensington", city="London", country="United Kingdom")
# {"lat": 51.49..., "lon": -0.17..., "precision": "name", ...}
if hit and hit["precision"] in ("name", "district"):
listing["lat"], listing["lon"] = hit["lat"], hit["lon"]
reap.geocode(name=None, area=None, city=None, country=None) does the same with a shared default Geocoder.
The cascade
geocode(name=None, area=None, city=None, country=None) builds candidate queries and stops at the first one that returns a result:
- The name with area, city and country, then the name with city and country. Only when a name and a city or area are given.
- The same two queries with the name cleaned: trailing generic words such as “Serviced Apartments”, “Aparthotel”, “Hotel”, “Suites” or
by <brand>are stripped. Only when cleaning changes the name. - The area with city and country, labeled
district. - The city with country, labeled
city.
The result is a dict with lat and lon (floats), precision ("name", "district" or "city"), importance and display_name from Nominatim, and the query that matched. It is None when nothing matched. Keep the name and district hits and drop or flag the coarse city ones.
clean_name(name) is available on its own if you want the cleaned name.
Options
Geocoder(endpoint="https://nominatim.openstreetmap.org/search", user_agent=..., cache=".reap_geocode.json", min_interval=1.1, profile="chrome", min_importance=0.0):
endpoint: a Nominatim-compatible search Uniform Resource Locator (URL).user_agent: sent on every request. The default iscurl_reap/2.0 (+https://github.com/anishfyi/curl_reap).cache: a JavaScript Object Notation (JSON) file of past lookups, keyed by the four inputs. Misses are cached too, so repeat runs make no requests.min_interval: minimum seconds between requests, enforced across threads.profile: the transport profile for the underlyingSession.min_importance: ignore hits whose Nominatimimportanceis lower than this and try the next candidate.
Be a good citizen
Nominatim’s public service asks for an identifying User-Agent and at most one request per second. The defaults set both. Set your own user_agent with contact details for larger jobs, or point endpoint at your own Nominatim instance.