REST APIs to perform Geospatial operations using MongoDB, Google Geocoding API, GeoJSON and Spring Data

This project is an extension to the previous project “How to build CRUD APIs using Apache CXF, Spring Data and MongoDB to manage a set of employees”. The objective of this project is to extend the APIs that were built out of the previous project, to deal mainly with employees’ address information. To accomplish this objective, it utilizes and thereby demonstrates the power of MongoDB’s geospatial features, Google’s geocoding apis and GeoJSON. Some of the real world use cases like the below are something that is possible using this project

a) List all employees that live within a 50 mile radius of a given address
b) List all employees that live within a given set of polygon co-ordinates

Before proceeding further with the implementation, here is a brief introduction about the supporting geo-technologies used in this project

GeoJSON
GeoJson is an open standard format developed to represent spatial geometries using standard JSON name/value pair convention. Geometries could be a simple spatial Point, LineString or a complex Polygon. In addition that, GeoJson also supports geo-features and feature collections. A “geo-feature” is typically a geometry combined with its additional properties and a feature collection is a multitude of features bundled as a single representation.

A geometry object mostly contains two members – “type” and “coordinates”. Type represents the name of the geojson object and coordinates represents an array of longitude and latitude pairs. There can be many such co-ordinate pairs depending on the type of geometry being represented. A couple of sample GeoJSONs are shown below

{ "type": "Point", "coordinates": [100.0, 0.0] }

{ "type": "Polygon",
    "coordinates": [
      [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ]
     ]
}

More about GeoJson and its examples can be found here GeoJSON Specification - http://geojson.org/geojson-spec.html#appendix-a-geometry-examples

MongoDB’s Geospatial features
MongoDB supports a handful of geospatial operations on both Euclidean plane and spherical surface. These operations are possible because it is equipped with two types of geo-indexes – 2dindex and 2dsphere. 2d index supports operations on an Euclidean plane and allows mongo documents to have geo-information as legacy co-ordinates [latitude, longitude]. 2dsphere index supports operations on earth like sphere and supports geo-data as both legacy co-ordinates and as GeoJSON.

Based on the type of operations that might require for an application, a choice needs to be made from either of two structures and that in turn will decide the type of geo-index that needs to be used. More about the different types of operators available in MongoDB is over here MongoDB GeoSpatial Query operators - http://docs.mongodb.org/manual/reference/operator/query-geospatial

Google’s geocoding Api
Geocoding is the process of converting a regular postal address into geographic co-ordinates. Reverse geocoding is the reverse process of converting geo coordinates back to human readable addresses. Using these processes, it is possible to encode addresses to coordinates and perform geo-operations on the coordinates.

More about Google’s Geocoding API can be found here Google developer site - Geocoding - https://developers.google.com/maps/documentation/geocoding/. An OAuth application (Google calls it project) has to be created and activated in Google's developer site to use this api. The server key of the created OAuth project will be used in our sample application to geocode addresses.

Having talked about the geo-technologies used in this project, let’s move on to discuss about how to incorporate these features and perform a real world usecase of "finding employees who work within a given radius of 50 miles from a given address".

Modifications to MongoDB document structure
As the employee’s work address information needs to be persisted along with the rest of employee information, the existing “employees” mongo document structure (the one that was used here “How to build CRUD APIs using Apache CXF, Spring Data and MongoDB to manage a set of employees”) is extended to comprise the "address" geojson. The existing documents in the store are updated to have their corresponding employee's address so that they qualify as candidates for the usecase that we are working towards. In addition to that, the documents that will be created in future needs to have the address information. Again, it is not mandatory that every employee’s document needs to contain their work address, but if they have, such documents will qualify as a candidate for geo operations.

The “address” object in our case is not a straight forward geojson. It contains two other objects – “postalAddress” and “location”. Postal address carries the work address of the employee. Location is the actual GeoJson object that carries the geocoded co-ordinate information of the employee work address. These two objects are bundled together into a single address object. Below is a sample address json object

"address" :{
    "loc": {
        "type": "Point",
        "coordinates": [
            -122.0829,
            37.4211
        ]
    },
    "postalAddress": {
        "street": "1600 Amphitheatre Parkway",
        "city": "Mountain View",
        "state": "California",
        "zip": "94043",
        "country": "UnitedStates"
    }
}

Following mongo query inserts documents that includes the above address structure into the employees collection of organization database

> use organization;
> db.employees.insert({
    "empId": NumberLong(35),
    "fname": "Richard",
    "lname": "Stallman",
    "deptName": "CS",
    "salary": 1000,
    "address": {
        "loc": {
            "type": "Point",
            "coordinates": [
                -122.3947492,
                37.7899519
            ]
        },
        "postalAddress": {
            "street": "199 Fremont Street",
            "city": "San Francisco",
            "state": "CA",
            "zip": "94105",
            "country": "United States"
        }
    }
});

Once we have a handful of employees along with their work address in mongo store, it is now time to query them using mongo console (or any favorite mongo client of yours) to find whether they are really query-able. MongoDB cannot support any geo-operations without having a geo-index. If done so, it says there is no index. Below is a snippet to illustrate that

Query before Indexing

> db.employees.find({
    "address.loc": {
        $near: {
            $geometry: {
                type: "Point",
                coordinates: [
                    -122.08,
                    37.42
                ]
            },
            $maxDistance: 80467.2
        }
    }
});
error: {
        "$err" : "Unable to execute query: error processing query: ns=organization.employees limit=0 skip=0\nTree: GEONEAR  
field=loc maxdist=35000 isNearSphere=0\nSort: {}\nProj: {}\n planner returned error: unable to find index for $geoNear query",
        "code" : 17007
}

Create a 2dsphere geo index
A 2dsphere index is created on the GeoJSON object. In this case, it is “address.loc” which is an embedded document.

> db.employees.createIndex({"address.loc": "2dsphere"});
{
    "createdCollectionAutomatically": false,
    "numIndexesBefore": 1,
    "numIndexesAfter": 2,
    "ok": 1
}

> db.employees.getIndexes();
[
    {
        "v": 1,
        "key": {
            "_id": 1
        },
        "name": "_id_",
        "ns": "organization.employees"
    },
    {
        "v": 1,
        "key": {
            "address.loc": "2dsphere"
        },
        "name": "address.loc_2dsphere",
        "ns": "organization.employees",
        "2dsphereIndexVersion": 2
    }
]

Query after Indexing
After creating a 2dsphere index, the same geo query operation is successful

> db.employees.find({
    "address.loc": {
        $near: {
            $geometry: {
                type: "Point",
                coordinates: [
                    -122.08,
                    37.42
                ]
            },
            $maxDistance: 80467.2
        }
    }
});

{
    "_id": ObjectId("55227f844beece3fbb066d7e"),
    "empId": NumberLong(33),
    "fname": "Sundar",
    "lname": "Pichai",
    "deptName": "CS",
    "salary": 1000,
    "address": {
        "loc": {
            "type": "Point",
            "coordinates": [
                -122.0829,
                37.4211
            ]
        },
        "postalAddress": {
            "street": "1600 Amphitheatre Parkway",
            "city": "Mountain View",
            "state": "California",
            "zip": "94043",
            "country": "United States"
        }
    }
}

The next set of steps is to make modifications to our java Crud apis to support and utilize these features

POM changes
The POM of this project is almost the same like the previous project apart from the fact that this one includes google’s geocoding library. Below is a snippet to do that

0.1.6
 
  com.google.maps
  google-maps-services
  ${google.maps.services.version}
 

Create Employee
The create employee api request is added with a new property called “Address”. This address will represent the work address of the employee. The below implementation geo-codes this given address into latitude and longitude information and then persisted into mongo store using the above document structure.

public EmployeeResponse createEmployee(EmployeeRequest employeeRequest) throws AddressException, Exception {
 EmployeeResponse employeeResponse = null;
 Employee employee = null;
 Double[] latlng = null;
 
 if (employeeRequest != null) {
  Long empId = employeeRepository.getNextId();
  log.info("new employeeId={}", empId);

  latlng = employeeHelper.getGeoResult(employeeRequest.getAddress());
  employee = employeeAdapter.buildDocument(empId, employeeRequest, latlng);
  employeeRepository.createEmployee(employee);
  if (employee.get_id() != null) {
  employeeResponse = employeeAdapter.convertToEmployeeResponse(employee);
  }
 }
 
 log.info("employeeResponse={}", employeeResponse);
 return employeeResponse;
}

The given postal address is converted to a string representation which is then used by the geocode call. The geocode call also uses something called as “geocontext”. It is the context created using the server key that was generated from your OAuth project in google’s developer site.

private static final GeoApiContext geoContext = new GeoApiContext().setApiKey("xxx");


/**
 * Gets geo information for a given postal address using google geocoding api
 * @param address the address information
 * @return GeocodingResult[] an array of GeocodingResult
 * @throws Exception
 */
public Double[] getGeoResult(PostalAddress pa) throws AddressException, Exception {
 log.info("get geoinfo for address={}", pa);

 if (pa != null) {
 String address = employeeAdapter.convertPostalAddressToTextAddress(pa);
 return geoCode(address);
 }
 return null;
}

 /**
  * Gets geo information for a given address using google geocoding api
  * @param address
  * @return Double[]
  * @throws AddressException
  * @throws Exception
  */
 public Double[] geoCode(String address) throws AddressException, Exception {
  log.info("geocode address={}", address);
  GeocodingResult[] geoResult = null;
  
  geoResult =  GeocodingApi.geocode(geoContext, address).await();
  // if no matches or more than 1 match throw address exception
  if (geoResult == null
    || geoResult.length != 1
    //|| geoResult[0].geometry.locationType != LocationType.ROOFTOP
    ) {
   throw new AddressException("INVALID_ADDRESS");
  }
  log.info("received geoInfo={}", geoResult[0].geometry.location);
  return new Double[] {geoResult[0].geometry.location.lat, geoResult[0].geometry.location.lng};
 }

Get and Get all employees
The most interesting usecase in get and get all employees apis is to find out all the employees in the organization living within a given radius (in miles). To elaborate a bit more, this operation finds all employees who live within a given radius of a circle whose center falls at the address that was provided in the request. This is accomplished by geo-coding the provided address into lat-long co-ordinates using geocoding api. Once we have the co-ordinates, they can be used to query mongodb using $near query operator in such a way that those co-ordinates are the center of a circle. The $near query operator in turn uses the geospatial index created on the "employees document collection" to find whether such an employee exists.

else if (latlng != null && lname == null) {
   // get employees that live within a given radius of the given address
   NearQuery nq = NearQuery.near(latlng[1], latlng[0], Metrics.MILES).maxDistance(new Double(radius));
   GeoResults<Employee> empGeoResults = mongoTemplate.geoNear(nq, Employee.class);
   if (empGeoResults != null) {
    empList = new ArrayList<Employee>();
    for (GeoResult<Employee> e : empGeoResults) {
     empList.add(e.getContent());
    }
   }

The below shown code snippet is to find all employees that have a given last name and live within a given radius (in miles) of the given address. The below method is different in a way where the query is completely created manually using Spring data's BasicDBObjectBuilder. The fetched DBObject is iterated using DBCursor and converted to "Employee" mongo document automatically using MongoConverter.

else if (latlng != null && lname != null) {
   // get employees that have a given last name and live within a given radius of the given address
   DBObject geoQuery = buildGeoQuery(latlng, radius);
   DBObject addressLoc = BasicDBObjectBuilder.start().add("address.loc", geoQuery).add("lname", lname).get();
   DBCursor cursor = mongoTemplate.getCollection("employees").find(addressLoc);
   empList = cursor.hasNext() ? new ArrayList<Employee>() : null;
   while (cursor.hasNext()) {
    DBObject empDBObject = (DBObject)cursor.next();
    Employee e = mongoTemplate.getConverter().read(Employee.class, empDBObject);
    empList.add(e);
   }
  }


 /**
  * Builds geo query using latitude, longitude co-ordinates and radius. Given
  * radius is in miles and needs to be converted into meters.
  * @param lname
  * @param latlng
  * @param radius
  * @return
  */
 private DBObject buildGeoQuery(Double[] latlng, String radius) {
  // Mongo uses longitude, latitude combination
  Double[] coordinates = new Double[] {latlng[1], latlng[0]};
  DBObject geometryContent = BasicDBObjectBuilder.start().add("type", "Point")
    .add("coordinates", coordinates).get();
  DBObject nearContent = BasicDBObjectBuilder.start().add("$geometry", geometryContent)
    .add("$maxDistance", (Double.valueOf(radius) * 1609.34)).get();
  DBObject addressLocContent = BasicDBObjectBuilder.start().add("$near", nearContent).get();
  return addressLocContent;
 }

The complete source code of this project has been hosted in my GitHub repository – REST APIs to perform geospatial operations using MongoDB. Please feel free to peek and play around.