Tripping the Bits
t @

Spatial REST API

There was news yesterday was that Esri is "giving" their REST API to Open Geospatial Consortium. While Esri's API is better than WMS queries, it's not perfect. It's also not the only option.

It's not perfect

JSON does not make a good REST API

I love JSON. It's a great web response format that can be parsed with hardware speed by modern web browsers. However, it is in no way geared toward being part of a clean REST API specification.

REST queries are typically based on URLs. A URL (or, more formally, URI) consists of three types of characters: unreserved, reserved and other.

You may type unreserved characters into the address bar of your favorite browser as-is and that's how they will be sent to the server.

Reserved characters is a group of 18 characters divided into general delimiters:

":", "/", "?", "#", "[", "]", "@"

and sub delimiters:

"!", "$", "&", "'", "(", ")", "*", "+", ",", ";", "="

Reserved characters may sometimes need to be percent-encoded before being sent to a web server.

When not used for their special purpose, most general delimiters require encoding. The exceptions are that the path portion of a URL can contain unencoded colon characters and the query portion can contain both colon and forward slash characters. The query portion can also contain the entire set of sub delimiters, unencoded.

All other characters are in the other group and must be percent-encoded.

Six of the eight special characters in the JSON spec (curly braces, square brackets, double quote, and back slash) either do not have an exception to encoding or fall into a URL's other group. This means they look ugly in a URL.

The browser will usually hide this ugliness from you if you type it in by hand or form an Ajax request in JavaScript, but when it's time to send that query to your web server, even if you remove all whitepace, your pretty GeoJSON polygon:

{
  "type": "Polygon",
  "coordinates": [ [
    [100.0, 0.0], [101.0, 0.0], [101.0, 1.0],
    [100.0, 1.0], [100.0, 0.0]
  ] ]
}

looks a little different to the web server:

%7B%22type%22:%22Polygon%22,%22coordinates%22:%5B%5B%5B100.0,0.0%5D,%5B101.0,0.0%5D,%5B101.0,1.0%5D,%5B100.0,1.0%5D,%5B100.0,0.0%5D%5D%5D%7D

Well-Known Text does make a good REST API

WKT has only four delimiters: left & right parentheses, comma and space. None of these characters are in the other category and when used in the query portion of a URL only one, the space, must be encoded. The polygon above looks like this to the web server when sent as WKT:

POLYGON((100.0%200.0,101.0%200.0,101.0%201.0,100.0%201.0,100.0%200.0))

It's not uncommon to encode space characters as plus signs. Plus signs are also a sub delimiter and do not need to be encoded. Google does this all the time with their APIs. Replacing the spaces with plus signs results in our web browser sending the polygon as:

POLYGON((100.0+0.0,101.0+0.0,101.0+1.0,100.0+1.0,100.0+0.0))

It's a bit shorter and it's a little more readable. It just so happens that there is an API defined that already has this plus sign transition as part of the spec and I talk about that below.

OGC's current effort

OGC is a professional organization that develops "standards for geospatial and location based services." That is a very noble goal and for the most part the REST API for WMS and WFS works well. However, the standards OGC defined for performing spatial queries and retrieving spatial data over the web are not reasonable for developers that want to use them.

As I have stated, JSON is not designed to be part of a clean REST specification but it's certainly better than XML, XML is an even less reasonable approach because the OGC specification requires you to not only send the shape as XML but an entire GML filter including XML namespace definitions. GeoServer's documentation gives an example of a simple query to get any shapes that intersect a point (as seen by your web server):

filter=%3CFilter%20xmlns:gml=%22http://www.opengis.net/gml%22%3E%3CIntersects%3E%3CPropertyName%3Ethe_geom%3C/PropertyName%3E%3Cgml:Point%20srsName=%224326%22%3E%3Cgml:coordinates%3E-74.817265,40.5296504%3C/gml:coordinates%3E%3C/gml:Point%3E%3C/Intersects%3E%3C/Filter%3E

At the very least, Esri's REST API breaks out the filter parts into different query arguments. Beyond that, it replaces XML with JSON and includes a couple shortcuts for bounding box and point queries.

Esri prefixes in an open standard?

Esri's API has "esri" written all over it. If OGC were to adopt Esri's REST API, I would hope they could scrub out this branding. When using an open standard, I do not expect to ask the web server whether my esriGeometryPoint data esriSpatialRelIntersects any esriGeometryPolygons. [James Fee linked to an OSGeo list message from someone with similar concerns.]

If OGC were to adopt the REST API, but on the condition that the esri prefix be removed, I would be confused if they didn't also go the extra step and adopt GeoJSON as the return format.

GeoJSON returns

While JSON is not a good request format but it is an incredibly good response format. It is simple and very clearly defined at geojson.org.

Esri's REST API does return JSON, which is good, but the objects it returns are a bit heavy and patterned after Esri's own data formats. Two examples jump out immediately:

The first is that Esri's response includes the field names of a result set three times: in the fieldAliases property, in the fields property and again in every feature returned. It also lists the field aliases twice. GeoJSON does not specify aliases and only returns the field names with each feature. The web developers decide if they need this metadata and if so, manage it however they want.

The second quickly noticeable problem is that all results must be from the same layer and of the same type. It's one thing if this results from a limitation of your back end storage but another if it's part of your API specification, forcing all future implementations to follow suit. All responses in the Esri specification return a single geometryType property. No modern spatial database servers require this and it is not a limitation in GeoServer's layer groups. Each feature in GeoJSON has its own geometry with its own geometry type.

It's not the only option

It's good to see that OGC is looking to pave cowpaths, especially given W3C's recent success with garnering interest in HTML5. However, I'm concerned they will choose a single company's offering, ignoring other efforts within the spatial community. I would like to remind developers that other groups and specifications exist and are being used in open source communities, two of which I describe below.

There are definitely parts of the Esri REST API worth looking at and incorporating. While it's important that OGC define a more useful API than what they currently recommend, I suggest taking Esri's approach as a basis for research and either extending one of the existing open source options or defining an new one rather than fully adopting Esri's.

GeoServer

GeoServer returns GeoJSON and allows OGC's XML-based queries. However, they also support a Common Query Language filter which uses Well-Known Text for a "much more compact and human readable syntax compared to OGC filters." This specification was, in fact, created by OGC for use with managing catalog services. GeoServer has repurposed it for spatial data queries. It has limitations but it is a very good start that should be given a chance.

GeoCouch

I am new to GeoCouch, but every time I hear a new detail about how it works, I realize it's very close to what I want as a web developer.

GeoCouch can return GeoJSON. Because of their design document approach, it can easily return your source data in any format you want. For a spatial query API, GeoCouch chose the OpenSearch Geo Extension API which uses primarily Well-Known Text but has shorthand queries for bounding box, point and radius searches. It also allows a plus instead of a space to make WKT queries look just a little nicer.

Will anyone notice?

The open source community appears to be perfectly happy doing their own thing if they don't like a specification. Had the W3C chosen to document only IE's implementations of CSS3 and HTML5, the other browser vendors and many web designers would continue to ignore them in favor of WHATWG.org. In the open spatial community, GeoServer & GeoCouch can both return GeoJSON and allow spatial queries in a format better suited to a REST API. OpenSearch could (and maybe should) submit their implementation to OGC, but again, I'm not sure how much it will matter. If OGC adopts the Esri REST API as a standard, GeoServer will implement it but I don't think they will throw away CQL. I don't think anyone will implement it for GeoCouch, which is fine.

Round trip

The open source spatial community should decide on a better REST API. Esri offered theirs but I don't think it's the best option for various reasons. I've mentioned two great projects to use as starting points and I am sure there are more out there. Regardless of API, they should settle on GeoJSON as a response format, as many individual spatial developers have already done.

I welcome your comments below, or by email or Twitter.

blog comments powered by Disqus

References