ArcGIS Open Data

For a year, ESRI’s open data initiative is online. As I’ve heard about it for the first time I was just thinking: nice move to get people into their ArcGIS online environment. But I have to admit: I was not looking and exploring their platform. Now I stumbled upon a recent post on reddit and there it was again:

opendata.arcgis.com has 25,000+ open datasets all accessible by a common API. Full disclosure: I work on the team that builds the product.

ArcGIS Open Data – landing page

The landing page is a beauty and provides you direct access to a vast amount of data. The search function is the center point of the webpage and is accompanied with a small selection of data providers (institutions that import their data to the open data platform). Furthermore you see “Noteworthy Data” and “Newest Data”. My only critique about this: Why are the building outlines of the City of Chesapeake (233k population) noteworthy if you can have the number of floors of each building in a city with nearly 4milion inhabitants….

ArcGIS Open Data – the search

The search is fast and results are well presented in a map (coverage) as well as in a list with a small description (if and only if the data provider provides this as well).

You can select your dataset by clicking on the map coverage or on one of the list items to see more data. Due to the fact that not all regions of the world are able or willing to provide data in an open form the quality and quantity of search results depends of the region. Where searching for “Berlin” gave me nearly 50 results the search for “Paris” yielded 2 results only. Furthermore the map usage is a bit hard as larger datasets might “cover” datasets with a smaller spatial extent and also leads to a click/select event on the map.

ArcGIS Open Data – the result

Here come the great and somehow crucial part. The presentation on screen is great! You have a map view with either points, lines or polygons on the map. ESRI included a hover function and a popup which shows the attribute table of each object/feature so you can get an idea about the features itself.

Furthermore you can view the whole attribute table and filter either by extent or by attributes. This can take some time (in this example up to 2 minutes: 380 objects out of 530.000) to see the table in the UI. The great disadvantage: You can’t abort the selection query and you are better in closing the tab and using a new one for a new query. The table view supports pie, line and bar charts as well as histograms but is also limited on the query. So it is easy to kill your browser and the working scripts with it!

ArcGIS Open Data – getting data

As it comes to open data my first thought is always: can I download th data and store it physically. Especially in this casemy second question was: Do I need to have an AGOL account to import the data into my analysis. As a big surprise you don’t need to have an AGOL account neither an ESRI account nor any other account to get the data. You just simply click on download and decide whether you want all the data or just the filtered one as shapefile, KML, spreadsheet or . Once you’ve done this, your task is added to the process list and you will be notified when the task is progressed. Additionally you can choose to get data via geojson or via WMS/WFS by clicking on API. You will see the links which you should use in your GIS of choice. This is the theoretical part. But whatch out, the user is coming… I was not able to download my wanted subset of “numbers of floor of each house in Berlin”. Off course i also failed in downloading the whole dataset. I have to admit that this dataset is quite large but downloading 380 features shouldn’t be this damn tough:

The export of smaller datasets went fine and also getting the JSON back from the API was no problem for these smaller datasets. When it comes to the GIS server option I was only able to get the data into ArcGIS but ffailed miserably in QGIS. But once in ArcGIS I was able to store a selection as shapefiles/geojson and we are free and open again: But I’ve heard, there is an indirect way to consume the ArcGIS REST API in QGIS using some GDAL magic.

ArcGIS Open Data – summary

The whole platform describes the standard for open data platform quite good: easy “search and get data” capabilities, support for open standards like geoJSON. But they really need to work on the performance when it comes to big data. Despite this I would like to work on with this portal and I prefer it much more compared to other data portals I have seen already (keeping the German ones in mind 😉 ). You want to judge by your own? Try it with some nice data from Maryland