Creating a heatmap – looks easy, becomes difficult

posted in: Practical Course | 0

For our App, we tried to get theft data from different stakeholders like the BKA. They provided us a huge table containing different kinds of thefts on a level of “province keys”, so we wanted to illustrate it on a map of Germany. With some research, we managed to map it on a postal code level because most of the possible tools would be able to work with postal codes. On a personal advice, we decided on ArcGIS as the tool of our choice for visualization of the data. As a positive point, TUM is giving away one-year student licenses, but we will later find out, that it is a very limited kind of license. For the first tool, ArcMap Online we came quickly to the first problems. The data formats usable for importing data are csv files or gpx files, so we decided to simply export our table to csv. On import screen, we were informed, that only 250 lines at once could be imported, but we had over 7000 lines of data in our table. Furthermore, we could identify, that postal codes are supported, but many of them either were not recognized or their GPS position in its backend seeming to be wrong. As a result of it, we did some research to map the postal codes to their center GPS position on our own. Due to the fact of limited lines on import with csv files, we tested the gpx format regarding the appointment of unlimited import from the import dialog of ArcMap Online. Thanks to some online conversion websites, it seems to get an easy alternative, but while importing it to ArcMap Online, we figured out, that only one GPS position was imported. So we decided to test some Desktop alternatives.

The student download page of Esri, the parent organization of ArcGIS, it looks like we could decide on a wide range of different products for this. Sadly, only ArcMap Desktop is usable with the license. After some struggling with the program- it crashes often on many different tasks- we managed to import an Excel table via conversion manager. The next problem followed ad-hoc, our GPS positions are not working with every coordinate system representation. After finding out, that the representation we are looking for would be the ETRF representation, we managed to get the different centers shown on a map. After another research, it turned out, that the classical heatmap feature is only implemented in ArcGIS Pro, which is not included in our license.  As a workaround, the community presented the approach of using the point density from the spatial analyst tools of ArcMap Desktop, but even this Addon was not covered by our license.

So in the end, we came back to ArcMap Online, the product with the 250 lines of data per file limit. After some time, the initial file with GPS positions was split into a huge amount of files with a maximum of 250 line per file. After importing them we came to the next problem. Due to the fact, that the points are now in many files, it is impossible to apply the same value range. Furthermore, Germany has an asymmetrical distribution of postal codes. As a result of it, we decided to only map the “province keys” to its center GPS position, which results in a file with about 400 lines of data. Whereas every file should have a limit of about 250 lines, we had to split it. We tried a splitting on west and east and one on south and north. Through viewing the result of both of them, we chose the north/south variant.

After that, we came to the result, that many of the problems would have been cut out if the license would content ArcGIS Pro.

Leave a Reply

You must be logged in to post a comment.