Error when importing into GeoDjango

[scalar]

Hello,

I am just starting a geodjango project that imports all the data into a spatial index. I will post it on github soon, but I am still having problems importing 10m-admin-1-states-provinces-shp.shp.

I get the following error:

Saved: United States

Saved: United States

Saved: United States

Warning 1: organizePolygons() received an unexpected geometry.

Either a polygon with interior rings, or a polygon with less than 4 points,

or a non-Polygon geometry. Return arguments as a collection.

Warning 1: Geometry of polygon of fid 22 cannot be translated to Simple Geometry. All polygons will be contained in a multipolygon.

GEOS_ERROR: IllegalArgumentException: Number of points must be 0 or >3

Failed to save the feature (id: 22) into the model with the keyword arguments:

{‘name_1′: u’Hawaii’, ‘region_cod’: u”, ‘prov_number’: 7, ‘nl_name_1’: u”, ‘first_fips’: u”, ‘shape_leng’: 13.354688171999999, ‘type_1′: u’State’, ‘country_pr’: u”, ‘gadm_level’: 1.0, ‘scale_rank’: 1, ‘validto_1′: u’Present’, ‘remarks_1’: u”, ‘region_c_1’: u”, ‘region_var’: u”, ‘varname_1′: u’HI|Hawaii’, ‘engtype_1′: u’State’, ‘hasc_1′: u’US.HI’, ‘region_c_2’: u”, ‘check_me’: 0, ‘validfr_1′: u’18980812’, ‘first_hasc’: u”, ‘region_c_3’: u”, ‘shape_area’: -0.35784454833599999, ‘name’: u’United States’, ‘region’: u”, ‘vertex_cou’: 550.0, ‘nev_countr’: u’United States’, ‘fips_1′: u’US15’, ‘mpoly’:

…..

File “/home/philipp/pinax0.7-env/lib/python2.6/site-packages/django/contrib/gis/geos/prototypes/errcheck.py”, line 32, in check_geom

raise GEOSException(‘Error encountered checking Geometry returned from GEOS C function “%s”.’ % func.__name__)

django.contrib.gis.geos.error.GEOSException: Error encountered checking Geometry returned from GEOS C function “GEOSWKTReader_read”.

My model looks like:

class County(models.Model):

vertex_cou = models.FloatField()

name = models.CharField(max_length=75)

name_1 = models.CharField(max_length=75)

varname_1 = models.CharField(max_length=75)

nl_name_1 = models.CharField(max_length=75)

hasc_1 = models.CharField(max_length=15)

type_1 = models.CharField(max_length=50)

engtype_1 = models.CharField(max_length=50)

validfr_1 = models.CharField(max_length=25)

validto_1 = models.CharField(max_length=15)

remarks_1 = models.CharField(max_length=125)

region = models.CharField(max_length=100)

region_var = models.CharField(max_length=254)

region_cod = models.CharField(max_length=50)

region_c_1 = models.CharField(max_length=50)

region_c_2 = models.CharField(max_length=50)

region_c_3 = models.CharField(max_length=50)

country_pr = models.CharField(max_length=128)

scale_rank = models.IntegerField()

check_me = models.IntegerField()

gadm_level = models.FloatField()

fips_1 = models.CharField(max_length=254)

first_fips = models.CharField(max_length=254)

first_hasc = models.CharField(max_length=254)

nev_countr = models.CharField(max_length=100)

prov_number = models.IntegerField()

shape_leng = models.FloatField()

shape_area = models.FloatField()

mpoly = models.MultiPolygonField(srid=4326)

objects = models.GeoManager()

def __unicode__(self):

return self.name

and the layer mapping:

county_mapping = {

‘name’ : ‘NAME_0’,

‘name_1’: ‘NAME_1’,

‘varname_1’ : ‘VARNAME_1’,

‘nl_name_1’ : ‘NL_NAME_1’,

‘vertex_cou’ : ‘VertexCou’,

‘hasc_1’: ‘HASC_1’,

‘type_1′:’TYPE_1’,

‘engtype_1’: ‘ENGTYPE_1’,

‘validfr_1’: ‘VALIDFR_1’,

‘validto_1’: ‘VALIDTO_1’,

‘remarks_1′:’REMARKS_1’,

‘region’: ‘Region’,

‘region_var’:’RegionVar’,

‘region_cod’: ‘Region_Cod’,

‘region_c_1’: ‘Region_C_1’,

‘region_c_2’: ‘Region_C_2’,

‘region_c_3’: ‘Region_C_3’,

‘country_pr’: ‘Country_Pr’,

‘scale_rank’: ‘ScaleRank’,

‘check_me’: ‘CheckMe’,

‘gadm_level’: ‘gadm_level’,

‘fips_1’: ‘FIPS_1’,

‘first_fips’: ‘FIRST_FIPS’,

‘first_hasc’: ‘FIRST_HASC’,

‘nev_countr’: ‘NEV_Countr’,

‘prov_number’: ‘ProvNumber’,

‘shape_leng’: ‘Shape_Leng’,

‘shape_area’ : ‘Shape_Area’,

‘mpoly’ : ‘MULTIPOLYGON’,

}

should I maybe be using another geometry instead of MultiPolygon?

Once this works I will post my project on github.

Thanks,

Philipp

[Nathaniel]

@ Philipp: I’ve since run a “Repair Geometry” routine on the admin-1 that should remove some of these errors. You can find that beta file here:

http://www.nacis.org/naturalearth/temp/10m_admin_1_version1b1.zip

Let me know if this new file imports for you.

_Nathaniel

[scalar]

thanks for the help, it worked now.

I posted the first part of my work here:

http://github.com/philippWassibauer/django-natural-earth

I hope other people are interested too who will help me keeping this up to date.

I would also like to incorporate Geoname Data. Eg.: I would need altitude numbers globally. Are you going to incorporate geoname data into your DB or should I create an app that imports data from both sources?

Thanks,

Philipp Wassibauer

[Nathaniel]

@ Philipp: Yes, I would like to incorporate GeoNames feature IDs on all Natural Earth features. Yes, please try creating an app that imports from both sources! Preserve the original NE feature IDs so we can bring your linkages back into the main data branch. I’ve also reached out to Yahoo! GeoID folks, I’d like to get both, will do wonders for semantic web. I’m also curious about GeoNames as they seem to link to Wikipedia. Is that hard coded or softcoded in their DB? Might be nice to get that as well as the GeoNames ID.

Please describe more: I would need altitude numbers globally. Do you mean elevations above sea level or something else? Is this important for matching with GeoNames? Thanks!

[Nathaniel]

Also, by mid January I should have cleaned the geometry on several of the other polygons files in Natural Earth so it’s easier to import into GeoDjango and other more strict geo database apps.

[dhilton]

I tried using the amended data but to no avail.

I got the following error on django 1.1.1:

./manage.py ne_import_countries ./data/10m_admin_1_version1b1.shp

Datasource has 1 layers

Importing Layer 10m_admin_1_version1b1 of type Polygon, size: 3786

Traceback (most recent call last):

File "./manage.py", line 16, in <module>

execute_manager(settings)

File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/core/management/__init__.py", line 362, in execute_manager

utility.execute()

File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/core/management/__init__.py", line 303, in execute

self.fetch_command(subcommand).run_from_argv(self.argv)

File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/core/management/base.py", line 195, in run_from_argv

self.execute(*args, **options.__dict__)

File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/core/management/base.py", line 222, in execute

output = self.handle(*args, **options)

File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/core/management/base.py", line 322, in handle

label_output = self.handle_label(label, **options)

File "./django-natural-earth/natural_earth/management/commands/ne_import_countries.py", line 35, in handle_label

transform=False, encoding='iso-8859-1')

File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/contrib/gis/utils/layermapping.py", line 187, in __init__

self.check_layer()

File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/contrib/gis/utils/layermapping.py", line 319, in check_layer

idx = check_ogr_fld(ogr_name)

File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/contrib/gis/utils/layermapping.py", line 265, in check_ogr_fld

raise LayerMapError('Given mapping OGR field "%s" not found in OGR Layer.' % ogr_map_fld)

django.contrib.gis.utils.layermapping.LayerMapError: Given mapping OGR field "SHAPE_AREA" not found in OGR Layer.

[Nathaniel]

Do other layers like the rivers import for you? I don’t have any experience with GeoDjango

The admin_0 and admin_1 are tough eggs to crack. The topology is a little messy right now. We hope to have that fixed in January.

[dhilton]

It’s not so much a geodjango issue as it is a PostGIS / property issue. The django app uses a specfic mapping of shapefile properties to internal database columns; change the fields and the db schema and it will load the data fine. The problem comes when you try and use the data in a meaningful way. I’ve used ogrinfo and qGis to inspect the properties of the shapefile but can’t seem to find much to help me. At the moment the app loads all of the counties but they’re not within their countries nor have their administrative names attached to them. Any ideas or pointers about where I can learn more about this so I can help?

[Nathaniel]

For the admin-0 the columns with the administrative names of the countries are in the Subunit, MapUnit, Country, and Sov columns. Do those load into your system? A forthcoming update will include more thematic data for each admin-0 feature.

[Author]

Posts