Error when importing into GeoDjango

This topic contains 8 replies, has 1 voice, and was last updated by  Nathaniel 7 years, 11 months ago.

Viewing 9 posts - 1 through 9 (of 9 total)
  • Author
    Posts
  • #3352

    scalar
    Participant

    Hello,

    I am just starting a geodjango project that imports all the data into a spatial index. I will post it on github soon, but I am still having problems importing 10m-admin-1-states-provinces-shp.shp.

    I get the following error:

    Saved: United States

    Saved: United States

    Saved: United States

    Warning 1: organizePolygons() received an unexpected geometry.

    Either a polygon with interior rings, or a polygon with less than 4 points,

    or a non-Polygon geometry. Return arguments as a collection.

    Warning 1: Geometry of polygon of fid 22 cannot be translated to Simple Geometry. All polygons will be contained in a multipolygon.

    GEOS_ERROR: IllegalArgumentException: Number of points must be 0 or >3

    Failed to save the feature (id: 22) into the model with the keyword arguments:

    {‘name_1′: u’Hawaii’, ‘region_cod’: u”, ‘prov_number’: 7, ‘nl_name_1’: u”, ‘first_fips’: u”, ‘shape_leng’: 13.354688171999999, ‘type_1′: u’State’, ‘country_pr’: u”, ‘gadm_level’: 1.0, ‘scale_rank’: 1, ‘validto_1′: u’Present’, ‘remarks_1’: u”, ‘region_c_1’: u”, ‘region_var’: u”, ‘varname_1′: u’HI|Hawaii’, ‘engtype_1′: u’State’, ‘hasc_1′: u’US.HI’, ‘region_c_2’: u”, ‘check_me’: 0, ‘validfr_1′: u’18980812’, ‘first_hasc’: u”, ‘region_c_3’: u”, ‘shape_area’: -0.35784454833599999, ‘name’: u’United States’, ‘region’: u”, ‘vertex_cou’: 550.0, ‘nev_countr’: u’United States’, ‘fips_1′: u’US15’, ‘mpoly’:

    …..

    File “/home/philipp/pinax0.7-env/lib/python2.6/site-packages/django/contrib/gis/geos/prototypes/errcheck.py”, line 32, in check_geom

    raise GEOSException(‘Error encountered checking Geometry returned from GEOS C function “%s”.’ % func.__name__)

    django.contrib.gis.geos.error.GEOSException: Error encountered checking Geometry returned from GEOS C function “GEOSWKTReader_read”.

    My model looks like:

    class County(models.Model):

    vertex_cou = models.FloatField()

    name = models.CharField(max_length=75)

    name_1 = models.CharField(max_length=75)

    varname_1 = models.CharField(max_length=75)

    nl_name_1 = models.CharField(max_length=75)

    hasc_1 = models.CharField(max_length=15)

    type_1 = models.CharField(max_length=50)

    engtype_1 = models.CharField(max_length=50)

    validfr_1 = models.CharField(max_length=25)

    validto_1 = models.CharField(max_length=15)

    remarks_1 = models.CharField(max_length=125)

    region = models.CharField(max_length=100)

    region_var = models.CharField(max_length=254)

    region_cod = models.CharField(max_length=50)

    region_c_1 = models.CharField(max_length=50)

    region_c_2 = models.CharField(max_length=50)

    region_c_3 = models.CharField(max_length=50)

    country_pr = models.CharField(max_length=128)

    scale_rank = models.IntegerField()

    check_me = models.IntegerField()

    gadm_level = models.FloatField()

    fips_1 = models.CharField(max_length=254)

    first_fips = models.CharField(max_length=254)

    first_hasc = models.CharField(max_length=254)

    nev_countr = models.CharField(max_length=100)

    prov_number = models.IntegerField()

    shape_leng = models.FloatField()

    shape_area = models.FloatField()

    mpoly = models.MultiPolygonField(srid=4326)

    objects = models.GeoManager()

    def __unicode__(self):

    return self.name

    and the layer mapping:

    county_mapping = {

    ‘name’ : ‘NAME_0’,

    ‘name_1’: ‘NAME_1’,

    ‘varname_1’ : ‘VARNAME_1’,

    ‘nl_name_1’ : ‘NL_NAME_1’,

    ‘vertex_cou’ : ‘VertexCou’,

    ‘hasc_1’: ‘HASC_1’,

    ‘type_1′:’TYPE_1’,

    ‘engtype_1’: ‘ENGTYPE_1’,

    ‘validfr_1’: ‘VALIDFR_1’,

    ‘validto_1’: ‘VALIDTO_1’,

    ‘remarks_1′:’REMARKS_1’,

    ‘region’: ‘Region’,

    ‘region_var’:’RegionVar’,

    ‘region_cod’: ‘Region_Cod’,

    ‘region_c_1’: ‘Region_C_1’,

    ‘region_c_2’: ‘Region_C_2’,

    ‘region_c_3’: ‘Region_C_3’,

    ‘country_pr’: ‘Country_Pr’,

    ‘scale_rank’: ‘ScaleRank’,

    ‘check_me’: ‘CheckMe’,

    ‘gadm_level’: ‘gadm_level’,

    ‘fips_1’: ‘FIPS_1’,

    ‘first_fips’: ‘FIRST_FIPS’,

    ‘first_hasc’: ‘FIRST_HASC’,

    ‘nev_countr’: ‘NEV_Countr’,

    ‘prov_number’: ‘ProvNumber’,

    ‘shape_leng’: ‘Shape_Leng’,

    ‘shape_area’ : ‘Shape_Area’,

    ‘mpoly’ : ‘MULTIPOLYGON’,

    }

    should I maybe be using another geometry instead of MultiPolygon?

    Once this works I will post my project on github.

    Thanks,

    Philipp

    #3933

    Nathaniel
    Keymaster

    @ Philipp: I’ve since run a “Repair Geometry” routine on the admin-1 that should remove some of these errors. You can find that beta file here:

    http://www.nacis.org/naturalearth/temp/10m_admin_1_version1b1.zip

    Let me know if this new file imports for you.

    _Nathaniel

    #3934

    scalar
    Participant

    thanks for the help, it worked now.

    I posted the first part of my work here:

    http://github.com/philippWassibauer/django-natural-earth

    I hope other people are interested too who will help me keeping this up to date.

    I would also like to incorporate Geoname Data. Eg.: I would need altitude numbers globally. Are you going to incorporate geoname data into your DB or should I create an app that imports data from both sources?

    Thanks,

    Philipp Wassibauer

    #3935

    Nathaniel
    Keymaster

    @ Philipp: Yes, I would like to incorporate GeoNames feature IDs on all Natural Earth features. Yes, please try creating an app that imports from both sources! Preserve the original NE feature IDs so we can bring your linkages back into the main data branch. I’ve also reached out to Yahoo! GeoID folks, I’d like to get both, will do wonders for semantic web. I’m also curious about GeoNames as they seem to link to Wikipedia. Is that hard coded or softcoded in their DB? Might be nice to get that as well as the GeoNames ID.

    Please describe more: I would need altitude numbers globally. Do you mean elevations above sea level or something else? Is this important for matching with GeoNames? Thanks!

    #3936

    Nathaniel
    Keymaster

    Also, by mid January I should have cleaned the geometry on several of the other polygons files in Natural Earth so it’s easier to import into GeoDjango and other more strict geo database apps.

    #3937

    dhilton
    Participant

    I tried using the amended data but to no avail.

    I got the following error on django 1.1.1:

    ./manage.py ne_import_countries ./data/10m_admin_1_version1b1.shp
    Datasource has 1 layers
    Importing Layer 10m_admin_1_version1b1 of type Polygon, size: 3786
    Traceback (most recent call last):
    File "./manage.py", line 16, in <module>
    execute_manager(settings)
    File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/core/management/__init__.py", line 362, in execute_manager
    utility.execute()
    File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/core/management/__init__.py", line 303, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
    File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/core/management/base.py", line 195, in run_from_argv
    self.execute(*args, **options.__dict__)
    File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/core/management/base.py", line 222, in execute
    output = self.handle(*args, **options)
    File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/core/management/base.py", line 322, in handle
    label_output = self.handle_label(label, **options)
    File "./django-natural-earth/natural_earth/management/commands/ne_import_countries.py", line 35, in handle_label
    transform=False, encoding='iso-8859-1')
    File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/contrib/gis/utils/layermapping.py", line 187, in __init__
    self.check_layer()
    File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/contrib/gis/utils/layermapping.py", line 319, in check_layer
    idx = check_ogr_fld(ogr_name)
    File "/var/www/python/mapy/lib/python2.6/site-packages/Django-1.1.1-py2.6.egg/django/contrib/gis/utils/layermapping.py", line 265, in check_ogr_fld
    raise LayerMapError('Given mapping OGR field "%s" not found in OGR Layer.' % ogr_map_fld)
    django.contrib.gis.utils.layermapping.LayerMapError: Given mapping OGR field "SHAPE_AREA" not found in OGR Layer.

    #3938

    Nathaniel
    Keymaster

    Do other layers like the rivers import for you? I don’t have any experience with GeoDjango :

    The admin_0 and admin_1 are tough eggs to crack. The topology is a little messy right now. We hope to have that fixed in January.

    #3939

    dhilton
    Participant

    It’s not so much a geodjango issue as it is a PostGIS / property issue. The django app uses a specfic mapping of shapefile properties to internal database columns; change the fields and the db schema and it will load the data fine. The problem comes when you try and use the data in a meaningful way. I’ve used ogrinfo and qGis to inspect the properties of the shapefile but can’t seem to find much to help me. At the moment the app loads all of the counties but they’re not within their countries nor have their administrative names attached to them. Any ideas or pointers about where I can learn more about this so I can help?

    #3940

    Nathaniel
    Keymaster

    For the admin-0 the columns with the administrative names of the countries are in the Subunit, MapUnit, Country, and Sov columns. Do those load into your system? A forthcoming update will include more thematic data for each admin-0 feature.

Viewing 9 posts - 1 through 9 (of 9 total)

You must be logged in to reply to this topic.