Geek Answers Handbook

Download AppEngine GeoPt Data

I am writing a GAE application in Java and use only Python for the data to load. I am trying to import a CSV file that looks like this:

POSTAL_CODE_ID,PostalCode,City,Province,ProvinceCode,CityType,Latitude,Longitude 1,A0E2Z0,Monkstown,Newfoundland,NL,D,47.150300000000001,-55.299500000000002

I managed to import this file into the data warehouse if I import Latitude and longitude as it floats, but I find it difficult to understand how import lat and lng are like GeoPt. Here is my loader.py file:

 import datetime from google.appengine.ext import db from google.appengine.tools import bulkloader class PostalCode(db.Model): id = db.IntegerProperty() postal_code = db.PostalAddressProperty() city = db.StringProperty() province = db.StringProperty() province_code = db.StringProperty() city_type = db.StringProperty() lat = db.FloatProperty() lng = db.FloatProperty() class PostalCodeLoader(bulkloader.Loader): def __init__(self): bulkloader.Loader.__init__(self, 'PostalCode', [('id', int), ('postal_code', str), ('city', str), ('province', str), ('province_code', str), ('city_type', str), ('lat', float), ('lng', float) ]) loaders = [PostalCodeLoader]

I think the two lines of db.FloatProperty () should be replaced with a db.GeoPtProperty (), but where my trail ends. I am very new to Python, so any help would be greatly appreciated.

+4
source share
5 answers

Ok, I got a response from the Google Group (thanks to Takashi Matsuo and Mike Armstrong). The solution is to modify the CSV file and concatenate lat and lng into a string with two quotation marks. A comma in a double-quoted string will not be considered a CSV delimiter.

 POSTAL_CODE_ID,PostalCode,City,Province,ProvinceCode,CityType,Point 1,A0E 2Z0,Monkstown,Newfoundland,NL,D,"47.150300000000001,-55.299500000000002"

Also, here is my new loader.py. Note that GeoPtProperty accepts a string with "00.0000,00.0000":

 import datetime from google.appengine.ext import db from google.appengine.tools import bulkloader class PostalCode(db.Model): id = db.IntegerProperty() postal_code = db.PostalAddressProperty() city = db.StringProperty() province = db.StringProperty() province_code = db.StringProperty() city_type = db.StringProperty() geo_pt = db.GeoPtProperty() class PostalCodeLoader(bulkloader.Loader): def __init__(self): bulkloader.Loader.__init__(self, 'PostalCode', [('id', int), ('postal_code', str), ('city', str), ('province', str), ('province_code', str), ('city_type', str), ('geo_pt', str) ]) loaders = [PostalCodeLoader]
+3
source

I don't know what your bootloader code is, but ...

 # given this class PostalCode(db.Model): id = db.IntegerProperty() postal_code = db.PostalAddressProperty() city = db.StringProperty() province = db.StringProperty() province_code = db.StringProperty() city_type = db.StringProperty() geoLocation = db.GeoPtProperty() # you should be able to do this myPostalCode.geoLocation = db.GeoPt(-44.22, -33.55)

more here

0
source

Avoid casting types and test instances. I use both geotarg and geohash http, as well as similar ones where deafness values ​​are recommended:

  geopt=db.GeoPtProperty(verbose_name="geopt")

...

  article.geopt = db.GeoPt(self.request.POST.get('lat'),self.request.POST.get('lng')) article.geohash = Geohash.encode(float(lat),float(lng), precision=2)#evalu8 precision variable

code disponible

demo application

0
source

You can define your own loader that will combine the two columns from cvs into one value, and then write a converter function that parses that value on db.GeoPt. In this solution, you do not need to change the csv file. Here is an example (assuming the csv file has only three columns - lat, lng and some name):

 import csv from google.appengine.ext import db from google.appengine.tools import bulkloader class GeoPoint(db.Model): name = db.StringProperty() location = db.GeoProperty() class GeoFileLoader(bulkloader.Loader): ''' Loader class processing input csv file and merging two columns into one ''' def __init__(self, kind_name, converters): bulkloader.Loader.__init__(self, kind_name, converters) def generate_records(self, filename): csv_reader = csv.reader(open(filename), delimiter=',') for row in csv_reader: if row: lat = row[0] lng = row[1] # Here we yield only one value for geo coordinates and name unchanged yield '%s,%s' % (lat, lng), row[2] def geo_converter(geo_str): ''' Converter function - return db.GeoPt from str ''' if geo_str: lat, lng = geo_str.split(',') return db.GeoPt(lat=float(lat), lon=float(lng)) return None # Loader that uses our GeoFileLoader to load data from csv class PointLoader(GeoFileLoader): def __init__(self): GeoFileLoader.__init__(self, 'GeoPoint', [('location', geo_converter), ('name', str)]) loaders = [PointLoader]

You can find more information on Nick Johnson’s blog.

0
source

This question should be deleted / outdated. Python is no longer used for the loader. Now only yaml files are used. See Q: Importing GeoPt Data Using Google AppEngine BulkLoader NMR for the answer to this using a modern bootloader.

0
source

More articles:


All Articles