I have 1048 csv data lines that I need to read. The data is as follows:
\N,Olenevka,,2596 \N,Urzuf,,2904 \N,Lebedyn,,27695 \N,Staryy Dobrotvor,Staryy Dobrotvor,6436
Here is the table:
+------------+------------------+------+-----+---------+----------------+ | Field | Type | Null | Key | Default | Extra | +------------+------------------+------+-----+---------+----------------+ | city_id | int(10) unsigned | NO | PRI | NULL | auto_increment | | name | text | YES | | NULL | | | alt_name | text | YES | | NULL | | | population | int(10) unsigned | NO | | NULL | | +------------+------------------+------+-----+---------+----------------+
Here is the code:
db.query(""" LOAD DATA INFILE '%s' INTO TABLE %s CHARACTER SET utf8 FIELDS TERMINATED BY ',' (@skip, name, alt_name, population) """ % (temp_file, table)) class Database(object): def __init__(self, database): self.database = database def query(self, cmd): return subprocess.call(['mysql -u root -e "%s" %s' % (cmd, self.database)], shell=True)
When I issue a command from python, it only reads in 1025 lines. But if I issue the same command from the mysql prompt, I get them all.
Here are the last lines from SELECT * after running the code:
| 1023 | Alchevs'k | | 116000 | | 1024 | Yakymivka | і | 12353 | | 1025 | Okhtyrka | | 49818 | | 1026 | Adzhamka | | 0 | +---------+------------------+-------------------------------------+------------+
The last line is only partially read. Here is the rest of the file.
\N,Okhtyrka,,49818 \N,Adzhamka,,3859 \N,Malynivka,і,7502 \N,Mykhaylivka,Mykhaylivka,3352 \N,Kopashnovo,,3010 \N,Narkevychi,і,1631 \N,Kirove,і,3291 \N,Orikhove,і,3000 \N,Krasni Okny,і ,5461 \N,Pavlivka,Pavlovka,738 \N,Kuchurhan,,3183 \N,Smyga,,2800 \N,Tayirove,ї,1849 \N,Teplodar,,8502 \N,Komsomolsk,,51740 \N,,,25000 \N,,,566 \N,Orzhiv,і,4125 \N,Shkil'ne,і,2127 \N,Denyshi,,1164 \N,Chetfalva,,755 \N,Stepove,,1549 \N,Scholkine,,11677 \N,Yuzhnoukrains'k,ї,39430
Ideas?