Geek Answers Handbook

What is the way to embed huge JSON data in SQLite DB in android

I have a large json query, that is, it contains about 50 thousand rows with 15 columns, which I need to insert into SQLite DB with the same structure. This, I have to copy the same data allocated in postgres db to my sqlite db in my application. Is there an effective way to do this? is there any api or something that could help with the job?

I have to say that I can do with OMRLite with JSON data that is small, but when I try to do this with large ones, my application crashes and has a memory error.

Please, if you have any idea or some example that I could follow, I will be very grateful! thanks in advance!

+4
source share
3 answers

You can also use the official Google Gon Streaming library.

Gson Streaming: Gson Streaming

JsonReader plays a very important role for parsing json using the Streaming library.

+2
source

Since JSON is so large, you cannot completely load it into memory. Even using standard JSONObject will result in low memory on many devices.

What I did in a similar situation is to use Jackson to parse it. Jackson can do this from a thread, so memory usage is not a problem at all. The disadvantage is the API, which is not so easy to use compared to the usual parameters.

Here is an example I found: Jackson Stream

+1
source

GSON stream works for me Gson is streaming , but I have to say that it takes its time, about 4 minutes for 68K rows with 10 columns for example. But still solves my problem. So I have a JSON response:

 - preciosArtPK: { codLista: 1, codArticulo: 11348, cansiVenta: 1, fecVigencia: 1435781252000 }, siglaVenta: "UN", precioVenta: 0, margenPct: 100, codUsuario: 1, vigente: "S", nomModulo: "MIGRACION"

The above JSON is part of the array response, but I have these "preciosArtPK" for inclusion in serialization using gson. How could I do this? I have a class that handles my serialization:

 @DatabaseTable(tableName = "preciosart") public class PreciosArt { public static final String PRECIOS_COD_LISTA = "_id"; public static final String PRECIOS_COD_ARTICULO = "cod_articulo"; public static final String PRECIOS_CANSI_VENTA = "cansi_venta"; public static final String PRECIOS_FEC_VIGENCIA = "fec_vigencia"; public static final String PRECIOS_SIGLA_VENTA = "sigla_venta"; public static final String PRECIOS_PRECIO_VENTA = "precio_venta"; public static final String PRECIOS_MARGEN_PCT = "margen_pct"; public static final String PRECIOS_COD_USUARIO = "cod_usuario"; public static final String PRECIOS_VIGENTE = "vigente"; public static final String PRECIOS_NOM_MODULO = "nom_modulo"; @DatabaseField(id = true, unique = true, columnName = PRECIOS_COD_LISTA) private Integer codLista; @DatabaseField(unique = true, columnName = PRECIOS_COD_ARTICULO) @SerializedName("codArticulo") // probar private Integer codArticulo; @DatabaseField(unique = true, columnName = PRECIOS_CANSI_VENTA) private Integer cansiVenta; @DatabaseField(unique = true, columnName = PRECIOS_FEC_VIGENCIA) private Long fecVigencia; @DatabaseField(columnName = PRECIOS_SIGLA_VENTA) @SerializedName("siglaVenta") private String siglaVenta; @DatabaseField(columnName = PRECIOS_PRECIO_VENTA) @SerializedName("precioVenta") private Double precioVenta; @DatabaseField(columnName = PRECIOS_MARGEN_PCT) @SerializedName("margenPct") private Float margenPct; @DatabaseField(columnName = PRECIOS_COD_USUARIO) @SerializedName("codUsuario") private Integer codUsuario; @DatabaseField(columnName = PRECIOS_VIGENTE) @SerializedName("vigente") private String vigente; @DatabaseField(columnName = PRECIOS_NOM_MODULO) @SerializedName("nomModulo") private String nomModulo;

but it does not fill in these fields (codArticulo, cansiVenta and fecVigencia). I read about deserializing these json formats creating another class, so I did the same:

 @SerializedName("codLista") private Integer codLista; @SerializedName("codArticulo") private Integer codArticulo; @SerializedName("cansiVenta") private Integer cansiVenta; @SerializedName("fecVigencia") private Long fecVigencia;

The problem is this: how could I populate this field using json deserialized? I am using OMRLite to do this job, this class is for this purpose:

 public long updatePreciosart(PreciosArt preciosArt){ long resultUpdate = -1; try { getHelper().getPreciosartDao().createOrUpdate(preciosArt); resultUpdate = 0; } catch (SQLException e) { e.printStackTrace(); resultUpdate = -1; } return resultUpdate; }

I hope you understand what the problem is, and I hope that you will help me! Thanks again!

0
source

More articles:


All Articles