Smart or Not: Persist serialized data (dotnet-protobuf, protobuf-net, json) in a relational database in CF

Question

I started reading some posts related to protocol buffers. The serialization method seems to be very suitable for transferring data to and from web servers. Has anyone considered using this method to save and retrieve data on a mobile device itself? (i.e. replacing the traditional / orm database layer). We are currently using custom reflection-based orm. We want to get away from using reflection on mobile devices. And, since we still have to send / receive serialized data, this seems like a good option.

• Where will the data be stored?
• How to request data?

Does it make sense to store data in a traditional database (SqlCE or SqlLite) with multiple searchable columns and then one column for serialized data?

Thoughts? Am I on a horse here?

Update: the same “theory” can work for other types of serialized data ... JSON for example. I could not find the NoSQL parameter to store and query serialized data on the Compact Framework. I would be interested in this option if anyone knows about this.

Comment on object databases I tried both db4o and Perst. To work with db4o was absolutely wonderful. I used it in “real life,” and the performance, usability, and serviceability were excellent. Their licensing fees for our situation were what I find outrageous. The finger was a step down from db4o, but also worked great. This “just worked” and was quick (although not very pleasant to request). Their licensing was very affordable, but something in their licensing was unacceptable to the (large, well-known) corporation with whom I contract. This brings me to where I am now ...

Answer 1

Well, since no one else tried to answer this question, I will express my opinion. Keep in mind that I never used protobuf (which is why I did not answer already), so all this is based on my free understanding of things and how, if I solved this problem, I would continue.

Firstly, I have reservations about sticking blobs into a relational database. This may come back to my bad experience, returning to the era of VB6 stones and may be invalid, but it still gives me a pause. So, I would probably investigate some other mechanisms for storing objects (like Perst or db4o ) before I try.

As a due diligence, I would at least profile blobs stuffing into the SQLCE database. Why SQLCE instead of SQlLite or some other RDMS? Well, because SQLCE supports TableDirect, which I am a huge fan of. Every time you can access data without using a slow query processor, you move forward.

So, I would create a table, give it some small search columns that I could look for, and then a column of images with a large binary blob (the actual blob is not relevant for the test, but it should be close enough in size to what you expect in production). Then I add indexes for each search column.

I would fill the table with several thousand records, and then profile the speed with which I could search for the desired key (I know it quickly) and, more importantly, the speed with which I can rehydrate my objects.

I would then weigh the costs (time for development, opportunity costs, opportunities for reuse, etc.) of the options and make a decision. Probably, I would also talk about the results, since this seems like a problem that will be of rather broad interest.

Answer 2

I didn’t see this at the time (sorry, but I don’t see any questions, and while I look at the protobuf-net tag, I don’t stay as vigilant on protocol-buffers . Question: Compared to the document database (and not relational ), I think this type of approach has great potential. It seems like a slightly overwhelming look at relational, if you just drop the drops, but it should work at least. ve, of course, used the same settings to load data into offline applications - not specifically for mobile devices, but the same concepts apply.

In particular, this type of document-oriented approach is useful when you want to rehydrate a “system”; those. completely. They are not so simple and flexible to query ad-hoc than a relational database. Of course, one option is rehydration and a query in memory (for example, in C #, querying LINQ-to-objects for re-hydrated data will be almost indistinguishable from a LINQ-to-SQL query for data in a relational database).

Answer 3

What is important, if you look at MariaDB 10, we introduce dynamic columns in which the database of structured objects is stored in the block https://mariadb.com/kb/en/dynamic-columns/ Conversion on the client side from blob to whatever can be easily done on the client, because we embed the API in the C-driver. I think we only enable json export, but with a little effort this can probably be extended to protobuff, serialized, deserialized.