Delphi: store data in some structure

Question

For a simulation program, I work in Delphi 2010. Simulation is not a problem, but I need to use a large dataset, which gives a problem. The data is available on an excel sheet, so there is no need to edit this data in Delphi, but collecting this data from excel sheets takes about 10 minutes. This is not a problem if you do not need to collect data every time the program starts. So I made a program that collects all the data, makes it visible, not a problem here, and then saves it. However, I cannot save it in the "Delphi" format without losing structure, so it can be downloaded in a few seconds.

I am not so experienced in Delphi, and I have been looking for a solution for a long time, but I could not figure out which is better. I think my way of structuring data is wrong, but it is simple and works. However, if there are better ways to store data, say so, but remember that I need more explanation than just using the "xml file", "generict" or "Ttreeview". (read, but could not use it).

Data for: I made this product. The next product I do is, so, do I need to clean? True or false.

The data is stored as a class (TObject) with the product number (integer) and a list containing all the products that can be made as follows. This list contains another class (TObject) with the product number (integer) and I need to clear (boolean). I want to save this structure in a file without losing data and returning it to the same structure.

Hope someone can help. Thank you in advance.

Update: code to provide a little more information (changed to English)

Clean_from = class(TObject) public myfromNumber : Integer; mylist : TList; published constructor Create; End Clean_To = class(TObject) public myToNumber : Integer; Clean : Boolean; End; constructor Clean_from.Create; begin inherited Create; myList := Tlist.Create; end; For i = 0 to 100 do begin From:= Clean_from.create; for j := 0 to 10 do begin To := Clean_To.create; To.clean := true or false; From.myList.add(To); end; GlobalList.add(from); end; 

And now I want to save the global list with all the contents so that I can load it with the same structure.

Answer 1

You need the so-called "serialization" mechanism.

1. Standard way

1.1 SaveToStream

In Delphi, we usually implement the SaveToStream method, which will save the contents of each object in the target TStream (either TFileStream or TMemoryStream ).

You will have to write serialization manually.

1.2 streaming in DFM format

See TWriter / TReader classes.

If you define your data in published properties, you can serialize them using these standard Delphi classes.

For some methods that can serialize any TCollection to and from JSON content, see this blog post .

2. RTTI

See for example this SO question .

In particular, the new extended RTTI (available since Delphi 2010) opens up new possibilities for serialization.

3. Use records instead of classes

If each element does not store a lot of content (some integer / logical), it makes sense to use records instead of objects. For speed and memory / fragmentation, it can be worth it.

Here is some shell capable of serializing any dynamic array , even containing nested records or dynamic arrays.

4. Use the database engine

Perhaps the best approach is not to get your data stuck in non-expandable binary form, which is the property of your application. If you want to add a property, you will have to manage it manually. Or, if you want to access your data from other applications, this can be difficult.

There are many database solutions - instead of using an external database (for example, MS SQL, FireBird or Oracle), it would be nice to embed the database in your application (much easier to install). It is worth mentioning SQLite , in which there are many wrappers , including our version (which will allow you to switch to any other database if you want to use MS SQL or Oracle).

You have other solutions - see this SO question - and if you need performance, take a look at our Big Table Library .

Answer 2

Add the SaveToStream() and LoadFromStream() methods to your data object, which, well, save the data in the stream and load the data from the stream.

 type TMyData = class(TObject) private FChildProducts: TList; FProductnumber : integer; FClean: boolean; public procedure LoadFromStream(const aStream: TStream); procedure SaveToStream(const aStream: TStream); published property Productnumber: Integer read FProductnumber write FProductnumber; property Clean: Boolean reas FClean write FClean; end; procedure TMyData.LoadFromStream(const aStream: TStream); var x, cnt: Integer; cD: TMyData; begin aStream.Read(FProductnumber, SizeOf(FProductnumber)); aStream.Read(FClean, SizeOf(FClean)); // read number of child products aStream.Read(cnt, SizeOf(cnt)); // load child objects for x := 1 to cnt do begin cD := TMyData.create; cD.LoadFromStream(aStream); FChildProducts.Add(cD); end; end; procedure TMyData.SaveToStream(const aStream: TStream); var x: Integer; begin aStream.Write(FProductnumber, SizeOf(FProductnumber)); aStream.Write(FClean, SizeOf(FClean)); // save number of child products x := FChildProducts.Count; aStream.Write(x, SizeOf(x)); // save child objects for x := 0 to FChildProducts.Count - 1 do (FChildProducts[x] as TMyData).SaveToStream(aStream); end; 

I assume that you have a list of "root objects", so you can create a function or method that saves / loads them to / from the stream, i.e.

 function SaveDataList(const List: TList;const aFileName: string); var x: Integer; FS: TFileStream; begin FS := TFileStream.Create(aFileName, ...); try // save file version x := 1; FS.Write(x, SizeOf(x)); // save number of products x := List.Count; FS.Write(x, SizeOf(x)); // save objects for x := 0 to List.Count - 1 do (List[x] as TMyData).SaveToStream(FS); finally FS.Free; end; end; 

This is a general idea ... how to load data back should also be clear. The fact is that when a data object changes (i.e. you add some property), you can increase the version number so that you can load data into the right version of the data object in the download code.

Answer 3

I would choose Arnoud option No. 4, but use ClientDataSet with nested ClientDataSets. This will allow you to load and save data in a flexible structure, as well as display them quickly. Check out this page , and also here for some nested data and sample data.