Delphi - Split Lines (the program does not respond), but does not split the file size (TTHREAD)

Question

Hope everyone does a great job. I have only one question, in Delphi programming ... I have a memo that downloads a .txt file and contains large lines, for example, 50,000 lines ....

SO I want to split them into 5000line and then load them into a new Memo. for example, for the first split, it will split 5000 lines into a new file, then load it into a new note after downloading (deleting the 5000 file), and, of course, for large LINES it will be 45000.

the second split will divide 5000 with 45 000, it will be 40 000 on large lines, 5000 on new lines (file text).

THE REASON, why I want to split the lines into the text of the file, and then load it, because the program does not respond (does not respond) when I separate files into a note.

procedure TForm1.Button1Click(Sender: TObject); var count,i ,X,m:integer; begin Memo2.Clear; Label1.Caption:=IntToStr(Memo1.Lines.Count) ; if Memo1.Lines.Count > 5000 then X:=5000 else X:= Memo1.Lines.Count ; for count:=0 to x do begin Memo2.lines.add(Memo1.lines.Strings[0]); Memo1.Lines.Delete(0); Memo2.Text:=Trim(Memo2.text); end; end; 

This is the code that I use to split small lines into small other lines ...

In a note, but when you have 1 million lines, the program will stop responding.

I added cropping (memo2.text); To delete an empty line at the end.

So, how can I do line splitting, not from a SIZE file (because it will kill lines) ... how can I do big line splitting, as I said to the file, then load it and delete it, then when I press the button again, it will perform the same operation with other lines ....

I know that we should use the TThread class, but I do not know how this happened with my code ... Thank you!

Thanks.

Answer 1

Threading is not the answer to the problem. You do not need threads for this task. The problem is here:

 Memo1.Lines.Delete(0); 

Doing this in a loop is exceptionally expensive for a note. The first part of the text is deleted, and the rest moves up.

The best approach:

• Move the entire source to a TStringList .
• Process the first 5000 lines of this list of lines in one line.
• Add this thread to a single entry in a note using SelText := ...

When finished, clear the original note with one call.

Answer 2

As mentioned earlier

Use TStringList to load and process your data, TMemo to display it.

And remember: if you can’t do it, fake it!

And you cannot load 50,000 lines into standard VCL control in a reasonable amount of time.

So, I made you a small sample application showing you how to fake it:

You need to add ERROR to demonstrate, I just took the shortest and most optimistic path.

At first I needed some data, so I created a file with random data:

 procedure TForm25.GenereateTextFile(const aLines: Integer); var Buffer: TStringlist; Line: string; i, j: Integer; begin Buffer := TStringlist.Create; try for i := 1 to aLines do begin SetLength(Line, Random(200)); for j := 1 to Length(Line) do Line[j] := Chr(ord('a') + Random(26)); Buffer.Add(Line); end; Buffer.SaveToFile('dummy.txt'); finally Buffer.Free; end; end; 

And I call it FormCreate:

 procedure TForm25.FormCreate(Sender: TObject); begin GenereateTextFile(50000); end; 

Then we need to generate some GUI to display the file:

Put a TTimer in your form and set the Interval to 50 and assign it an OnTimer event

procedure TForm25.Timer1Timer (sender: TObject); start Timer1.Enabled: = False; GenerateGUI (Self, 'dummy.txt'); end;

It remains only to create the actual graphical interface:

 procedure TForm25.GenerateGUI(const aForm: TForm; const aFileName: TFilename; aLinesPerPage: Integer = 5000); var Buffer: TStringlist; Tmp: TStringlist; i, j: Integer; PageControl: TPageControl; TabSheet: TTabSheet; Memo: TMemo; begin Buffer := TStringlist.Create; Buffer.LoadFromFile(aFileName); Buffer.BeginUpdate; PageControl := TPageControl.Create(aForm); PageControl.Parent := aForm; PageControl.Align := alClient; i := 0; for i := 0 to (Buffer.Count div aLinesPerPage) - 1 do begin TabSheet := TTabSheet.Create(PageControl); TabSheet.PageControl := PageControl; TabSheet.Caption := Format('Lines %d to %d', [i * aLinesPerPage, (i + 1) * aLinesPerPage - 1]); Memo := TMemo.Create(TabSheet); Memo.Parent := TabSheet; Memo.Align := alClient; j := 0; Tmp := TStringlist.Create; Tmp.BeginUpdate; Application.ProcessMessages; while j < aLinesPerPage do begin Tmp.Add(Buffer[0]); Buffer.Delete(0); inc(j); end; Memo.Lines.Assign(Tmp); Tmp.Free; Application.ProcessMessages; end; FreeAndNil(Buffer); end; 

You will see that the pages are displayed one by one and the data is displayed.

The reason it appears one after another is because I call Application.ProcessMessages; that it slows down a little bot a little, and you tell your user that the program is alive.

Answer 3

I'm not sure how important the use of TMemo is for your application, but if all you want to do is split the text file into several files with the same number of lines in them (except maybe the final one, of course), you don’t need TMemo in general, and not any Delphi container class such as TStringList or TStream. The fact is that you already have a good container, namely the file system.

The following is a Delphi console application that uses file access methods that have been in Object Pascal since Delphi (I can hear some Delphi fans fool me by showing you this way, but imo doesn’t hurt to find others that are completely correct ways to do things, especially since this method avoids the semantics and use of Delphi container classes).

Hope this works, self-evidently. After creating a test file, it reads it back, one at a time, and outputs it to a sequentially numbered output file until it contains as many lines as you need or there are no more lines to read. It takes 0.078 seconds from start to finish on my laptop.

code:

 program FileSplit; {$APPTYPE CONSOLE} uses SysUtils; const MaxLines = 100000; LinesPerFile = 5000; BufferSize = 32768; var MainFileName : String; procedure CreateFile(AFileName : String); var AFile : Text; Line : Integer; Buffer : Array[1..BufferSize] of Char; S : String; begin AssignFile(AFile, AFileName); SetTextBuf(AFile, Buffer, SizeOf(Buffer)); Rewrite(AFile); Line := 1; while Line <= MaxLines do begin S := Format('This is line: %d', [Line]); writeln(AFile, S); Inc(Line); end; Close(AFile); end; procedure SplitFile(AFileName : String); var AFile : Text; SubFileNumber : Integer; Buffer : Array[1..BufferSize] of Char; procedure CreateSubFile(SubFileNumber : Integer); var OutFile : Text; OutBuffer : Array[1..BufferSize] of Char; SubFileName : String; S : String; Line : Integer; begin SubFileName := ExtractFilePath(AFileName) + IntToStr(SubFileNumber) + '.Txt'; writeln(SubFileName); Assign(OutFile, SubFileName); SetTextBuf(OutFile, OutBuffer, SizeOf(OutBuffer)); Rewrite(OutFile); Line := 1; // the following reads the input file a line at a time // and outputs it to the current output file // until there nothing left to read or the lines // per output file is reached while (Line <= LinesPerFile) and not Eof(AFile) do begin readln(AFile, S); writeln(OutFile, S); Inc(Line); end; Close(OutFile); end; begin Assign(AFile, AFileName); SetTextBuf(AFile, Buffer, SizeOf(Buffer)); Reset(AFile); SubFileNumber := 1; while not Eof(AFile) do begin CreateSubFile(SubFileNumber); Inc(SubFileNumber); end; Close(AFile); end; begin MainFileName:= GetEnvironmentVariable('Temp'); MainFileName := IncludeTrailingPathDelimiter(MainFileName); MainFileName := MainFileName + 'Main.Txt'; CreateFile(MainFileName); writeln(Format('File %s created', [MainFileName])); SplitFile(MainFileName); readln; end. 

Answer 4

You may encounter tmemo limitations (it depends on which version of Delphi you are using). Of course, it will take time to remove the first line from tmemo, mostly 5,000 times.

Could you use readln and writeln for memo 2 for the first 5000 lines and memo 2 for the rest, if any?