How to speed up responsiveness when displaying large amounts of text

Question

I am trying to create a Python script to highlight specific templates in a .txt file. To do this, I changed the script that Tkinter used to highlight the given data set. However, the files that I'm trying to process are about 10,000 lines, which leads to a slow scroll, because I think it displays everything - whether on the screen or not (correct me if I am wrong). Is it possible to change my code so that it outputs the result in a more efficient way? I tried to find a remedy for this, but I did not find anything myself.

My code is as follows:

from Tkinter import * class FullScreenApp(object): def __init__(self, master, **kwargs): self.master=master pad=3 self._geom='200x200+0+0' master.geometry("{0}x{1}+0+0".format( master.winfo_screenwidth()-pad, master.winfo_screenheight()-pad)) master.bind('<Escape>',self.toggle_geom) def toggle_geom(self,event): geom=self.master.winfo_geometry() print(geom,self._geom) self.master.geometry(self._geom) self._geom=geom root = Tk() app = FullScreenApp(root) t = Text(root) t.pack() #Import file with open('data.txt') as f: for line in f: t.insert(END, line) #Search terms - Leave blank if not required search_term0 = '0xCAFE' search_term1 = '0x0011' search_term2 = '0x961E' search_term3 = '0x0000' search_term4 = '' #Assigns highlighted colours for terms not blank t.tag_config(search_term0, background='red') if search_term1 != '': t.tag_config(search_term1, background='red') if search_term2 != '': t.tag_config(search_term2, background='red') if search_term3 != '': t.tag_config(search_term3, background='red') if search_term4 != '': t.tag_config(search_term4, background='red') #Define search #Requires text widget, the keyword, and a tag def search(text_widget, keyword, tag): pos = '1.0' while True: idx = text_widget.search(keyword, pos, END) if not idx: break pos = '{}+{}c'.format(idx, len(keyword)) text_widget.tag_add(tag, idx, pos) #Search for terms that are not blank search(t, search_term0, search_term0) if search_term1 != '': search(t, search_term1, search_term1) if search_term2 != '': search(t, search_term2, search_term2) if search_term3 != '': search(t, search_term3, search_term3) if search_term4 != '': search(t, search_term4, search_term3) root.mainloop() 

An example of the data in the file is given in the following link: here

Thanks so much for your time, it is really appreciated.

Answer 1

So, this is solved by what is called multithreaded. A computer can perform several tasks at the same time, otherwise your web experience will be different. Here is a simple function demonstrating mulit-threading

 from threading import Thread def execOnDifferentThread(funct=print, params=("hello world",)): t = Thread(target=funct, args=params) t.start() 

Now note that this may not be the best example, but all you need to do to run the function now in parallel is execOnDifferentThread(funct=A, params=B) , where A is the name of the function and B is tuple of arguments to be passed to your function. Now I don’t want to write my code for you, but using this function, you can multi-thread certain parts of your code to make it faster. If you're really stuck, just comment on where and what to do. But first try it yourself, now that you have the power of multi-threading on your hands.

Answer 2

Assuming MCVE is as follows:

 import tkinter as tk def create_text(text_len): _text = list() for _ in range(text_len): _text.append("{}\n".format(_)) _text = "".join(_text) return _text if __name__ == '__main__': root = tk.Tk() txt = tk.Text(root) txt.text = create_text(10000) txt.insert('end', txt.text) txt.pack() root.mainloop() 

Analysis

Based on this I don't think this is a rendering problem. I think this is a problem with the <KeyPress> fixed signup speed. This means that the number of events recorded per second is fixed, although the hardware can be recorded faster. A similar regulation should be true for the mouse scroll event.

---

Rendering solutions

It is possible to cut the text for part of the txt['height'] buffer. But isn't that the way TK should be rendering?

---

Solutions for rendering an unrelated problem

If the step is defined as moving the cursor to the previous or next line, for each registered event, Up or Down ; then scrolling_speed = step * event_register_frequency .

---

Increasing step size

An easy workaround would be to simply increase the step size, as with increasing the number of lines to jump, for each key binding registration.

But this is already the default behavior, assuming the page length is> 1 line, Page Up or Page Down has the page step size. This increases the scroll speed, even if the speed of event logging remains unchanged.

Alternatively, a new large-step event handler can be defined to invoke several cursor movements for each Up and Down registration, for example:

 import tkinter as tk def create_text(text_len): _text = list() for _ in range(text_len): _text.append("{}\n".format(_)) _text = "".join(_text) return _text def step(event): if txt._step_size != 1: _no_of_lines_to_jump = txt._step_size if event.keysym == 'Up': _no_of_lines_to_jump *= -1 _position = root.tk.call('tk::TextUpDownLine', txt, _no_of_lines_to_jump) root.tk.call('tk::TextSetCursor', txt, _position) return "break" if __name__ == '__main__': root = tk.Tk() txt = tk.Text(root) txt.text = create_text(10000) txt.insert('end', txt.text) txt._step_size = 12 txt.bind("<Up>", step) txt.bind("<Down>", step) txt.pack() root.mainloop() 

By comparing the speed of registration of keypress events :

As mentioned in, the actual change in registration speed in keypress is beyond the scope of Tk. Instead, it can be emulated:

 import tkinter as tk def create_text(text_len): _text = list() for _ in range(text_len): _text.append("{}\n".format(_)) _text = "".join(_text) return _text def step_up(*event): _position = root.tk.call('tk::TextUpDownLine', txt, -1) root.tk.call('tk::TextSetCursor', txt, _position) if txt._repeat_on: root.after(txt._repeat_freq, step_up) return "break" def step_down(*event): _position = root.tk.call('tk::TextUpDownLine', txt, 1) root.tk.call('tk::TextSetCursor', txt, _position) if txt._repeat_on: root.after(txt._repeat_freq, step_down) return "break" def stop(*event): if txt._repeat_on: txt._repeat_on = False root.after(txt._repeat_freq + 1, stop) else: txt._repeat_on = True if __name__ == '__main__': root = tk.Tk() txt = tk.Text(root) txt.text = create_text(10000) txt.insert('end', txt.text) txt._repeat_freq = 100 txt._repeat_on = True txt.bind("<KeyPress-Up>", step_up) txt.bind("<KeyRelease-Up>", stop) txt.bind("<KeyPress-Down>", step_down) txt.bind("<KeyRelease-Down>", stop) txt.pack() root.mainloop() 

Increasing step size and increasing registry registration speed

 import tkinter as tk def create_text(text_len): _text = list() for _ in range(text_len): _text.append("{}\n".format(_)) _text = "".join(_text) return _text def step_up(*event): _no_of_lines_to_jump = -txt._step_size _position = root.tk.call('tk::TextUpDownLine', txt, _no_of_lines_to_jump) root.tk.call('tk::TextSetCursor', txt, _position) if txt._repeat_on: root.after(txt._repeat_freq, step_up) return "break" def step_down(*event): _no_of_lines_to_jump = txt._step_size _position = root.tk.call('tk::TextUpDownLine', txt, _no_of_lines_to_jump) root.tk.call('tk::TextSetCursor', txt, _position) if txt._repeat_on: root.after(txt._repeat_freq, step_down) return "break" def stop(*event): if txt._repeat_on: txt._repeat_on = False root.after(txt._repeat_freq + 1, stop) else: txt._repeat_on = True if __name__ == '__main__': root = tk.Tk() txt = tk.Text(root) txt.text = create_text(10000) txt.insert('end', txt.text) txt._step_size = 1 txt._repeat_freq = 100 txt._repeat_on = True txt.bind("<KeyPress-Up>", step_up) txt.bind("<KeyRelease-Up>", stop) txt.bind("<KeyPress-Down>", step_down) txt.bind("<KeyRelease-Down>", stop) txt.pack() root.mainloop()