Editing programs while they work? What for?

Question

Lately I have been getting more on Lisp and Lispy languages, and I find them quite powerful.

One thing that I read on the net is that you can write in Lisp, Clojure, etc., that you can edit your program "while it is running."

Maybe something is missing me, but what's the point?

Of course, this can save a few seconds, but is that all? Whenever I make changes to my program, I just stop it and then run it again, and it has been working fine for decades.

There must be a reason besides saving time - what is it?

Can someone give me a good study that will make me drool this function? :)

Looking forward to drooling!

Answer 1

There are several extremely useful use cases. One example — in the GUI — I saw this while developing the GUI application in real time, as it worked alongside my Emacs: I added the code for the new button and clicked “Cc Cc” to compile this single function, and the button appeared in the window! There was no need to close and reopen the application. Then I started customizing widgets and manipulating the layout, and the open window instantly regrouped - the buttons moved, new text fields would just appear, etc., As soon as I made every small change I made.

Another example is a great overview of the PenGra OpenGL Clojure library, where a programmer creates a 3D Tetris game in real time. He starts with an empty OpenGL window next to his emacs. It defines the cube object - CMx - and it is on the screen. Executes a command to rotate it, immediately begins to rotate. It starts a loop that defines 5 more cubes in different places, pop-pop-pop-pop-pop-pop they appear. All this instantly reacts, the complete OpenGL toolkit is right here to play. Add the surface texture to your cube and immediately see how it appears. It becomes a flexible 3D world - the code dynamically changes the existing world, but does not close and reopens a three-dimensional canvas with each change.

Penumbra Livecoding Screencast - Download the HD version for a better experience.

There is also an excellent presentation / screencast about the Overtone audio library for Clojure. A library is a set of synthesizer tools in which you have a set of synthesizer functions for controlling a sound wave. During the presentation, the developer writes some code that begins to reproduce the tone. Then he spends ten seconds recording a cycle that reproduces this sound 10 times, but each time it increases the frequency, and again CMx, and you hear it, the notes rise higher. Within 20 minutes in real time he receives a song. It seems like a lot of fun.

Overtone Presentation Link

Other uses would be, for example: web scanning / data mining - to develop and improve algorithms for extracting information in real time, seeing the data returned at each step; Robotics programming - sending commands to a robot during its life; Face / image recognition - using a library such as OpenCV, make sure that your changes instantly update what the library recognizes in the image / video when you develop the code; Mathematics work (Clojure has an "Incanter" for statistics); and in any environment where you want to immediately see what impact your changes have on the data you work with.

So, the funniest aspect of having REPL is in front of you. Things that were not tangible, obedient, interactive, begin to become. GUI design, 3D graphics, software sound production, data extraction and conversion, these things were usually done at a distance of the hand. But with Clojure (and to some extent with other dynamic languages ​​too) it became really tangible and immediate; you see each change as soon as you write the code, and if something does not work or you don’t get the result you expected, you just change what you missed and run it immediately.

Clojure is very suitable for this. It's amazing that you can use Java libraries in real time the same way - even though Java itself cannot! Thus, Overtone uses the real-time Java synth library, even though you never could in Java, Penumbra uses OpenGL Java bindings, etc. This is because Rich Hickey developed Clojure so that it can compile the JVM bytecode on the fly. This is an amazing language - Clojure has made a huge contribution to the incredible pleasure and productive programming.

Answer 2

There must be a reason besides saving time - what is it?

No no. I mean, it never happens: the whole reason to use a computer at all is to save time. Nothing can make a computer that you cannot do manually. It will take a little longer.

In this case, I would not fire “a few seconds,” given that this is one of the things that I do most often, all day, my entire programming career. After a few seconds for re-compilation, a few seconds before restarting, a few seconds to restore the state that my program had in the previous time - even on a fast workstation, there may be a minute between iterations. (Previously, it was much worse, but faster hardware only made it less terrible and not good. Recompiled files are generally or worse associated with I / O binding and may never match the speed of more detailed compilation.)

In Lisp, recompiling one function in an already running process is almost instantaneous (I never saw it even for 0.1 seconds even on my 5-year-old laptop), and restarting means that I should not recreate my state even when something is signaling.

Here's a tool that gives me over 100x acceleration of one of the slowest and most common things that I do as a programmer. I do not know what else you need. We can probably come up with some reasons, but if this is not enough, I do not know what will happen. Um, is that pretty cool too ?:-)

(* Whenever someone says “never” about something related to technology, that person invariably ends up looking like a complete jerk 2 years later, and despite Lisp's longevity, I'm not ruled out.)

Answer 3

There is a marketing slogan for Lisp:

With Lisp and its incremental development method, the cost of software change depends on the size of the change, and not on the size of the entire software.

Even if we have a large software system, the cost (time, ...) for the change remains depending on the size of the change. If we add a new method or change a new method, the efforts will continue to involve trying to change the method, compile the method in stages, and gradually load the method.

In many traditional software environments, changing the method may require partial recompilation, a new associated executable, reboot, reboot, etc. The larger the software, the longer it takes.

For humans, this means that we may be leaving the state of flow . This part of the performance of good Lisp environments: in a short time, you can make many changes to the software system as soon as the programmer feels comfortable and enters this stream state. I think many have experienced this when the work is completed in a short time - against the time when one is sitting in front of a system that does not respond, and we are faced with a wait time.

There is also little cognitive distance between us and the program we work on. For example, if you are editing a class in a batch environment, you must imagine the impact of the changes. In Lisp, you are editing a class while modifying the objects themselves. This means that you change the behavior of objects directly, and not a new version of them after a cycle of batch editing-compilation-link-run-test.

In the Lisp system, you change the class in the CAD system, and then you can immediately activate it. When people ask if Lisp works for large program groups, the answer may be that a large team of software developers is not needed if you work gradually. The problem then was that the really good experienced software developers familiar with the gradual development were (rarely?).

Many applications have a separate scripting language level, sometimes for original developers (and not for users). In Lisp, this is optional; Lisp is its own extension language .

Answer 4

In the real world, this is mainly used in development, and, like many other functions, its only value comes down to confusion in the right context.

• personal enlightener for programmers *
• True continuous deployment.
• zero planned downtime Service Level Agreements.
• debugging servers.

* not a guarantee.

---

for me, and I suspect that some of the others here, the real advantage of this REPL driven development is that it can be indescribably fun. addictive even. This can sometimes make sense of the code. give it a try ... come try, first REPL is always free :)

---

currently one big draw is continuous deployment.

Currently, the idea of ​​continuous deployment is that you change one thing, build everything (or pack it faster), and then deploy. with the lisp model, you can actually edit a deployed one (usually a box that receives a mirror of real client sessions) when it is in deployment.

just a pedantic note. you are not actually editing running classes. you compile a new copy of the class and leave it in a known place (var), then the next time it is used, a new copy will be found and used. its not really editing at startup and more like new code takes effect immediately , this reduces the volume of the devlopment process from programs to expressions (usually functions).

---

another drooling point is the idea of ​​getting bennefit from security fixes without having to declare downtime . you can upgrade without it, which would cost your SLA any of your precious “planned outages.” If you need to plan your planned downtime six months in advance, and you only get two hours after that (for these poor souls), this can really make them drool.

---

If you have replicated access to your running application, since it is deployed (potentially (with permission) on the client’s site), you can connect to the application during its launch and run tests on the existing code in the existing context without stopping and connecting the debugger. You also won’t get any speed loss from the debugger. This can be done using REPL, although when you get a replica there, you can easily create new code (some will say that injecting dynamic class loaders through the debugger is simple) and then correct the situation. Thus, you can connect to a running server. find that after a short failure the function was unable to reconnect to the database, and then reconnect it back and forth.

---

as with all programming constructs, it will never be a silver mark , and this constant deployment / development has an interesting drawback: the program may be correct in memory and incorrect on disk. if you compile a function, then you break it and save it, then only a working copy of the code will work. It’s useful for me to know about this and re-update files immediately after saving them.

---

This might seem fancy, so see how to Embed Clojure REPL in your production application.

Answer 5

I remember that someone from NASA talked about his experience. His team realized the soft use of the spacecraft back in the 70s. And they effectively modified their soft remotely on the fly when some errors were discovered.

Or imagine that you have a long process that takes several days to complete, and in the end it cannot write the results due to permissions or other minor problems.

One more example. You are in the integration phase and you need to make small changes. And again there are a lot of them. I dream of such an opportunity in Java, because currently it takes me 30-40 minutes to rebuild and reinstall my application (to restore it again in 10 minutes).

Answer 6

If you look at something like Erlang, you need to avoid downtime.

It works on things like telephone switches, which you can't just turn off for a few seconds.

For more normal use, however, it is a “good to have” function, but yes, it may not be critical.

Answer 7

You see the real data. This is a great advantage. Then you do not need to think.

Answer 8

Because you can?

Seriously, just try this time and you will feel pain when you return to your old programming language without REPL.

Instant feedback, easy execution of quick tests without the need to configure the fake state of the program in your device under test. The ability to check the status of a running program (what is the value of this variable). All these are lifeguards in real time.

Answer 9

This is mainly for development, where it just stores time.

But time savers are overwhelming.

Once you get used to it, returning to the old way is like moving from flying to swimming in tar.

Answer 10

In industrial systems this is used for PLC programming to facilitate downtime and unsafe conditions.

These are systems that are used in nuclear power plants, production systems, steel mills, etc. The process always works, constantly, and downtime is very expensive or unsafe. Imagine a system that controls the cooling of a nuclear reactor, you cannot turn off this system to deploy new code, you should be able to change it when it works.

This is similar to Erlang's answer for telephone switch systems.

Answer 11

Well, imagine you need to fix the server and not stop it.

If you do this in a “typical” language, it will be associated with some kind of heavy magic. You must mask the executable code for it. I think that this will require correction of function tables, etc., Everything in assembling and manipulating function pointers. Good place for mistakes.

In Lisp, the idea of ​​updating without downtime is built into the language model. Despite some difficulties of updating that you cannot get away from (how you work with a long-term connection), this does not require heavy magic of the compiled language.

Although I did not spend significant time on this (i.e., something useful), I developed a server prototype in Common Lisp that would make at least some updating on the network without downtime.

Answer 12

Another good thing, besides modifying the program on the fly without having to restart everything (having done it for decades, does not mean that this is the best thing, right?), Is that you can check your program in its current state and an opportunity to find out what is happening.

Answer 13

Casey Muratori just did a few lessons on how to do this using C and the Microsoft C / C ++ compiler. This is actually quite simple, with only a few dozen lines of code. Watch the videos 22/24/25:

https://www.youtube.com/watch?v=WMSBRk5WG58

In game design, the rationale is to be able to tune constants more quickly to find the emotional tenor you are aiming for. Things such as a sense of play, non-player behaviors, and lighting / surroundings benefit from this.

Answer 14

You can also take a look at:

[ http://cvberry.com/tech_writings/howtos/remotely_modifying_a_running_program_using_swank.html]

which deals with remote modification of a running program using Swank.