Only inexperienced developers use infinite loops

A language agnostic look at this common programming pattern. My hope is that once you've read my post you'll never write an infinite loop again.

My favourite number: 8

What are infinite loops

I've chosen Python as the language to demonstrate due to its readability.

1. Loops

The simplest loop is the while loop with a condition that doesn't change, to demonstrate;

while True:
  """do code things"""
  pass

In C or C-like languages you may see many syntax variants like for (;;) {} and do {} while but if they're infinite there's no notable difference.

The issue with such infinite loops is we are trained to see these as a pattern and not a bug, and few developers are aware of any caveats or only know of a few. All of the associated caveats I'll soon explain are actually software defects, and the way to solve the defect is to refactor away from the infinite loop.

Infinite loops are software defects

2. Iterators

The next common technique is mutating an iterable, like so;

l = [1]
for x in l:
    l.append(x + 1)

The iterable being iterated inevitably grows in each iterations preventing the program to complete. The obvious problem with this is the iterable memory space will grow and may even exceed memory constraints causing a panic or exception. So when this technique is observed we often associate it with being a bug immediately, appropriately.

3. Generators

A new technique has come about with the growing popularity of generators.

def infinity():
    while True:
        yield
for _ in infinity():
    pass

This is particularly dangerous, we turn to generators due to their efficiency and often they are amazing for this. But when we turn that into an infinite loop it becomes the source of the worst of both examples 1 and 2 above.

Generators as loops leak memory and are defects

Use cases that work

So why do we use a infinite loop in any language? What benefits do we get using this and what caveats does this pattern bring to our programs?

There are only 2 cases where an infinite loop might arguably make sense.

1. Limited scope

If the entire scope of the program is to operate the loop, and only the loop itself, then there is reason to argue that the program can do that well. But what a silly program that would be.

There are successful programs that have used infinite loops. Such as the Node.js event loop, and a game engine (like Phaser.io), but developers shouldn't choose to write an infinite loop unless we are ready to accept the responsibility and consequences.

"Infinite loops" with great power comes great responsibility

Mozilla has created circusd to fill the place of the now dead supervisord. Its purpose is to keep your program alive or give you finer control of a long-running workload often implemented with an infinite loop. With such programs available, developers don't need to write their own infinite loops anymore.

2. Advanced Programming Languages

Rust to my knowledge (and I don't know everything) is the only memory safe programming language available to mere mortals.

In a language that gives us control over memory management (like Rust) and the ability to manage fully any external file descriptors, we may avoid the caveats associated with infinity loops.

Rust; memory safe programming for mere mortals

Caveats

These are common use cases for developers to resort to infinity loops and the associated software defects they introduce.

Daemons

Whenever we are given a requirement that leads us to use a daemon pattern in our programs we immediately and amateurishly turn to infinity loops.
At the risk of stating the obvious to those more experienced, a daemon is not actually a pattern, it isn't even intended as a noun, it's an adjective to describe a process that has no ability to interact with or be interacted with its executor. In no way is a daemon directly associated with long-running processes, so why do many developers make this immediate association? If you fall into this misconception you're just hacking together some code example you stumbled across probably written by someone equally or less knowledgeable as yourself in what they're doing.

The misconception that infinity loops are daemons

If you have a use case for a program to be a daemon you would need to first defer that program execution to a child process which becomes its own session leader allowing the original process to exit without killing the child containing the program you want to continue executing, thus the program is now detached from its executor and is, therefore, a daemon. Notice there is no mention of infinite loops?

If you don't understand process, session, and leaders yet start here

To win software development; understand processes

Many termination conditions

What is a termination condition? Here are a few from various languages;

• return
• exit
• die
• break
• continue
• goto

With so many exit statements available it is often that we change a conditional loop to an infinite loop for readability or simplicity sake, which is quite possibly the most ill-conceived approach. Try to convince me that it is easier to search a block of code for any of the listed exit statements over looking at the single loop condition and I'll happily eat my words, literally, I'll print the whole post and eat it for your entertainment.

Understand your exit condition/s, then depending on that analysis you can produce a single conditional to track and use for your loop condition so it is no longer an infinite loop. If that sounds challenging here is an example;

condition = None
break_condition = [42, 67, 12]
while condition not in break_condition:
    """Run some task and assign a value to represent that outcome"""
    condition = random.randint(0, 100)
    """As long as the outcomes are appropriate to continue the loop"""
    pass

Now all you need to do is track one variable for all of your complex exit statements while avoiding using infinite loops!

Understand your exit conditions for clean code

Defects introduced by infinite loops

Earlier I promised to explain that infinite loop caveats are actually software defects, which shouldn't have ever been introduced.

Memory leaks

Many reported bugs that get traced back to a memory leak are often addressed by optimising code to work more efficiently with the languages GC.
If this occurs in an infinite loop you've not fixed the defect but rather resolved the bug report symptom only. For the memory leak to accumulate data over time the root cause is the infinite loop, you might of today found one leak and patched it, good for you, but the root cause (the infinite loop) remained and the defect prevails to strike again when new code is introduced by an unsuspecting junior developer who inevitably will get blamed.

A junior developer's regression bug is your fault

If you consider yourself a senior developer and you find a memory leak within an infinite loop it is your responsibility to refactor to remove the infinite loop, you don't need it, or that junior developer's regression bug is your fault.

Incomplete execution

When the executing program is killed or forced to terminate by an external source (like a load balancer or a terminal session being interrupted) your infinite loop and all of the logic inside is immediately abandoned.
You may reach this conclusion through debugging a number of bug reports, and when you reach this conclusion hopefully you learn that it can be managed using signals.
I've seen this happen in almost every using containers like Docker, but it has nothing to do with containers or Docker, it's only come into the view of developers since ops engineers are becoming less concerned with program development.
If signals are not already managed and you're program is not designed or effective when terminated at any stage, you should implement signal handling immediately. It's uncommon for APIs or web server scripts to need this so coming from a web developer background you'd be excused for not knowing about signals, but if you're not from a web background shame on you.

Implement signal handling immediately, shame on you

In an infinite loop, signals become more difficult to manage and I often see a loop short-circuit condition only at the top of the loop, and only handles one of the signals (usually Interrupt or Terminate).
Although this common and most developers wouldn't see the problem with it, here is a quick question you should consider; how do you handle unknown state?

By not handling all signals, or not responding to the signal promptly, the program enters an unknown state which is not good, to say the least.
You may be addressing another symptom but again not the root cause. Unless you react to all signals promptly, not just at the top of your infinite loop, you create a different race condition to the one you tried to solve and introduce an unknown state condition that didn't exist before you fixed the bug that led you to this.

Experienced developers either found themselves doing this and learned not to do it ever, or were thorough enough to learn about signals, daemons, and infinite loops in all of their hellishness before falling for their allures.

Inconsistent state

We just explored unknown state, where the program received a signal and suppressed that signal, an inconsistent state is not that.

An inconsistent state is part of your program logic, it's where you have an expectation of the program state to be one thing but you've been given bug reports that somehow it has entered something else entirely. The most common inconsistent state defect with infinity loops is where the program was not able to complete all logic within its loop.

Inconsistencies may be related to data in your database/s, files (local or remote), or even termination of some remote resource.

Ops engineers can help with causes for inconsistent state

This may sound similar to the problem faced with signals, but the root cause of inconsistent state is not signals.

If you've had an experienced ops engineer lead you to solve this symptom by looking into connection timeouts or locked files you'd normally resolve the bug reports by working around a connection timeout - reconnecting or something similar, again addressing a symptom without understanding it.

File descriptors are the things that a program obtains from the system to open files and remote connections, but due to infinite loops being long-running processes the program expects a file or remote connection to be in a certain state when it reuses the same resource it had opened earlier. But as you'll learn from your debugging, things change, get locked or deleted, and connections timeout. Simply reconnecting doesn't stop the problem happening because things like database connections and file handling are done via a library or some other abstraction it is often impossible for programs to manage their own file descriptors for these resources, leaving you with only an option to reconnect or some other useless abstraction that doesn't actually prevent the problem occurring only react to the symptoms.

Learn about file descriptors to achieve greatness

Now you've learned about file descriptors you should resolve your bug at the root cause, refactor your code to remove the infinite loop so your program never enters an inconsistent state where it would ever have any stale file descriptor references in memory.

Refactor your loop

How to refactor away from infinite loops?

Above I gave an example of using a single conditional with multiple exit statements, this method takes all of the exit conditions within the loop and replaces them with a variable that takes a unique value corresponding to that exit condition. You can then turn your infinite loop into a loop with one conditional, therefore one place to track your exit conditions.
You'll find that now you have an easy way to reason with your loop and it's exit conditions, you'll start identifying previously hard to locate exit conditions in your code. What you previously had was a pseudo-daemon with a big defect commonly called an infinite loop, and now you have a program you can reason with.

Another way to refactor might be using an application like circusd to take the place of the infinite loop altogether making it redundant to have one in your code at all. All your program will need to do is execute the program itself (the code you had in your loop).

You'll find that something like circusd will also make many (or all) your previous infinite loop exit condition redundant in your code, as it provides you ways to run your program, recover or keep it running, delay start, delay executions, and many many more programming complexities so you can just write your program and forget you ever heard the word daemon.

Mozilla's circus saves lives