What is Strict Structured Concurrency?

Consider this code that fetches user information and shows a spinner in order to visually cue the latency:

await run(function* getUserInfo(userId) {
  let spinner = yield* spawn(function* () {
    yield* showSpinner({ style: "circle" });
  });

  let [user, groups] = yield* all([fetchUser(userId), fetchGroups(userId)]);

  yield* spinner.halt();

  return { ...user, groups };
});

This works by starting the spinner in the background, fetching user information in parallel, and then, once it has been fetched, stopping the spinner before returning the combined results. This works well, and the effortless cancellation that Effection provides ensures that all traces of the spinner are gone before we are ready to return.

Let's add a timeout to make sure that things don't take too long:

await run(function* getUserInfo(userId) {
  let spinner = yield* spawn(function* () {
    yield* showSpinner({ style: "circle" });
  });

  let timeout = yield* spawn(function* () {
    yield* sleep(1000);
    throw new Error(`timeout: took tool long!`);
  });

  let [user, groups] = yield* all([fetchUser(userId), fetchGroups(userId)]);

  yield* timeout.halt();
  yield* spinner.halt();

  return { ...user, groups };
});

Along with the spinner, it starts a countdown in the background that will blow up the whole operation if it takes too long. And that's ok, because once we have the user info safely in hand, we can halt that task too.

For good measure, we can tack on by adding a heartbeat to the remote server to make sure that it stays alive while we fetch this userinfo.

await run(function* getUserInfo(userId) {
  let spinner = yield* spawn(function* () {
    yield* showSpinner({ style: "circle" });
  });

  let timeout = yield* spawn(function* () {
    yield* sleep(1000);
    throw new Error(`timeout: took tool long!`);
  });

  let heartbeat = yield* spawn(function* () {
    yield* sendHeartBeat({ milliseconds: 100 });
  });

  let [user, groups] = yield* all([fetchUser(userId), fetchGroups(userId)]);

  yield* heartbeat.halt();
  yield* timeout.halt();
  yield* spinner.halt();

  return { ...user, groups };
});

Foreground vs Background

By now we can see an interesting dynamic at play. There are actually two different kinds of operations that we're running. The first type are those that are evaluated directly in order to produce the result of our computation. In this case, it is the combination of fetchUser(), fetchGroups(), and all() which form the literal definition of what it means to getUserInfo(). They are the components used to express the core algorithm which is why we call them the foreground.

Then there are the other tasks: the spinner, the timeout, the heartbeat. These don’t participate in the algorithm, and they don't produce a value on which the return value depends. Instead they are there to produce a persistent side-effect that supports the foreground while it runs. That’s what makes them background tasks.

More formally, a foreground task is one on which the return value depends. A background task is one on which it does not.

Notice how in the code, there is no need to manage the lifecycle of the foreground tasks. It happens naturally because the foreground requires the values of all its tasks to be fully computed before it can compute its own return value. In other words, the lifetime of the foreground is always naturally aligned with the lifetime of its scope.

On the other hand, the lifetime of a background task is not naturally aligned with the lifetime of its scope. It could be long, or it could be short. Quite often, as in the case of our spinner, it could be infinite. Whatever the case, once the foreground is complete, it means that the return value has by definition been computed and therefore spending any further work on the background is incorrect and wasteful of resources.

The principle here is that background work does not have equal structural standing with foreground work, and therefore once the forgeground is complete, a correct program must shutdown the background immediately.

The Teardown Tax

Which brings us back to our example. It is not just that it can be annoying to have to include lines like these in order to destroy the backround (although it can be).

yield * heartbeat.halt();
yield * timeout.halt();
yield * spinner.halt();

Rather, it is that to not do so would be incorrect.

The programmer must always remember to both identify and teardown the background in every single task. Otherwise, they will squander resources doing work that has no bearing on the outcome. Even in cases where the background will eventually settle, every moment spent beyond the lifetime of the foreground is wasted.

To forget is at best to introduce an unknown amount of excess work into your program. At worst, it results in an outright deadlock.

Where "classic" structured concurrency stops

When Nathaniel Smith brought structured concurrency into the mainstream back in 2018, he argued that, to paraphrase, "a task cannot finish until all of the child tasks that it created have also finished." If you have not read it, I highly recommend you stop reading this right now and go have a sit with it.

He shows how within a structurally concurrent system, control flows in to the top of a scope, and control flows out from the bottom. But in all cases the ledger of concurrent tasks is exactly the same as when it entered. This simple constraint is a straight up super-power because it allows us to build and compose abstractions that can nevertheless contain all kinds of side-effects and state. The safety is real.

Its thesis, which he establishes very effectively, serves as the bedrock upon which most structured concurrency implementations rest, Effection included. However, it remains silent on the question of how to handle the interaction between background and foreground. It's in this unspecified gap that Effection doubles down on program structure as a means towards guaranteeing program correctness.

Strict Structured Concurrency

To see this in action, let's revisit the code sample not as written before, but as you would actually write it using idiomatic Effection:

await run(function* getUserInfo(userId) {
  let spinner = yield* spawn(function* () {
    yield* showSpinner({ style: "circle" });
  });

  let timeout = yield* spawn(function* () {
    yield* sleep(1000);
    throw new Error(`timeout: took tool long!`);
  });

  let heartbeat = yield* spawn(function* () {
    yield* sendHeartBeat({ milliseconds: 100 });
  });

  let [user, groups] = yield* all([fetchUser(userId), fetchGroups(userId)]);

  return { ...user, groups };
});

This effectively has the same runtime profile as before, and it has the same properties of correctness, but the section of code that halts all of the background tasks is notably missing.

This is because Effection takes care of what is already known to be needed to ensure program correctness. Namely, to ensure that the background goes away as soon as the foreground is finished computing and it is no longer needed. We call this additional safety "strict" structured concurrency. It says that not only must a task not finish until all its children finish, but also that the background is halted once the foreground completes.

One way to think about it is like the memory resources that hold variable references in a function’s stack frame. When a function exits, those memory resources are automatically recycled. You don’t have to deallocate them explicitly; you don’t even have to think about them because the lifetime of the memory is tied to the function’s scope. Strict structured concurrency does the same thing for tasks. The background tasks are automatically reclaimed when the foreground moves on, so that it's one less thing that the programmer has to carry.

The guarantees still hold

Strict structured concurrency is still structured concurrency.

When a background task is shut down automatically, it is not terminated outright. The parent still waits, just as it would under the standard model, for every child to run all of its cleanup paths. The result computed by the foreground will not be reported to the caller until they are fully complete.

To demonstrate this, here is an example that starts a task running in the background, and then promptly finishes.

import { main, sleep, spawn } from "effection";

await main(function* () {
  yield* spawn(function* () {
    try {
      yield* sleep(2000);
    } finally {
      yield* sleep(500);
      console.log("cleanup complete");
    }
  });
  console.log("done");
});

This will print “done” immediately, wait 500 milliseconds, and then print "cleanup complete" before exiting. This is because upon exit, the background task will be halted and its finally {} block must be run. The strict refinement doesn't weaken the guarantee. It just makes it more intuitive with orderly shutdown as the default rather than something you have to manage by hand.

Focus on the algorithm

The deepest consequence of strict structured concurrency is where it focuses attention. The foreground is in the foreground. The background is just allowed to be the background. And the structure of your program guarantees that the latter will gracefully disappear once the former is complete.

Perhaps a more philosophical way to put it: if a concurrent operation computes, but there is no one there to consume its result, does it actually exist? Under the guarantees of strict structured concurrency, the answer is no. And the code you would have written to make it stop? That doesn't need to exist either.

The end. PRIOR DRAFT BEGINS HERE LEFT HERE FOR EDITING PURPOSES.

People who come to Effection for the first time, even those who are already familiar with structured concurrency, are often surprised by how aggressively it tears down child tasks. They'll spawn a few concurrent operations, return from the parent, and discover that every single child has been cancelled. Not joined. Not awaited. Cancelled.

import { run, sleep, spawn } from "effection";

await run(function* () {
  yield* spawn(function*(){
	  yield* sleep(1000);
	  console.log('one second');
  );

  yield* spawn(function*(){
	  yield* sleep(2000);
	  console.log('two seconds');
  );

  console.log("done");
});

This program prints done and exits right away. Both sleepers? Gone. If you're coming from a structured concurrency background like Python or Swift, this might feel wrong. In those systems, a parent scope waits for all of its children to complete before it exits. That's the guarantee. That's what makes it structured. So what is Effection doing here?

The answer is that Effection is applying a refined version of structured concurrency; one that imposes more rigid constraints on the lifetime of each task. We arrived at this behavior after years of iteration, and we call it strict structured concurrency.

Structured concurrency as we know it

Nathaniel J. Smith changed the game back in 2018 when he published "Notes on structured concurrency, or: Go statement considered harmful." (If you haven’t read it, then you should right now. It’s so good!) Its core insight, which is now widely accepted, is that concurrent tasks, like local variables, should have their lifetimes aligned with the lexical scope in which they appear. In other words, every child task lives inside a parent, and the parent does not exit until every child is accounted for without exception.

This is the guarantee that defines structured concurrency. But what does it mean in practice?

In Smith’s original conception, it means that when an open scope has tasks spawned into it, it will wait until every task is finished before closing.

// pseudocode
with classic {
  scope.start(taskA)  // runs for 1 second
  scope.start(taskB)  // runs for 2 seconds
  scope.start(taskC)  // runs for 3 seconds
}
// ← doesn't reach here until all three are done (3 seconds)

Control flows in the top, stuff happens, control flows out the bottom, but in all cases the ledger of concurrent tasks is exactly the same as when it entered. This simple constraint is a straight up super-power because it allows us to build and compose abstractions that can nevertheless contain all kinds of side-effects and state. The safety is real.

The cancellation tax

But what happens when you want to leave a scope before its children are done?

Under the standard model, it’s roll your own. The scope implicitly holds the door open until everyone finishes, so if you want an early exit, you need to reach for a cancellation mechanism to wind down the children yourself. Each system is different (In Effection, you generally use Task.halt())

For example, this code shows a spinner while downloading and returning user information.

function* getUserInfo(userId) {
  let spinner = yield* spawn(function* () {
    yield* showSpinner({ style: "circle" });
  });

  let [user, groups] = yield* all([fetchUser(userId), fetchGroups(userId)]);

  yield* spinner.halt();

  return { ...user, groups };
}

To recap, this code:

1. shows the spinner
2. grabs the user info
3. halts the spinner
4. return the user info

This works, and it is safe, and it is correct. But in practice, we found that we were having to explicitly manage the lifetime of tasks like the spinner constantly.

Imagine we extended our operation by adding a timeout.

function* getUserInfo(userId, timeoutMs) {
  let spinner = yield* spawn(function* () {
    yield* showSpinner({ style: "circle" });
  });

  let timeout = yield* spawn(function* () {
    yield* sleep(timeoutMS);
    throw new Error("timed out");
  });

  let [user, groups] = yield* all([fetchUser(userId), fetchGroups(userId)]);

  yield* spinner.halt();
  yield* timeout.halt();

  return { ...user, groups };
}

This begins the countdown right after the spinner. Then, right before we’re ready to return, just as with the spinner, we tear it down.

Additions like this kept happening over and over again. From maintaining a “keep alive” heartbeat on a web socket, to periodically flushing OTEL metrics, the pattern that began to emerge was that there were actually two types of tasks: one whose lifecycle always seemed to just work itself out, and another that always had to be managed.

Foreground and background

The tasks whose lifecycles "just worked out" were the ones whose values represented the heart of the computation. In the example above, to return a combination of fetchUsers() and fetchGroups() is the literal definition of what it means to getUserInfo(). They are the pure components used to express the scope’s algorithm which is why we call them foreground tasks.

Then there are the other tasks: the spinner, the timeout, the heartbeat. These don’t participate in the algorithm, and they don't produce a value that the scope consumes. Instead they are there to produce a persistent side-effect that supports the foreground while it runs. That’s what makes them background tasks.

Stated more formally, a foreground task is one whose result is explicitly consumed by the foreground, whereas a background task is one whose result is never consumed by the foreground.

To see this difference in action, let’s take our original example, but instead of consuming the user info directly, let’s start by spawning the fetch in a background task first.

function* getUserInfo(userId) {
  let spinner = yield* spawn(function* () {
    yield* showSpinner({ style: "circle" });
  });

  let info = yield* spawn(function* () {
    return yield* all([fetchUser(userId), fetchGroups(userId)]);
  });

  // both `spinner` and `info` are now running in the background.

  let [user, groups] = yield* info; // ← `info` is "pulled" into forgeground

  // only `spinner` remains in the background... shut it down
  yield* spinner.halt();

  return { ...user, groups };
}

Notice how there is no need to manage the lifecycle of the info task. It just happened naturally because the foreground requires the values of user and groups in order compute its return value. In other words, the info task must be completed by the time we get to the end of the function. If it weren’t, then we wouldn’t be at the end of the function, now would we? As a result, the lifetime of a foreground task is always naturally aligned with the lifetime of its scope.

On the other hand, the lifetime of a background task is not naturally aligned with the lifetime of its scope. A background task’s natural lifetime can be long. It can be short. Quite often it is infinite. But whatever the case, what sets it apart from the foreground is that once its scope completes, it has no further reason to exist.

A spinner whose downloads are complete? A heartbeat nobody is listening for? A collector with no more metrics to flush? These aren't tasks that need to be “managed.” These are tasks that just need to go away.

Under the standard model, background tasks hold the scope open just like foreground tasks do, which means that the programmer is required to explicitly manage the lifecycle of each and every task in the background. That's the cancellation tax. It's a tax on your algorithm paid in the form of the stuff that isn’t in it.

The “strict” refinement

Strict structured concurrency, the kind built into Effection, adds a new constraint to the existing guarantee: a child may not outlive its parent. When a scope reaches its end, anything remaining in the background is instructed to immediately shut down.

What does this mean for our getUserInfo example? It means the teardown of the spinner and the timeout just disappear from the code:

function* getUserInfo(userId, timeoutMs) {
  yield* spawn(function* () {
    yield* showSpinner({ style: "circle" });
  });

  yield* spawn(function* () {
    yield* sleep(timeoutMs);
    throw new Error("timed out");
  });

  let [user, groups] = yield* all([fetchUser(userId), fetchGroups(userId)]);

  return { ...user, groups };
}

The foreground expresses the algorithm, and the background is spun up to support it. When the foreground completes and its scope exits, the background, which is no longer needed, is automatically reclaimed. What's left is just the code that matters.

One way to think about it is like the memory resources that hold variable references in a function’s stack frame. When a function exits, those memory resources are automatically recycled. You don’t have to deallocate them explicitly, you don’t even have to think about them because the lifetime of the memory is tied to the function’s scope. Strict structured concurrency does the same thing for tasks. The background tasks are automatically reclaimed when the foreground moves on, so that it's one less thing you carry.

The guarantees still hold

Strict structured concurrency is still structured concurrency.

When a background task is shut down automatically, it is not terminated outright. The parent still waits, just as it would under the standard model, for every child to run all of its cleanup paths. The result computed by the foreground will not be reported to the caller until they are complete.

Consider this example that starts a task running in the background, and then promptly finishes.

import { main, sleep, spawn } from "effection";

await main(function* () {
  yield* spawn(function* () {
    try {
      yield* sleep(2000);
    } finally {
      yield* sleep(500);
      console.log("cleanup complete");
    }
  });
  console.log("done");
});

This will print “done” immediately, wait 500 milliseconds, and then print "cleanup complete" before exiting. This is because upon exit, the background task will be halted and its finally {} block must be run. The strict refinement doesn't weaken the guarantee. It just makes it more intuitive by making orderly shutdown the default rather than something you have to arrange by hand.

Focus on the algorithm

However the deepest consequence of the strict variant of structured concurrency is where it focuses attention. The foreground is in the foreground. The background is just allowed to be the background. And the structure of your program guarantees that the latter will gracefully disappear once the former is complete.

Perhaps a more philosophical way to put it: if a concurrent operation computes, but there is no one there to consume its result, does it exist? Under strict structured concurrency, the answer is no. And the code you would have written to make it stop? That doesn't need to exist either.