DEV Community

Cover image for Step Functions Activity vs. Callback: When to Poll
Sascha Mirow-Stanitzok
Sascha Mirow-Stanitzok

Posted on Originally published at sascha-ms.de

Step Functions Activity vs. Callback: When to Poll

A Step Functions activity might be the most underrated feature in AWS Step Functions. Most teams know the callback with a task token: the workflow drops a message with a token into a queue, and some worker reports back later with the result. An activity flips that around. The worker asks for work on its own, picks it up and reports the result. No queue, no inbound connection, just outbound HTTPS.

In my post on Step Functions timeouts and heartbeats, activities only got a passing mention. I want to fix that here, because the question “activity or callback?” keeps coming up in architecture discussions and rarely gets a clean answer. You get the differences, a clear decision guide and a complete TypeScript worker you can try right away.

In short

A Step Functions activity is a queue with an ARN: the workflow places a task there, a worker fetches it via long polling and reports the result back with the task token. That fits especially well for workers that cannot accept inbound connections, such as computers in a retail shop or containers that run all the time. The biggest pitfall is polling itself: too few open polls, polls cut off during shutdown and missing timeouts lead to tasks that hang for minutes.

What you will learn in this article

  • How a Step Functions activity works and how it differs from a callback
  • When a polling worker pays off and when a callback is the better choice
  • How to set up the activity, state machine and alarm with AWS CDK
  • What a TypeScript worker with several pollers, heartbeats and clean shutdown looks like
  • Which typical mistakes leave tasks hanging

What is a Step Functions activity?

In Step Functions, an activity is first of all just a name with an ARN. It contains no code and no runtime. You create it (in the console, with the CLI or as infrastructure as code) and reference its ARN in the Resource field of a task state. When an execution reaches that state, Step Functions schedules a task and waits.

The actual work is done by a so-called activity worker. That can be any program able to open HTTPS connections to AWS: a process on an EC2 instance, a container in ECS, a computer in the office or a mini PC in a shop. The worker calls the GetActivityTask API and receives two things: the input as a JSON string and a task token. Later it uses this token to report back via SendTaskSuccess or SendTaskFailure. For longer jobs it sends SendTaskHeartbeat in between as a sign of life.

Long polling in a nutshell

GetActivityTask uses long polling. In other words, the request does not return empty right away when there is nothing to do. Step Functions keeps the connection open for up to 60 seconds and answers as soon as a task shows up. If nothing arrives in that time, the API returns an empty task token and the worker simply asks again. A waiting worker therefore uses hardly any compute time, and new tasks still arrive with almost no delay.

Activity or callback: the difference

Both patterns use the same mechanism under the hood: a task token that uniquely identifies a waiting execution. The difference lies in how the task reaches the worker. With a .waitForTaskToken callback, Step Functions pushes the work out through an integration, for example as an SQS message, a Lambda invocation or an EventBridge event, and includes the token from the context object ($$.Task.Token). With an activity, the worker pulls the work itself.

Diagram: a callback with task token pushes work via SQS/Lambda, while an activity worker fetches it via GetActivityTask

Callback and activity compared: who moves the task to the worker?

In day-to-day use, the two options differ in a few points that matter for your decision:

Criterion Step Functions activity Callback with task token
Who moves the task? The worker fetches it with GetActivityTask Step Functions sends it through an integration
Extra resource None, just the activity itself Queue, topic, event bus or function
Lambda as worker Poor fit, because Lambda would have to poll constantly Great fit, for example via SQS with an event source mapping
Buffering and dead-letter queue No DLQ of its own, tasks wait in the activity Everything SQS or EventBridge offers
Express workflows Not supported Not supported
Typical use Long-running workers without inbound connections Event-driven processing, approvals, third-party systems

One point is often overlooked: a callback via SQS is a pull for the worker as well. It reads from the queue, after all. So the real difference is whether there is one more service between Step Functions and the worker that you have to run, monitor and pay for. With an activity, Step Functions takes on the role of the queue as well.

When a Step Functions activity pays off

I reach for an activity when at least one of the following cases applies. If none of them does, the callback with a task token is usually the better choice.

Workers without inbound connections

The classic case is a computer that AWS cannot reach and should not reach: a PC in a retail shop behind a router, a machine on a factory floor or a server in your own data center. The worker only needs outbound HTTPS to the Step Functions endpoint. There are no open ports, no port forwarding and no VPN tunnel. For many security teams, that is a strong argument.

Long-running containers and special hardware

If your worker runs around the clock anyway, say an ECS service with lots of memory or an EC2 instance with a GPU, polling costs practically nothing extra. You skip the queue in between and have one less mechanism that can go wrong.

When to stick with the callback

If the worker should be a Lambda function, use the callback. A Lambda function that polls in a loop pays for every second of waiting. AWS itself points out in its best practices that 100 polling functions cost a hundred times as much as one function with 100 polling threads. You should also stay with the callback if you need the strengths of a queue: a dead-letter queue, redrive, fan-out to several receivers or metrics on queue length. Not sure which service fits at all? The AWS service finder helps with a rough first choice.

🎬 How a Step Functions activity flows – this animated diagram only runs on my site: watch it on sascha-ms.de

The example: printing contracts at a bike rental

For the example, we stay with the fictional bike rental “Speichenwerk”. When someone books a bike online, a workflow starts. One of its steps prints the rental contract on the printer at the shop counter. The printer is attached to the shop PC, and the shop PC sits behind a simple internet router that allows no inbound connections.

This is exactly where a Step Functions activity fits. In the morning, the shop PC starts a small Node.js worker that polls the activity speichenwerk-print-contract. When a task comes in, it prints the contract, sends heartbeats while doing so and reports the result. If the PC fails, Step Functions notices the missing heartbeat after 30 seconds and schedules the task again.

Step by step: Step Functions activity with CDK and TypeScript

You build the infrastructure with AWS CDK and the worker with TypeScript and the AWS SDK for JavaScript v3. I checked all files with tsc --noEmit, cdk synth and Vitest.

Prerequisites

  • Node.js 22 or newer and npm
  • An AWS account where you may create Step Functions, IAM and CloudWatch resources
  • AWS CLI v2 with a configured profile
  • Basic knowledge of Step Functions (states, executions)

Step 1: Create the project

Create an empty directory and install the dependencies. These are the versions I used to test the example.

mkdir speichenwerk-activity && cd speichenwerk-activity
npm init -y
npm install @aws-sdk/client-sfn@3.1169.0 @smithy/node-http-handler@4.14.2
npm install -D aws-cdk-lib@2.277.0 aws-cdk@2.1148.1 constructs@10.8.1 typescript@7.0.2 tsx@4.23.15 vitest@5.1.0 aws-sdk-client-mock@4.1.0 @types/node@26.6.6
mkdir bin lib src test
Enter fullscreen mode Exit fullscreen mode

In package.json, add "type": "module" and the scripts "test": "vitest run" and "worker": "tsx src/main.ts". The tsconfig.json and cdk.json look like this:

{
  "compilerOptions": {
    "target": "ES2023",
    "module": "NodeNext",
    "moduleResolution": "NodeNext",
    "strict": true,
    "esModuleInterop": true,
    "skipLibCheck": true,
    "types": ["node"]
  },
  "include": ["bin", "lib", "src", "test"]
}
Enter fullscreen mode Exit fullscreen mode
{
  "app": "npx tsx bin/app.ts"
}
Enter fullscreen mode Exit fullscreen mode

Step 2: The CDK stack

The stack creates four things: the activity, a standard state machine with the print step, an IAM role for the shop PC and a CloudWatch alarm.

// lib/contract-print-stack.ts
import { CfnOutput, Duration, Stack, type StackProps } from 'aws-cdk-lib';
import * as cloudwatch from 'aws-cdk-lib/aws-cloudwatch';
import * as iam from 'aws-cdk-lib/aws-iam';
import * as sfn from 'aws-cdk-lib/aws-stepfunctions';
import * as tasks from 'aws-cdk-lib/aws-stepfunctions-tasks';
import type { Construct } from 'constructs';

export class ContractPrintStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    // The activity is just a "queue with an ARN" – it contains no code
    const printActivity = new sfn.Activity(this, 'PrintContractActivity', {
      activityName: 'speichenwerk-print-contract',
    });

    const prepare = new sfn.Pass(this, 'PrepareContract', {
      parameters: {
        'rentalId.$': '$.rentalId',
        'customerName.$': '$.customerName',
        'bikeId.$': '$.bikeId',
      },
    });

    const printContract = new tasks.StepFunctionsInvokeActivity(this, 'PrintContract', {
      activity: printActivity,
      // No sign of life for 30 s means the worker is considered lost
      heartbeatTimeout: sfn.Timeout.duration(Duration.seconds(30)),
      // Hard upper limit, including the wait time in the activity queue
      taskTimeout: sfn.Timeout.duration(Duration.minutes(10)),
      resultPath: '$.print',
    });

    printContract.addRetry({
      errors: ['States.HeartbeatTimeout'],
      maxAttempts: 2,
      interval: Duration.seconds(5),
      backoffRate: 2,
    });

    const printFailed = new sfn.Fail(this, 'PrintFailed', {
      error: 'ContractNotPrinted',
      cause: 'The rental contract could not be printed',
    });
    printContract.addCatch(printFailed, { resultPath: '$.error' });

    const stateMachine = new sfn.StateMachine(this, 'RentalCheckout', {
      stateMachineName: 'speichenwerk-rental-checkout',
      definitionBody: sfn.DefinitionBody.fromChainable(
        prepare.next(printContract).next(new sfn.Succeed(this, 'ReadyForPickup')),
      ),
      stateMachineType: sfn.StateMachineType.STANDARD,
      timeout: Duration.minutes(30),
    });

    // Role for the shop PC: may only poll and answer this one activity
    const workerRole = new iam.Role(this, 'ShopWorkerRole', {
      assumedBy: new iam.AccountRootPrincipal(),
      description: 'Activity worker in the shop (print station)',
    });
    printActivity.grant(
      workerRole,
      'states:GetActivityTask',
      'states:SendTaskSuccess',
      'states:SendTaskFailure',
      'states:SendTaskHeartbeat',
    );

    // Tasks waiting longer than 60 s for a worker? Workers are missing or overloaded
    printActivity
      .metricScheduleTime({ statistic: 'Maximum', period: Duration.minutes(5) })
      .createAlarm(this, 'WorkerMissingAlarm', {
        threshold: 60_000,
        evaluationPeriods: 1,
        treatMissingData: cloudwatch.TreatMissingData.NOT_BREACHING,
        alarmDescription: 'Print tasks are waiting too long for the shop worker',
      });

    new CfnOutput(this, 'ActivityArn', { value: printActivity.activityArn });
    new CfnOutput(this, 'StateMachineArn', { value: stateMachine.stateMachineArn });
    new CfnOutput(this, 'WorkerRoleArn', { value: workerRole.roleArn });
  }
}
Enter fullscreen mode Exit fullscreen mode
// bin/app.ts
import { App } from 'aws-cdk-lib';
import { ContractPrintStack } from '../lib/contract-print-stack.js';

const app = new App();
new ContractPrintStack(app, 'ContractPrintStack');
Enter fullscreen mode Exit fullscreen mode

A few places deserve a second look. The PrintContract state has two time limits. heartbeatTimeout with 30 seconds detects a crashed worker quickly. taskTimeout with ten minutes limits the total duration of the state, including the time a task waits for a free worker. Without this limit and without an overall timeout on the state machine, an execution would wait until the maximum run time if the shop PC is switched off. For standard workflows that is one year.

The retry on States.HeartbeatTimeout schedules the task again after a crash, so another worker (or the same one after a restart) can take over. All other errors go through the catch into the PrintFailed state.

The ShopWorkerRole may only poll and answer this single activity. The alarm uses the ActivityScheduleTime metric, which is the time a task stays scheduled before a worker picks it up. If it rises above one minute, the workers cannot keep up or only run part of the time. If no worker runs at all, a task is never picked up. That is why you should also monitor the ActivitiesTimedOut metric or failed executions.

Step 3: The print logic

The print logic is deliberately simple. It validates the input and simulates printing with a delay. The AbortSignal matters here: if the task token expires, printing stops cleanly instead of carrying on into the void.

// src/print-contract.ts
import { setTimeout as sleep } from 'node:timers/promises';

export interface PrintJob {
  rentalId: string;
  customerName: string;
  bikeId: string;
}

export class InvalidPrintJobError extends Error {
  override readonly name = 'InvalidPrintJob';
}

export function parsePrintJob(input: unknown): PrintJob {
  const job = input as Partial<PrintJob> | null;
  for (const field of ['rentalId', 'customerName', 'bikeId'] as const) {
    if (typeof job?.[field] !== 'string' || job[field].length === 0) {
      throw new InvalidPrintJobError(`Field "${field}" is missing or empty`);
    }
  }
  return job as PrintJob;
}

// Simulates the printer at the shop counter (takes a few seconds)
export async function printContract(input: unknown, signal: AbortSignal) {
  const job = parsePrintJob(input);
  const printMs = Number(process.env.PRINT_DURATION_MS ?? 3000);
  await sleep(printMs, undefined, { signal });
  return {
    rentalId: job.rentalId,
    printer: process.env.PRINTER_NAME ?? 'counter-1',
    printedAt: new Date().toISOString(),
  };
}
Enter fullscreen mode Exit fullscreen mode

The custom error type InvalidPrintJob later ends up as the error name in SendTaskFailure. That lets you react to it specifically in the state machine, for example with a dedicated catch that reports the case to the counter.

Step 4: The activity worker

Now for the core piece. The worker starts several poll loops in parallel and at the same time limits how many tasks it processes at once. Each task gets its own heartbeat interval.

// src/activity-worker.ts
import {
  GetActivityTaskCommand,
  SendTaskFailureCommand,
  SendTaskHeartbeatCommand,
  SendTaskSuccessCommand,
  SFNClient,
  TaskTimedOut,
} from '@aws-sdk/client-sfn';
import { NodeHttpHandler } from '@smithy/node-http-handler';
import { Agent } from 'node:https';
import { setTimeout as sleep } from 'node:timers/promises';

export type JobHandler = (input: unknown, signal: AbortSignal) => Promise<unknown>;

export interface WorkerOptions {
  activityArn: string;
  workerName: string;
  /** How many GetActivityTask calls may be open at the same time */
  pollers: number;
  /** How many tasks the machine processes at the same time */
  maxParallelJobs: number;
  /** Heartbeat interval, well below heartbeatTimeout */
  heartbeatMs: number;
}

export function createSfnClient(region?: string, maxSockets = 50): SFNClient {
  return new SFNClient({
    region,
    requestHandler: new NodeHttpHandler({
      connectionTimeout: 5_000,
      // Step Functions holds a poll open for up to 60 s – leave some headroom
      requestTimeout: 70_000,
      httpsAgent: new Agent({ keepAlive: true, maxSockets }),
    }),
  });
}

class Slots {
  private waiters: Array<() => void> = [];
  constructor(private free: number) {}

  async acquire(): Promise<void> {
    if (this.free > 0) {
      this.free--;
      return;
    }
    await new Promise<void>((resolve) => this.waiters.push(resolve));
  }

  release(): void {
    const next = this.waiters.shift();
    if (next) next();
    else this.free++;
  }
}

export class ActivityWorker {
  private stopping = false;
  private readonly running = new Set<Promise<void>>();
  private readonly slots: Slots;

  constructor(
    private readonly client: SFNClient,
    private readonly opts: WorkerOptions,
    private readonly handler: JobHandler,
  ) {
    this.slots = new Slots(opts.maxParallelJobs);
  }

  async run(): Promise<void> {
    const loops = Array.from({ length: this.opts.pollers }, (_, i) => this.pollLoop(i));
    await Promise.all(loops);
    await Promise.allSettled([...this.running]);
  }

  /** Start no new polls; let running polls and tasks finish cleanly */
  stop(): void {
    this.stopping = true;
  }

  private async pollLoop(index: number): Promise<void> {
    let failures = 0;
    while (!this.stopping) {
      // Only poll when we can process a task right away
      await this.slots.acquire();
      if (this.stopping) {
        this.slots.release();
        break;
      }

      let taskToken: string | undefined;
      let input = '{}';
      try {
        const task = await this.client.send(
          new GetActivityTaskCommand({
            activityArn: this.opts.activityArn,
            workerName: `${this.opts.workerName}-${index}`,
          }),
        );
        taskToken = task.taskToken || undefined;
        input = task.input ?? '{}';
        failures = 0;
      } catch (err) {
        this.slots.release();
        failures++;
        console.error(`[poller ${index}] GetActivityTask failed:`, (err as Error).name);
        await sleep(Math.min(30_000, 500 * 2 ** failures));
        continue;
      }

      if (!taskToken) {
        // No work for 60 s: simply poll again
        this.slots.release();
        continue;
      }

      const job = this.process(taskToken, input).finally(() => {
        this.slots.release();
        this.running.delete(job);
      });
      this.running.add(job);
    }
  }

  private async process(taskToken: string, rawInput: string): Promise<void> {
    const abort = new AbortController();
    const heartbeat = setInterval(() => {
      this.client.send(new SendTaskHeartbeatCommand({ taskToken })).catch((err) => {
        // Token expired: Step Functions is no longer waiting for us
        if (err instanceof TaskTimedOut) abort.abort(err);
      });
    }, this.opts.heartbeatMs);

    try {
      const output = await this.handler(JSON.parse(rawInput), abort.signal);
      await this.client.send(
        new SendTaskSuccessCommand({ taskToken, output: JSON.stringify(output ?? {}) }),
      );
    } catch (err) {
      if (err instanceof TaskTimedOut || abort.signal.aborted) {
        console.warn('Task token expired, discarding result');
        return;
      }
      const error = err instanceof Error ? err : new Error(String(err));
      await this.client
        .send(
          new SendTaskFailureCommand({
            taskToken,
            error: error.name.slice(0, 256),
            cause: error.message.slice(0, 32_768),
          }),
        )
        .catch((sendErr) => console.error('SendTaskFailure failed:', sendErr.name));
    } finally {
      clearInterval(heartbeat);
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Three decisions are built into this code. First, the SDK client gets a requestTimeout of 70 seconds. According to the API reference, the client timeout should be at least 65 seconds, which is five seconds more than the maximum time the service holds a poll. Second, a loop only polls when a processing slot is free. Otherwise the worker would accept tasks it cannot work on immediately, and their clock would already be running. Third, the heartbeat reacts to TaskTimedOut: if Step Functions reports that the token has expired, the worker aborts the work via the AbortSignal and does not send a result anymore.

Step 5: Start script with clean shutdown

The start script reads its configuration from environment variables and handles SIGINT and SIGTERM.

// src/main.ts
import { hostname } from 'node:os';
import { ActivityWorker, createSfnClient } from './activity-worker.js';
import { printContract } from './print-contract.js';

const activityArn = process.env.ACTIVITY_ARN;
if (!activityArn) {
  console.error('Please set ACTIVITY_ARN');
  process.exit(1);
}

const pollers = Number(process.env.POLLERS ?? 4);
const worker = new ActivityWorker(
  createSfnClient(process.env.AWS_REGION, pollers + 10),
  {
    activityArn,
    workerName: process.env.WORKER_NAME ?? hostname(),
    pollers,
    maxParallelJobs: Number(process.env.MAX_PARALLEL_JOBS ?? 2),
    heartbeatMs: 10_000,
  },
  printContract,
);

for (const signal of ['SIGINT', 'SIGTERM'] as const) {
  process.once(signal, () => {
    console.log(`${signal} received – open polls may take up to 60 s to finish`);
    worker.stop();
  });
}

console.log(`Worker started, ${pollers} pollers`);
await worker.run();
console.log('Worker stopped');
Enter fullscreen mode Exit fullscreen mode

When shutting down, the worker deliberately does not abort running polls. A poll that is being assigned a task at the very moment you cut the connection loses the token. The task then hangs until the heartbeat timeout kicks in. It is better to wait up to 60 seconds until all open polls return on their own.

Step 6: Tests with Vitest

With aws-sdk-client-mock you can test the worker without an AWS account. The three tests cover success, invalid input and an expired token.

// test/activity-worker.test.ts
import {
  GetActivityTaskCommand,
  SendTaskFailureCommand,
  SendTaskSuccessCommand,
  SFNClient,
  TaskTimedOut,
} from '@aws-sdk/client-sfn';
import { mockClient } from 'aws-sdk-client-mock';
import { beforeEach, describe, expect, it } from 'vitest';
import { ActivityWorker, type JobHandler } from '../src/activity-worker.js';
import { printContract } from '../src/print-contract.js';

const sfnMock = mockClient(SFNClient);
const options = {
  activityArn: 'arn:aws:states:eu-central-1:111122223333:activity:demo',
  workerName: 'test',
  pollers: 1,
  maxParallelJobs: 1,
  heartbeatMs: 60_000,
};

function runOnce(handler: JobHandler) {
  const worker = new ActivityWorker(new SFNClient({ region: 'eu-central-1' }), options, async (input, signal) => {
    worker.stop();
    return handler(input, signal);
  });
  return worker.run();
}

describe('ActivityWorker', () => {
  beforeEach(() => {
    sfnMock.reset();
    process.env.PRINT_DURATION_MS = '1';
  });

  it('reports the result with SendTaskSuccess', async () => {
    sfnMock.on(GetActivityTaskCommand).resolvesOnce({
      taskToken: 'token-1',
      input: JSON.stringify({ rentalId: 'r-42', customerName: 'Mia', bikeId: 'b-7' }),
    });
    sfnMock.on(SendTaskSuccessCommand).resolves({});

    await runOnce(printContract);

    const calls = sfnMock.commandCalls(SendTaskSuccessCommand);
    expect(calls).toHaveLength(1);
    expect(JSON.parse(calls[0].args[0].input.output!)).toMatchObject({ rentalId: 'r-42' });
  });

  it('reports invalid tasks with SendTaskFailure', async () => {
    sfnMock.on(GetActivityTaskCommand).resolvesOnce({
      taskToken: 'token-2',
      input: JSON.stringify({ rentalId: 'r-43' }),
    });
    sfnMock.on(SendTaskFailureCommand).resolves({});

    await runOnce(printContract);

    const calls = sfnMock.commandCalls(SendTaskFailureCommand);
    expect(calls[0].args[0].input).toMatchObject({ taskToken: 'token-2', error: 'InvalidPrintJob' });
  });

  it('silently discards the result when the token has expired', async () => {
    sfnMock.on(GetActivityTaskCommand).resolvesOnce({ taskToken: 'token-3', input: '{}' });
    sfnMock
      .on(SendTaskSuccessCommand)
      .rejects(new TaskTimedOut({ message: 'expired', $metadata: {} }));

    await runOnce(async () => ({ ok: true }));

    expect(sfnMock.commandCalls(SendTaskFailureCommand)).toHaveLength(0);
  });
});
Enter fullscreen mode Exit fullscreen mode
npx tsc --noEmit
npx vitest run
Enter fullscreen mode Exit fullscreen mode

Expected result: Test Files 1 passed (1) and Tests 3 passed (3).

Step 7: Deploy and try it out

Deploy the stack and note the three outputs ActivityArn, StateMachineArn and WorkerRoleArn.

# only needed for the first CDK deployment in an account and Region
npx cdk bootstrap
npx cdk deploy

# profile that assumes the worker role (values from the stack output)
aws configure set profile.shop.role_arn "<WorkerRoleArn>"
aws configure set profile.shop.source_profile default

# start the worker (first terminal)
ACTIVITY_ARN="<ActivityArn>" AWS_PROFILE=shop AWS_REGION=eu-central-1 npm run worker
Enter fullscreen mode Exit fullscreen mode

In a second terminal, start an execution and check the result after a few seconds:

STATE_MACHINE_ARN="<StateMachineArn>"
EXECUTION_ARN=$(aws stepfunctions start-execution \
  --state-machine-arn "$STATE_MACHINE_ARN" \
  --input '{"rentalId":"r-1001","customerName":"Mia Example","bikeId":"b-17"}' \
  --query executionArn --output text)

sleep 10
aws stepfunctions describe-execution --execution-arn "$EXECUTION_ARN" \
  --query '{status: status, output: output}'
Enter fullscreen mode Exit fullscreen mode

The status should be SUCCEEDED, and under print in the output you will find the worker’s response with rentalId, printer and printedAt. Then try the failure case: stop the worker with Ctrl+C, start an execution and watch it in the console. Without a worker it ends in the PrintFailed state with the error ContractNotPrinted once the time limits run out. To clean up, npx cdk destroy is all you need.

How many pollers does an activity worker need?

This question is trickier than it looks. GetActivityTask hands out each task token exactly once. If a token gets lost on its way to the worker, for example because of a network error, further polls can be blocked for up to 60 seconds. For production systems, AWS therefore recommends keeping at least 100 polls open for each activity ARN, with pollers that run separately from the actual work.

For a shop PC that prints a few contracts per hour, that is oversized. My worker does separate polling and work cleanly, but it only opens as many polls as there are free processing slots. For a throughput worker in ECS, I would calculate differently: raise the number of pollers significantly, set maxSockets in the HTTP agent to match and run several instances. Node.js works well for this, because thanks to asynchronous I/O many open long polls need hardly any resources.

Typical mistakes with Step Functions activities

The following mistakes cause tasks to hang or run twice. Almost all of them can be avoided with a few lines of code.

A client timeout that is too short

In its Node handler, the AWS SDK for JavaScript v3 sets no request timeout by default. Trouble starts when you configure a short timeout of, say, ten seconds for all clients, or when a proxy on the network closes idle connections early. Then every empty poll fails with an error, and in the worst case tokens get lost. Set the timeout for the Step Functions client explicitly to at least 65 seconds.

No timeout on the task state

Without TimeoutSeconds, an activity state waits until the maximum execution duration is reached. If no worker is running, you might not notice for days. Always set both limits and additionally monitor ActivitiesTimedOut or the wait time.

Duplicate work after a heartbeat timeout

If the worker crashes after printing but before SendTaskSuccess, the retry schedules the task again. The result: two printed contracts. With paper that is annoying, with a payment it would be expensive. So make the work idempotent, for example with a task ID that the worker checks in a small table or file before doing the work.

Activities in Express workflows

Express workflows support neither activities nor .waitForTaskToken or .sync. Such a definition simply does not work there. If you need a waiting step, a standard workflow is a must.

One activity for everything

Activities are not versioned. If you change the input format in an incompatible way, old workers no longer understand the new tasks. In that case AWS advises creating a new activity with its own name. So plan names like print-contract-v2 from the start instead of quietly changing the format.

Cost and security

Standard workflows are billed per state transition, not by waiting time. An activity state that waits five minutes for the printer therefore costs no more than one that finishes after a second. Calls to GetActivityTask and SendTaskHeartbeat are subject to throttling limits per account and Region. In Europe (Ireland), for example, each API has a bucket of 3,000 requests with a refill rate of 500 requests per second, while Regions such as Frankfurt get 1,500 with 300 per second. Do the math before you start hundreds of workers with short heartbeat intervals.

Two points matter for security. The shop PC needs AWS credentials. Do not store long-lived access keys there; for computers outside AWS, IAM Roles Anywhere with certificates is the better option. Also, the input of every execution ends up in the execution history. So only pass the data the worker really needs. For a real contract in the example, the booking ID would probably be enough, and the worker would load the details itself.

My take

At first glance, activities look old-fashioned. They date back to the early days of Step Functions, and in most serverless architectures a callback via SQS or EventBridge really is the better choice. Still, I think it is a mistake to ignore them completely. As soon as part of your process runs on hardware or in networks you do not want reachable from outside, the activity is the leanest tool AWS offers for it.

What convinces me is the simplicity. There is no queue, no event source mapping and no dead-letter queue to configure. What makes me careful is the responsibility you take on in return: polling, heartbeats, shutdown and idempotency are entirely up to you. If you do not build them properly, you get hanging tasks that are hard to track down. My advice: use activities at the edges of your system and stick with events at its core.

FAQ

What is the difference between an activity and waitForTaskToken?

With an activity, the worker fetches the task itself via GetActivityTask. With .waitForTaskToken, Step Functions sends the task with the token out through an integration such as SQS, Lambda or EventBridge. In both cases the response goes back via SendTaskSuccess or SendTaskFailure.

Can a Lambda function be an activity worker?

Technically yes, economically it rarely makes sense. The function would have to poll constantly and would be billed for the waiting time. For Lambda, the callback via SQS is the better choice.

How long can an activity task wait?

In standard workflows up to one year, limited by the maximum execution duration. Still, always set your own TimeoutSeconds, so executions do not stay open for months when no worker is running.

Do activities work in Express workflows?

No. Express workflows support neither activities nor callbacks with a task token. You need a standard workflow for that.

Your next step

Take a look at your existing workflows: is there a callback that only exists because a worker somewhere outside AWS has to fetch work? Then try the example and compare how much infrastructure you save. And if you already run activities in production: how many pollers do you run per activity, and have you followed the recommendation of 100 open polls? Let me know in the comments.

If you want to dig deeper into time limits, my post on heartbeats in Step Functions covers the details of HeartbeatSeconds. What a callback with human approval looks like is shown in Human in the loop with Step Functions.

All code from this article, ready to run: smirowstanitzok/sascha-ms-examples

Sources

This post was created with AI assistance.


👉 Next step: Working with AWS CDK? Grab my printable AWS CDK cheat sheet with commands, concepts and a stack skeleton.

📬 Stay updated: New articles on serverless, AI and leading tech teams via RSS, browser push or the weekly newsletter (in German).

Originally published on sascha-ms.de. More articles on serverless, AI in everyday development and leading tech teams: sascha-ms.de/en

Top comments (0)