Building a Robust Request Queue for Concurrent API calls in Flutter

Introduction: The Reality of Connectivity

In Conakry, the dream of universal financial inclusion through mobile money is often tempered by the harsh reality of network latency. When I design apps for our users, I do not have the luxury of assuming stable 4G connections or high-end device processing power. For many of our users, the app is a lifeline, and a failed transaction—a stalled balance check or a pending money transfer—is not just a technical error; it is a loss of trust in digital finance.

Building a robust request queue for concurrent API calls in Flutter is not about optimizing performance for the sake of speed. It is about architectural resilience. When a user in a low-coverage area taps 'Send Money,' the request must eventually reach our servers, even if the connection drops three times in the process. We are building for the margin, and in the margin, the network is always the enemy. This article explores how to build a queuing mechanism that treats every API call as a durable task rather than a fleeting impulse.

The Problem: When Concurrency Fails

Flutter’s http or dio packages are excellent tools, but they are often used in isolation. A standard implementation simply fires an asynchronous request and hopes for the best. In an unstable environment, this is insufficient. When a user performs multiple actions—refreshing a transaction list, checking a balance, and initiating a transfer—these requests can collide, create race conditions, or worse, exhaust the device's limited memory.

On low-end feature phones running our bridged applications, memory is scarce. If we don’t manage the concurrency of these requests, the Dart VM may experience significant pressure. Furthermore, if the user leaves the app or the OS kills the process due to a connection timeout, pending requests are often lost. A robust queueing system acts as a buffer. It serializes necessary operations, manages retries with exponential backoff, and ensures that the state of our financial ledger is consistent across the network.

Designing the Queue Layer

To build a queue that respects these constraints, we need a service that acts as the gatekeeper for all outgoing traffic. This service must track the status of each operation: pending, in_progress, completed, or failed.

We start by defining a QueuedRequest model. This model stores the payload, the endpoint, and the metadata required to reconstruct the request after a failure. By wrapping our API calls in a dedicated QueueManager, we ensure that even if the app UI is interrupted, the logic governing the request remains intact. We prioritize transactions over metadata requests, ensuring that critical money transfers are never blocked by secondary UI updates like profile picture loads.

// A basic structure for a queued network operation
class QueuedRequest {
  final String id;
  final String endpoint;
  final Map<String, dynamic> data;
  final int priority;
  int retryCount;

  QueuedRequest({
    required this.id,
    required this.endpoint,
    required this.data,
    this.priority = 1,
    this.retryCount = 0,
  });
}

Implementing the Concurrency Handler

Managing concurrency requires more than just a list of tasks. We need a mechanism to limit the number of active calls (the parallelism limit). For our users on edge networks, flooding the connection with ten concurrent requests often leads to zero successful completions due to socket timeouts. We throttle our output to a maximum of two concurrent requests, favoring quality of service over raw throughput.

Here is how we implement the internal loop for the QueueManager using a StreamController or a simple state-based ticker. The queue consumes the pending list, executes the request within a try-catch block, and handles the backoff logic internally.

class QueueManager {
  final List<QueuedRequest> _queue = [];
  bool _isProcessing = false;

  Future<void> add(QueuedRequest request) async {
    _queue.add(request);
    _queue.sort((a, b) => b.priority.compareTo(a.priority));
    _processQueue();
  }

  Future<void> _processQueue() async {
    if (_isProcessing || _queue.isEmpty) return;
    _isProcessing = true;

    while (_queue.isNotEmpty) {
      final current = _queue.removeAt(0);
      try {
        await _executeRequest(current);
      } catch (e) {
        if (current.retryCount < 3) {
          current.retryCount++;
          _queue.insert(0, current); // Re-queue at the front
          await Future.delayed(Duration(seconds: 2 * current.retryCount));
        }
      }
    }
    _isProcessing = false;
  }
}

Strategies for Heterogeneous Connectivity

When we deploy to heterogeneous devices, we must account for the fact that some users are on high-end smartphones while others use our USSD-bridged apps. The bridge itself relies on our backend receiving signals in a specific order. If our Flutter app sends a payment request before the login authentication state is fully synced, the backend will reject the request.

By using the queue, we can enforce 'sequence locks.' A sequence lock ensures that a request with a dependency—such as 'transfer money'—cannot execute until the 'fetch balance' request has successfully completed and returned the latest state. This isn't just a technical convenience; it is a security necessity for financial applications. We must ensure that the user’s view of their wallet is accurate before allowing them to move funds.

Troubleshooting and Optimization

One common issue we face is the 'phantom request.' This happens when a user taps a button multiple times due to slow UI feedback. If the queue doesn't have an idempotent key—a unique identifier for that specific transaction—the user might inadvertently trigger multiple transfers.

Pro-tips for robust request management:

1. Idempotency Keys: Always send a unique client-side UUID with every transaction request. This allows the server to recognize duplicate requests and ignore them if a retry occurs due to a response timeout.
2. Prioritization: Use a simple integer ranking system. Financial transactions must be set to priority: 10, while UI analytics should be priority: 1.
3. Connection Observability: Use the connectivity_plus package to pause the queue when the internet is entirely offline. There is no need to fire requests into a void; wait for the system to signal a change in network state.
4. Persistence: For critical apps, persist the queue to disk (using Hive or SQFlite) before sending. If the app crashes, the queue should resume on the next launch.
5. Exponential Backoff: Never retry immediately. In unstable networks, immediate retries only increase congestion. Use a delay that grows: 2s, 4s, 8s, 16s.

Conclusion: Designing with Empathy

Building for the unbanked is a continuous exercise in empathy. When we write code for these environments, we are not just solving a technical puzzle; we are acknowledging the user's struggle. The request queue is a silent servant. It keeps the transaction moving when the signal fades. It waits patiently for the connection to return, and it handles the retries so the user doesn't have to watch a spinning progress indicator until they feel despair.

In my experience at the fintech lab, I have learned that the most elegant code is the code that is never noticed by the user. A queue that successfully persists through a network drop, silently resolving the request when the signal returns, is a piece of craftsmanship that directly contributes to the digital empowerment of our community. As developers, we must prioritize this level of resilience in every layer of our stack. Whether you are targeting high-end smartphones or bridging to USSD-enabled feature phones, the principles of durability, serialization, and observability remain the same. Build for the connection you have, but architect for the connection you wish you had, and your users will find the reliability they need to participate in the digital economy.