Thinking About Performance in Real-Time Applications

About this article. This is a general technical explainer, not an implementation log. The project-specific facts are limited to those in the Copartner case study: React and D3.js trading dashboards, Node.js services for trading and risk metrics, a reported 30% reduction in backend data latency, and live-streaming infrastructure supporting a reported 10,000+ concurrent users. Measurement methods, baselines, and techniques are not described and are not implied.

In a real-time interface, being correct and being on time are the same requirement. A chart that shows last minute’s figure is not slightly wrong; it is wrong. And an interface that works for one user can fail for ten thousand for reasons that have nothing to do with its logic.

The thesis of this article is that performance work in real-time systems starts with deciding what to measure, because latency, throughput, and concurrency pull in different directions and a single “faster” does not exist.

Three words that are not synonyms

• Latency is how long one piece of data takes to get from where it originates to where it is used. It is felt by the individual user.
• Throughput is how much data the system moves per unit of time. It is felt by the system’s capacity.
• Concurrency is how many users or connections are active at once. It multiplies everything else.

A system can have excellent throughput and poor latency (batching), or the reverse (handling each item immediately at a high per-item cost). Naming which one you are improving prevents the common mistake of optimizing one and degrading another.

Follow the data end to end

The delay a user perceives is the sum of every stage between the source and the screen:

1. Data arrives from its source.
2. A backend processes it, for example computing metrics.
3. The result is delivered to clients.
4. The client updates its state.
5. The interface draws it.

Each stage can dominate. An engineer who tunes the fourth stage while the second takes most of the time has improved nothing the user can feel. Measure each stage separately before deciding where to work.

Measurement comes first

Performance claims are only as useful as the method behind them. A few principles:

• Define the interval. “Latency” means nothing until you say from where to where.
• Look at distributions, not averages. Averages hide the slow tail that users actually notice. Percentiles such as the 95th and 99th say more.
• Measure under realistic load. A system that is quick at idle may degrade under concurrency.
• Change one thing at a time, and compare against the same baseline.
• Keep the method with the number. A percentage improvement is only interpretable alongside what was measured and how.

This is why a reported figure, such as a 30% reduction in backend data latency, is best read as an outcome of a particular measurement, and why this article does not guess at the details of one.

The backend: Node.js and processing

Node.js runs JavaScript on an event loop. That model is well suited to I/O-heavy work (many connections, many small waits) and poorly suited to long CPU-bound computation, which blocks everything else sharing the loop. For a service computing metrics from live data, the questions that matter are:

• Which work is I/O and which is CPU?
• Is anything expensive being repeated that could be computed once and reused?
• Is data being shaped in ways the client does not need?

// Pseudocode — illustrative, not production source.
on new data:
    update only the metrics affected by this data
    publish the result to subscribers

// Compare with:
on every request:
    recompute all metrics from scratch

The first form does work in proportion to what changed. The second does work in proportion to what exists. Which one is right depends on the data, and the difference grows with scale.

The interface: React and D3.js

On the client, the usual tension is between how often data arrives and how often the screen needs to change. A feed can deliver many updates per second, while the display refreshes a limited number of times per second and a human reads far fewer.

React manages application state and component structure. D3.js is a toolkit for binding data to visual elements and computing scales and shapes, so it is a natural fit for charts. Combining them means deciding who owns the DOM for a chart: letting both manipulate the same elements invites bugs and wasted work.

Practical considerations:

• Batch updates so several data points produce one redraw.
• Redraw only what changed instead of rebuilding a whole chart.
• Limit the amount of data drawn. Thousands of points on a small chart are invisible to the eye and costly to render.
• Keep data and view separate so that a slow render does not stall data handling.

Concurrency and fan-out

When many clients consume the same live data, the system’s job becomes fan-out: take one result and deliver it to many recipients. Two principles help:

• Do shared work once. If every client’s data derives from the same computation, compute it once and distribute it.
• Keep per-client work small. Whatever happens per connection is multiplied by the number of connections.

Concurrency also exposes limits that single-user testing never touches: memory per connection, open sockets, and how the system behaves when many clients arrive or disconnect together. A reported scale such as 10,000+ concurrent users describes the capacity a system supported, not a recipe for reaching it.

Streaming design considerations

For live streaming, as for live data generally, a few design questions recur:

• What is the freshness requirement? Real-time is a spectrum; knowing the target tells you what to trade.
• What happens to slow consumers? A client that cannot keep up should not hold back everyone else.
• How does the system degrade? Under overload, deciding what to drop or delay is better than failing unpredictably.
• Where are the single points of failure? Anything every client depends on deserves attention first.

Finding the bottleneck

A reliable routine is to measure, locate the largest contributor, change it, and measure again. The bottleneck moves: once the slowest stage improves, another becomes the slowest. Stop when the result meets the requirement, not when the number looks impressive.

Conclusion

Real-time performance is less about clever tricks than about discipline: define what you are measuring, find where the time actually goes, and respect that latency, throughput, and concurrency are different problems. Improvements that cannot be measured cannot be trusted, and numbers without a method cannot be compared.

For the confirmed scope of one such engagement, see the Copartner case study.