Improve Duplicate Suppression with Time Windows and ACK/Data Partitioning

[robekl]

Summary

This change upgrades duplicate suppression from a pure cyclic overwrite model to a bounded time-window model, and separates ACK dedupe from non-ACK packet dedupe. It keeps fixed memory usage and simple lookup behavior while improving resilience under bursty traffic.

Problem

The previous duplicate suppression behavior depended mainly on table capacity and overwrite order. Under sustained or bursty traffic, duplicate keys could be evicted quickly and then treated as new again. ACK-heavy periods could also consume dedupe capacity in ways that indirectly reduced protection for non-ACK packets.

Impact When It Happens

When dedupe protection is pressured, the mesh can experience:

• duplicate/replayed packets being re-accepted,
• unnecessary retransmission/forwarding load,
• increased airtime contention and collision likelihood,
• degraded delivery reliability precisely during high-load periods.

Scope of the Problem

This problem primarily appears in:

• high-traffic windows,
• bursty ACK-heavy exchanges,
• denser topologies where duplicate opportunities are common.

It is less visible in quiet or lightly loaded meshes but still a latent reliability risk.

Description of the Change

The dedupe table now uses:

1. Traffic-class partitioning
   • Separate handling paths for ACK and non-ACK packets.
2. Time-windowed dedupe
   • Per-entry timestamp tracking for both ACK and hash entries.
   • Duplicate match is valid only within a configured dedupe time window.
3. Improved slot selection
   • Prefer empty slots first.
   • Then prefer expired slots.
   • Fall back to cyclic overwrite only when needed.
4. Additional lightweight observability
   • ACK-hit counter,
   • data-hit counter,
   • overwrite-under-pressure counter.

How This Addresses the Problem

• Time windows provide a predictable dedupe horizon independent of immediate table churn.
• Partitioning prevents ACK pressure from directly degrading non-ACK dedupe behavior.
• Expired-slot reuse improves retention of relevant recent entries before cyclic overwrite.
• Overwrite-pressure stats make contention behavior more visible for tuning.

Scope of the Fix

In scope:

• Duplicate suppression internals for ACK and non-ACK packet classes.
• Entry aging logic and bounded reuse/overwrite strategy.
• Related dedupe observability counters.

Out of scope:

• Protocol/wire-format changes.
• Dynamic/adaptive dedupe sizing.
• Multi-partition route-mode (flood/direct) dedupe specialization.
• Broad telemetry/UI exposure of new counters.

Benefits

• Better duplicate/replay resistance under bursty conditions.
• More stable behavior during ACK-heavy traffic.
• Reduced avoidable forwarding work during load spikes.
• Maintains embedded constraints (fixed-size memory, simple bounded loops).

Drawbacks / Tradeoffs

• Slightly higher per-entry state (timestamps).
• Additional branching in dedupe lookups.
• Dedupe semantics are now explicitly window-based rather than purely capacity/overwrite based.
• Startup restore behavior remains conservative (timestamp state is not persisted as active runtime age).

New Complexity Introduced

• Low to moderate:
  • timestamp bookkeeping per dedupe entry,
  • two dedupe windows (ACK/data),
  • expired-slot selection before overwrite.
• Complexity is localized to dedupe internals and remains straightforward.

ROI

High.

• Small implementation surface area.
• No protocol migration or compatibility cost.
• Meaningful reliability gain in real-world high-load scenarios.
• Good balance of robustness improvement versus added complexity.