fix(index,router): keep SemanticRouter in step with the keyspace when a cluster UNLINK fails

[vishal-bala]

Motivation

On Redis Cluster, SemanticRouter could persist a config claiming a reference was deleted while its hash was still indexed and still matched. SearchIndex deletes cluster keys one at a time and logs per-key failures rather than raising, so drop_keys returned a short count with no record of which keys survived, and delete_route_references and remove_route removed every reference regardless. Two defects sit on the same path: the per-key handler missed SlotNotCoveredError, which derives from RedisClusterException rather than RedisError (verified on redis-py 8.1.0), and drop_by_filter never returned when a key would not unlink.

Changes

Batch deletes report what they could not remove

_unlink_batch and _delete_batch return (count, failed_keys), aggregated across batches by a private _drop_keys, and the per-key handler also catches RedisClusterException. drop_keys, drop_documents, clear and drop_by_filter keep their signatures and return types.

The router reconciles, persists, then raises

delete_route_references and remove_route remove only the references whose keys actually went away, persist the config, and then raise the new PartialDeletionError. It carries failed_keys and deleted, so a retry targets exactly what is left. The route name now comes from the route_name field stored in each hash rather than from splitting the key, which broke on names containing the separator.

Deleting a route's last reference removes the route

delete_route_references could persist a route with references: [], which Route rejects on load, so SemanticRouter.from_existing failed with ValidationError: References must not be empty. Such a route is now removed, as remove_route already does.

drop_by_filter stops when a batch makes no progress

drop_by_filter stops once a fruitless batch returns only keys that have already failed, reports completed=False, and logs whether the deletes failed or the index lists keys that no longer exist. A single fruitless batch does not end the run, because another client may have deleted exactly those documents in the meantime.

Secondary changes

• BulkResult is exported from redisvl.index, and it and PartialDeletionError are documented in the API reference.
• Per-key failures are logged with %r, so a newline in a key cannot forge a log line.
• Hermetic unit coverage in tests/unit/test_unlink_failure_reconciliation.py, because requires_cluster tests are skipped everywhere including CI, plus an integration test that empties a route and reloads it with from_existing.

Release Notes

On Redis Cluster, SemanticRouter.remove_route and delete_route_references now raise PartialDeletionError when some reference keys cannot be deleted, instead of returning normally. Deleting a route's last reference now removes the route, so code that reads to_dict()["routes"] by position should look routes up by name.

---

Note

Medium Risk
Changes deletion semantics on Redis Cluster (logged partial failures, new PartialDeletionError from router APIs) and drop_by_filter completion behavior; public drop_keys signatures unchanged but callers must handle new router exceptions.

Overview
Fixes Redis Cluster partial-delete behavior so callers that mirror keyspace state (especially SemanticRouter) cannot silently diverge from what is still indexed.

Index layer: Per-key DELETE/UNLINK on cluster now returns (count, failed_keys) (via _drop_keys), catches RedisClusterException (e.g. SlotNotCoveredError) alongside RedisError, and documents that drop_keys may return a short count with only logs—no exception. drop_by_filter detects batches that remove nothing, sets BulkResult.completed=False, and logs whether deletes failed vs. stale index entries. BulkResult is exported from redisvl.index.

Semantic router: remove_route and delete_route_references unlink via _drop_keys, reconcile in-memory/persisted references to keys that actually disappeared, then raise PartialDeletionError (failed_keys, deleted) if any remain. Route names are read from stored hashes (not key splitting). Deleting a route’s last reference drops the route so configs stay loadable by from_existing.

Docs and unit/integration tests cover the new exception, stall behavior, and router reconciliation.

^{Reviewed by Cursor Bugbot for commit 67a6128. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 7d1d500. Configure here.}