Any desire to define behavior of add_on_miss=true table WITHOUT data plane deletions of expired entries?

[jfingerh]

Currently if we have a table T1 that uses this combination of features:

• add_on_miss = true; in table properties
• idle_timeout_with_auto_delete = true; in table properties
• add_entry used to add entries to the table in the data plane

Then T1 seems to be a very useful kind of table. The data plane can add entries without the control plane having to do so, presumably at very high rate in the data plane, and the control plane need not be bothered to add such table entries.

Similarly, the data plane will auto-delete entries that have expired, without the control plane having to bother to do so. Presumably high quality PNA implementations would support as high a rate for auto-deleting expired entries as it supports for adding new entries in the data plane, at least long term if not over arbitrary duration time intervals.

The question raised by this issue is whether we wish to expect PNA implementations to support the following slightly different combination of features for a table (which I will call T2 for an example name):

• add_on_miss = true; in table properties
• idle_timeout_with_auto_delete = false; in table properties, either explicitly, or implicitly because no such table property is mentioned in the definition of the table (the default value of this property is false if not explicitly given, according to current plans)
• psa_idle_timeout = true; in table properties
• add_entry used to add entries to the table in the data plane

T2 is similar to T1 in that the data plane can add entries, without the control plane being required to do so.

T2 is different than T1 in that the data plane WILL NOT auto-delete expired entries. Instead it will notify the control plane of expired entries, each notification containing some identifying information about the key of the expired entry. It is up to the control plane to decide what to do about such notifications. For example, the control plane writer could choose to delete every entry for which it receives such an idle timeout notification as soon as possible. Another control plane writer could choose to perform some additional sanity checks on the entry, and delete some such entries quickly, but leave other entries in table T2 for an arbitrarily long period of time.

The main reason I ask is that for such a table, the control plane now has a potential burden to try to either:

• keep up with the high rate of data-plane-added entries
• OR, don't bother trying to keep up, but if the data plane adds entries to the point where the table's capacity is reached, it will likely remain at its maximum capacity for long periods of time, while the control plane deletes entries relatively slowly compared to the data plane attempted add rate. Also during such times, many data plane calls to add_entry would fail to add new entries.

Is this combination worth supporting? The behavior seems clear to define, but the usefulness seems questionable.

[jfingerh]

More generally, it seems we should document one of the alternatives below about tables with data plane add-on-miss, and also data plane delete-on-expire:

• PNA implementations need not provide any way to guarantee exporting of information from the data plane about all entries added, or deleted. They can export such information in a best-effort fashion via mechanisms like mirrored packets and digest messages.
• PNA implementation should provide ways to guarantee exporting of information from the data plane about all entries added, or deleted. That mechanism is .

[jfingerh]

Note that the current pna.p4 definition of the add_entry() extern function currently requires an ExpireTimeProfileId_t parameter, which really only makes sense if the table also has pna_idle_timeout not equal to NO_TIMEOUT.

If anyone wants to support add-on-miss tables with pna_idle_timeout = NO_TIMEOUT, that parameter is unnecessary, and there should probably be another extern function that adds an entry without that parameter.

As mentioned above, though, this seems unlikely to be desired, at least for the most common use cases of add-on-miss tables.