Pagination on created schemas and cred defs?

[dbluhm]

See: https://github.com/hyperledger/aries-cloudagent-python/blob/anoncreds-rs/aries_cloudagent/anoncreds/issuer.py#L232-L255
And: https://github.com/hyperledger/aries-cloudagent-python/blob/anoncreds-rs/aries_cloudagent/anoncreds/issuer.py#L417-L446

This is similar in behavior to the current interface but as you can see it is using fetch_all to retrieve all of the matching records. In the event your issuer has created lots of schemas/cred defs, this might be a slow operation. Should it use pagination?

[dbluhm]

Perhaps it is best to just use the other values to narrow the results rather than to paginate?

[genaris]

Are you using pagination for other kind of records in ACA-Py? We had recently a discussion regarding this in an AFJ call and we figured out that, even if it's certainly possible to add offset and limit, special care must be taken in case new records are added between calls.

An issue we have is about sorting: it seems that Indy's WQL was not meant to store a massive amount of data (and possibly Askar inherited this design constraints as well). For server deployments like ACA-Py it's a bit tricky because we would probably like to sort records by creation date or last activity date or whatever, and this will not be possible if using encrypted tags.

[swcurran]

Askar does a better job at handling large numbers of records because of improvements in indexing and the like. That said, it does still use encrypted tags. Not using encrypted tags is a big change in the security model. Note that there is a way to specifically use unencrypted tags for specific purposes, and then to use those with ordering and filtering. Is that enough of a compromise, or would you wind up with plaintext tags everywhere?

I’ve heard that in AFJ you are considering going to database engine-managed encryption at rest. Any progress on that? What are the ramifications of enabling that support — impact on the security model, compatibility across DB engines in particular.

[swcurran]

Closing in favor of #3001