Upgrade cloudant and couchdb lucene indexes

[GMishx]

The current plugin used to communicate with couchdb (cloudant) is no longer maintained. There is another package from IBM called cloudant-java-sdk which supports Java 8, 11, 17, 21! We should plan the migration as early as possible (and it appears to be straight from the readme), but not clashing with ongoing removal of Liferay.

In addition, couchdb-lucene is also archived and no longer maintained, I tried to find an alternative. There is a server recently merged in couchdb codebase called Nouveau which provides the same functionality with lucene query engine, just on different endpoints. However, it being an experimental feature, one has to compile the couchdb codebase to get the server jar. But the community has already planned it for the 3.4 release.
P.s., the server was proposed by the same person who wrote the couchdb-lucene 😃

With the current integration of couchdb-lucene, the plugin ektorplucene is used for searching. But with Nouveau, this plugin will not work out of the box and there is no plugin for nouveau available for now. However, it is simple collection of pojo classes and can simply be rewritten in SW360 using coudant APIs. But removing ektorplucene, we can also remove the ektorp plugin lurking around 😛

[GMishx]

I tried to setup the Nouveau server locally and created the index for user database. It appears to be working as expected.

Here is my design doc with the indexes, experimenting with different type of analyzers available:

{
  "_id": "_design/nouveau_user",
  "nouveau": {
    "users": {
      "index": "function(doc) {\n  if (doc.type == 'user' ) {\n    if (typeof(doc.givenname) == 'string') {\n        index(\"string\", \"givenname\", doc.givenname, {\"store\": true});\n    }\n    if (typeof(doc.email) == 'string') {\n        index(\"string\", \"email\", doc.email, {\"store\": true});\n    }\n    if (typeof(doc.lastname) == 'string') {\n        index(\"text\", \"lastname\", doc.lastname, {\"store\": true});\n    }\n  }\n}",
      "default_analyzer": "english",
      "field_analyzers": {
        "email": "email"
      }
    }
  }
}

And running the queries with lucene syntax:

curl --user "admin:admin" -X POST -H 'Content-Type: application/json' \
  'http://localhost:5984/sw360users/_design/nouveau_user/_nouveau/users' \
  -d '{"q": "lastname:admin? OR lastname:user1", "include_docs": true}'

{
  "total_hits_relation": "EQUAL_TO",
  "total_hits": 3,
  "ranges": null,
  "hits": [
    {
      "order": [
        {
          "value": 0.54726034,
          "@type": "float"
        },
        {
          "value": "3d1a44c19219433da0cf5a35da7e0843",
          "@type": "string"
        }
      ],
      "id": "3d1a44c19219433da0cf5a35da7e0843",
      "fields": {
        "lastname": "User1",
        "givenname": "Test",
        "email": "[email protected]"
      },
      "doc": {
        "_id": "3d1a44c19219433da0cf5a35da7e0843",
        "_rev": "19-e780244d0a32e94aee80923e983a383f",
        "type": "user",
        "email": "[email protected]",
        "userGroup": "USER",
        "externalid": "sw3602",
        "fullname": "Test User1",
        "givenname": "Test",
        "lastname": "User1",
        "department": "DEPARTMENT",
        "wantsMailNotification": true,
        "restApiTokens": [
          {
            "token": "$2a$04$Software360RestApiSaleQ53Zkq1OGwMHX4Y5kUDlIhRzS3/rMam",
            "name": "20240220",
            "createdOn": "2024-02-20 07:38:50",
            "numberOfDaysValid": 80,
            "authorities": [
              "READ",
              "WRITE"
            ],
            "__isset_bitfield": 1
          }
        ],
        "deactivated": false,
        "issetBitfield": "1"
      }
    },
    {
      "order": [
        {
          "value": 1,
          "@type": "float"
        },
        {
          "value": "2bf782c4714a43b2a3c3a469f49139d3",
          "@type": "string"
        }
      ],
      "id": "2bf782c4714a43b2a3c3a469f49139d3",
      "fields": {
        "lastname": "admin1",
        "givenname": "Test",
        "email": "[email protected]"
      },
      "doc": {
        "_id": "2bf782c4714a43b2a3c3a469f49139d3",
        "_rev": "5-88e406b7ffb77f68660fa305018d9c5e",
        "type": "user",
        "email": "[email protected]",
        "userGroup": "ADMIN",
        "externalid": "z0030000",
        "fullname": "Test Admin1",
        "givenname": "Test",
        "lastname": "admin1",
        "department": "DEPARTMENT",
        "wantsMailNotification": true,
        "deactivated": true,
        "issetBitfield": "1"
      }
    },
    {
      "order": [
        {
          "value": 1,
          "@type": "float"
        },
        {
          "value": "c927955df75642c383131bb308e7cfe9",
          "@type": "string"
        }
      ],
      "id": "c927955df75642c383131bb308e7cfe9",
      "fields": {
        "lastname": "Admin2",
        "givenname": "Test",
        "email": "[email protected]"
      },
      "doc": {
        "_id": "c927955df75642c383131bb308e7cfe9",
        "_rev": "2-e95f7566bfc5996a88425dded72d2a68",
        "type": "user",
        "email": "[email protected]",
        "userGroup": "ADMIN",
        "externalid": "z003u7px",
        "fullname": "Test Admin2",
        "givenname": "Test",
        "lastname": "Admin2",
        "department": "DEPARTMENT2",
        "wantsMailNotification": true,
        "deactivated": false,
        "issetBitfield": "1"
      }
    }
  ],
  "counts": null,
  "bookmark": "W1t7InZhbHVlIjowLjU0NzI2MDM0LCJAdHlwZSI6ImZsb2F0In0seyJ2YWx1ZSI6IjNkMWE0NGMxOTIxOTQzM2RhMGNmNWEzNWRhN2UwODQzIiwiQHR5cGUiOiJzdHJpbmcifV0sW3sidmFsdWUiOjEuMCwiQHR5cGUiOiJmbG9hdCJ9LHsidmFsdWUiOiJjOTI3OTU1ZGY3NTY0MmMzODMxMzFiYjMwOGU3Y2ZlOSIsIkB0eXBlIjoic3RyaW5nIn1dXQ=="
}