From 004e943242c9f979d8555cca42b6daf40732cd99 Mon Sep 17 00:00:00 2001 From: jzonthemtn Date: Mon, 27 May 2024 09:15:52 -0400 Subject: [PATCH 1/5] Updating documentation. --- README.md | 323 +++++++++++++++++- documentation/documentation.md | 159 --------- documentation/queries/dsl_queries.md | 81 ----- documentation/queries/sql_queries.md | 310 ----------------- documentation/query_id.md | 37 -- documentation/schemas.md | 214 ------------ .../org/opensearch/ubi/UbiActionFilter.java | 2 +- src/main/resources/queries-mapping.json | 2 +- 8 files changed, 315 insertions(+), 813 deletions(-) delete mode 100644 documentation/documentation.md delete mode 100644 documentation/queries/dsl_queries.md delete mode 100644 documentation/queries/sql_queries.md delete mode 100644 documentation/query_id.md delete mode 100644 documentation/schemas.md diff --git a/README.md b/README.md index b259ee8..45c10a7 100644 --- a/README.md +++ b/README.md @@ -1,28 +1,331 @@ # OpenSearch User Behavior Insights -This project is the OpenSearch plugin for storing and managing search requests and responses along with events produced client-side. +This repository contains the OpenSearch plugin for the User Behavior Insights (UBI) capability. This plugin +facilitates persisting client-side events (e.g. item clicks, scroll depth) and OpenSearch queries for the purpose of analyzing the data +to improve search relevance and user experience. -Related Links: -* OpenSearch UBI RFC - https://github.com/opensearch-project/OpenSearch/issues/12084 +UBI and this plugin project was originally proposed in the [OpenSearch UBI RFC](https://github.com/opensearch-project/OpenSearch/issues/12084). -## Building the Plugin +## UBI Schemas -Build the plugin +Please note that this repository is the implementation of the UBI plugin for OpenSearch. For details on the JSON Schema used by UBI to send and receive queries and events please see the [UBI](https://github.com/o19s/ubi) repository and the links below. +* [Query Request Schema](https://o19s.github.io/ubi/docs/html/query.request.schema.html) +* [Query Response Schema](https://o19s.github.io/ubi/docs/html/query.response.schema.html) +* [Event Schema](https://o19s.github.io/ubi/docs/html/event.schema.html) + +## Getting Help + +* For questions or help getting started, please find us in the [OpenSearch Slack](https://opensearch.org/slack.html) in the `#plugins` channel. +* For bugs or feature requests, please create [a new issue](https://github.com/o19s/opensearch-ubi/issues/new/choose). + +## User Quick Start + +### Installing the Plugin + +To get started, download the plugin zip file from the [releases](https://github.com/o19s/opensearch-ubi/releases). Next, install the plugin into OpenSearch with the following command: + +``` +bin/opensearch-plugin install file:/opensearch-ubi-1.0.0-os2.14.0.zip +``` + +To create the UBI indexes called `ubi_queries` and `ubi_events`, send a query to an OpenSearch index: + +``` +curl -s http://localhost:9200/your-index/_search -H "Content-Type: application/json" -d' + { + "ext": { + "ubi": { + } + }, + "query": { + "match": { + "name": "toner" + } + } + }' +``` + +These indexes can also be created manually by using the [mapping files](https://github.com/o19s/opensearch-ubi/tree/2.14.0/src/main/resources): + +``` +curl -s -X PUT "http://localhost:9200/ubi-events" -H "Content-Type: application/json" --data-binary @/path/to/events-mapping.json +curl -s -X PUT "http://localhost:9200/ubi-queries" -H "Content-Type: application/json" --data-binary @/path/to/events-queries.json +``` + +### Capturing Queries + +Queries sent to OpenSearch containing a `ubi` section in the `ext` block will be captured by the plugin and stored in the `ubi_queries` index. For example: + +``` +curl -s http://localhost:9200/your-index/_search -H "Content-Type: application/json" -d' + { + "ext": { + "ubi": { + "query_id": "12300d16cb-b6f1-4012-93ebcc49cac90426" + } + }, + "query": { + "match": { + "name": "toner" + } + } + }' +``` + +As shown in the query above, the presence of the `ubi` block in the `ext` section causes the UBI plugin to capture the query and store it in the `ubi_queries` index. +The `ubi` block can contain the following parameters. All parameters are optional. + +* `query_id` - A unique identifier for the query. If not provided, the plugin will generate a random UUID for the query and return the UUID in the query response. The `query_id` can be in any format but we use UUIDs in the examples. +* `user_query` - The user-entered query. This is meant to be the actual text the user provided to initiate the search. +* `client_id` - A unique identifier for the originator of the query. The client may be a user, an application, or any other originator of the query. The `client_id` can be in any format but we use UUIDs in the examples. +* `object_id` - The name of a field in the index that contains a unique identifier for each result. If not provided, the `_id` field is used. + +Following is an example query that provides the parameters: + +``` +curl -s http://localhost:9200/your-index/_search -H "Content-Type: application/json" -d' + { + "ext": { + "ubi": { + "query_id": "12300d16cb-b6f1-4012-93ebcc49cac90426", + "user_query": "small basketball", + "client-id": "c4af7ca2-d6d2-4326-a41f-e616ebdd3d7b", + "object_id": "product_name" + } + }, + "query": { + "match": { + "name": "toner" + } + } + }' +``` + +Similar to a query response, query responses will also include a `ubi` section in the `ext` block. + +``` +{ + "took": 3, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 2.2413535, + "hits": [ + { + "_index": "ecommerce", + "_id": "968447", + "_score": 2.2413535, + "_source": { + "id": "968447", + "name": "Cyan Toner Cartridge" + } + } + ] + }, + "ext": { + "ubi": { + "query_id": "12300d16cb-b6f1-4012-93ebcc49cac90426" + } + } +} +``` + +The only field present in the query response will be the `query_id`. The value of the `query_id` is the same as the `query_id` provided in the query, or, if not provided, a random UUID. + +### Indexed Queries + +To see how the query is indexed, you can search the `ubi_queries` index: + +``` +curl -s http://localhost:9200/ubi_queries/_search -H "Content-Type: application/json" | jq +{ + "took": 3, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 1, + "hits": [ + { + "_index": "ubi_queries", + "_id": "6CrooY8BzxoaOvIPKtWj", + "_score": 1, + "_source": { + "query_response_id": "e4bdd289-0875-421f-bc34-aa71eb8e1cb3", + "user_query": "toner", + "query_id": "12300d16cb-b6f1-4012-93ebcc49cac90426", + "query_response_object_ids": [ + "9" + ], + "client_id": null, + "query": "{\"query\":{\"match\":{\"name\":{\"query\":\"toner\",\"operator\":\"OR\",\"prefix_length\":0,\"max_expansions\":50,\"fuzzy_transpositions\":true,\"lenient\":false,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"boost\":1.0}}},\"ext\":{\"query_id\":\"12300d16cb-b6f1-4012-93ebcc49cac90426\",\"user_query\":\"toner\",\"client_id\":null,\"object_id\":null,\"query_attributes\":{\"system\":\"my_system\",\"experiment\":\"exp_1\"}}}", + "timestamp": 1716408298072 + } + } + ] + } +``` + +Each indexed query will have the following fields: + +* `query_response_id` - A unique identifier for the query response. +* `user_query`- Corresponds to the `user_query` in the query request. +* `query_id` - Corresponds to the `query_id` in the query request, or a random UUID if a `query_id` was not provided in the query request. +* `query_response_object_ids` - A list of the values of the `object_id` field in the document. +* `client_id` - Corresponds to the `client_id` in the query request. +* `query` - The raw query that was provided to OpenSearch. +* `timestamp` - The Unix timestamp when the query was indexed. + +### Capturing Events + +The UBI plugin does *not* provide a way to capture client-side events. Sending client-side events requires action on the client to send the events to OpenSearch for indexing. + +The following is an example of a client-side event. [Additional examples are available.](https://github.com/o19s/ubi/tree/main/samples) + +``` +curl -s -X POST http://localhost:9200/ubi_events/_doc/ -H "Content-Type: application/json" -d' + { + "action_name": "page_exit", + "user_id": "1821196507152684", + "query_id": "00112233-4455-6677-8899-aabbccddeeff", + "session_id": "c3d22be7-6bdc-4250-91e1-fc8a92a9b1f9", + "page_id": "/docs/latest/", + "timestamp": "2024-05-16T12:34:56.789Z", + "message_type": "INFO", + "message": "On page /docs/latest/ for 3.35 seconds", + "event_attributes": { + "position":{}, + "object": { + "idleTimeoutMs": 5000, + "currentIdleTimeMs": 250, + "checkIdleStateRateMs": 250, + "isUserCurrentlyOnPage": true, + "isUserCurrentlyIdle": false, + "currentPageName": "http://localhost:4000/docs/latest/", + "timeElapsedCallbacks": [], + "userLeftCallbacks": [], + "userReturnCallbacks": [], + "visibilityChangeEventName": "visibilitychange", + "hiddenPropName": "hidden" + } + } + }' +``` + +## Analyzing Queries and Client-Side Events + +With the queries and client-side events we can use OpenSearch's SQL capability to analyze the data. + +### Queries with zero results + +We can identify queries with zero results by querying either the `ubi_queries` or `ubi_events` indexes as shown below. Both queries should return the same value. + +```sql +select + count(0) +from ubi_queries +where query_response_object_ids is null +``` + +```sql +select + count(0) +from ubi_events +where action_name='on_search' and event_attributes.data.data_detail.query_data.query_response_object_ids is null +order by timestamp +``` + +### Most Common Client-Side Events + +Find the most common client-side events: + +```sql +select + action_name, count(0) Total +from ubi_events +group by action_name +order by Total desc +``` + +action_name|Total +|---|---| +on_search|3199 +brand_filter|3112 +button_click|3150 +type_filter|3149 +product_hover|3132 +product_sort|3115 +login|2458 +logout|1499 +new_user_entry|208 + +### Client-Side Events Associated ith Queries + +All client-side events that are associated with a query should have the same `query_id`. + +```sql +select + action_name, count(0) Total +from ubi_events +where query_id is not null +group by action_name +order by Total desc +``` +action_name|Total +|---|---| +on_search|1329 +brand_filter|669 +button_click|648 +product_hover|639 +product_sort|625 +type_filter|613 +logout|408 + +## Development + +If you find bugs or want to request a feature, please create [a new issue](https://github.com/o19s/opensearch-ubi/issues/new/choose). For questions or to discuss how UBI works, please find us in the [OpenSearch Slack](https://opensearch.org/slack.html) in the `#plugins` channel. + +The plugin provides an implementation of an `ActionFilter` plugin that can capture and index queries, a `SearchExtBuilder` that provides the UBI parameters, and the object classes used to index the queries. Testing is done by YAML rest tests and unit tests. + +### Building and Testing + +The plugin can be built using Gradle: ``` ./gradlew build ``` -Build the OpenSearch docker image and add the plugin, then start the containers: +To test and debug, build the OpenSearch docker image that contains the built plugin and then start the containers: ``` -docker compose build -docker compose up +docker compose build && docker compose up ``` -See the [Quick Start](documentation/documentation.md#quick-start) section in the documentation for a complete example of how to initialize the plugin and store events and queries. +Or to start a three-node OpenSearch cluster: + +``` +docker compose build +docker compose -f docker-compose-cluster.yaml up +``` -## Releasing +### Releasing Run `tag-and-release.sh` to create and push a tag to GitHub. The tag will kick off the GitHub Action to build the release. After running `tag-and-release.sh`, update the version numbers in `gradle.properties` and the `Dockerfile` and commit the changes. ## License diff --git a/documentation/documentation.md b/documentation/documentation.md deleted file mode 100644 index ad74672..0000000 --- a/documentation/documentation.md +++ /dev/null @@ -1,159 +0,0 @@ -# OpenSearch User Behavior Insights - -This repository contains the OpenSearch plugin for the User Behavior Insights (UBI) capability. This plugin -facilitates persisting client-side events (e.g. item clicks, scroll depth) and OpenSearch queries for the purpose of analyzing the data -to improve search relevance and user experience. - -## Quick Start - -Build the plugin. Note that you will have to match up the JDK 11 on your system to java home in the `gradle.properties` file: - -`./gradlew build` - -Build the OpenSearch docker image and add the plugin, then start the containers: - -``` -docker compose build -docker compose up -``` - -Or to start a three-node OpenSearch cluster: - -``` -docker compose build -docker compose -f docker-compose-cluster.yaml up -``` - -Initialize the `awesome` UBI store: - -``` -curl -X PUT "http://localhost:9200/_plugins/ubi/awesome?index=ecommerce&object_id=id" -``` - -Send an event to the `awesome` store: - -``` -curl -X POST http://localhost:9200/_plugins/ubi/mystore -H "Content-Type: application/json" -d ' -{ - "action_name": "search", - "user_id": "98fcf189-4fa8-4322-b6f5-63fbb6b556c9", - "timestamp": 1705596607509 -}' -``` - -Get events: - -``` -curl -s http://localhost:9200/.awesome_events/_search | jq -``` - -Do a search of the `ecommerce` index: - -``` -curl -s http://localhost:9200/ecommerce/_search -H "X-ubi-store: awesome" | jq -``` - -Get queries: - -``` -curl -s http://localhost:9200/.awesome_queries/_search | jq -``` - -Delete the store: - -``` -curl -X DELETE http://localhost:9200/_plugins/ubi/awesome -``` - -## UBI Store - -The plugin has a concept of a "store", which is a logical collection of the events and queries. A store consists of two indices. One -index is used to store events, and the other index is for storing queries. - -### OpenSearch Data Mappings -Ubi has 2 primary indices: -- **UBi Queries** stores all queries and results. -- **UBi Events** store that the Ubi client writes events to. -*Please follow the [schema deep dive](./schemas.md) to understand how these two indices make Ubi into a causal framework for search.* - -## Plugin API - -The plugin exposes a REST API for managing UBI stores and persisting events. - -| Method | Endpoint | Purpose | -|--------|-----------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `PUT` | `/_plugins/ubi/{store}?index={index}&object_id={object_id}` |

Initialize a new UBI store for the given index. The `object_id` is optional and allows for providing the name of a field in the `index`'s schema to be used as the unique result/item ID for each search result. If not provided, the `_id` field is used.

| -| `DELETE` | `/_plugins/ubi/{store}` | Delete a UBI store | -| `GET` | `/_plugins/ubi` | Get a list of all UBI stores | -| `POST` | `/_plugins/ubi/{store}` | Index an event into the UBI store | -| `TRACE` | `/_plugins/ubi` | For temporary developer debugging | - -### Creating a UBI Store - -To create a UBI store to contain events and queries, send a `PUT` request: - -``` -curl -X PUT http://localhost:9200/_plugins/ubi/mystore?index=ecommerce -``` - -### Deleting a UBI Store - -To delete a store, send a `DELETE` request: - -``` -curl -X DELETE http://localhost:9200/_plugins/ubi/mystore -``` - -This will delete the UBI store and all contained events and queries. Please use this with caution. - -### Get a List of UBI Stores - -To get a list of stores, send a `GET` request: - -``` -curl -X GET http://localhost:9200/_plugins/ubi -``` - -### Persist a Client-Side Event into a UBI Store - -To persist a client-side event into a store, send a `POST` request where the body of the request is the event: - -``` -curl -X POST http://localhost:9200/_plugins/ubi/mystore -H "Content-Type: application/json" -d ' -{ - "action_name": "search", - "user_id": "98fcf189-4fa8-4322-b6f5-63fbb6b556c9", - "timestamp": 1705596607509 -}' -``` - -## Capturing Queries with UBI - -### Associating a Query with Client-Side Events - -The plugin passively listens to query requests passing through OpenSearch. Without any extra information, -the plugin cannot associate a query with the client-side events associated with the query. (What user clicked on what to make this query?) - -To make this association, queries need to have a header value that indicates the user ID. - -#### Required Headers - -|Header|Purpose|Required?|Detail| -|---|---|---|---| -|`X-ubi-store`|Tells the plugin which store this query should be persisted to.|Required|| -|`X-ubi-user-id`|Allow for associating client-side user events with the query|Required|| -|`X-ubi-query-id`|The client can provide a query ID. If not provided, the plugin will generate a random query ID. The purpose of the query ID is to uniquely identify the query.|No|[query_id](query_id.md)| -|`X-ubi-session-id`|A session ID to associate with the query.|No|| - -### Example Queries - -The following query tells the plugin that the query being run should be persisted to the store `mystore` and be associated with user ID `john`: - -``` -curl http://localhost:9200/ecommerce/_search -H "X-ubi-store: mystore" -H "X-ubi-user-id: 12345" -``` - -With this query, when the plugin sees a query, the plugin will be able to associate the query with an individual user and know to persist the query in the UBI store `mystore`. - -[Sample SQL queries](documentation\queries\sql_queries.md) -[Sample OpenSearch queries](documentation\queries\dsl_queries.md) diff --git a/documentation/queries/dsl_queries.md b/documentation/queries/dsl_queries.md deleted file mode 100644 index eaf0750..0000000 --- a/documentation/queries/dsl_queries.md +++ /dev/null @@ -1,81 +0,0 @@ - - -```json -GET .ubi_log_events/_search -{ - "size":0, - "aggs":{ - "event_types":{ - "terms": { - "field":"action_name", - "size":10 - } - } - } -} -``` -returns -```json -{ - "took": 1, - "timed_out": false, - "_shards": { - "total": 1, - "successful": 1, - "skipped": 0, - "failed": 0 - }, - "hits": { - "total": { - "value": 10000, - "relation": "gte" - }, - "max_score": null, - "hits": [] - }, - "aggregations": { - "event_types": { - "doc_count_error_upper_bound": 0, - "sum_other_doc_count": 0, - "buckets": [ - { - "key": "brand_filter", - "doc_count": 3084 - }, - { - "key": "product_hover", - "doc_count": 3068 - }, - { - "key": "button_click", - "doc_count": 3054 - }, - { - "key": "product_sort", - "doc_count": 3012 - }, - { - "key": "on_search", - "doc_count": 3010 - }, - { - "key": "type_filter", - "doc_count": 2925 - }, - { - "key": "login", - "doc_count": 2433 - }, - { - "key": "logout", - "doc_count": 1447 - }, - { - "key": "new_user_entry", - "doc_count": 207 - } - ] - } - } -} -``` \ No newline at end of file diff --git a/documentation/queries/sql_queries.md b/documentation/queries/sql_queries.md deleted file mode 100644 index 68096c8..0000000 --- a/documentation/queries/sql_queries.md +++ /dev/null @@ -1,310 +0,0 @@ -# Sample UBI SQL queries -These can be performed on the OpenSearch Dashboards/Query Workbench: -http(s):\//`{server}`:5601/app/opensearch-query-workbench - -## Queries with zero results -Although it's trivial on the server side to find queries with no results, we can also get the same answer by querying the event side. -### Server-side -```sql -select - count(0) -from .ubi_log_queries -where query_response_objects_ids is null -order by user_id -``` - -### Client event-side -```sql -select - count(0) -from .ubi_log_events -where action_name='on_search' and event_attributes.data.data_detail.query_data.query_response_objects_ids is null -order by timestamp -``` - -Both client and server-side queries should return the same number. - - - -## Trending queries -```sql -select - message, count(0) Total -from .ubi_log_events -where - action_name='on_search' -group by message -order by Total desc -``` -message|Total -|---|---| -Virtual flexibility systematic|143 -Virtual systematic flexibility|89 -discrete desk service Cross group|75 -Blanditiis quo sint repudiandae a sit.|75 -Optio id quis alias at.|75 -Consectetur enim sunt laborum adipisci occaecati reiciendis.|70 -Like prepare trouble consider.|68 -User Behavior Insights|65 -cheapest laptop with i9|64 -Cross group discrete service desk|63 -Laptop|61 -Amet maxime numquam libero ipsam amet.|59 -fastest laptop|54 -Voluptas iusto illum eum autem cum illum.|51 -fortalece relaciones e-business|2 -evoluciona comercio electrónico en tiempo real|2 -incentiva canales escalables|1 -incentiva ancho de banda inalámbrica|1 -implementa sistemas de siguiente generación|1 -implementa marcados eficientes|1 - - - -## Event type distribution counts -To make a pie chart like widget on all the most common events: -```sql -select - action_name, count(0) Total -from .ubi_log_events -group by action_name -order by Total desc -``` -action_name|Total -|---|---| -on_search|3199 -brand_filter|3112 -button_click|3150 -type_filter|3149 -product_hover|3132 -product_sort|3115 -login|2458 -logout|1499 -new_user_entry|208 - -### Events associated with queries -Since the `query_id` is set during client-side search, all events that are associated with a query will have that same `query_id`. -To make a pie chart like widget on the most common events preceded by a query: -```sql -select - action_name, count(0) Total -from .ubi_log_events -where query_id is not null -group by action_name -order by Total desc -``` -action_name|Total -|---|---| -on_search|1329 -brand_filter|669 -button_click|648 -product_hover|639 -product_sort|625 -type_filter|613 -logout|408 - - -## Sample Search Odyssy -Find a search in the query log: -```sql -select * -from .ubi_log_queries -where query_id ='1065c70f-d46a-442f-8ce4-0b5e7a71a892' -order by timestamp -``` -(In this generated data, the `query` field is plain text; however in the real implementation the query will be in the internal DSL of the query and parameters.) -query_response_id|query_id|user_id|query|query_response_objects_ids|session_id|timestamp ----|---|---|---|---|---|--- -1065c70f-d46a-442f-8ce4-0b5e7a71a892|1065c70f-d46a-442f-8ce4-0b5e7a71a892|155_7e3471ff-14c8-45cb-bc49-83a056c37192|Blanditiis quo sint repudiandae a sit.|8659955|fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|2027-04-17 10:16:45 - -In the event log -Search for the events that correspond to the query above, `1065c70f-d46a-442f-8ce4-0b5e7a71a892`. -```sql -select - query_id, action_name, message_type, message, event_attributes.data.data_id, event_attributes.data.description, session_id, user_id -from .ubi_log_events -where query_id = '1065c70f-d46a-442f-8ce4-0b5e7a71a892' -order by timestamp -``` -query_id|action_name|message_type|message|event_attributes.data.data_id|event_attributes.data.description|session_id|user_id ----|---|---|---|---|---|---|--- -1065c70f-d46a-442f-8ce4-0b5e7a71a892|product_hover|INQUERY|Focused logistical policy|1692104|HP LaserJet Color CP3525dn Printer Colour 600 x 1200 DPI A4|fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|brand_filter|INFO||||fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|type_filter|INFO|Multi-tiered client-server software|||fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|product_sort|PURCHASE||77499830|SES Creative Charm bracelets|fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|logout|ERROR||||fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|on_search|QUERY|Blanditiis quo sint repudiandae a sit.|1065c70f-d46a-442f-8ce4-0b5e7a71a892||fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|product_purchase|REJECT||1377181|Matrox G55-MDDE32LPDF graphics card GDDR|fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|brand_filter|WARN||||fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|product_sort|PURCHASE|Object-based upward-trending policy|137688|HERMA CD labels A4 Ø 116 mm white paper matt opaque 200 pcs.|fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|product_click|INQUERY||4534016|ASUS BX700 mouse Bluetooth Laser 1200 DPI Right-hand|fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|product_hover|INQUERY||78314263|Tripp Lite DMCCASTER flat panel mount accessory|fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|product_click|INQUERY||2073|HP LaserJet 5100tn 1200 x 1200 DPI A3|fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 -1065c70f-d46a-442f-8ce4-0b5e7a71a892|button_click|WARN||||fa6e3b1c-3212-44d2-b16b-690b4aeddbba_1975|155_7e3471ff-14c8-45cb-bc49-83a056c37192 - -## User sessions -To look at more sessions from the same user above, `155_7e3471ff-14c8-45cb-bc49-83a056c37192`. -```sql -select - user_id, session_id, query_id, action_name, message_type, message, event_attributes.data.data_type, timestamp -from .ubi_log_events -where user_id ='155_7e3471ff-14c8-45cb-bc49-83a056c37192' -order by timestamp -``` -Results are truncated to a few sessions: - -user_id|session_id|query_id|action_name|message_type|message|event_attributes.data.data_type|timestamp ----|---|---|---|---|---|---|--- -user_id|session_id|query_id|action_name|message_type|message|event_attributes.data.data_type|timestamp -155_7e3471ff-14c8-45cb-bc49-83a056c37192|2465d7cf-7123-499c-a510-f5681db2bad8_1967||new_user_entry|INFO|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|2465d7cf-7123-499c-a510-f5681db2bad8_1967||login|ERROR|iniciativa potenciada centrado en el usuario||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|2465d7cf-7123-499c-a510-f5681db2bad8_1967|11a40012-8e70-4cb8-afcb-b7d1214aa0b0|on_search|QUERY|Blanditiis quo sint repudiandae a sit.|query|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||login|ERROR|Switchable actuating methodology||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||product_purchase|INQUERY|Enterprise-wide high-level circuit|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||product_purchase|REJECT||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||product_sort|PURCHASE|Enhanced content-based protocol|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||button_click|INFO|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||brand_filter|INFO|Automated solution-oriented firmware||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||product_sort|PURCHASE||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||button_click|INFO|sinergia dedicada mandatorio||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||product_sort|PURCHASE||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||product_click|INQUERY||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||type_filter|INFO|actitud maximizada virtual||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||product_sort|INQUERY||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|f8500640-1a69-41f0-b2ea-c7b2d7af5ab1_1970||brand_filter|ERROR|Re-contextualized zero administration complexity||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968|71b2903a-9108-4829-96f1-a675e7a635d8|brand_filter|INFO|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968||login|INFO|Optional context-sensitive system engine||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968||button_click|INFO|Sharable background knowledge user||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968||product_hover|PURCHASE|política basado en necesidades multicanal|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968||product_click|INQUERY||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968||button_click|WARN|Customer-focused exuding policy||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968||button_click|WARN|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968||product_sort|INQUERY|paradigma basado en el contexto opcional|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968|71b2903a-9108-4829-96f1-a675e7a635d8|on_search|QUERY|what is ubi?|query|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968|71b2903a-9108-4829-96f1-a675e7a635d8|product_purchase|INQUERY|Ergonomic 24/7 solution|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968|71b2903a-9108-4829-96f1-a675e7a635d8|product_purchase|REJECT|Enhanced uniform methodology|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968|71b2903a-9108-4829-96f1-a675e7a635d8|type_filter|WARN|Seamless didactic info-mediaries||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|8819a35a-7cd8-4365-8c29-6b07fe46f073_1968|71b2903a-9108-4829-96f1-a675e7a635d8|product_sort|REJECT|algoritmo direccional visionario|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||login|INFO|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||button_click|WARN|Enterprise-wide 24hour focus group||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||type_filter|WARN|Balanced cohesive adapter||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||product_click|INQUERY|Ergonomic hybrid instruction set|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||product_purchase|PURCHASE||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||product_sort|INQUERY|Automated zero administration encoding|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||product_sort|INQUERY|conjunto de instrucciones multitarea de tamaño adecuado|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||type_filter|WARN|enfoque heurística opcional||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||button_click|INFO|Multi-channeled optimizing neural-net||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||product_hover|INQUERY||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||product_click|INQUERY|Programmable intangible product|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||product_hover|INQUERY||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||product_purchase|PURCHASE|Grass-roots client-server conglomeration|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||brand_filter|ERROR|Implemented real-time standardization||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|dc7984c6-11b1-40ad-b6a5-b96717139da2_1969||product_sort|PURCHASE|función modular progresivo|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||login|INFO|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||type_filter|WARN|conglomeración maximizada seguro||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||button_click|INFO|Focused regional portal||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||button_click|INFO|definición sistémica virtual||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||type_filter|INFO|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||type_filter|INFO|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||product_purchase|INQUERY||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||product_hover|INQUERY|Seamless directional database|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||product_hover|REJECT|aplicación dinámica robusto|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||product_click|INQUERY|aplicación 4ta generación personalizable|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971||product_click|INQUERY|alianza holística administrado|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971|0e360673-46fe-4912-a10c-0ab90bbb0513|on_search|QUERY|what is ubi?|query|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971|0e360673-46fe-4912-a10c-0ab90bbb0513|product_purchase|REJECT|Diverse intermediate hardware|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971|0e360673-46fe-4912-a10c-0ab90bbb0513|product_click|INQUERY|Advanced contextually-based Graphical User Interface|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971|0e360673-46fe-4912-a10c-0ab90bbb0513|product_purchase|PURCHASE|Ergonomic mission-critical ability|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971|0e360673-46fe-4912-a10c-0ab90bbb0513|product_purchase|INQUERY|Visionary discrete groupware|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|86f9c1e4-428f-4520-acef-883770c0f541_1971|0e360673-46fe-4912-a10c-0ab90bbb0513|logout|WARN|Compatible composite process improvement||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||login|INFO|Upgradable interactive analyzer||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||product_hover|REJECT|Horizontal modular database|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||product_click|INQUERY|Re-engineered interactive knowledge user|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||product_hover|INQUERY|caja de herramientas holística orgánico|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||brand_filter|INFO|Public-key neutral infrastructure||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||brand_filter|INFO|software 24 horas programable||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||product_click|INQUERY||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||product_purchase|REJECT||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||button_click|WARN|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||product_hover|REJECT||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||type_filter|INFO|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|c83a0f59-1ae5-4f63-a45f-b0dcefc7a7d5_1972||logout|INFO|Sharable discrete policy||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|5fb20967-14fe-49e1-93e4-5ab54b0d54a7_1973||login|ERROR|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|5fb20967-14fe-49e1-93e4-5ab54b0d54a7_1973|898fdbfb-ee8f-4a21-a0e6-8acbc46e45f6|on_search|QUERY|Amet maxime numquam libero ipsam amet.|query|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|5fb20967-14fe-49e1-93e4-5ab54b0d54a7_1973|898fdbfb-ee8f-4a21-a0e6-8acbc46e45f6|product_sort|INQUERY||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|5fb20967-14fe-49e1-93e4-5ab54b0d54a7_1973|e3b7319c-2517-4375-bc66-7ff50bfd37f5|on_search|QUERY|Laptop|query|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|5fb20967-14fe-49e1-93e4-5ab54b0d54a7_1973|e3b7319c-2517-4375-bc66-7ff50bfd37f5|brand_filter|INFO|||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|5fb20967-14fe-49e1-93e4-5ab54b0d54a7_1973|e3b7319c-2517-4375-bc66-7ff50bfd37f5|product_sort|REJECT||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|5fb20967-14fe-49e1-93e4-5ab54b0d54a7_1973|e3b7319c-2517-4375-bc66-7ff50bfd37f5|brand_filter|INFO|capacidad 3ra generación multi-capas||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|5fb20967-14fe-49e1-93e4-5ab54b0d54a7_1973|e3b7319c-2517-4375-bc66-7ff50bfd37f5|button_click|INFO|base de trabajo nueva generación distribuido||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|5fb20967-14fe-49e1-93e4-5ab54b0d54a7_1973|e382687a-a853-460a-99e3-9fec9806875e|on_search|QUERY|Cross group discrete service desk|query|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|4aca7a9c-895f-481c-86a0-1419cec4fbcc_1974||login|INFO|Horizontal full-range framework||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|4aca7a9c-895f-481c-86a0-1419cec4fbcc_1974||button_click|INFO|Vision-oriented motivating matrix||2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|4aca7a9c-895f-481c-86a0-1419cec4fbcc_1974||product_hover|PURCHASE|Cross-platform cohesive product|product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|4aca7a9c-895f-481c-86a0-1419cec4fbcc_1974||product_sort|INQUERY||product|2027-04-17 10:16:45 -155_7e3471ff-14c8-45cb-bc49-83a056c37192|4aca7a9c-895f-481c-86a0-1419cec4fbcc_1974||brand_filter|INFO|Multi-layered next generation process improvement||2027-04-17 10:16:45 - - -## List user sessions that logged out without any queries -- This query denotes users without a query_id. Note that this could happen if the client side is not passing the returned query to other events. -```sql -select - user_id, session_id, count(0) EventTotal -from .ubi_log_events -where action_name='logout' and query_id is null -group by user_id, session_id -order by EventTotal desc -``` -user_id|session_id|EventTotal ----|---|--- -100_15c182f2-05db-4f4f-814f-46dc0de6b9ea|1c36712c-44b8-4fdd-8f0d-fdfeab5bd794_1290|1 -175_e5f262f1-0db3-4948-b349-c5b95ff31259|816f94d6-8966-4a8b-8984-a2641d5865b2_2251|1 -175_e5f262f1-0db3-4948-b349-c5b95ff31259|314dc1ff-ef38-4da4-b4b1-061f62dddcbb_2248|1 -175_e5f262f1-0db3-4948-b349-c5b95ff31259|1ce5dc30-31bb-4759-9451-5a99b28ba91b_2255|1 -175_e5f262f1-0db3-4948-b349-c5b95ff31259|10ac0fc0-409e-4ba0-98e9-edb323556b1a_2249|1 -174_ab59e589-1ae4-40be-8b29-8efd9fc15380|dfa8b38a-c451-4190-a391-2e1ec3c8f196_2228|1 -174_ab59e589-1ae4-40be-8b29-8efd9fc15380|68666e11-087a-4978-9ca7-cbac6862273e_2233|1 -174_ab59e589-1ae4-40be-8b29-8efd9fc15380|5ca7a0df-f750-4656-b9a5-5eef1466ba09_2234|1 -174_ab59e589-1ae4-40be-8b29-8efd9fc15380|228c1135-b921-45f4-b087-b3422e7ed437_2236|1 -173_39d4cbfd-0666-4e77-84a9-965ed785db49|f9795e2e-ad92-4f15-8cdd-706aa1a3a17b_2206|1 -173_39d4cbfd-0666-4e77-84a9-965ed785db49|f3c18b61-2c8a-41b3-a023-11eb2dd6c93c_2207|1 -173_39d4cbfd-0666-4e77-84a9-965ed785db49|e12f700c-ffa3-4681-90d9-146022e26a18_2210|1 -173_39d4cbfd-0666-4e77-84a9-965ed785db49|da1ff1f6-26f1-49d4-bd0d-d32d199e270e_2208|1 -173_39d4cbfd-0666-4e77-84a9-965ed785db49|a1674e9d-d2dd-4da9-a4d1-dd12a401e8e7_2216|1 -172_875f04d6-2c35-45f4-a8ac-bc5b675425f6|cc8e6174-5c1a-48c5-8ee8-1226621fe9f7_2203|1 -171_7d810730-d6e9-4079-ab1c-db7f98776985|927fcfed-61d2-4334-91e9-77442b077764_2189|1 -16_581fe410-338e-457b-a790-85af2a642356|83a68f57-0fbb-4414-852b-4c4601bf6cf2_156|1 -16_581fe410-338e-457b-a790-85af2a642356|7881141b-511b-4df9-80e6-5450415af42c_162|1 -16_581fe410-338e-457b-a790-85af2a642356|1d64478e-c3a6-4148-9a64-b6f4a73fc684_158|1 - -Since some of these query-less logouts repeat with some users, here is a query to see which users do this the most: -```sql -select - user_id, count(0) EventTotal -from .ubi_log_events -where action_name='logout' and query_id is null -group by user_id -order by EventTotal desc -``` -user_id|EventTotal ----|--- -87_5a6e1f8c-4936-4184-a24d-beddd05c9274|8 -127_829a4246-930a-4b24-8165-caa07ee3fa47|7 -49_5da537a3-8d94-48d1-a0a4-dcad21c12615|6 -56_6c7c2525-9ca5-4d5d-8ac0-acb43769ac0b|6 -140_61192c8e-c532-4164-ad1b-1afc58c265b7|6 -149_3443895e-6f81-4706-8141-1ebb0c2470ca|6 -196_4359f588-10be-4b2c-9e7f-ee846a75a3f6|6 -173_39d4cbfd-0666-4e77-84a9-965ed785db49|5 -52_778ac7f3-8e60-444e-ad40-d24516bf4ce2|5 -51_6335e0c3-7bea-4698-9f83-25c9fb984e12|5 -175_e5f262f1-0db3-4948-b349-c5b95ff31259|5 -61_feb3a495-c1fb-40ea-8331-81cee53a5eb9|5 -181_f227264f-cabd-4468-bfcc-4801baeebd39|5 -185_435d1c63-4829-45f3-abff-352ef6458f0e|5 -100_15c182f2-05db-4f4f-814f-46dc0de6b9ea|5 -113_df32ed6e-d74a-4956-ac8e-6d43d8d60317|5 -151_0808111d-07ce-4c84-a0fd-7125e4e33020|5 -204_b75e374c-4813-49c4-b111-4bf4fdab6f26|5 -29_ec2133e5-4d9b-4222-aa7c-2a9ae0880ddd|5 -41_f64abc69-56ea-4dd3-a991-7d1fd292a530|5 \ No newline at end of file diff --git a/documentation/query_id.md b/documentation/query_id.md deleted file mode 100644 index e7d68fe..0000000 --- a/documentation/query_id.md +++ /dev/null @@ -1,37 +0,0 @@ -# Basic Ubi flow -**Executive Summary**: Once a user performs search, that search is tied to a `query_id`. Then any following user events until the next search are logged and indexed by the search's `query_id`. - -### Ubi Roles -- **Ubi Plugin**: in charge of initiating an Ubi store, saving all incoming queries associated with an Ubi store, and logging the Ubi events directly. -- **Search Client**: in charge of searching and recieving a `query_id` from the **Ubi Plugin**. This `query_id` is then passed to the **Ubi Client** -- **Ubi Client**: in charge of logging user events, such as onClick, passing the data to the **Ubi Plugin**, with the appropriate `query_id`, if the event follows a search event that produced a `query_id`. - -### Ubi `query_id` Sequence -```mermaid -sequenceDiagram - - User-123->>Search Client: "Show me cute puppies" - box Application - participant Search Client - participant Ubi Client - end - box OpenSearch - participant OpenSearch - participant Ubi Plugin - end - Search Client->>OpenSearch: "puppies" {X-ubi-store: cute-things-index, X-ubi-user-id: user-123} - OpenSearch->> Ubi Plugin: save user query and hits to the query store - Ubi Plugin->>OpenSearch: User-123's query_id is 15c182f2-05db-4f4f-814f-46dc0de6b9ea - OpenSearch->>Search Client: {...results...} query_id: 15c182f2-05db-4f4f-814f-46dc0de6b9ea - Search Client->> Ubi Client: The new query_id is 15c182f2-05db-4f4f-814f-46dc0de6b9ea - User-123->>Ubi Client: item_onClick, user_id:User-123, query_id: 15c182f2-05db-4f4f-814f-46dc0de6b9ea - Ubi Client->> Ubi Plugin: save item_onClick, query_id & user_id to the event store - User-123->>Ubi Client: item_Checkout, user_id:User-123, query_id: 15c182f2-05db-4f4f-814f-46dc0de6b9ea - Ubi Client->> Ubi Plugin: save item_Checkout, query_id & user_id to the event store -``` -1) The user searches for something like "cute puppies" -2) The search client forwards that request to OpenSearch - - The query is transformed under the hood and saved to the query store with the hits - - If no `query_id` is passed in with the request headers, the server generates a new `query_id` and sends it back to the client. - - If the search client passes in a `query_id` with the search, the server will associate the query in the query store with the client's id -3) Once the Ubi Client has a `query_id`, associated with the user's query, all subsequent user events are logged with this `query_id` until the user initiates a new search. \ No newline at end of file diff --git a/documentation/schemas.md b/documentation/schemas.md deleted file mode 100644 index 234e013..0000000 --- a/documentation/schemas.md +++ /dev/null @@ -1,214 +0,0 @@ - -# Key User Behavior Insights concepts -**User Behavior Insights** (Ubi) **Logging** is really just a matter of linking and indexing queries, results and events within OpenSearch. -## Key ID's -Ubi is not functional unless the links between the following are consistently maintained within your Ubi-enabled application: - -- [`user_id`](#user_id) represents a unique user. -- [`object_id`](#object_id) represents an id for whatever item the user is searching for, such as *epc*, *isbn*, *ssn*, *handle*, etc. -- [`query_id`](#query_id) is a unique id for the raw query language executed and the resultant `object_id`'s that the query returned. \ -- [`action_name`](#action_name), though not technically an *id*, the `action_name` tells us what exact action (such as `click` or `add_to_cart`) was taken (or not) with this `object_id`. - -To summarize: the `query_id` signals the beginning of a `user_id`'s *Search Journey*, the `action_name` tells us how the user is interacting with the query results within the application, and [`event_attributes.object.object_id`](#object_id) is referring to the precise query result that the user interacts with. - -## Ubi Roles -- **Search Client**: in charge of searching, and then recieving *objects* from some document index in OpenSearch. -  (1, 2, *5* & 7, below) -- **User Behavior Insights** module: once activated, manages the **Ubi Queries** store in the background, indexing each underlying, technical, DSL, index query with a unique [`query_id`](#query_id) along with all returned resultant [`object_id`](#object_id)'s, and then passing the `query_id` back to the **Search Client** so that events can be linked to this query. -  (3, 4 & *5*, below) -- **objects**: are whatever items the user is searching for with the queries. Activating Ubi involves mapping your real-world objects (via its *isbn*, etc...) to the [`object_id`](#object_id) fields in the schemas below. -- The **Search Client**, if separate from the **Ubi Client**, forwards the indexed [`query_id`](#query_id) to the **Ubi Client**. -   *Note:* We break out the roles of *search* and *Ubi event indexing* here, but many implementations will likely use the same OpenSearch client instance for both roles of searching and index writing. -  (6, below) -- The **Ubi Client** then indexes all user events with this [`query_id`](#query_id) until a new search is performed, and a new `query_id` is generated by **User Behavior Insights** and passed back to the **Ubi Client** -- If the **Ubi Client** interacts with a result *object*, such as `onClick`, that [`object_id`](#object_id), *onClick* [`action_name`](#action_name) and `query_id` are all indexed together, signalling the causal link between the *search* and the *object*. -  (8 & 9, below) - - - -```mermaid -graph LR -style L fill:none,stroke-dasharray: 5 5 -subgraph L["`*Legend*`"] - style ss height:150px - subgraph ss["Standard Search"] - direction LR - - style ln1a fill:blue - ln1a[ ]--->ln1b[ ]; - end - subgraph ubi-leg["Ubi data flow"] - direction LR - - ln2a[ ].->|"`**Ubi interaction**`"|ln2b[ ]; - style ln1c fill:red - ln1c[ ]-->|query_id flow|ln1d[ ]; - end -end -linkStyle 0 stroke-width:2px,stroke:#0A1CCF -linkStyle 2 stroke-width:2px,stroke:red -``` -```mermaid -%%{init: { - "flowchart": {"htmlLabels": false}, - - } -}%% -graph TB - -User--1) raw search string-->Search; -Search--2) search string-->Docs -style OS stroke-width:2px, stroke:#0A1CCF, fill:#62affb, opacity:.5 -subgraph OS[OpenSearch Cluster fa:fa-database] - style E stroke-width:1px,stroke:red - E[( Ubi Events )] - style Docs stroke-width:1px,stroke:#0A1CCF - style Q stroke-width:1px,stroke:red - Docs[(Document Index)] -."3) {DSL...} & [object_id's,...]".-> Q[( Ubi Queries )]; - Q -.4) query_id.-> Docs ; -end - -Docs -- "5) return both query_id & [objects,...]" --->Search ; -Search-.6) query_id.->U; -Search --7) [results, ...]--> User - -style *client-side* stroke-width:1px, stroke:#D35400 -subgraph "`*client-side*`" - style User stroke-width:4px, stroke:#EC636 - User["`**User**`" fa:fa-user] - App - Search - U - style App fill:#D35400,opacity:.35, stroke:#0A1CCF, stroke-width:2px - subgraph App[       UserApp fa:fa-store] - style Search stroke-width:2px, stroke:#0A1CCF - Search( Search Client ) - style U stroke-width:1px,stroke:red - U( Ubi Client ) - end -end - -User -.8) selects object_id:123.->U; -U-."9) index event:{query_id, onClick, object_id:123}".->E; - -linkStyle 1,2,0,6 stroke-width:2px,fill:none,stroke:#0A1CCF -linkStyle 3,4,5,8 stroke-width:2px,fill:none,stroke:red -``` - -## Ubi Stores -There are 2 separate stores for Ubi: -### 1) **Ubi Queries** -All underlying query information and results ([`object_id`](#object_id)'s) are stored in the **Ubi Queries** store, and remains largely invisible in the background. -The only obvious difference will be in the `ubi` stanze of the json response, *which could cause index bloat if one forgets that this is enabled*. - -**Ubi Queries** [schema](../src/main/resources/queries-mapping.json): -Since Ubi manages the **Ubi Queries** store, the developer should never have to write directly to this store (except for importing data). - -- `timestamp` -   A unix timestamp of when the query was received - -- `query_id` -   A unique ID of the query provided by the client or generated automatically. The same query text issued multiple times would generate different `query_id`. - - - `user_id` -   A user ID provided by the client - -- `session_id` -   An optional session ID provided by the client. _This is currently under review of if we keep this_. - -- `query_response_objects_ids` -   This is an array of the `object_id`'s. This *could* be the same id as the `_id` but is meant to be the externally valid id of document/item/product. - - - -### 2) **Ubi Events** -This is the event store that the client side directly indexes events to, linking the event [`action_name`](#action_name), [`object_id`](#object_id)'s and [`query_id`](#query_id)'s together with any other important event information. -Since this schema is dynamic, the developer can add any new fields and structures (such as *user* information, *geo-location* information, etc.) at index time that are not in the current **Ubi Events** [schema](../src/main/resources/events-mapping.json): -- `application` -

- -   (size 100) - name of the application tracking UBI events (e.g. *amazon-shop*, *ABC-microservice*) -- `action_name` -

- -   (size 100) - any name you want to call your event. For example, with *javascript* events, you could include `on_click`, `logon`, `add_to_cart`, `page_scroll`.... _This should be formalized. A list of standard ones and then custom ones._ - -- `query_id` -

- -   (size 100) - ID for some query. Either the client provides this, or the `query_id` is generated at index time by **Ubi Queries**. -- `user_id`. `session_id`, `source_id`

- -   (size 100) - are id's largely at the calling client's discretion for tracking users, sessions and sources (i.e. pages) of the event. - The `user_id` must be consistent in both the **Ubi Queries** and **Ubi Events** stores. - -- `timestamp`: -   UTC-based, unix epoch time. - -- `message_type` - -   (size 100) - originally thought of in terms of ERROR, INFO, WARN, but could be anything useful such as `QUERY` or `CONVERSION`. - Can be used to group `action_name` together in logical bins. _Thinking this should be backend logic in analysis_ - -- `message` - -   (size 256) - optional text message for the log entry. For example, with a `message_type` of `INFO`, people might expect an informational or debug type text for this field, but a `message_type` of `QUERY`, we would expect the text to be more about what the user is searching on. - - -- `event_attributes`'s structure is where any relevant information about the event can be stored. - There are two primary structures in the `event_attributes`: - - **`event_attributes.position`** - structure that contains information on the location of the event origin, such as screen *x,y* coordinates, or the *n*th object out of 10 results, .... - - - `event_attributes.position.ordinal` - -   tracks the *n*th item within a list that a user could select, click (i.e. selecting the 3rd element could be event{`onClick, results[4]`}) - - - `event_attributes.position.{x,y}` - -   tracks x and y values, that the client defines - - - `event_attributes.position.page_depth` - -   tracks page depth of results - - - `event_attributes.position.scroll_depth` - -   tracks scroll depth of page results - - - `event_attributes.position.trail` - -   text field for tracking the path/trail that a user took to get to this location - -

- - - **`event_attributes.object`**, which contains identifying information of the object returned from the query that the user interacts with (i.e.: a book, a product, a post, etc..). - The `object` structure has two ways to refer to the object, with `object_id` being the id that links prior queries to this object: - - - `event_attributes.object.internal_id` is a unique id that OpenSearch can use to internally to index the object, think the `_id` field in the indices. - - `event_attributes.object.object_id` -   is the id that a user could look up amd find the object instance within the **document corpus**. Examples include: *ssn*, *isbn*, *primary_ean*, etc. Variants need to be incorporated in the `object_id`, so for a t-shirt that is red, you would need SKU level as the `object_id`. - Initializing Ubi requires mapping from the **Document Index**'s primary key to this `object_id` - - - `event_attributes.object.object_type` - -   indicates the type/class of object. - - - `event_attributes.object.description` - -   optional description of the object - - - `event_attributes.object.transaction_id` - -   optionally points to a unique id representing a successful transaction - - - `event_attributes.object.to_user_id` - -   optionally points to another user, if they are the recipient of this object, perhaps as a gift, from the user's `user_id` - - `event_attributes.object.object_detail` - -   optional text for further data object details - - - `event_attributes.object.object_detail.json` - -   if the user has a json object representing what was acted upon, it can be stored here; however, note that that could lead to index bloat if the json objects are large. -- *extensible fields*: any new fields by any other names in the json objects that one indexes will dynamically expand this schema to that use-case. diff --git a/src/main/java/org/opensearch/ubi/UbiActionFilter.java b/src/main/java/org/opensearch/ubi/UbiActionFilter.java index 5823cdb..ea1a1af 100644 --- a/src/main/java/org/opensearch/ubi/UbiActionFilter.java +++ b/src/main/java/org/opensearch/ubi/UbiActionFilter.java @@ -200,7 +200,7 @@ public void onResponse(IndicesExistsResponse indicesExistsResponse) { source.put("query_id", queryRequest.getQueryId()); source.put("query_response_id", queryRequest.getQueryResponse().getQueryResponseId()); source.put("query_response_object_ids", queryRequest.getQueryResponse().getQueryResponseObjectIds()); - source.put("user_id", queryRequest.getUserId()); + source.put("client_id", queryRequest.getUserId()); source.put("user_query", queryRequest.getUserQuery()); // The query can be null for some types of queries. diff --git a/src/main/resources/queries-mapping.json b/src/main/resources/queries-mapping.json index 298a18c..8906b52 100644 --- a/src/main/resources/queries-mapping.json +++ b/src/main/resources/queries-mapping.json @@ -16,7 +16,7 @@ "query_response_object_ids": { "type": "keyword" }, - "user_id": { + "client_id": { "type": "keyword" }, "user_query": { From 83f01f5fa518df3623a68852b4d48b0707b6065c Mon Sep 17 00:00:00 2001 From: jzonthemtn Date: Mon, 27 May 2024 09:17:28 -0400 Subject: [PATCH 2/5] Fixing word. --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 45c10a7..d5a6a9b 100644 --- a/README.md +++ b/README.md @@ -101,7 +101,7 @@ curl -s http://localhost:9200/your-index/_search -H "Content-Type: application/j }' ``` -Similar to a query response, query responses will also include a `ubi` section in the `ext` block. +Similar to a query request, query responses will also include a `ubi` section in the `ext` block. ``` { From 9195a5a264af0d0473f5c1119efdcdb37663c3b8 Mon Sep 17 00:00:00 2001 From: jzonthemtn Date: Mon, 27 May 2024 09:40:30 -0400 Subject: [PATCH 3/5] Removing unneeded files. --- README.md | 5 +++++ get-queries.sh | 3 --- install-plugin.sh | 6 ------ search.sh | 16 ---------------- 4 files changed, 5 insertions(+), 25 deletions(-) delete mode 100755 get-queries.sh delete mode 100755 install-plugin.sh delete mode 100755 search.sh diff --git a/README.md b/README.md index d5a6a9b..4d13b3e 100644 --- a/README.md +++ b/README.md @@ -18,6 +18,11 @@ Please note that this repository is the implementation of the UBI plugin for Ope * For questions or help getting started, please find us in the [OpenSearch Slack](https://opensearch.org/slack.html) in the `#plugins` channel. * For bugs or feature requests, please create [a new issue](https://github.com/o19s/opensearch-ubi/issues/new/choose). +## Useful Commands + +* Get the indexed queries: `curl http://localhost:9200/ubi_queries/_search | jq` +* + ## User Quick Start ### Installing the Plugin diff --git a/get-queries.sh b/get-queries.sh deleted file mode 100755 index 1700f16..0000000 --- a/get-queries.sh +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/bash -e - -curl http://localhost:9200/ubi_queries/_search | jq diff --git a/install-plugin.sh b/install-plugin.sh deleted file mode 100755 index aa817f1..0000000 --- a/install-plugin.sh +++ /dev/null @@ -1,6 +0,0 @@ -#!/bin/bash -e - -# Use this convenience script to install the built plugin in a locally running Opensearch. - -DIR=`pwd` -sudo /usr/share/opensearch/bin/opensearch-plugin install file:${DIR}/build/distributions/opensearch-rest-plugin.zip diff --git a/search.sh b/search.sh deleted file mode 100755 index e494566..0000000 --- a/search.sh +++ /dev/null @@ -1,16 +0,0 @@ -#!/bin/bash -e - -curl -s http://localhost:9200/ecommerce/_search -H "Content-Type: application/json" -d' - { - "ext": { - "ubi": { - "query_id": "12300d16cb-b6f1-4012-93ebcc49cac90426", - "user_query": "toner" - } - }, - "query": { - "match": { - "name": "toner" - } - } - }' | jq From ee38ba7e5362ec5339c4479d048aa20ab6939b9f Mon Sep 17 00:00:00 2001 From: jzonthemtn Date: Tue, 28 May 2024 13:31:53 -0400 Subject: [PATCH 4/5] Renaming object_id to object_id_field. --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 4d13b3e..318be7a 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ The `ubi` block can contain the following parameters. All parameters are optiona * `query_id` - A unique identifier for the query. If not provided, the plugin will generate a random UUID for the query and return the UUID in the query response. The `query_id` can be in any format but we use UUIDs in the examples. * `user_query` - The user-entered query. This is meant to be the actual text the user provided to initiate the search. * `client_id` - A unique identifier for the originator of the query. The client may be a user, an application, or any other originator of the query. The `client_id` can be in any format but we use UUIDs in the examples. -* `object_id` - The name of a field in the index that contains a unique identifier for each result. If not provided, the `_id` field is used. +* `object_id_field` - The name of a field in the index that contains a unique identifier for each result. If not provided, the `_id` field is used. Following is an example query that provides the parameters: @@ -95,7 +95,7 @@ curl -s http://localhost:9200/your-index/_search -H "Content-Type: application/j "query_id": "12300d16cb-b6f1-4012-93ebcc49cac90426", "user_query": "small basketball", "client-id": "c4af7ca2-d6d2-4326-a41f-e616ebdd3d7b", - "object_id": "product_name" + "object_id_field": "product_name" } }, "query": { @@ -180,7 +180,7 @@ curl -s http://localhost:9200/ubi_queries/_search -H "Content-Type: application/ "9" ], "client_id": null, - "query": "{\"query\":{\"match\":{\"name\":{\"query\":\"toner\",\"operator\":\"OR\",\"prefix_length\":0,\"max_expansions\":50,\"fuzzy_transpositions\":true,\"lenient\":false,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"boost\":1.0}}},\"ext\":{\"query_id\":\"12300d16cb-b6f1-4012-93ebcc49cac90426\",\"user_query\":\"toner\",\"client_id\":null,\"object_id\":null,\"query_attributes\":{\"system\":\"my_system\",\"experiment\":\"exp_1\"}}}", + "query": "{\"query\":{\"match\":{\"name\":{\"query\":\"toner\",\"operator\":\"OR\",\"prefix_length\":0,\"max_expansions\":50,\"fuzzy_transpositions\":true,\"lenient\":false,\"zero_terms_query\":\"NONE\",\"auto_generate_synonyms_phrase_query\":true,\"boost\":1.0}}},\"ext\":{\"query_id\":\"12300d16cb-b6f1-4012-93ebcc49cac90426\",\"user_query\":\"toner\",\"client_id\":null,\"object_id_field\":null,\"query_attributes\":{\"system\":\"my_system\",\"experiment\":\"exp_1\"}}}", "timestamp": 1716408298072 } } @@ -193,7 +193,7 @@ Each indexed query will have the following fields: * `query_response_id` - A unique identifier for the query response. * `user_query`- Corresponds to the `user_query` in the query request. * `query_id` - Corresponds to the `query_id` in the query request, or a random UUID if a `query_id` was not provided in the query request. -* `query_response_object_ids` - A list of the values of the `object_id` field in the document. +* `query_response_object_ids` - A list of the values of the `object_id_field` field in the document. * `client_id` - Corresponds to the `client_id` in the query request. * `query` - The raw query that was provided to OpenSearch. * `timestamp` - The Unix timestamp when the query was indexed. From 19bdb5d453a768ce9d78fccdc55ff6a5d78d4bfc Mon Sep 17 00:00:00 2001 From: Eric Pugh Date: Tue, 28 May 2024 13:35:45 -0400 Subject: [PATCH 5/5] some tweaks --- README.md | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-) diff --git a/README.md b/README.md index 318be7a..1a01ad4 100644 --- a/README.md +++ b/README.md @@ -8,7 +8,11 @@ UBI and this plugin project was originally proposed in the [OpenSearch UBI RFC]( ## UBI Schemas -Please note that this repository is the implementation of the UBI plugin for OpenSearch. For details on the JSON Schema used by UBI to send and receive queries and events please see the [UBI](https://github.com/o19s/ubi) repository and the links below. +Please note that this repository is the implementation of the UBI plugin for OpenSearch. + +[!IMPORTANT] This release targets to the 1.0 version of the UBI Specification, which will be released when this plugin is released and the below links will be updated. + +For details on the JSON Schema used by UBI to send and receive queries and events please see the [UBI](https://github.com/o19s/ubi) repository and the links below. * [Query Request Schema](https://o19s.github.io/ubi/docs/html/query.request.schema.html) * [Query Response Schema](https://o19s.github.io/ubi/docs/html/query.response.schema.html) * [Event Schema](https://o19s.github.io/ubi/docs/html/event.schema.html) @@ -21,7 +25,7 @@ Please note that this repository is the implementation of the UBI plugin for Ope ## Useful Commands * Get the indexed queries: `curl http://localhost:9200/ubi_queries/_search | jq` -* +* ## User Quick Start @@ -33,7 +37,7 @@ To get started, download the plugin zip file from the [releases](https://github. bin/opensearch-plugin install file:/opensearch-ubi-1.0.0-os2.14.0.zip ``` -To create the UBI indexes called `ubi_queries` and `ubi_events`, send a query to an OpenSearch index: +To create the UBI indexes called `ubi_queries` and `ubi_events`, send a query to an OpenSearch index with the `ubi` query block added: ``` curl -s http://localhost:9200/your-index/_search -H "Content-Type: application/json" -d' @@ -93,8 +97,8 @@ curl -s http://localhost:9200/your-index/_search -H "Content-Type: application/j "ext": { "ubi": { "query_id": "12300d16cb-b6f1-4012-93ebcc49cac90426", - "user_query": "small basketball", - "client-id": "c4af7ca2-d6d2-4326-a41f-e616ebdd3d7b", + "user_query": "Toner", + "client_id": "c4af7ca2-d6d2-4326-a41f-e616ebdd3d7b", "object_id_field": "product_name" } }, @@ -190,7 +194,7 @@ curl -s http://localhost:9200/ubi_queries/_search -H "Content-Type: application/ Each indexed query will have the following fields: -* `query_response_id` - A unique identifier for the query response. +* `query_response_id` - A unique identifier for the query response. * `user_query`- Corresponds to the `user_query` in the query request. * `query_id` - Corresponds to the `query_id` in the query request, or a random UUID if a `query_id` was not provided in the query request. * `query_response_object_ids` - A list of the values of the `object_id_field` field in the document. @@ -250,7 +254,7 @@ where query_response_object_ids is null ``` ```sql -select +select count(0) from ubi_events where action_name='on_search' and event_attributes.data.data_detail.query_data.query_response_object_ids is null @@ -262,7 +266,7 @@ order by timestamp Find the most common client-side events: ```sql -select +select action_name, count(0) Total from ubi_events group by action_name @@ -286,7 +290,7 @@ new_user_entry|208 All client-side events that are associated with a query should have the same `query_id`. ```sql -select +select action_name, count(0) Total from ubi_events where query_id is not null @@ -338,4 +342,3 @@ This code is licensed under the Apache 2.0 License. See [LICENSE.txt](LICENSE.tx ## Copyright Copyright OpenSearch Contributors. See [NOTICE.txt](NOTICE.txt) for details. -