Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update script supports java.lang.String.sha1() and java.lang.String.sha256() methods #16923

Merged
merged 3 commits into from
Jan 2, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
- Support prefix list for remote repository attributes([#16271](https://github.com/opensearch-project/OpenSearch/pull/16271))
- Add new configuration setting `synonym_analyzer`, to the `synonym` and `synonym_graph` filters, enabling the specification of a custom analyzer for reading the synonym file ([#16488](https://github.com/opensearch-project/OpenSearch/pull/16488)).
- Add stats for remote publication failure and move download failure stats to remote methods([#16682](https://github.com/opensearch-project/OpenSearch/pull/16682/))
- Update script supports java.lang.String.sha1() and java.lang.String.sha256() methods ([#16923](https://github.com/opensearch-project/OpenSearch/pull/16923))
- Added a precaution to handle extreme date values during sorting to prevent `arithmetic_exception: long overflow` ([#16812](https://github.com/opensearch-project/OpenSearch/pull/16812)).
- Add search replica stats to segment replication stats API ([#16678](https://github.com/opensearch-project/OpenSearch/pull/16678))
- Introduce a setting to disable download of full cluster state from remote on term mismatch([#16798](https://github.com/opensearch-project/OpenSearch/pull/16798/))
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@
import org.opensearch.script.ScriptContext;
import org.opensearch.script.ScriptEngine;
import org.opensearch.script.ScriptService;
import org.opensearch.script.UpdateScript;
import org.opensearch.search.aggregations.pipeline.MovingFunctionScript;
import org.opensearch.threadpool.ThreadPool;
import org.opensearch.watcher.ResourceWatcherService;
Expand Down Expand Up @@ -109,6 +110,11 @@ public final class PainlessModulePlugin extends Plugin implements ScriptPlugin,
ingest.add(AllowlistLoader.loadFromResourceFiles(Allowlist.class, "org.opensearch.ingest.txt"));
map.put(IngestScript.CONTEXT, ingest);

// Functions available to update scripts
List<Allowlist> update = new ArrayList<>(Allowlist.BASE_ALLOWLISTS);
update.add(AllowlistLoader.loadFromResourceFiles(Allowlist.class, "org.opensearch.update.txt"));
map.put(UpdateScript.CONTEXT, update);

// Functions available to derived fields
List<Allowlist> derived = new ArrayList<>(Allowlist.BASE_ALLOWLISTS);
derived.add(AllowlistLoader.loadFromResourceFiles(Allowlist.class, "org.opensearch.derived.txt"));
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#
# SPDX-License-Identifier: Apache-2.0
#
# The OpenSearch Contributors require contributions made to
# this file be licensed under the Apache-2.0 license or a
# compatible open source license.
#

# This file contains an allowlist for the update scripts

class java.lang.String {
String org.opensearch.painless.api.Augmentation sha1()
String org.opensearch.painless.api.Augmentation sha256()
}
Original file line number Diff line number Diff line change
Expand Up @@ -123,3 +123,39 @@
- match: { error.root_cause.0.type: "illegal_argument_exception" }
- match: { error.type: "illegal_argument_exception" }
- match: { error.reason: "Iterable object is self-referencing itself" }

# update script supports java.lang.String.sha1() and java.lang.String.sha256() methods
# related issue: https://github.com/opensearch-project/OpenSearch/issues/16423
---
"Update script supports sha1() and sha256() method for strings":
- skip:
version: " - 2.18.99"
reason: "introduced in 2.19.0"
- do:
index:
index: test_1
id: 1
body:
foo: bar

- do:
update:
index: test_1
id: 1
body:
script:
lang: painless
source: "ctx._source.foo_sha1 = ctx._source.foo.sha1();ctx._source.foo_sha256 = ctx._source.foo.sha256();"

- match: { _index: test_1 }
- match: { _id: "1" }
- match: { _version: 2 }

- do:
get:
index: test_1
id: 1

- match: { _source.foo: bar }
- match: { _source.foo_sha1: "62cdb7020ff920e5aa642c3d4066950dd1f01f4d" }
- match: { _source.foo_sha256: "fcde2b2edba56bf408601fb721fe9b5c338d10ee429ea04fae5511b68fbf8fb9" }
Original file line number Diff line number Diff line change
Expand Up @@ -440,3 +440,41 @@
lang: painless
source: syntax errors are fun!
- match: {error.reason: 'compile error'}

# script in reindex supports java.lang.String.sha1() and java.lang.String.sha256() methods
# related issue: https://github.com/opensearch-project/OpenSearch/issues/16423
---
"Script supports sha1() and sha256() method for strings":
- skip:
version: " - 2.18.99"
reason: "introduced in 2.19.0"
- do:
index:
index: twitter
id: 1
body: { "user": "foobar" }
- do:
indices.refresh: {}

- do:
reindex:
refresh: true
body:
source:
index: twitter
dest:
index: new_twitter
script:
lang: painless
source: ctx._source.user_sha1 = ctx._source.user.sha1();ctx._source.user_sha256 = ctx._source.user.sha256()
- match: {created: 1}
- match: {noops: 0}

- do:
get:
index: new_twitter
id: 1

- match: { _source.user: foobar }
- match: { _source.user_sha1: "8843d7f92416211de9ebb963ff4ce28125932878" }
- match: { _source.user_sha256: "c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2" }
Original file line number Diff line number Diff line change
Expand Up @@ -432,3 +432,38 @@
lang: painless
source: syntax errors are fun!
- match: {error.reason: 'compile error'}

# script in update_by_query supports java.lang.String.sha1() and java.lang.String.sha256() methods
# related issue: https://github.com/opensearch-project/OpenSearch/issues/16423
---
"Script supports sha1() and sha256() method for strings":
- skip:
version: " - 2.18.99"
reason: "introduced in 2.19.0"
- do:
index:
index: twitter
id: 1
body: { "user": "foobar" }
- do:
indices.refresh: {}

- do:
update_by_query:
index: twitter
refresh: true
body:
script:
lang: painless
source: ctx._source.user_sha1 = ctx._source.user.sha1();ctx._source.user_sha256 = ctx._source.user.sha256()
- match: {updated: 1}
- match: {noops: 0}

- do:
get:
index: twitter
id: 1

- match: { _source.user: foobar }
- match: { _source.user_sha1: "8843d7f92416211de9ebb963ff4ce28125932878" }
- match: { _source.user_sha256: "c3ab8ff13720e8ad9047dd39466b3c8974e592c2fa383d4a3960714caef0c4f2" }
Loading