SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr #1999

epugh · 2023-10-10T20:08:34Z

https://issues.apache.org/jira/browse/SOLR-17023

Description

Code and BATS test that demonstrates downloading a sentiment classification model from Huggingface, converting it to Onnx model, uploading the model to Solr's FileStore, and then configuring a collection and associated pipeline to use the model for enriching content.

Solution

New DocumentCategorizerUpdateProcessorFactory that interacts with the model.

Tests

BATS test. Check this PR out and run: ./gradlew integrationTests --tests test_opennlp.bats

Checklist

Please review the following and check all that apply:

[x ] I have reviewed the guidelines for How to Contribute and my code conforms to the standards described there to the best of my ability.
I have created a Jira issue and added the issue ID to my pull request title.
[x ] I have given Solr maintainers access to contribute to my PR branch. (optional but recommended)
[ x] I have developed this patch against the main branch.
I have run ./gradlew check.
I have added tests for my changes.
I have added documentation for the Reference Guide
Look at GPU support
Integrate with Films demo.
Can we have BATS tests that depend on external resources, or do I check in the ONNX model instead? (make a stupid one???)
Write a unit test?
Provide both a label but ALSO a score!

epugh · 2023-10-10T20:10:03Z

Check it out @jzonthemtn

risdenk · 2023-10-10T20:16:44Z

solr/modules/analysis-extras/build.gradle

+configurations.all {
+  resolutionStrategy {
+    force 'org.apache.opennlp:opennlp-tools:2.2.0'
+    force 'org.apache.opennlp:opennlp-dl:2.2.0'
+  }
+}
+


Why do you need for force a resolution? This shouldn't be necessary at all...

We need apache/lucene#448 to be merged to get us to 2.2, or we ahve to work around Lucene... Is there a better way? This is what Jeff and I hacked in..

try org.apache.opennlp:opennlp*=2.2.0 in versions.props and remove this from here. It should force the upgrade to 2.2.0.

lots of errors when I took out the resolutionStrategy:

* What went wrong: Could not determine the dependencies of task ':solr:modules:analysis-extras:compileJava'. > Could not resolve all task dependencies for configuration ':solr:modules:analysis-extras:compileClasspath'. > Could not resolve org.apache.opennlp:opennlp-tools:2.2.0. Required by: project :solr:modules:analysis-extras project :solr:modules:analysis-extras > org.apache.opennlp:opennlp-dl:2.2.0 > Cannot find a version of 'org.apache.opennlp:opennlp-tools' that satisfies the version constraints: Dependency path 'org.apache.solr:analysis-extras:10.0.0-SNAPSHOT' --> 'org.apache.opennlp:opennlp-tools:2.2.0' Constraint path 'org.apache.solr:analysis-extras:10.0.0-SNAPSHOT' --> 'org.apache.solr:core:10.0.0-SNAPSHOT' (apiElements) --> 'org.apache.opennlp:opennlp-tools:1.9.4' because of the following reason: Computed from com.palantir.consistent-versions' versions.lock in solr-root Constraint path 'org.apache.solr:analysis-extras:10.0.0-SNAPSHOT' --> 'org.apache.solr:solrj:10.0.0-SNAPSHOT' (apiElements) --> 'org.apache.opennlp:opennlp-tools:1.9.4' because of the following reason: Computed from com.palantir.consistent-versions' versions.lock in solr-root Dependency path 'org.apache.solr:analysis-extras:10.0.0-SNAPSHOT' --> 'org.apache.lucene:lucene-analysis-opennlp:9.8.0' (compile) --> 'org.apache.opennlp:opennlp-tools:1.9.1'

also, feel free to push up changes to this branch ;-)

make sure to run ./gradlew --write-locks to make sure it takes into account versions.props changes.

risdenk · 2023-10-10T20:30:08Z

versions.props

@@ -70,3 +70,4 @@ org.semver4j:semver4j=5.2.1
 org.slf4j:*=2.0.9
 org.xerial.snappy:snappy-java=1.1.10.5
 software.amazon.awssdk:*=2.20.155
+org.apache.opennlp:opennlp-dl=2.2.0


try org.apache.opennlp:opennlp*=2.2.0

…t 600 mb file.

epugh · 2023-10-10T21:13:37Z

Thanks @risdenk , your suggestions worked. One slight wrinkle that I don't know how to deal with is that the com.microsoft.onnxruntime:onnxruntime_gpu:1.14.0 file doesn't work on OSX, only linux and windows. So if you look at the BATS test, you'll see some code where I replace it with com.microsoft.onnxruntime:onnxruntime:1.14.0. I don't really know how to handle that going forward.. I mean, swapping a magic jar is probably a bad idea ;-).

epugh · 2023-10-10T21:13:56Z

./gradlew integrationTests --tests test_opennlp.bats

epugh · 2023-10-10T22:46:36Z

> Task :solr:packaging:integrationTests
Running BATS tests with Solr base port 53065
1..1
ok 1 Check lifecycle of sentiment classification in 20000ms

This is after uploading the model to the PackageStore and reference it in there!

versions.props

risdenk · 2023-10-11T00:44:50Z

solr/packaging/test/test_opennlp.bats

+@test "Check lifecycle of sentiment classification" {
+
+  # GPU versions is linux and windows only, not OSX.  So swap jars.
+  rm -f ${SOLR_TIP}/modules/analysis-extras/lib/onnxruntime_gpu-1.14.0.jar


Why not just always run on cpu? Why need to test if there is a capable GPU or not? Could be on a VM without a GPU too even w/ linux and windows...

for the purposes of this, I totally agree.. I think I need some Gradle help to figure this out ;-( I've banged at it with no luck.

Just CPU is definitely a better choice here. And just a note, ONNX Runtime does not currently support Mac silicon so this will only work on x64 machines.

OpenNLP brings in the onnxruntime-gpu dependency. I will open a PR in OpenNLP to use onnxruntime instead for all of the stated reasons.

if there is a workaround, please let me know...!

Release is done. New artifacts should be available soon ;-) (not depending on gpu anymore)

Thank you!!!

risdenk · 2023-10-11T00:44:58Z

solr/packaging/test/test_opennlp.bats

+  rm -f ${SOLR_TIP}/modules/analysis-extras/lib/onnxruntime_gpu-1.14.0.jar
+  # restore 
+  #curl --insecure -o ${SOLR_TIP}/modules/analysis-extras/lib/onnxruntime-1.14.0.jar https://repo1.maven.org/maven2/com/microsoft/onnxruntime/onnxruntime/1.14.0/onnxruntime-1.14.0.jar
+  cp /Users/epugh/Documents/projects/solr-epugh/onnxruntime-1.14.0.jar ${SOLR_TIP}/modules/analysis-extras/lib/


This is using your path...

yeah... I was in the back of David Smiley's car in the wilds of New Brunswick Canada while I was working on this and didn't want to burn up my data plan ;-). Likewise I need to figure out a better apporoach with the model since it's 600 mb.... I don't know if I can find a toy one that is super small, or if I add docs to download it from hugging facet etc. Longer term, I doin't even think this bats test gets committed the way it is because of all the external dependencies.. Though.. if I had a super small model that I could check into Git, maybe that owuld be okay? Like we have a binary package for some of the package tests..

epugh · 2023-10-11T15:29:06Z

@risdenk @jzonthemtn so, any ideas how to force the onnxruntime instead of the onnxruntime_gpu? I regenerated the licenses, and noticed that it created a "onnxruntime_gpu-1.14.0.jar.sha1".

epugh · 2023-10-11T15:29:55Z

also, i am seeing two errors on looking up licenses...

Execution failed for task ':solr:modules:analysis-extras:validateJarLicenses'.
> Certain license/ notice files are missing:
    - License file missing ('com.microsoft.onnxruntime:onnxruntime_gpu:1.14.0'), expected it at: /Users/epugh/Documents/projects/solr-epugh/solr/licenses/onnxruntime_gpu-LICENSE-[type].txt, where [type] can be any of [ASL, BSD, BSD_LIKE, CDDL, CPL, EPL, MIT, MPL, PD, SUN, COMPOUND].
    - License file missing ('org.apache.opennlp:opennlp-dl:2.2.0'), expected it at: /Users/epugh/Documents/projects/solr-epugh/solr/licenses/opennlp-dl-LICENSE-[type].txt or /Users/epugh/Documents/projects/solr-epugh/solr/licenses/opennlp-LICENSE-[type].txt, where [type] can be any of [ASL, BSD, BSD_LIKE, CDDL, CPL, EPL, MIT, MPL, PD, SUN, COMPOUND].

epugh · 2023-10-11T16:05:55Z

@jzonthemtn did open https://issues.apache.org/jira/browse/OPENNLP-1515, but that will require a release of OpenNLP before we can piggy back on it

epugh · 2023-10-11T16:21:28Z

I think the first draft is done, and it "works". There are some caveats and things to be figured out, like the onnxruntime_gpu versus onnxruntime issue. Based on my work so far, we definitly need tutorials, there are too many moving pieces. The PackageStore is great, at least as far as I've worked with it, but it does expect to load the data in memory as it moves it around, so we need to figure that out. Right now we only get sentiment labels back, and we really need sentiment scores, i.e a number. I don't know if that can be a 0 to 1, of if it has to be 1,2,3,4,5 labels. I'd like to experiment with the saem thing as a copyField into a text analyzer, so you copy "review" into "review_sentiment_score" and get a number there. Or, do we have a streaming expression you run that updates the review_sentiment_score for all docs that don't have it with an atomic update?

…n the OpenNLP project

cpoerschke · 2024-01-18T16:15:44Z

Looking at these build failures and the error message generated, it appears that it may be caused by us using Java 11 and OpenNLP being compiled with Java 17?? ... Is this a deal breaker for this PR?

Hmm, interesting. So we have:

OpenNLP as minimum Java17 as you mention -- https://github.com/apache/opennlp/blob/opennlp-2.3.1/pom.xml#L167
Lucene as minimum Java11 -- https://github.com/apache/lucene/blob/releases/lucene/9.9.1/build.gradle#L75-L76
Solr as minimum Java11 -- https://github.com/apache/solr/blob/releases/solr/9.4.1/build.gradle#L88

So if the classes here were built as an independent plugin (with minimum Java17) and then deployed (with the relevant dependencies) into a Solr setup running Java17 with the original Solr artefacts (built with Java11) -- I wonder if that would work?

Also noting that apache/lucene#579 bumped Lucene to Java17 on main branch i.e. presumably then a future Lucene10 will be minimum Java17 version.

rzo1 · 2024-01-18T16:24:22Z

So if the classes here were built as an independent plugin (with minimum Java17) and then deployed (with the relevant dependencies) into a Solr setup running Java17 with the original Solr artefacts (built with Java11) -- I wonder if that would work?

I guess this should work.

epugh · 2024-01-18T16:41:40Z

Feels like what we should be doing is having Solr 10 target Java 17 since Lucene 10 will require it, and then this code goes on Solr 10, but not on Solr 9. This lets us have some more time to experiment with out dealing with the headaches of supporting an official release in the 9.x line (backcompat and the rest)....??

cpoerschke · 2024-01-18T17:52:04Z

Feels like what we should be doing is having Solr 10 target Java 17 since Lucene 10 will require it, and then this code goes on Solr 10, but not on Solr 9. This lets us have some more time to experiment with out dealing with the headaches of supporting an official release in the 9.x line (backcompat and the rest)....??

I concur. Also a nice motivation for targeting Java 17 i.e. specific example of functionality that it would unlock. And in the meantime "independent plugin" approaches remain a possibility in the community, perhaps even in the https://github.com/apache/solr-sandbox if someone wanted to pursue that (haven't checked how that is built, just kinda "name dropping" solr-sandbox here).

epugh · 2024-02-13T13:55:44Z

Lucene 9 requires older version of Java than the minimum required version that OpenNLP requires. That means that this PR is pending a release of Lucene 10, and the adoption of Lucene 10 by Solr. It would be interesting to think about if there was a way for Solr main branch to somehow depend on the Lucene main branch release that jumps the minimum Java versions all around, and would allow this PR to be merged.

Resolved Conflicts: versions.lock versions.props

cpoerschke · 2024-02-13T15:57:00Z

versions.props

@@ -49,6 +49,7 @@ org.apache.httpcomponents:httpmime=4.5.14
 org.apache.kerby:*=1.0.1
 org.apache.logging.log4j:*=2.21.0
 org.apache.lucene:*=9.9.2


Temporarily within this pull request (pre-merge) we could change this to a Lucene 10 prerelease based on the "Update Lucene prerelease" steps in https://github.com/apache/solr/blob/main/help/dependencies.txt ...

... might be worth waiting though until Lucene 9.10 is out and https://issues.apache.org/jira/browse/SOLR-17157 has upgraded Solr to use it i.e. then solr/main will be closer to lucene/main than it is right now.

cpoerschke · 2024-02-13T16:09:42Z

... It would be interesting to think about if there was a way for Solr main branch to somehow depend on the Lucene main branch release that jumps the minimum Java versions all around, and would allow this PR to be merged.

Technically I guess Solr main could continue to depend on whatever Lucene version and just jumping up the minimum Java version for Solr main to 17 would be sufficient? With all the ups-and-downs of main and branch_9x having different minimums.

risdenk · 2024-02-13T16:29:12Z

@cpoerschke #1510 might be helpful here. I have a few wip prs for newer jdks

github-actions · 2024-04-17T00:00:43Z

This PR had no visible activity in the past 60 days, labeling it as stale. Any new activity will remove the stale label. To attract more reviewers, please tag someone or notify the [email protected] mailing list. Thank you for your contribution!

github-actions · 2024-10-07T00:00:54Z

This PR is now closed due to 60 days of inactivity after being marked as stale. Re-opening this PR is still possible, in which case it will be marked as active again.

epugh · 2024-10-07T10:15:00Z

this is still valid ;-)

epugh added 3 commits October 10, 2023 12:31

first version w bats test

f19f660

not positive I need this

48fd37b

new name, and OpenNLP is kind of a implmentation detail ;-).

cecd4f6

epugh marked this pull request as draft October 10, 2023 20:08

risdenk requested changes Oct 10, 2023

View reviewed changes

risdenk reviewed Oct 10, 2023

View reviewed changes

epugh added 2 commits October 10, 2023 18:04

baby steps, found that packagesstore blows up killing solr when i pos…

10abbd0

…t 600 mb file.

remove resolutionStraregy force from gradle build..

dd4cc89

epugh requested a review from risdenk October 10, 2023 21:16

match name in the underlying OpenNLP project. bikeshedding!

af18d0c

tidy

6950629

risdenk requested changes Oct 11, 2023

View reviewed changes

epugh added 3 commits October 11, 2023 12:22

reorder params

1adba22

log formatting

bcbf16d

regenerate...

e634b1c

dynamically grab the models from hugging face.

778e3f4

use logging structure for stack traces

405153b

cpoerschke mentioned this pull request Oct 11, 2023

[experimental] start BERT module #1073

Draft

jzonthemtn mentioned this pull request Oct 12, 2023

OPENNLP-1515: Changing onnx dependency to onnxruntime. apache/opennlp#551

Merged

10 tasks

download the correct jar, and document the work to remove this need i…

4e54090

…n the OpenNLP project

epugh changed the title ~~SOLR-17023: Use Modern NLP Models from Apache OpenNLP with Solr~~ SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr Oct 26, 2023

epugh mentioned this pull request Oct 30, 2023

SOLR-16835: Generate Python client from OpenAPI spec #1681

Merged

7 tasks

Merge remote-tracking branch 'origin/main' into SOLR-17023

3a760da

Resolved Conflicts: versions.lock versions.props

github-actions bot added dependencies Dependency upgrades tool:build labels Feb 13, 2024

cpoerschke added 2 commits February 13, 2024 15:38

upgrade OpenNLP from 2.3.1 to 2.3.2 (to match Lucene main branch)

bdee568

tentative: minimum Java17 for this PR

541dceb

cpoerschke reviewed Feb 13, 2024

View reviewed changes

Update gradle-precommit.yml - Java 11 --> 17

633391e

cpoerschke added 3 commits February 13, 2024 16:11

Update solrj-test.yml - Java 11 --> 17

e2fe1fe

Update docker-test.yml - Java 11 --> 17

855b5ea

Update bin-solr-test.yml - Java 11 --> 17

9c16a8b

undo 'tentative: minimum Java17 for this PR' -- see PR 1510 instead

476a7d4

github-actions bot added the stale PR not updated in 60 days label Apr 17, 2024

github-actions bot added the closed-stale Closed after being stale for 60 days label Oct 7, 2024

github-actions bot closed this Oct 7, 2024

epugh reopened this Oct 7, 2024

github-actions bot added the cat:index label Oct 7, 2024

epugh added exempt-stale Prevent a PR from going stale and removed stale PR not updated in 60 days cat:index closed-stale Closed after being stale for 60 days labels Oct 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr #1999

SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr #1999

epugh commented Oct 10, 2023 •

edited

Loading

epugh commented Oct 10, 2023

risdenk Oct 10, 2023

epugh Oct 10, 2023

risdenk Oct 10, 2023

epugh Oct 10, 2023

risdenk Oct 10, 2023

risdenk Oct 10, 2023

epugh commented Oct 10, 2023

epugh commented Oct 10, 2023

epugh commented Oct 10, 2023

risdenk Oct 11, 2023

epugh Oct 11, 2023

jzonthemtn Oct 11, 2023 •

edited

Loading

epugh Oct 11, 2023

rzo1 Nov 27, 2023

epugh Nov 27, 2023

risdenk Oct 11, 2023

epugh Oct 11, 2023

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

cpoerschke commented Jan 18, 2024

rzo1 commented Jan 18, 2024

epugh commented Jan 18, 2024

cpoerschke commented Jan 18, 2024

epugh commented Feb 13, 2024

cpoerschke Feb 13, 2024

cpoerschke commented Feb 13, 2024

risdenk commented Feb 13, 2024

github-actions bot commented Apr 17, 2024

github-actions bot commented Oct 7, 2024

epugh commented Oct 7, 2024

SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr #1999

Are you sure you want to change the base?

SOLR-17023: Use Modern NLP Models via ONNX and Apache OpenNLP with Solr #1999

Conversation

epugh commented Oct 10, 2023 • edited Loading

Description

Solution

Tests

Checklist

epugh commented Oct 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

epugh commented Oct 10, 2023

epugh commented Oct 10, 2023

epugh commented Oct 10, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jzonthemtn Oct 11, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

epugh commented Oct 11, 2023

cpoerschke commented Jan 18, 2024

rzo1 commented Jan 18, 2024

epugh commented Jan 18, 2024

cpoerschke commented Jan 18, 2024

epugh commented Feb 13, 2024

Choose a reason for hiding this comment

cpoerschke commented Feb 13, 2024

risdenk commented Feb 13, 2024

github-actions bot commented Apr 17, 2024

github-actions bot commented Oct 7, 2024

epugh commented Oct 7, 2024

epugh commented Oct 10, 2023 •

edited

Loading

jzonthemtn Oct 11, 2023 •

edited

Loading