-
Notifications
You must be signed in to change notification settings - Fork 673
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SOLR-17062: Create JS client for Admin UI use #2050
SOLR-17062: Create JS client for Admin UI use #2050
Conversation
This uses the 'javascript-closure-angular' generator from the OpenAPI Generators project. Not sure if this is the right choice - they also offer a vanilla 'javascript' option that might be more suitable. As a JS novice, I'm not in a great position to evaluate the client itself. But I can offer it up as a draft PR and see whether others more familiar with the ecosystem can vet the generated code at all.
@MarcusSorealheis care to take a look? |
I'll take a look now. |
solr/api/build.gradle
Outdated
// Non-Java client generation tasks below: | ||
|
||
task buildJSClient(type: org.openapitools.generator.gradle.plugin.tasks.GenerateTask) { | ||
generatorName.set("javascript-closure-angular") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@gerlowskija a few thoughts:
-
I strongly recommend to use the
typescript
orjavascript
generators rather than one that ties users to a specific vendor. Specifically, AngularJS is a deprecated project and the version Solr was built on doesn't even exist anymore. The Angular that sort of lives on is written in Typescript. -
The
typescript
option is listed in the docs asexperimental
, though I doubt it would be very experimental andtypescript-node
is labeledstable
. In reality, both are probably more stable than thejavascript
option. Thejavascript
option may offer a slightly lower barrier to entry for future contributors for now, while thetypescript
one will have a growing ecosystem and offer far better maintainability in the future. -
The biggest benefits of the Typescript language are maintenance and developer experience, especially for a team of Java users.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given we want to dog-food our current Admin UI first, perhaps the pure javascript
client fits best. But for sure, once we plan to release something for customer use, TS bindings would likely be requested. Do we know whether the "typescript" generator is identical to the javascript
generator plus types, or are they separate templates?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I strongly recommend to use the typescript or javascript generators
👍 - I chose 'javascript-closure-angular' somewhat arbitrarily. I knew in a vague sense that the Admin UI used Angular, and just went with that. But I like your point about vendor-neutrality. Will update this to use the 'javascript' generator.
(I was under the impression that typescript was a distinct language from Javascript, and that the two weren't necessarily interchangeable. But I'm getting the sense from you guys above that there's more interplay there than I'd imagined. I've got some reading to do I think...)
Do we know whether the "typescript" generator is identical to the javascript generator plus types, or are they separate templates?
I don't know, but it's something I can check on. The default templates for each generator are in subdirectories here, so we'd just need to find the two relevant dirs and diff them.
One final point: Overall, this is a very positive and important effort. There's a wave of Typescript and Python developers at companies using Solr that would love to extend it directly, and a generated client would facilitate an easy integration point with low maintenance. |
I liked that the http client that https://github.com/OpenAPITools/openapi-generator/blob/master/modules/openapi-generator/src/main/resources/Javascript/libraries/javascript/package.mustache uses has had a recent release.... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've thought about it. Here's my final recommendation, though you can do as you please:
- Create a JS Client using
javascript
(this PR) - Create a TS Client using
typescript
(a new PR) - Create a Python Client (this PR)
That way, we support the current UI, leave room for and encourage a new UI, and all the data scientists writing Python can extend Solr in interesting ways.
Alright, I've updated the PR to use the 'javascript' generator. The generated code looks simple enough, but I'm no JS expert. I think my next steps here would be to look into whatever build logic is necessary to make this generated code available to the Admin UI, and then maybe change one API call in the admin UI to use the new JS client. That gives us a minimal bit of dog-fooding, and does much of the plumbing so that anyone else interested in the Admin UI can do more. Notably, that doesn't involve publishing the client, making it a release artifact, etc. We can go down those roads in a subsequent ticket if we start getting demand for a JS client or if we're comfortable enough with the dog-fooding that we're confident it'll be worth the maintenance burden to publish in some broader way. |
I think your idea about not publishing it as an artifact till we've tried it out makes sense! |
@@ -17,6 +17,7 @@ | |||
|
|||
plugins { | |||
id 'io.swagger.core.v3.swagger-gradle-plugin' version '2.2.2' | |||
id "org.openapi.generator" version "6.0.1" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
BTW, I looked and there is a 7.0.1 versin https://plugins.gradle.org/plugin/org.openapi.generator, maybe we have to update gradle?
NOTE: DOESN'T WORK! Currently on loading the Admin UI, solr/index.js triggers a number of import errors. At a glance, this appears to be because the generated Javascript file uses a style of import that omits the file-extension (i.e. '.js'). AFAICT, this extension-less style of import is common in node.js apps, or when using a 'npm' packaged module, but doesn't work well when using "unbundled" JS files. We can change the generation template to include the '.js' suffix in imports, but I'm not sure if that's the right approach. Once the import issues are resolved, this commit includes code changes to switch the Admin UI's collection-deletion over to using the v2 API. Hard to test or validate this code until the import issues are resolved, but for now it shows what example usage should looks like.
Alright, made some good progress here:
But.... I'm struggling to actually test this because I can't figure out how to correctly import the library/module. 😬 I've added a script tag (
It looks like these errors correspond to imports near the top of Possibly related, the README produced with the generated client includes some instructions on building and publishing the JS code as an NPM package. We don't run a Node server or use npm in the Admin UI as far as I can tell, so I was hoping that we could avoid any sort of "packaging", beyond copying the JS files into the webapp's Would love some help from anyone more familiar with JS (and maybe Node) development. Maybe @janhoy or @MarcusSorealheis if either of you get a few minutes? We're so close to getting this in, I hate to see this blocked so near to the finish line. |
Alright, great news: with a good deal of help from @HoustonPutman I've been able to resolve the packaging/import issues mentioned in my last comment. Despite its name, it looks like the 'javascript' generator produces JS code that relies on a few server-side-NodeJS features (e.g. using If we eventually publish this JS client we'll need to decide whether to publish in NodeJS/npm-ready or the browserify-bundled form. That'd be another great question for @MarcusSorealheis or anyone else more familiar with the JS ecosystem. With these problems solved, the PR is ready to be merged! |
I was initially planning on using the delete-collection button as the inaugural use of our v2 client, but ended up opting for the "Reload Collection" button instead. Currently, the v2 client is exposed as the "CollectionsV2" service defined in "services.js". This PR should be a good example for anyone interested in toying with the admin UI and expanding the v2 footprint there. Most of our Solr API calls live in in JS "controller" files. Just add the "CollectionsV2" service as an arg for the controller in question (if not already present), and Angular will call the right factory functions under the hood. |
I just got a look at the most recent question as I had some challenges to work through.
NodeJS/npm-ready is the way to go long term. Browserify is declining rapidly, though it was important for years after it was released. Webpack is a common replacement for Browserify today, and you can also do some of the bundling now without either. Webpack usually makes it easier. |
A little late to the party, but I wanted to say how nice the V2 js code integrated! I expected something much more intrusive... |
This commit adds build code to generate a JS client (using the OpenAPI Generator's 'javascript' template) and adds the necessary plumbing to bundle the client into our Admin UI. See 'CollectionsV2' in services.js as an example. Note that nothing in this commit adds this JS client as a release artifact, publishes it to npm, etc. --------- Co-authored-by: Houston Putman <[email protected]>
https://issues.apache.org/jira/browse/SOLR-17062
Description
Currently, Solr only offers one "first-party" client, SolrJ, which can only be used within JVM environments. This is obviously limiting for our users.
Historically, keeping client bindings in multiple languages up to date has been too daunting for the project to undertake. But now that Solr produces an OpenAPI spec for its upcoming "v2" API, we have the ability to generate client bindings from that for a variety of languages with very little additional maintenance cost.
There's extra incentive for us to experiment with languages that we ourselves use internally, as that gives us the opportunity to vet/dogfood the generated clients in our own code.
Solution
This PR uses the 'openapi-generator' gradle plugin to create a Javascript client based on our OpenAPI spec. A few details:
./gradlew solr:api:buildJSClient
to generate the clientsolr/api/build/generated/js
. Not sure how packaging works in Javascript code, but the generated code doesn't appear to be packaged in any way - it's "just" a bundle of JS files.Tests
The code-generator has an option to generate tests for the generated client code. I've disabled that currently, trusting the generator to do its thing, but we can enable that if it's something we'd feel more comfortable using.
Checklist
Please review the following and check all that apply:
main
branch../gradlew check
.