Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FCREPO-3834 Enable Camel toolbox to send xml records to Solr indexing service #191

Merged
merged 5 commits into from
Feb 15, 2024

Conversation

Surfrdan
Copy link
Contributor

@Surfrdan Surfrdan commented Mar 2, 2023

** FCREPO-3834 Enable Camel toolbox to send xml records to Solr indexing service **


JIRA Ticket: https://fedora-repository.atlassian.net/browse/FCREPO-3834

What does this Pull Request do?

Removes the ldpath related code and replaces the Solr requirement on LDPath with a Solr XSLT transform implementation

What's new?

Replaced LDPath transform with an XSLT based transform which can be supplied per object via the same hasIndexingTransformation as the previous LDPath implementation. Alternatively, if no transofmation is specified, a bare-boines default transform is supplied on the classpathj which replicates the functaionality of the LDPath version.

How should this be tested?

A description of what steps someone could take to:

  • Make a call to the solr indexing endpoint and examing the resulting solr document whihc should be indexed on the configured solr core.

Additional Notes:README.md should probably be updated too to reflect the new changes and removal of the LDPath functionality, however, I wasn't sure whether that should wait until after a release as I believe the README.md is the main dicumentation for this repo and relates to the current release.

Interested parties

@fcrepo/committers

Copy link
Contributor

@whikloj whikloj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works just needs a few tweaks.

I do think the default LDPath gave a lot more indexing by default. Perhaps we could try to duplicate that with the default XSLT?

For example, with LDPath the resource

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix fedora: <http://fedora.info/definitions/v4/repository#> .
@prefix ldp: <http://www.w3.org/ns/ldp#> .
@prefix dc: <http://purl.org/dc/elements/1.1/> .

<http://localhost:8080/fcrepo/rest/simple_container>
        dc:title               "A simple container" ;
        fedora:created         "2024-01-16T17:09:14.829081Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
        fedora:lastModified    "2024-01-16T17:09:14.829081Z"^^<http://www.w3.org/2001/XMLSchema#dateTime> ;
        fedora:createdBy       "fedoraAdmin" ;
        fedora:lastModifiedBy  "fedoraAdmin" ;
        rdf:type               ldp:BasicContainer ;
        rdf:type               ldp:Resource ;
        rdf:type               fedora:Resource ;
        rdf:type               ldp:RDFSource ;
        rdf:type               ldp:Container ;
        rdf:type               fedora:Container .

results in

{
   "id":"http://fcrepo:8080/fcrepo/rest/simple_container",
   "createdBy_s":"fedoraAdmin",
   "lastModified_l":1705424954829,
   "created_l":1705424954829,
   "lastModifiedBy_s":"fedoraAdmin",
   "type_ss":["http://fedora.info/definitions/v4/repository#Container",
        "http://fedora.info/definitions/v4/repository#Resource",
        "http://www.w3.org/ns/ldp#BasicContainer",
        "http://www.w3.org/ns/ldp#Container",
        "http://www.w3.org/ns/ldp#RDFSource",
        "http://www.w3.org/ns/ldp#Resource"],
    "title":["A simple container"],
    "_version_":1788267677844766720}]
}

With this PR the same resource (with slightly different created/modified datetimes) results in

{
    "id":"http://fcrepo:8080/fcrepo/rest/simple_container",
    "_version_":1788269396095926272}]
}

@Surfrdan
Copy link
Contributor Author

I've pushed a number of changes which should address those points.

Copy link
Contributor

@whikloj whikloj left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@whikloj whikloj merged commit e71b893 into fcrepo-exts:main Feb 15, 2024
3 checks passed
whikloj added a commit that referenced this pull request Sep 12, 2024
* Initial commit of solr basic auth functionality

* reducing log level of solr indexer when using auth

* FCREPO-3834 Enable Camel toolbox to send xml records to Solr indexing service (#191)

* replacing ldpath service with XSLT processing solr indexer

* no longer need HTTP_URI

* added properties for new solr xsl transforms to docker-compose properties file

* returning config to original state rather than my dev stack

* adding fields to solr xsl

---------

Co-authored-by: Dan Field <[email protected]>

* [maven-release-plugin] prepare release fcrepo-camel-toolbox-6.1.0

* [maven-release-plugin] prepare for next development iteration

* Stop using onSpinWait (#203)

Alters Thread.onSpinWait to use latch and await to avoid using CPU cycles to wait.
Also adds additional debug/trace logging to Solr route
Adds the default transform as the default on the property for Solr indexing
Adds some additional documentation on Solr indexing.

* Alterations at Jared's request on PR

* Initial commit of solr basic auth functionality

* reducing log level of solr indexer when using auth

* Alterations at Jared's request on PR

---------

Co-authored-by: Jared Whiklo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants