Problem running DBpedia_music example

[DougalW]

Hi, when I build the specification for the DBpedia_music example it completes with success. When I generate the server it also completes successfully. When I run it either via Python3 or via the docker build, it generates this error:

dwatt$ python3 -m openapi_server
INFO:rdflib:RDFLib Version: 4.2.2
2020-09-23 10:25:00,804 - root - ERROR - /Users/dwatt/Downloads/OBA/outputs/dbpedia_music/servers/python/server/contexts/context_overwrite.json missing
2020-09-23 10:25:01,415 - openapi_server.cached - WARNING - Cache file does not exist: [Errno 2] No such file or directory: '/Users/dwatt/Downloads/OBA/outputs/dbpedia_music/servers/python/server/openapi_server/openapi/openapi.yaml.cache'
2020-09-23 10:25:01,598 - root - ERROR - /Users/dwatt/Downloads/OBA/outputs/dbpedia_music/servers/python/server/contexts/context_overwrite.json missing
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/utils.py", line 120, in get_function_from_name
    function = deep_getattr(module, attr_path)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/utils.py", line 68, in deep_getattr
    return functools.reduce(getattr, attrs, obj)
AttributeError: module 'openapi_server.controllers' has no attribute 'user_controller'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/dwatt/Downloads/OBA/outputs/dbpedia_music/servers/python/server/openapi_server/__main__.py", line 20, in <module>
    main()
  File "/Users/dwatt/Downloads/OBA/outputs/dbpedia_music/servers/python/server/openapi_server/__main__.py", line 13, in main
    app.add_api('openapi.yaml',
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/apps/flask_app.py", line 57, in add_api
    api = super(FlaskApp, self).add_api(specification, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/apps/abstract.py", line 144, in add_api
    api = self.api_cls(specification,
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/apis/abstract.py", line 111, in __init__
    self.add_paths()
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/apis/abstract.py", line 219, in add_paths
    self._handle_add_operation_error(path, method, sys.exc_info())
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/apis/abstract.py", line 231, in _handle_add_operation_error
    raise value.with_traceback(traceback)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/apis/abstract.py", line 209, in add_paths
    self.add_operation(path, method)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/apis/abstract.py", line 175, in add_operation
    self._add_operation_internal(method, path, operation)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/apis/flask_api.py", line 113, in _add_operation_internal
    function = operation.function
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/operations/abstract.py", line 368, in function
    security_decorator = self.security_decorator
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/operations/secure.py", line 123, in security_decorator
    bearer_info_func = get_bearerinfo_func(security_scheme)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/decorators/security.py", line 101, in get_bearerinfo_func
    return get_function_from_name(func)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/utils.py", line 123, in get_function_from_name
    raise last_import_error
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/connexion/utils.py", line 111, in get_function_from_name
    module = importlib.import_module(module_name)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/dwatt/Downloads/OBA/outputs/dbpedia_music/servers/python/server/openapi_server/controllers/user_controller.py", line 9, in <module>
    from openapi_server.models.user import User
ModuleNotFoundError: No module named 'openapi_server.models.user'

The only other issue I could see in the log output was when I run the generate-server.sh script it thinks I don't have yapf installed. I installed it via pip but the script still returns this error. I even created a sim link in /usr/local/bin/yapf to point to the binary in my python3.8 installation, but same message is returned.

dwatt$ bash generate-server.sh 
Docker is installed
[main] INFO  o.o.codegen.DefaultGenerator - OpenAPI Generator: python-flask (server)
[main] INFO  o.o.codegen.DefaultGenerator - Generator 'python-flask' is considered stable.
[main] INFO  o.o.c.l.PythonAbstractConnexionServerCodegen - Environment variable PYTHON_POST_PROCESS_FILE not defined so the Python code may not be properly formatted. To define it, try 'export PYTHON_POST_PROCESS_FILE="/usr/local/bin/yapf -i"' (Linux/Mac)
[main] INFO  o.o.c.l.PythonAbstractConnexionServerCodegen - NOTE: To enable file post-processing, 'enablePostProcessFile' must be set to `true` (--enable-post-process-file for CLI).
[main] WARN  o.o.codegen.DefaultCodegen - Empty operationId found for path: GET /bands. Renamed to auto-generated operationId: bandsGET
[main] WARN  o.o.codegen.DefaultCodegen - Empty operationId found for path: POST /bands. Renamed to auto-generated operationId: bandsPOST
[main] WARN  o.o.codegen.DefaultCodegen - Empty operationId found for path: GET /bands/{id}. Renamed to auto-generated operationId: bandsIdGET
[main] WARN  o.o.codegen.DefaultCodegen - Empty operationId found for path: PUT /bands/{id}. Renamed to auto-generated operationId: bandsIdPUT
[main] WARN  o.o.codegen.DefaultCodegen - Empty operationId found for path: DELETE /bands/{id}. Renamed to auto-generated operationId: bandsIdDELETE
[main] WARN  o.o.codegen.DefaultCodegen - Empty operationId found for path: GET /genres. Renamed to auto-generated operationId: genresGET
[main] WARN  o.o.codegen.DefaultCodegen - Empty operationId found for path: POST /genres. Renamed to auto-generated operationId: genresPOST
[main] WARN  o.o.codegen.DefaultCodegen - Empty operationId found for path: GET /genres/{id}. Renamed to auto-generated operationId: genresIdGET
[main] WARN  o.o.codegen.DefaultCodegen - Empty operationId found for path: PUT /genres/{id}. Renamed to auto-generated operationId: genresIdPUT
[main] WARN  o.o.codegen.DefaultCodegen - Empty operationId found for path: DELETE /genres/{id}. Renamed to auto-generated operationId: genresIdDELETE
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/models/band.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/models/genre.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/controllers/band_controller.py
[main] INFO  o.o.codegen.DefaultGenerator - File exists. Skipped overwriting /local/server/openapi_server/test/test_band_controller.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/controllers/genre_controller.py
[main] INFO  o.o.codegen.DefaultGenerator - File exists. Skipped overwriting /local/server/openapi_server/test/test_genre_controller.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/README.md
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/test-requirements.txt
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/requirements.txt
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/__main__.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/util.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/typing_utils.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/controllers/__init__.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/controllers/security_controller_.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/models/__init__.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/models/base_model_.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/openapi/openapi.yaml
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/.gitignore
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/Dockerfile
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/.dockerignore
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/setup.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/tox.ini
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/git_push.sh
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/.travis.yml
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/encoder.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/test/__init__.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/openapi_server/__init__.py
[main] INFO  o.o.codegen.AbstractGenerator - writing file /local/server/.openapi-generator/VERSION
Copying query files
SUCCESS
```
`

[dgarijo]

@DougalW thanks for reporting this. We will have a look and come back to you.
Since everything is run through Docker, I believe you should not need to install anything locally.
Have you tried running with Docker as we suggest in: https://oba.readthedocs.io/en/latest/running_server/?

[DougalW]

@dgarijo Yes I tried both approaches.

[mosoriob]

Hi @DougalW

It seems a bug. I'm trying to reproduce the error. Did you use the master branch or the release 3.4.0?

Thanks

[mosoriob]

Unable to reproduce using the master branch.

I have improved the tests on our CI system. Now, the dbpedia music tests includes:

1. Generate the openapi.
2. Generate the Python server
3. Request a resource

You can see the test script here

[mosoriob]

Hey! I found it.

The following options are generated the error.

enable_post_paths: true
enable_delete_paths: true
enable_put_paths: true

In the dbpedia_music, these options must be false because we cannot edit the dbpedia endpoint.

@DougalW feel safe to test the version 3.4.0 or 3.5.0 using the updated configuration file

@dgarijo I'm going to verify the model_catalog.

[dgarijo]

Thanks @mosoriob!! will try to see if we need to update the doc

[ddtxra]

issue does still exist when setting true on put, post and delete even on local endpoint.

My yaml file

name: persontest
output_dir: outputs

## OpenAPI Section
### Name, version and URL of the OpenAPI
openapi:
  openapi: 3.0.1
  info:
    description: This is the API of the Persons test
    title: PersonTest
    version: v1.0.0
  servers:
    - url: http://localhost:3030/test/sparql

## Ontologies
### List of ontologies
ontologies:
  - person_test.ttl

endpoint:
  url: http://localhost:3030/test/sparql
  prefix: http://test.com/rdf/resource/

## Filter the paths by methods
enable_get_paths: true
enable_post_paths: true
enable_delete_paths: false
enable_put_paths: false

## Select the classes to add in the API
follow_references: false

My person_test.ttl file:

@prefix : <https://w3id.org/example#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix xml: <http://www.w3.org/XML/1998/namespace> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix dcterms: <http://purl.org/dc/terms/> .
@prefix skos: <http://www.w3.org/2004/02/skos/core#> .
@prefix prov: <http://www.w3.org/ns/prov#> .

:Person rdf:type owl:Class ;
rdfs:label "Person"@en.

:Professor rdf:type owl:Class ;
    rdfs:subClassOf :Person ;
    owl:disjointWith :Student ;
    rdfs:label "Professor"@en .

:Student rdf:type owl:Class ;
    rdfs:subClassOf :Person ;
    owl:disjointWith :Professor ;
    rdfs:label "Student"@en .

:identifier rdf:type owl:DatatypeProperty ;
  rdfs:domain :Person ;
  rdfs:range xsd:string ;
  rdfs:label "name"@en .

:gender rdf:type owl:DatatypeProperty ;
  rdfs:domain :Person ;
  rdfs:range [ rdf:type rdfs:Datatype ;
    owl:oneOf [ rdf:type rdf:List ;
      rdf:first "female" ;
      rdf:rest [ rdf:type rdf:List ;
        rdf:first "male" ;
        rdf:rest rdf:nil
        ]
      ]
    ] ;
  rdfs:label "gender"@en .


:researchField rdf:type owl:DatatypeProperty ;
    rdfs:domain :Professor ;
    rdfs:range xsd:string ;
    rdfs:label "research field"@en .


:teachesTo rdf:type owl:ObjectProperty ;
    rdfs:domain :Professor ;
    rdfs:range :Student ;
    rdfs:label "teaches to"@en .


:identifier rdf:type owl:DatatypeProperty , owl:FunctionalProperty ;
  rdfs:domain :Person ;
  rdfs:range xsd:string ;
  rdfs:label "identifier"@en .


:hasRecord rdf:type owl:ObjectProperty , owl:FunctionalProperty ;
    rdfs:domain :Student ;
    rdfs:range :StudentRecord ;
    rdfs:label "has record"@en .


:Professor rdf:type owl:Class ;
  rdfs:subClassOf :Person ,
    [ rdf:type owl:Restriction ;
      owl:onProperty :teachesCourse ;
      owl:someValuesFrom :Course
    ] .

Is there any plan to fix it?

[ddtxra]

I used the version 3.6.0 btw:
java -jar oba-3.6.0-jar-with-dependencies.jar -c config.yaml

[dgarijo]

@ddtxra,
I will reopen the issue, but at the moment we do not have the resources to continue fixing issues/new features in OBA related to the OpenAPI implementation, just fixing issues in the mapping.
Some of these problems may be caused by us using an older version of the OpenAPI server. We need to update the version and test whether the specification is supported successfully for the Python clients we were using (before there were some problems supporting unions).

If you dig in further and find the problem, please let us know.