Demonstrator - Reasoning WebAPI issueshttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues2021-06-13T23:44:10Zhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/65Endpoint /explain accepts malformatted request2021-06-13T23:44:10ZSiebers, MichaelEndpoint /explain accepts malformatted requestAs the endpoint `/irrelevant/file`, the endpoint `/explain` accepts absolute path names with a trailing slash (#64). This is not yet detected by system tests.As the endpoint `/irrelevant/file`, the endpoint `/explain` accepts absolute path names with a trailing slash (#64). This is not yet detected by system tests.Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/64Endpoint `irrelevant/file` accepts malformatted request2021-06-13T23:44:10ZSiebers, MichaelEndpoint `irrelevant/file` accepts malformatted request# Evidence
Job `system_test:irrelevant robust against non-path payload` of pipeline [#24446](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/pipelines/24446). Test "Querry Irrelevance[13/20]: Bad Request" indicates that t...# Evidence
Job `system_test:irrelevant robust against non-path payload` of pipeline [#24446](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/pipelines/24446). Test "Querry Irrelevance[13/20]: Bad Request" indicates that the endpoint accepted an abs_path with trailing slash!Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/63Endpoint irrelevant/file returns 5002021-06-13T22:01:23ZSiebers, MichaelEndpoint irrelevant/file returns 500Test have detected a case where the endpoint `irrelevant/file` responds with status code 500. It turned out that the predicate `irrelevant/1` throws an exception. So the behavior of the WebAPI is correct.
Source of the exception is the ...Test have detected a case where the endpoint `irrelevant/file` responds with status code 500. It turned out that the predicate `irrelevant/1` throws an exception. So the behavior of the WebAPI is correct.
Source of the exception is the predicate `manipulation_time/2'. When called with a directory as first argument the stored dict does not have the appropriate key. Thus an exception is thrown.
## Evidence
- Failed tests: `system_test:endpoint_irrelevant@file` of job https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/jobs/72393.
- [Loaded background theory database](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/blob/b6b23ae408491c55b3d7eb65e9bf84ed1c256cec/tests/api/endpoint_irrelevant@file-theory_bg.db)
- Request body: `{"abs_path": "dir_1/file_new"}`
## Tasks
- Refine the `irrelevance_common.plt` tests to catch this error in the future.
- Safeguard `manipulation_time/2' to fail when called with an directory at first argument.
- Test other `irrelevance_common.pl` predicates for similar errors.Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/62HTML error on wrong Content-Length2021-06-12T09:00:31ZSiebers, MichaelHTML error on wrong Content-LengthAccording to #28 all error messages are expected to be JSON.
When transmitting a non-numeric Content-Length header, the API correctly returns '400 Bad Request`. However, the transmitted error message is not a JSON object but an HTML docu...According to #28 all error messages are expected to be JSON.
When transmitting a non-numeric Content-Length header, the API correctly returns '400 Bad Request`. However, the transmitted error message is not a JSON object but an HTML document.
**Evidence**: Test `bg/add: Valid error response body` belonging to Job `system_test:robust against wrong payload` in [Pipeline #24261](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/pipelines/24261/test_report)https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/61Setting Content-Length header too high results in Server Error2021-06-14T19:13:35ZSiebers, MichaelSetting Content-Length header too high results in Server Error# Description
Requests to the API may contain a request body. Regardless whether it is required by the endpoint or not, request handling is controlled by the transmitted `Content-Length` header. If no such header is included in the reque...# Description
Requests to the API may contain a request body. Regardless whether it is required by the endpoint or not, request handling is controlled by the transmitted `Content-Length` header. If no such header is included in the request, the API returns `411 Length Required` (for endpoints requiring payload) or ignores the body gracefully (for endpoints requiring no payload). However when a `Content-Length` header is transmitted, there are three cases dependent on the transmitted length (`L`) and the actual body size (`B`).
| Condition | Behavior | Expected Behavior |
| --- | --- | --- |
| `L < B` | Only `L` bytes are read, the rest is discarded. Endpoints expecting a payload operate on the truncated data, succeeding or failing accordingly. Endpoints expecting no payload ignore the body. | as observed |
| `L == B` | The complete body is read. Endpoints expecting a payload operate accordingly. Endpoints expecting no payload ignore the body. | as observed |
| `L > B` | The first `B` bytes are read, then the server waits for the final bytes. Regardless whether the endpoint expects payload or not, the server finally returns `500 Internal Server Error` after timing out (after approx. 1 min.). | to discuss |
## Discussion
The problem is the final case, `L > B`. The server waits for the final bytes which will never arrive. In my opinion, this is no server error but a client-side error. Thus, the resulting status code should be in the 4xx region. Either a general `400 Bad Request` or the specific `408 Request Timeout`? Any thoughts?
## References
- [Failed Pipeline](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/pipelines/24239/failures)
- [Wikipedia Status Code descriptions](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes)https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/60API reacts to wrong methods2021-06-08T10:10:28ZSiebers, MichaelAPI reacts to wrong methodsJob [#72050](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/jobs/72050) failed for f90ec879d44e31742bd03512655a92798077fb0e:
Unlike described in the API endpoint documentation, the endpoint `/bg/clear` listens to the HT...Job [#72050](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/jobs/72050) failed for f90ec879d44e31742bd03512655a92798077fb0e:
Unlike described in the API endpoint documentation, the endpoint `/bg/clear` listens to the HTTP methods `GET`, `HEAD`, `PUT`, `DELETE`, `OPTIONS`, `TRACE`, and `PATCH`. It's unclear whether the behavior for such calls is identical to the expected `POST`-calls. Nevertheless, no other functionality than the one of the the `POST`-calls is indented.
The API should reject calls with the undocumented methods.https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/58Support for running as daemon under *nix2021-06-02T12:42:13ZSebastian SeufertSupport for running as daemon under *nixWhen the API is started as daemon it quits/fails.\
Assumption is this is due to SWI's expected environment during runtime / when starting as default REPL with stdio
(case at hand: this [student-project](https://gitlab.rz.uni-bamberg.de/c...When the API is started as daemon it quits/fails.\
Assumption is this is due to SWI's expected environment during runtime / when starting as default REPL with stdio
(case at hand: this [student-project](https://gitlab.rz.uni-bamberg.de/cogsys/its/its-subtraction-webinterface) for which @klaus.stein tried to run a stripped version of the API in a docker container).
@michael.siebers
this might be a somewhat easy fix, c.f. the library-support for http-daemons [here](https://github.com/SWI-Prolog/packages-http/blob/master/http_unix_daemon.pl).\
Do you see any problems I might not see if we used this as wrapper around `web_api.pl`?
-----------------------
Thinking ahead on what might be a problem:
We deliberately ignored anything related to multiuser/threading issues, so could it be possible for the container to be configured s.t. it spawns separate isolated daemon-processes if needed? @klaus.stein \
(I know this is not all that needs to be considered...)Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/57Number headers only for documentation2020-12-13T19:24:59ZSiebers, MichaelNumber headers only for documentationRegarding the numbering of headers introduced in commit 1dd752e6cf9ca4831eb9c1144ed5c887440dacd1: These are applied to all headlines (in our documentation, auto-generated directory views and the manual). In some places this looks rather ...Regarding the numbering of headers introduced in commit 1dd752e6cf9ca4831eb9c1144ed5c887440dacd1: These are applied to all headlines (in our documentation, auto-generated directory views and the manual). In some places this looks rather ugly, e.g. http://localhost:4444/doc/pldoc/man?section=preddesc.
Could we restrict this formatting to our documentation (directly below `/doc`)?
Implementation note: our documentation is surrounded with a custom wrapper div (id `wiki-wrapper`?)Sebastian SeufertSebastian Seuferthttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/56Cleanup2021-05-13T11:43:50ZSiebers, MichaelCleanupSmall tasks to be done before bumping the version number:
- change pldoc alias from `__API__` to `_API_` (consistent with default aliases)
- move `version.pl` to root directory (more prominent position)
- export version number under oth...Small tasks to be done before bumping the version number:
- change pldoc alias from `__API__` to `_API_` (consistent with default aliases)
- move `version.pl` to root directory (more prominent position)
- export version number under other name (`this_version/1` is rather uncommon), or export `api_version//0` instead
- remove mockups
- remove unused predicates?
- define and use location `background`?
- verify schemata
- provide curls?Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/55Automatically generate standalone documentation2021-06-04T21:54:14ZSiebers, MichaelAutomatically generate standalone documentationGenerate standalone documentation in form of local html pages. Generation may be done in CI/CD.Generate standalone documentation in form of local html pages. Generation may be done in CI/CD.Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/54review readme.md2020-12-19T04:18:55ZSiebers, Michaelreview readme.mdRevise main [readme.md](readme.md):
- requirements (SWI-Prolog 8.2.2.)
- short intro how to start server (without arguments or only with `--port`)
- refer to documentation server by the software itselfRevise main [readme.md](readme.md):
- requirements (SWI-Prolog 8.2.2.)
- short intro how to start server (without arguments or only with `--port`)
- refer to documentation server by the software itselfVersion 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/52revise API endpoint documentation2021-06-12T09:00:31ZSiebers, Michaelrevise API endpoint documentationreview and amend [api_endpoints.md](doc/api_endpoints.md):
- review `/bg (GET)`, `/bg (POST)`, `/clear`
- add `/clear/all`
- handle todos in file
- write short section introductionsreview and amend [api_endpoints.md](doc/api_endpoints.md):
- review `/bg (GET)`, `/bg (POST)`, `/clear`
- add `/clear/all`
- handle todos in file
- write short section introductionsVersion 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/51document JSON schemas2021-06-12T09:00:31ZSiebers, Michaeldocument JSON schemas- revise JSON schemas below `doc/schema`
- link JSON schemas in [api_endpoints.md](doc/api_endpoints.md) where appropriate
- write [schema overview](doc/schema/index.html)- revise JSON schemas below `doc/schema`
- link JSON schemas in [api_endpoints.md](doc/api_endpoints.md) where appropriate
- write [schema overview](doc/schema/index.html)Version 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/50describe module web_api2020-12-19T04:18:55ZSiebers, Michaeldescribe module web_apiWrite developer documentation for module web_api in [module_web_api.md](doc/module_web_api.md).Write developer documentation for module web_api in [module_web_api.md](doc/module_web_api.md).Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/49describe module theory_bg2021-05-13T11:30:31ZSiebers, Michaeldescribe module theory_bgWrite developer documentation for module theory_bg in [module_theory_bg.md](doc/module_theory_bg.md).Write developer documentation for module theory_bg in [module_theory_bg.md](doc/module_theory_bg.md).Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/48describe module types2020-12-19T04:18:55ZSiebers, Michaeldescribe module typesWrite developer documentation for module types in [module_types.md](doc/module_types.md).Write developer documentation for module types in [module_types.md](doc/module_types.md).Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/46write documentation overview2021-05-13T11:30:31ZSiebers, Michaelwrite documentation overviewwrite / amend the documentation's [landing page](doc/overview.md)write / amend the documentation's [landing page](doc/overview.md)Version 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/43Write module description for web_api2021-05-13T11:30:31ZSiebers, MichaelWrite module description for web_apiReword structured PLDoc comment in `/src/web_api.pl`.Reword structured PLDoc comment in `/src/web_api.pl`.Version 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/42Integrate PLDoc server2020-12-10T18:07:35ZSiebers, MichaelIntegrate PLDoc serverAn additional PLDoc server would autogenerate and serve HTML files for the prolog predicate definitions. This server shall be integrated into the main server.
- The endpoint `/doc` will serve the main documentation (started in the WIKI)...An additional PLDoc server would autogenerate and serve HTML files for the prolog predicate definitions. This server shall be integrated into the main server.
- The endpoint `/doc` will serve the main documentation (started in the WIKI).
- The endpoint `doc/pldoc` will serve the actual predicate documentation.
- Whether documentation is served is configurable using a CLI switch (default is yes)
- The served pages will have some custom look and feel (nice-to-have).
- Footer will display the used SWI-Prolog and WebAPI version.https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/41Contradicting returns of `/irrelevant/file` and `/explain`2020-12-04T19:38:07ZSiebers, MichaelContradicting returns of `/irrelevant/file` and `/explain`Given the example background knowledge, `/irrelevant/file` reports `/A/B/Filename2.exe` to be irrelevant and `/A/B/Filename.exe`to be not irrelevant. However, retrieving an explanation for `/A/B/Filename.exe` succeeds while requesting an...Given the example background knowledge, `/irrelevant/file` reports `/A/B/Filename2.exe` to be irrelevant and `/A/B/Filename.exe`to be not irrelevant. However, retrieving an explanation for `/A/B/Filename.exe` succeeds while requesting an explanation for `/A/B/Filename2.exe` results in a Bad Request response as `/A/B/Filename2.exe` is not irrelevant.Version 1.0Siebers, MichaelSiebers, Michael