Demonstrator - Reasoning WebAPI issueshttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues2021-06-13T23:44:10Zhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/65Endpoint /explain accepts malformatted request2021-06-13T23:44:10ZSiebers, MichaelEndpoint /explain accepts malformatted requestAs the endpoint `/irrelevant/file`, the endpoint `/explain` accepts absolute path names with a trailing slash (#64). This is not yet detected by system tests.As the endpoint `/irrelevant/file`, the endpoint `/explain` accepts absolute path names with a trailing slash (#64). This is not yet detected by system tests.Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/64Endpoint `irrelevant/file` accepts malformatted request2021-06-13T23:44:10ZSiebers, MichaelEndpoint `irrelevant/file` accepts malformatted request# Evidence
Job `system_test:irrelevant robust against non-path payload` of pipeline [#24446](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/pipelines/24446). Test "Querry Irrelevance[13/20]: Bad Request" indicates that t...# Evidence
Job `system_test:irrelevant robust against non-path payload` of pipeline [#24446](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/pipelines/24446). Test "Querry Irrelevance[13/20]: Bad Request" indicates that the endpoint accepted an abs_path with trailing slash!Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/63Endpoint irrelevant/file returns 5002021-06-13T22:01:23ZSiebers, MichaelEndpoint irrelevant/file returns 500Test have detected a case where the endpoint `irrelevant/file` responds with status code 500. It turned out that the predicate `irrelevant/1` throws an exception. So the behavior of the WebAPI is correct.
Source of the exception is the ...Test have detected a case where the endpoint `irrelevant/file` responds with status code 500. It turned out that the predicate `irrelevant/1` throws an exception. So the behavior of the WebAPI is correct.
Source of the exception is the predicate `manipulation_time/2'. When called with a directory as first argument the stored dict does not have the appropriate key. Thus an exception is thrown.
## Evidence
- Failed tests: `system_test:endpoint_irrelevant@file` of job https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/jobs/72393.
- [Loaded background theory database](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/blob/b6b23ae408491c55b3d7eb65e9bf84ed1c256cec/tests/api/endpoint_irrelevant@file-theory_bg.db)
- Request body: `{"abs_path": "dir_1/file_new"}`
## Tasks
- Refine the `irrelevance_common.plt` tests to catch this error in the future.
- Safeguard `manipulation_time/2' to fail when called with an directory at first argument.
- Test other `irrelevance_common.pl` predicates for similar errors.Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/62HTML error on wrong Content-Length2021-06-12T09:00:31ZSiebers, MichaelHTML error on wrong Content-LengthAccording to #28 all error messages are expected to be JSON.
When transmitting a non-numeric Content-Length header, the API correctly returns '400 Bad Request`. However, the transmitted error message is not a JSON object but an HTML docu...According to #28 all error messages are expected to be JSON.
When transmitting a non-numeric Content-Length header, the API correctly returns '400 Bad Request`. However, the transmitted error message is not a JSON object but an HTML document.
**Evidence**: Test `bg/add: Valid error response body` belonging to Job `system_test:robust against wrong payload` in [Pipeline #24261](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/pipelines/24261/test_report)https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/61Setting Content-Length header too high results in Server Error2021-06-14T19:13:35ZSiebers, MichaelSetting Content-Length header too high results in Server Error# Description
Requests to the API may contain a request body. Regardless whether it is required by the endpoint or not, request handling is controlled by the transmitted `Content-Length` header. If no such header is included in the reque...# Description
Requests to the API may contain a request body. Regardless whether it is required by the endpoint or not, request handling is controlled by the transmitted `Content-Length` header. If no such header is included in the request, the API returns `411 Length Required` (for endpoints requiring payload) or ignores the body gracefully (for endpoints requiring no payload). However when a `Content-Length` header is transmitted, there are three cases dependent on the transmitted length (`L`) and the actual body size (`B`).
| Condition | Behavior | Expected Behavior |
| --- | --- | --- |
| `L < B` | Only `L` bytes are read, the rest is discarded. Endpoints expecting a payload operate on the truncated data, succeeding or failing accordingly. Endpoints expecting no payload ignore the body. | as observed |
| `L == B` | The complete body is read. Endpoints expecting a payload operate accordingly. Endpoints expecting no payload ignore the body. | as observed |
| `L > B` | The first `B` bytes are read, then the server waits for the final bytes. Regardless whether the endpoint expects payload or not, the server finally returns `500 Internal Server Error` after timing out (after approx. 1 min.). | to discuss |
## Discussion
The problem is the final case, `L > B`. The server waits for the final bytes which will never arrive. In my opinion, this is no server error but a client-side error. Thus, the resulting status code should be in the 4xx region. Either a general `400 Bad Request` or the specific `408 Request Timeout`? Any thoughts?
## References
- [Failed Pipeline](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/pipelines/24239/failures)
- [Wikipedia Status Code descriptions](https://en.wikipedia.org/wiki/List_of_HTTP_status_codes)https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/60API reacts to wrong methods2021-06-08T10:10:28ZSiebers, MichaelAPI reacts to wrong methodsJob [#72050](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/jobs/72050) failed for f90ec879d44e31742bd03512655a92798077fb0e:
Unlike described in the API endpoint documentation, the endpoint `/bg/clear` listens to the HT...Job [#72050](https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/jobs/72050) failed for f90ec879d44e31742bd03512655a92798077fb0e:
Unlike described in the API endpoint documentation, the endpoint `/bg/clear` listens to the HTTP methods `GET`, `HEAD`, `PUT`, `DELETE`, `OPTIONS`, `TRACE`, and `PATCH`. It's unclear whether the behavior for such calls is identical to the expected `POST`-calls. Nevertheless, no other functionality than the one of the the `POST`-calls is indented.
The API should reject calls with the undocumented methods.https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/58Support for running as daemon under *nix2021-06-02T12:42:13ZSebastian SeufertSupport for running as daemon under *nixWhen the API is started as daemon it quits/fails.\
Assumption is this is due to SWI's expected environment during runtime / when starting as default REPL with stdio
(case at hand: this [student-project](https://gitlab.rz.uni-bamberg.de/c...When the API is started as daemon it quits/fails.\
Assumption is this is due to SWI's expected environment during runtime / when starting as default REPL with stdio
(case at hand: this [student-project](https://gitlab.rz.uni-bamberg.de/cogsys/its/its-subtraction-webinterface) for which @klaus.stein tried to run a stripped version of the API in a docker container).
@michael.siebers
this might be a somewhat easy fix, c.f. the library-support for http-daemons [here](https://github.com/SWI-Prolog/packages-http/blob/master/http_unix_daemon.pl).\
Do you see any problems I might not see if we used this as wrapper around `web_api.pl`?
-----------------------
Thinking ahead on what might be a problem:
We deliberately ignored anything related to multiuser/threading issues, so could it be possible for the container to be configured s.t. it spawns separate isolated daemon-processes if needed? @klaus.stein \
(I know this is not all that needs to be considered...)Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/57Number headers only for documentation2020-12-13T19:24:59ZSiebers, MichaelNumber headers only for documentationRegarding the numbering of headers introduced in commit 1dd752e6cf9ca4831eb9c1144ed5c887440dacd1: These are applied to all headlines (in our documentation, auto-generated directory views and the manual). In some places this looks rather ...Regarding the numbering of headers introduced in commit 1dd752e6cf9ca4831eb9c1144ed5c887440dacd1: These are applied to all headlines (in our documentation, auto-generated directory views and the manual). In some places this looks rather ugly, e.g. http://localhost:4444/doc/pldoc/man?section=preddesc.
Could we restrict this formatting to our documentation (directly below `/doc`)?
Implementation note: our documentation is surrounded with a custom wrapper div (id `wiki-wrapper`?)Sebastian SeufertSebastian Seuferthttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/56Cleanup2021-05-13T11:43:50ZSiebers, MichaelCleanupSmall tasks to be done before bumping the version number:
- change pldoc alias from `__API__` to `_API_` (consistent with default aliases)
- move `version.pl` to root directory (more prominent position)
- export version number under oth...Small tasks to be done before bumping the version number:
- change pldoc alias from `__API__` to `_API_` (consistent with default aliases)
- move `version.pl` to root directory (more prominent position)
- export version number under other name (`this_version/1` is rather uncommon), or export `api_version//0` instead
- remove mockups
- remove unused predicates?
- define and use location `background`?
- verify schemata
- provide curls?Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/55Automatically generate standalone documentation2021-06-04T21:54:14ZSiebers, MichaelAutomatically generate standalone documentationGenerate standalone documentation in form of local html pages. Generation may be done in CI/CD.Generate standalone documentation in form of local html pages. Generation may be done in CI/CD.Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/54review readme.md2020-12-19T04:18:55ZSiebers, Michaelreview readme.mdRevise main [readme.md](readme.md):
- requirements (SWI-Prolog 8.2.2.)
- short intro how to start server (without arguments or only with `--port`)
- refer to documentation server by the software itselfRevise main [readme.md](readme.md):
- requirements (SWI-Prolog 8.2.2.)
- short intro how to start server (without arguments or only with `--port`)
- refer to documentation server by the software itselfVersion 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/52revise API endpoint documentation2021-06-12T09:00:31ZSiebers, Michaelrevise API endpoint documentationreview and amend [api_endpoints.md](doc/api_endpoints.md):
- review `/bg (GET)`, `/bg (POST)`, `/clear`
- add `/clear/all`
- handle todos in file
- write short section introductionsreview and amend [api_endpoints.md](doc/api_endpoints.md):
- review `/bg (GET)`, `/bg (POST)`, `/clear`
- add `/clear/all`
- handle todos in file
- write short section introductionsVersion 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/51document JSON schemas2021-06-12T09:00:31ZSiebers, Michaeldocument JSON schemas- revise JSON schemas below `doc/schema`
- link JSON schemas in [api_endpoints.md](doc/api_endpoints.md) where appropriate
- write [schema overview](doc/schema/index.html)- revise JSON schemas below `doc/schema`
- link JSON schemas in [api_endpoints.md](doc/api_endpoints.md) where appropriate
- write [schema overview](doc/schema/index.html)Version 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/50describe module web_api2020-12-19T04:18:55ZSiebers, Michaeldescribe module web_apiWrite developer documentation for module web_api in [module_web_api.md](doc/module_web_api.md).Write developer documentation for module web_api in [module_web_api.md](doc/module_web_api.md).Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/49describe module theory_bg2021-05-13T11:30:31ZSiebers, Michaeldescribe module theory_bgWrite developer documentation for module theory_bg in [module_theory_bg.md](doc/module_theory_bg.md).Write developer documentation for module theory_bg in [module_theory_bg.md](doc/module_theory_bg.md).Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/48describe module types2020-12-19T04:18:55ZSiebers, Michaeldescribe module typesWrite developer documentation for module types in [module_types.md](doc/module_types.md).Write developer documentation for module types in [module_types.md](doc/module_types.md).Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/47write admin guide2020-12-08T16:20:16ZSiebers, Michaelwrite admin guidewrite [admin guide](doc/admin_guide.md)write [admin guide](doc/admin_guide.md)Version 1.1https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/46write documentation overview2021-05-13T11:30:31ZSiebers, Michaelwrite documentation overviewwrite / amend the documentation's [landing page](doc/overview.md)write / amend the documentation's [landing page](doc/overview.md)Version 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/45Write user guide2020-12-08T16:17:59ZSiebers, MichaelWrite user guidewrite [user guide](doc/user_guide.md)write [user guide](doc/user_guide.md)Version 1.1https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/44Write developer guide2021-05-10T18:01:41ZSiebers, MichaelWrite developer guidewrite [developer guide](doc/dev_guide.md)write [developer guide](doc/dev_guide.md)Version 1.1