Demonstrator - Reasoning WebAPI issueshttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues2020-12-04T19:38:07Zhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/41Contradicting returns of `/irrelevant/file` and `/explain`2020-12-04T19:38:07ZSiebers, MichaelContradicting returns of `/irrelevant/file` and `/explain`Given the example background knowledge, `/irrelevant/file` reports `/A/B/Filename2.exe` to be irrelevant and `/A/B/Filename.exe`to be not irrelevant. However, retrieving an explanation for `/A/B/Filename.exe` succeeds while requesting an...Given the example background knowledge, `/irrelevant/file` reports `/A/B/Filename2.exe` to be irrelevant and `/A/B/Filename.exe`to be not irrelevant. However, retrieving an explanation for `/A/B/Filename.exe` succeeds while requesting an explanation for `/A/B/Filename2.exe` results in a Bad Request response as `/A/B/Filename2.exe` is not irrelevant.Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/32Alternating error responses2020-12-04T17:46:03ZSiebers, MichaelAlternating error responses# Summary
The problem of alternating responses as expected and error responses, like described in #17 and #18, still persists. This happens if the HTTP stream is kept open (keep-alive), no `Content-Length` header is sent, the request bod...# Summary
The problem of alternating responses as expected and error responses, like described in #17 and #18, still persists. This happens if the HTTP stream is kept open (keep-alive), no `Content-Length` header is sent, the request body contains a valid payload, and there is more data after the valid payload.
# Steps to reproduce
Taking the `/irrelevant/file`-endpoint as example:
1. Send a POST request to endpoint `/irrelevant/file` ...
- with body `{"abs_path":"a"}_`,
- with header `Content-Type: application/json`,
- with header `Connection: keep-alive`,
- without header `Content-Length`
2. Send any request to any endpoint (may be identical)
cURL command to reproduce
```curl
curl --request POST 'http://localhost:4444/irrelevant/file' \
--header 'Content-Type: application/json' \
--header 'Content-Length:' \
--data-raw '{"abs_path":"a"}_' \
--next --request GET 'http://localhost:4444/bg'
```
## Observed behavior
- first request succeeds with `{"result":true}`
- second request fails with a HTML-formatted `400 Bad Request` message stating `Illegal HTTP request`
## Desired behavior
I'm not sure what this should be. Depending on the additional payload supplied, the first request could be a `400 Bad Request`. If it's only white space the request may be considered ok. But if it is _real_ garbage it should be dismissed.
Either way, the second request should succeed (if it is correctly formatted)
# Solution
Internally, the predicate `http_client:http_convert_data/4` (defined in library http/http_json) is used to parse the JSON payload. The behavior of this predicate depends on the `Content-Length` header:
- If it is not supplied, the stream is read (and parsed) until parsing the first JSON object succeeds. Further data is left on the stream.
- If such a header is supplied, the stream is read (and parsed) until the full content length is read or parsing the first JSON object succeeds. Further data (up to the content's length) is discarded.
The first case leads to the described problem, as additional payload will "start" the next request making it invalid. The second case is also problematic itself, but this is not relevant for this issue.
There are different ways to circumvent the problem:
1. Disallow POST requests without `Content-Length` header.
1. Always close the stream after each request (disregarding the `Connection` header)
1. After parsing the JSON payload, read the stream until the end and manually handle excess payload.Version 1.0Sebastian SeufertSebastian Seuferthttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/6Complete explanation templates2020-11-21T17:18:11ZSiebers, MichaelComplete explanation templatesThere should be explanation templates for all predicates which might be used in the irrelevance theory.There should be explanation templates for all predicates which might be used in the irrelevance theory.Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/40Bump version number2021-05-13T11:47:31ZSiebers, MichaelBump version numberChange version number in code and documentationChange version number in code and documentationVersion 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/39Decide on xxx_time semantics2020-12-04T18:17:58ZSiebers, MichaelDecide on xxx_time semanticsIn `irrelevance_common.pl`, `creation_time/2` works as (+,-), other `xxx_time/2` are implemented as (?,?).
The tests for all of these predicates are commented to test (+,+), (+,-), and (-,+). But actually only (+,+) and (+,-) is tested!In `irrelevance_common.pl`, `creation_time/2` works as (+,-), other `xxx_time/2` are implemented as (?,?).
The tests for all of these predicates are commented to test (+,+), (+,-), and (-,+). But actually only (+,+) and (+,-) is tested!Version 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/38Harmonize internal and extermal representation of items2020-12-04T19:38:07ZSiebers, MichaelHarmonize internal and extermal representation of items_The described status quo assumes !13 has been merged. Consequently, this issue should only be attacked after said merge._
# Current situation
The external (JSON) representation of files and directories must have the property "type" wit..._The described status quo assumes !13 has been merged. Consequently, this issue should only be attacked after said merge._
# Current situation
The external (JSON) representation of files and directories must have the property "type" with a value of `"file"` or `"directory"`, respectively. The internal (dict) representation does not have this key, but the tag if the dict is either `'file'` or `'directory'`.
When receiving external representations (via `/bg` POST):
- the external representation is read into a dict (where the tag is a variable)
- it is checked if the "type" key is present with a valid value (done before transformation to give meaningful error messages)
- the "type" key is removed from the dict
- the dict's tag is unified with the value of type
When sending external representations (via `/bg` Get):
- the internal representation is retrieved from background knowledge
- it is send as is (This a bug, as we claim that our result conforms to the input specification but we send no "type" property).
# Solution
Either
1. we add the "type" property/key before sending out internal representations,
2. we change our internal representation to include the type in addition to the tag value, or
3. we change our internal representation to include the type but use a uniform tag, for example `item`.Version 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/33Validate JSON input and throw exceptions2020-12-04T17:58:19ZSiebers, MichaelValidate JSON input and throw exceptionsOnly for the `/irrelevant/file` and the `/explain` endpoint, the JSON input is validated. And only these two endpoints throw appropriate exceptions (c.f. !6). These practices should be rolled out to the other endpoints:
- [ ] ~~`/`~~
- ...Only for the `/irrelevant/file` and the `/explain` endpoint, the JSON input is validated. And only these two endpoints throw appropriate exceptions (c.f. !6). These practices should be rolled out to the other endpoints:
- [ ] ~~`/`~~
- [ ] ~~`/add`~~
- [ ] ~~`/remove`~~
- [ ] ~~`/query`~~
- [x] `/state`
- [x] `/bg`
- [x] POST requests (c.f. #21)
- [x] GET requests
- [x] `/clear`
- [x] `/clear/all`
Some of these tasks might be done more easy after #27 is solved.Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/35Undocumented edge case for adding background knowledge2020-12-02T17:01:10ZSiebers, MichaelUndocumented edge case for adding background knowledge# Summary
Adding two items with the same absolute path succeeds if some other property is distinct.
# Steps to reproduce
Send a POST request to `/bg` requesting to add two items with identical path but at least one distinct property.
E...# Summary
Adding two items with the same absolute path succeeds if some other property is distinct.
# Steps to reproduce
Send a POST request to `/bg` requesting to add two items with identical path but at least one distinct property.
Example cURL call:
```shell
curl --request POST 'http://localhost:4444/bg' \
--header 'Content-Type: application/json' \
--data-raw '[{
"type" : "directory",
"abs_path": "/A/B",
"creation_time": 1
},
{
"type" : "directory",
"abs_path": "/A/B",
"creation_time": 2
}
]'
```
# Observed behavior
The server returns that both items have been added.
```json
{
"added": 2,
"received": 2,
"skipped": 0
}
```
# Expected behavior
Only the first addition should succeed. The second should fail as there is already an item known at this path.Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/28Force errors to be JSON2021-06-08T15:42:32ZSiebers, MichaelForce errors to be JSONThe http server library decides automatically if errors should be returned as formatted HTML or as JSON. This decision is made in `http_json:prefer_json/1`.
We always want JSON returns. However, we *may* get HTML returns, for example if...The http server library decides automatically if errors should be returned as formatted HTML or as JSON. This decision is made in `http_json:prefer_json/1`.
We always want JSON returns. However, we *may* get HTML returns, for example if the request does not contain an `Accept` header. Can we force the library to return JSON?Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/24Document theory_common and theory_templates2020-12-04T19:40:05ZSiebers, MichaelDocument theory_common and theory_templatesDescribe ...
- [x] the idea behind predicates in `theory_common.pl`
- [x] where these predicates are (or might be) used
- [x] the format of explanation templates
- [x] how to load explanation templates
- [x] where explanation templates a...Describe ...
- [x] the idea behind predicates in `theory_common.pl`
- [x] where these predicates are (or might be) used
- [x] the format of explanation templates
- [x] how to load explanation templates
- [x] where explanation templates are usedVersion 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/22Map http_reply exceptions to JSON reply2020-12-01T12:37:34ZSiebers, MichaelMap http_reply exceptions to JSON replyVersion 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/21Add key and type check for file/directory JSON payloads2020-12-04T17:46:03ZSiebers, MichaelAdd key and type check for file/directory JSON payloadsSeveral endpoints require information on files or directories as JSON objects. When parsing these objects we should check that
- no required key is missing,
- values are of appropriate / expected type and
- no unknown key is provided.
W...Several endpoints require information on files or directories as JSON objects. When parsing these objects we should check that
- no required key is missing,
- values are of appropriate / expected type and
- no unknown key is provided.
Without these checks _semantic_ errors may occur which are hard to debug / detect (c.f. #19).Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/56Cleanup2021-05-13T11:43:50ZSiebers, MichaelCleanupSmall tasks to be done before bumping the version number:
- change pldoc alias from `__API__` to `_API_` (consistent with default aliases)
- move `version.pl` to root directory (more prominent position)
- export version number under oth...Small tasks to be done before bumping the version number:
- change pldoc alias from `__API__` to `_API_` (consistent with default aliases)
- move `version.pl` to root directory (more prominent position)
- export version number under other name (`this_version/1` is rather uncommon), or export `api_version//0` instead
- remove mockups
- remove unused predicates?
- define and use location `background`?
- verify schemata
- provide curls?Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/26Startup parameters2020-11-26T13:39:06ZSiebers, MichaelStartup parametersStartup options for the server (like port to run on) seem to be managed in the predicate server_opts/1. However, I do not use how to use them. Regarding the port option, for instance, neither of the following works:
```
swipl -s run.pl ...Startup options for the server (like port to run on) seem to be managed in the predicate server_opts/1. However, I do not use how to use them. Regarding the port option, for instance, neither of the following works:
```
swipl -s run.pl --port 8976
```
```
swipl -s run.pl --port=8976
```
```
swipl -s run.pl port=8976
```
According to the swipl usage message, swi-options and programm arguments should (can?) be separated by `--`. This seems to kinda work. However, then our code complains that `--` is no valid argument.
Basically, I have the following questions:
1. How is it possible to start the server with arguments?
1. Which arguments are defined? Or rather, what do they do?
1. Can we print some usage message ourselves? For example with `-h` but without interfering with swipl's `-h` argument?Version 1.0Sebastian SeufertSebastian Seuferthttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/54review readme.md2020-12-19T04:18:55ZSiebers, Michaelreview readme.mdRevise main [readme.md](readme.md):
- requirements (SWI-Prolog 8.2.2.)
- short intro how to start server (without arguments or only with `--port`)
- refer to documentation server by the software itselfRevise main [readme.md](readme.md):
- requirements (SWI-Prolog 8.2.2.)
- short intro how to start server (without arguments or only with `--port`)
- refer to documentation server by the software itselfVersion 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/52revise API endpoint documentation2021-06-12T09:00:31ZSiebers, Michaelrevise API endpoint documentationreview and amend [api_endpoints.md](doc/api_endpoints.md):
- review `/bg (GET)`, `/bg (POST)`, `/clear`
- add `/clear/all`
- handle todos in file
- write short section introductionsreview and amend [api_endpoints.md](doc/api_endpoints.md):
- review `/bg (GET)`, `/bg (POST)`, `/clear`
- add `/clear/all`
- handle todos in file
- write short section introductionsVersion 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/51document JSON schemas2021-06-12T09:00:31ZSiebers, Michaeldocument JSON schemas- revise JSON schemas below `doc/schema`
- link JSON schemas in [api_endpoints.md](doc/api_endpoints.md) where appropriate
- write [schema overview](doc/schema/index.html)- revise JSON schemas below `doc/schema`
- link JSON schemas in [api_endpoints.md](doc/api_endpoints.md) where appropriate
- write [schema overview](doc/schema/index.html)Version 1.0https://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/50describe module web_api2020-12-19T04:18:55ZSiebers, Michaeldescribe module web_apiWrite developer documentation for module web_api in [module_web_api.md](doc/module_web_api.md).Write developer documentation for module web_api in [module_web_api.md](doc/module_web_api.md).Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/49describe module theory_bg2021-05-13T11:30:31ZSiebers, Michaeldescribe module theory_bgWrite developer documentation for module theory_bg in [module_theory_bg.md](doc/module_theory_bg.md).Write developer documentation for module theory_bg in [module_theory_bg.md](doc/module_theory_bg.md).Version 1.0Siebers, MichaelSiebers, Michaelhttps://gitlab.rz.uni-bamberg.de/cogsys/dare2del/demonstrator/-/issues/48describe module types2020-12-19T04:18:55ZSiebers, Michaeldescribe module typesWrite developer documentation for module types in [module_types.md](doc/module_types.md).Write developer documentation for module types in [module_types.md](doc/module_types.md).Version 1.0Siebers, MichaelSiebers, Michael