search issueshttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues2019-11-28T08:55:10Zhttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/97Upgrade dependencies2019-11-28T08:55:10ZGradl, TobiasUpgrade dependencies- Major upgrade so various issues should be expected to arise
- Spring and the like
*(from redmine: issue id 185, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #184- Major upgrade so various issues should be expected to arise
- Spring and the like
*(from redmine: issue id 185, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #1843.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/108Remove old packages and dependencies2019-11-28T08:55:24ZGradl, TobiasRemove old packages and dependencies- model, federation etc.
- just remove and see what happens
*(from redmine: issue id 489, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #496
* duplicates #494- model, federation etc.
- just remove and see what happens
*(from redmine: issue id 489, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #496
* duplicates #4943.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/110Upgrade to recent ES2019-11-28T08:55:26ZGradl, TobiasUpgrade to recent EStest the more recent elastic search versions
*(from redmine: issue id 491, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #496
* relates #492test the more recent elastic search versions
*(from redmine: issue id 491, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #496
* relates #4923.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/111Rework indexing strategy2019-11-28T08:55:28ZGradl, TobiasRework indexing strategyThis is required due to changes in ES 2.0
https://www.elastic.co/guide/en/elasticsearch/plugins/2.0/plugins-delete-by-query.html
- One index per mapping might be an option
- Saving indexed data to the filesystem in order to fast r...This is required due to changes in ES 2.0
https://www.elastic.co/guide/en/elasticsearch/plugins/2.0/plugins-delete-by-query.html
- One index per mapping might be an option
- Saving indexed data to the filesystem in order to fast reindex is
another option
*(from redmine: issue id 492, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #4913.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/112Remove relational DB completely and migrate data to ES2019-11-28T08:55:29ZGradl, TobiasRemove relational DB completely and migrate data to ESuse spring-data-elasticsearch
*(from redmine: issue id 493, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #496use spring-data-elasticsearch
*(from redmine: issue id 493, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #4963.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/113Cleanup unused dependencies2019-11-28T08:55:31ZGradl, TobiasCleanup unused dependenciesThere are some dependencies that I do not remember the use of ->test
with Tomcat, however!
*(from redmine: issue id 494, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #496
* duplicates #489There are some dependencies that I do not remember the use of ->test
with Tomcat, however!
*(from redmine: issue id 494, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #496
* duplicates #4893.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/114Migrate security from BaseController to Filter/Interceptor2019-11-28T08:55:32ZGradl, TobiasMigrate security from BaseController to Filter/Interceptor.
*(from redmine: issue id 495, created on 2018-01-22tgradl, closed on 2018-01-24)*.
*(from redmine: issue id 495, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/115Refactor and recreate search2019-11-28T08:55:34ZGradl, TobiasRefactor and recreate search- Start from scratch as persistence layer is swapped
- Include dependencies and functionality as indicated
*(from redmine: issue id 496, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #493
* relates...- Start from scratch as persistence layer is swapped
- Include dependencies and functionality as indicated
*(from redmine: issue id 496, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #493
* relates #489
* relates #491
* relates #494
* relates #4973.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/116Rebuild collection/endpoint management2019-11-28T08:55:35ZGradl, TobiasRebuild collection/endpoint management.
*(from redmine: issue id 497, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #496.
*(from redmine: issue id 497, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #4963.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/117Recreate/migrate collection registry sync behavior2019-11-28T08:55:36ZGradl, TobiasRecreate/migrate collection registry sync behaviortry to reuse functionality
*(from redmine: issue id 498, created on 2018-01-22tgradl, closed on 2018-01-24)*try to reuse functionality
*(from redmine: issue id 498, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/118Implement new metamodel for schemata/mappings2019-11-28T08:55:37ZGradl, TobiasImplement new metamodel for schemata/mappingsSave and load from filesystem ->physical independence of ScheReg
being live
*(from redmine: issue id 499, created on 2018-01-22tgradl, closed on 2018-01-24)*Save and load from filesystem ->physical independence of ScheReg
being live
*(from redmine: issue id 499, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/119Consume ScheReg API for schemata consumption2019-11-28T08:55:38ZGradl, TobiasConsume ScheReg API for schemata consumptionSync and refresh filesystem
*(from redmine: issue id 500, created on 2018-01-22tgradl, closed on 2018-01-24)*Sync and refresh filesystem
*(from redmine: issue id 500, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/120Move model to MongoDB2019-11-28T08:55:39ZGradl, TobiasMove model to MongoDBThe model in ES is actually ok, but backend access and (emergency)
administration is not so easy
*(from redmine: issue id 503, created on 2018-01-22tgradl, closed on 2018-01-24)*The model in ES is actually ok, but backend access and (emergency)
administration is not so easy
*(from redmine: issue id 503, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/121Reimplement cleaner version of datasource management dialogs2019-11-28T08:55:41ZGradl, TobiasReimplement cleaner version of datasource management dialogsd
*(from redmine: issue id 504, created on 2018-01-22tgradl, closed on 2018-01-24)*d
*(from redmine: issue id 504, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/122Logically connect datasources and schemas2019-11-28T08:55:42ZGradl, TobiasLogically connect datasources and schemas- what do we import?
- do we create ‘dummy’ schema entries?
- how are connection manifested?
*(from redmine: issue id 505, created on 2018-01-22tgradl, closed on 2018-01-24)*- what do we import?
- do we create ‘dummy’ schema entries?
- how are connection manifested?
*(from redmine: issue id 505, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/123Rework GS crawl concept2019-11-28T08:55:43ZGradl, TobiasRework GS crawl concept- how to relate with index and schema?
- Determine, when reindexing needs to be done and which crawl data to
reevaluate
*(from redmine: issue id 506, created on 2018-01-22tgradl, closed on 2018-01-24)*- how to relate with index and schema?
- Determine, when reindexing needs to be done and which crawl data to
reevaluate
*(from redmine: issue id 506, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/124Implement crawl state listener2019-11-28T08:55:44ZGradl, TobiasImplement crawl state listener- monitor crawls and update their status on error/complete
- hold state info on queue/progressing internally
*(from redmine: issue id 507, created on 2018-01-22tgradl, closed on 2018-01-24)*- monitor crawls and update their status on error/complete
- hold state info on queue/progressing internally
*(from redmine: issue id 507, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/125Migrate/implement resource consumption service2019-11-28T08:55:45ZGradl, TobiasMigrate/implement resource consumption service.
*(from redmine: issue id 508, created on 2018-01-22tgradl, closed on 2018-01-24)*.
*(from redmine: issue id 508, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/126Implement crawls as persisted entity2019-11-28T08:55:46ZGradl, TobiasImplement crawls as persisted entity- Easier CRUD
- Relation to any other object persisted with crawl
*(from redmine: issue id 509, created on 2018-01-22tgradl, closed on 2018-01-24)*- Easier CRUD
- Relation to any other object persisted with crawl
*(from redmine: issue id 509, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/127Implement OAI-PMH offline crawling/indexing2019-11-28T08:55:48ZGradl, TobiasImplement OAI-PMH offline crawling/indexing- Refactor OAIPMH / BaseXML processing services
- Introduce a generic online processing service?
- How could the generic xml file processing service include the oaipmh
specific metadata?
*(from redmine: issue id 510, created ...- Refactor OAIPMH / BaseXML processing services
- Introduce a generic online processing service?
- How could the generic xml file processing service include the oaipmh
specific metadata?
*(from redmine: issue id 510, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/128Explicitly define mapping when synchronizing schemata2019-11-28T08:55:49ZGradl, TobiasExplicitly define mapping when synchronizing schemata- Disable dynamic?!
*(from redmine: issue id 511, created on 2018-01-22tgradl, closed on 2018-01-24)*- Disable dynamic?!
*(from redmine: issue id 511, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/130Use SR mappings to configure GS2019-11-28T08:55:51ZGradl, TobiasUse SR mappings to configure GSMap fields to:
- data types (other than string, like e.g. geo)
- links
- titles
- analyzers
*(from redmine: issue id 513, created on 2018-01-22tgradl, closed on 2018-01-24)*Map fields to:
- data types (other than string, like e.g. geo)
- links
- titles
- analyzers
*(from redmine: issue id 513, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/131Implement simple search interface2019-11-28T08:55:52ZGradl, TobiasImplement simple search interfacetry to migrate from v2 branch
*(from redmine: issue id 514, created on 2018-01-22tgradl, closed on 2018-01-24)*try to migrate from v2 branch
*(from redmine: issue id 514, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/132Fix snippet generation for search results2019-11-28T08:55:54ZGradl, TobiasFix snippet generation for search resultscorrupted after migration
*(from redmine: issue id 515, created on 2018-01-22tgradl, closed on 2018-01-24)*corrupted after migration
*(from redmine: issue id 515, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/133Fix resource serialization2019-11-28T08:55:55ZGradl, TobiasFix resource serializationAlso adapt deserialization later
*(from redmine: issue id 516, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #517Also adapt deserialization later
*(from redmine: issue id 516, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #5173.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/135Provide search options box2019-11-28T08:55:58ZGradl, TobiasProvide search options boxMigrate from old search
*(from redmine: issue id 518, created on 2018-01-22tgradl, closed on 2018-01-24)*Migrate from old search
*(from redmine: issue id 518, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/136Consume grammars and functions with schema and mapping APIs and persist2019-11-28T08:56:00ZGradl, TobiasConsume grammars and functions with schema and mapping APIs and persist- Detection of freshness
*(from redmine: issue id 527, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #529- Detection of freshness
*(from redmine: issue id 527, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #5293.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/137Grammars in mappings not executed2019-11-28T08:56:01ZGradl, TobiasGrammars in mappings not executedProblem: Grammars and functions expect labels as targets…is this
restriction really important?
*(from redmine: issue id 529, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #527Problem: Grammars and functions expect labels as targets…is this
restriction really important?
*(from redmine: issue id 529, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #5273.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/138Refactor OAI-PMH online/offline crawling, XML crawling2019-11-28T08:56:03ZGradl, TobiasRefactor OAI-PMH online/offline crawling, XML crawlingRedundant code etc.
*(from redmine: issue id 530, created on 2018-01-22tgradl, closed on 2018-01-24)*Redundant code etc.
*(from redmine: issue id 530, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/139Configure indexing with dedicated schema/mappings2019-11-28T08:56:04ZGradl, TobiasConfigure indexing with dedicated schema/mappings- data types
- analyzers
etc.
*(from redmine: issue id 531, created on 2018-01-22tgradl, closed on 2018-01-24)*- data types
- analyzers
etc.
*(from redmine: issue id 531, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/141Upgrade to new metamodel components and processing2019-11-28T08:56:06ZGradl, TobiasUpgrade to new metamodel components and processing- metamodel
- processing
- gtf?
*(from redmine: issue id 576, created on 2018-01-22tgradl, closed on 2018-01-24)*- metamodel
- processing
- gtf?
*(from redmine: issue id 576, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/142Implement metadata prefix correlation for XML root namespaces2019-11-28T08:56:07ZGradl, TobiasImplement metadata prefix correlation for XML root namespaces- Serves to detect correct local prefix to use when crawling data
- Schemata do not have to be named after local OAI-PMH prefixes
*(from redmine: issue id 577, created on 2018-01-22tgradl, closed on 2018-01-24)*- Serves to detect correct local prefix to use when crawling data
- Schemata do not have to be named after local OAI-PMH prefixes
*(from redmine: issue id 577, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/143Setup analyzers as configured with indices and utilize2019-11-28T08:56:08ZGradl, TobiasSetup analyzers as configured with indices and utilize- Analyzers should be defined in generic search and have a good common
name
- Analyzers are then referenced in the schema definition of the
gs-index-model
- Individual analyzers are utilized once mapped by a source schema
-...- Analyzers should be defined in generic search and have a good common
name
- Analyzers are then referenced in the schema definition of the
gs-index-model
- Individual analyzers are utilized once mapped by a source schema
- No special treatment needs to be applied when processing and
indexing documents
*(from redmine: issue id 578, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/144Implement aggregations for wordcloud2019-11-28T08:56:10ZGradl, TobiasImplement aggregations for wordcloud- Home page on every index
- With collection results on regular search
*(from redmine: issue id 579, created on 2018-01-22tgradl, closed on 2018-01-24)*- Home page on every index
- With collection results on regular search
*(from redmine: issue id 579, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/145Implement home for MWW search2019-11-28T08:56:11ZGradl, TobiasImplement home for MWW search- search box
- wordcloud
- MWW colors and logos
*(from redmine: issue id 580, created on 2018-01-22tgradl, closed on 2018-01-24)*- search box
- wordcloud
- MWW colors and logos
*(from redmine: issue id 580, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/151Wordcloud does not load on browser start2019-11-28T08:56:18ZGradl, TobiasWordcloud does not load on browser startThis issue is based on jsessionid being appended in baseUrl ->use
util.composeUrl instead
*(from redmine: issue id 586, created on 2018-01-22tgradl, closed on 2018-01-24)*This issue is based on jsessionid being appended in baseUrl ->use
util.composeUrl instead
*(from redmine: issue id 586, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/153Implement file adapter in generic search2019-11-28T08:56:20ZGradl, TobiasImplement file adapter in generic search.
*(from redmine: issue id 588, created on 2018-01-22tgradl, closed on 2018-01-24)*.
*(from redmine: issue id 588, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/154Show document count in datasource viewer2019-11-28T08:56:21ZGradl, TobiasShow document count in datasource viewerper endpoint/mapping
*(from redmine: issue id 589, created on 2018-01-22tgradl, closed on 2018-01-24)*per endpoint/mapping
*(from redmine: issue id 589, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/155Implement rudimentary security2019-11-28T08:56:22ZGradl, TobiasImplement rudimentary securitylocal accounts
*(from redmine: issue id 590, created on 2018-01-22tgradl, closed on 2018-01-24)*local accounts
*(from redmine: issue id 590, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/156Remove MWW branding and create fork for such adaptions2019-11-28T08:56:23ZGradl, TobiasRemove MWW branding and create fork for such adaptionsfork?
*(from redmine: issue id 591, created on 2018-01-22tgradl, closed on 2018-01-24)*fork?
*(from redmine: issue id 591, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/157Prevent setup errors with empty databases2019-11-28T08:56:24ZGradl, TobiasPrevent setup errors with empty databasesmongo and es
*(from redmine: issue id 592, created on 2018-01-22tgradl, closed on 2018-01-24)*mongo and es
*(from redmine: issue id 592, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/159Separate crawling from processing/indexing2019-11-28T08:56:27ZGradl, TobiasSeparate crawling from processing/indexing- no more OAI PMH processing service
- fetch files depending on interface
- then iterate files
*(from redmine: issue id 594, created on 2018-01-22tgradl, closed on 2018-01-24)*- no more OAI PMH processing service
- fetch files depending on interface
- then iterate files
*(from redmine: issue id 594, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/165Refactor / implement mapping sync2019-11-28T08:56:33ZGradl, TobiasRefactor / implement mapping sync- performance and independence improvement
- sync often ~ every some minutes
- manual sync through interface possible
*(from redmine: issue id 601, created on 2018-01-22tgradl, closed on 2018-01-24)*- performance and independence improvement
- sync often ~ every some minutes
- manual sync through interface possible
*(from redmine: issue id 601, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/166Refactor ScheReg schema sync2019-11-28T08:56:34ZGradl, TobiasRefactor ScheReg schema synccheck what needs to be done in what circumstances ->mongo, index,
filesystem
*(from redmine: issue id 602, created on 2018-01-22tgradl, closed on 2018-01-24)*check what needs to be done in what circumstances ->mongo, index,
filesystem
*(from redmine: issue id 602, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/168Refactor ColReg sync2019-11-28T08:56:36ZGradl, TobiasRefactor ColReg syncthink of required actions also to the index
*(from redmine: issue id 604, created on 2018-01-22tgradl, closed on 2018-01-24)*think of required actions also to the index
*(from redmine: issue id 604, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/170Consequently implement managers as design pattern when using threading2019-11-28T08:56:39ZGradl, TobiasConsequently implement managers as design pattern when using threading- sync
- indexing
- crawling
*(from redmine: issue id 606, created on 2018-01-22tgradl, closed on 2018-01-24)*- sync
- indexing
- crawling
*(from redmine: issue id 606, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/172Upgrade to ES 5.02019-11-28T08:56:41ZGradl, TobiasUpgrade to ES 5.0- Upgrade to Java 8 required
- Only minor changes in code for compilation errors
- log4j 2 requires binding to slf4j
- new string mappings ->task 610
*(from redmine: issue id 608, created on 2018-01-22tgradl, closed on 2018-01-...- Upgrade to Java 8 required
- Only minor changes in code for compilation errors
- log4j 2 requires binding to slf4j
- new string mappings ->task 610
*(from redmine: issue id 608, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/173Rethink content keys as ES > 5.0 will not allow empty keys2019-11-28T08:56:42ZGradl, TobiasRethink content keys as ES > 5.0 will not allow empty keysback to <sub>?\ yes\ why\ not,\ and\ only</sub>
*(from redmine: issue id 609, created on 2018-01-22tgradl, closed on 2018-01-24)*back to <sub>?\ yes\ why\ not,\ and\ only</sub>
*(from redmine: issue id 609, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/174Adjust mapping creation to new string mappings2019-11-28T08:56:43ZGradl, TobiasAdjust mapping creation to new string mappingsShould we be less dependent on this? Other option: put, then get right
away and persist this get for further comparison
*(from redmine: issue id 610, created on 2018-01-22tgradl, closed on 2018-01-24)*Should we be less dependent on this? Other option: put, then get right
away and persist this get for further comparison
*(from redmine: issue id 610, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/175Encapsulate elasticsearch access in a new 'esx' layer2019-11-28T08:56:44ZGradl, TobiasEncapsulate elasticsearch access in a new 'esx' layer- Client like mongotemplate only used from within esx classes
*(from redmine: issue id 611, created on 2018-01-22tgradl, closed on 2018-01-24)*- Client like mongotemplate only used from within esx classes
*(from redmine: issue id 611, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/178Implement manual selection of oaipmh prefix for endpoints/schema assignments2019-11-28T08:56:48ZGradl, TobiasImplement manual selection of oaipmh prefix for endpoints/schema assignmentse.g. cmdi can often not automatically be detected
*(from redmine: issue id 614, created on 2018-01-22tgradl, closed on 2018-01-24)*e.g. cmdi can often not automatically be detected
*(from redmine: issue id 614, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/181Adapt to CLARIN use case2019-11-28T08:56:52ZGradl, TobiasAdapt to CLARIN use case- Redeclaration of namespaces
*(from redmine: issue id 617, created on 2018-01-22tgradl, closed on 2018-01-24)*- Redeclaration of namespaces
*(from redmine: issue id 617, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/182Migrate collection selection2019-11-28T08:56:53ZGradl, TobiasMigrate collection selectionas pre-stage to ownSearch
*(from redmine: issue id 618, created on 2018-01-22tgradl, closed on 2018-01-24)*as pre-stage to ownSearch
*(from redmine: issue id 618, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/183Prevent session id from being exposed as get parameter2019-11-28T08:56:54ZGradl, TobiasPrevent session id from being exposed as get parameter- update web.xml to java 7 ->3.1 schema
*(from redmine: issue id 619, created on 2018-01-22tgradl, closed on 2018-01-24)*- update web.xml to java 7 ->3.1 schema
*(from redmine: issue id 619, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/184Translate _any field query2019-11-28T08:56:55ZGradl, TobiasTranslate _any field queryequal in other schemas
*(from redmine: issue id 620, created on 2018-01-22tgradl, closed on 2018-01-24)*equal in other schemas
*(from redmine: issue id 620, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/186Move to a dedicated GS configuration schema for resources2019-11-28T08:56:58ZGradl, TobiasMove to a dedicated GS configuration schema for resourcesThere is no need or real benefit from converting indexed resources at
query time; instead, extract title, link etc. at indexing time and index
with resource
*(from redmine: issue id 622, created on 2018-01-22tgradl, closed on 2018-01-2...There is no need or real benefit from converting indexed resources at
query time; instead, extract title, link etc. at indexing time and index
with resource
*(from redmine: issue id 622, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #6133.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/187Refactor index mapping generation / conversion2019-11-28T08:56:59ZGradl, TobiasRefactor index mapping generation / conversion- analyzers
- data types
*(from redmine: issue id 623, created on 2018-01-22tgradl, closed on 2018-01-24)*- analyzers
- data types
*(from redmine: issue id 623, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/188Outdate schema definitions2019-11-28T08:57:00ZGradl, TobiasOutdate schema definitions- when changed in SR
- when mapping to GS Model changes as well!
- Not automations right now
*(from redmine: issue id 624, created on 2018-01-22tgradl, closed on 2018-01-24)*- when changed in SR
- when mapping to GS Model changes as well!
- Not automations right now
*(from redmine: issue id 624, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/189Implement data type handling (complete example: geo_points)2019-11-28T08:57:01ZGradl, TobiasImplement data type handling (complete example: geo_points)follow up in task 627
*(from redmine: issue id 627, created on 2018-01-22tgradl, closed on 2018-01-24)*follow up in task 627
*(from redmine: issue id 627, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/193Consolidate datasource info and actions under comprehensive new view2019-11-28T08:57:06ZGradl, TobiasConsolidate datasource info and actions under comprehensive new view- Crawls
- Schema info
- …
*(from redmine: issue id 631, created on 2018-01-22tgradl, closed on 2018-01-24)*- Crawls
- Schema info
- …
*(from redmine: issue id 631, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/200Implement prefix autodetection in OaiPmhCrawler / OaiPmhHarvestingService2019-11-28T08:57:13ZGradl, TobiasImplement prefix autodetection in OaiPmhCrawler / OaiPmhHarvestingServiceTry much in OaiPmhHarvestingService, only crawl related aspects
(persistence) in OaiPmhCrawler
*(from redmine: issue id 638, created on 2018-01-22tgradl, closed on 2018-01-24)*Try much in OaiPmhHarvestingService, only crawl related aspects
(persistence) in OaiPmhCrawler
*(from redmine: issue id 638, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/201Have OaiPmhHarvestingService fail if response is not ListRecords2019-11-28T08:57:14ZGradl, TobiasHave OaiPmhHarvestingService fail if response is not ListRecordstypically if no good metadata prefix is used or url wrong etc.
*(from redmine: issue id 639, created on 2018-01-22tgradl, closed on 2018-01-24)*typically if no good metadata prefix is used or url wrong etc.
*(from redmine: issue id 639, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/202Complete transparency for crawls2019-11-28T08:57:15ZGradl, TobiasComplete transparency for crawls- Crawl manager handles inquiries to threads
*(from redmine: issue id 640, created on 2018-01-22tgradl, closed on 2018-01-24)*- Crawl manager handles inquiries to threads
*(from redmine: issue id 640, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/203OaiPmhHarvester should know about record state2019-11-28T08:57:17ZGradl, TobiasOaiPmhHarvester should know about record stateTo provide size and progress
*(from redmine: issue id 641, created on 2018-01-22tgradl, closed on 2018-01-24)*To provide size and progress
*(from redmine: issue id 641, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/205Implement online file adapter2019-11-28T08:57:19ZGradl, TobiasImplement online file adapter- dowload
- unpack / untar
- chunk
*(from redmine: issue id 643, created on 2018-01-22tgradl, closed on 2018-02-09)*- dowload
- unpack / untar
- chunk
*(from redmine: issue id 643, created on 2018-01-22tgradl, closed on 2018-02-09)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/210Extract common functionality into search-commons library2019-11-28T08:57:25ZGradl, TobiasExtract common functionality into search-commons libraryreusable for Cosmotool and potentially other searches
*(from redmine: issue id 655, created on 2018-01-22tgradl, closed on 2018-01-24)*reusable for Cosmotool and potentially other searches
*(from redmine: issue id 655, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/212Implement search result preview2019-11-28T08:57:27ZGradl, TobiasImplement search result preview- mainly implemented for pictures in WissKI
- reusable for links (screenshots)
*(from redmine: issue id 678, created on 2018-01-22tgradl, closed on 2018-01-24)*- mainly implemented for pictures in WissKI
- reusable for links (screenshots)
*(from redmine: issue id 678, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/213Implement configured-by-mapping identifiers2019-11-28T08:57:28ZGradl, TobiasImplement configured-by-mapping identifiersIndexing ->Identifier
*(from redmine: issue id 679, created on 2018-01-22tgradl, closed on 2018-01-24)*Indexing ->Identifier
*(from redmine: issue id 679, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/214Implement configured-by-mapping imagelinks2019-11-28T08:57:30ZGradl, TobiasImplement configured-by-mapping imagelinksIndexing ->Image
*(from redmine: issue id 680, created on 2018-01-22tgradl, closed on 2018-01-24)*Indexing ->Image
*(from redmine: issue id 680, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/216Upgrade underlying frameworks2019-11-28T08:57:32ZGradl, TobiasUpgrade underlying frameworksbase like Cosmotool
*(from redmine: issue id 800, created on 2018-01-22tgradl, closed on 2018-01-24)*base like Cosmotool
*(from redmine: issue id 800, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/217Fix parallel threads for crawling2019-11-28T08:57:33ZGradl, TobiasFix parallel threads for crawling- seems to fail in Cosmotool currently
- every processing step set to prototype now =>possibly bad for
API based Dispatching (queuing with synchronization?)
*(from redmine: issue id 801, created on 2018-01-22tgradl, closed on 2...- seems to fail in Cosmotool currently
- every processing step set to prototype now =>possibly bad for
API based Dispatching (queuing with synchronization?)
*(from redmine: issue id 801, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #8043.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/218Investigate crawling memory leak2019-11-28T08:57:35ZGradl, TobiasInvestigate crawling memory leakprofile Java?
*(from redmine: issue id 802, created on 2018-01-22tgradl, closed on 2018-01-24)*profile Java?
*(from redmine: issue id 802, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/223Upgrade dependencies for major release2019-11-28T08:57:40ZGradl, TobiasUpgrade dependencies for major release- MInfBA-DARIAH
- Elasticsearch 6
*(from redmine: issue id 870, created on 2018-01-22tgradl, closed on 2018-01-24)*- MInfBA-DARIAH
- Elasticsearch 6
*(from redmine: issue id 870, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/224Reimplement YAML-config-based logging2019-11-28T08:57:41ZGradl, TobiasReimplement YAML-config-based logginglog4j 2?
*(from redmine: issue id 871, created on 2018-01-22tgradl, closed on 2018-01-24)*log4j 2?
*(from redmine: issue id 871, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/225Share reference handling between webapps2019-11-28T08:57:43ZGradl, TobiasShare reference handling between webappsmigrate respective functionality to core
*(from redmine: issue id 874, created on 2018-01-22tgradl, closed on 2018-01-24)*migrate respective functionality to core
*(from redmine: issue id 874, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/226Adapt to current DME model API2019-11-28T08:57:44ZGradl, TobiasAdapt to current DME model API- renamed
- DatamodelReferenceContainer
*(from redmine: issue id 875, created on 2018-01-22tgradl, closed on 2018-01-24)*- renamed
- DatamodelReferenceContainer
*(from redmine: issue id 875, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/227Support recursive datamodel structures2019-11-28T08:57:45ZGradl, TobiasSupport recursive datamodel structures- Serialization
- Mapping generation
*(from redmine: issue id 877, created on 2018-01-22tgradl, closed on 2018-01-24)*- Serialization
- Mapping generation
*(from redmine: issue id 877, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/228Refactor mapping generation service2019-11-28T08:57:46ZGradl, TobiasRefactor mapping generation service- retain element ids??
*(from redmine: issue id 878, created on 2018-01-22tgradl, closed on 2018-01-24)*- retain element ids??
*(from redmine: issue id 878, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/230Render and index integrations metamodel with associated datemodels2019-11-28T08:57:48ZGradl, TobiasRender and index integrations metamodel with associated datemodelsPut in -integrations next to -content
*(from redmine: issue id 880, created on 2018-01-22tgradl, closed on 2018-01-24)*Put in -integrations next to -content
*(from redmine: issue id 880, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/231Restrict or predefine search when integrations model exists2019-11-28T08:57:49ZGradl, TobiasRestrict or predefine search when integrations model existsShow as main faceting datamodel
*(from redmine: issue id 881, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #882Show as main faceting datamodel
*(from redmine: issue id 881, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #8823.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/232Query by-indexed integrations model directly2019-11-28T08:57:51ZGradl, TobiasQuery by-indexed integrations model directlyinstead of translating the query
*(from redmine: issue id 882, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #881instead of translating the query
*(from redmine: issue id 882, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* relates #8813.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/233Sammlungsbilder mit Suchergebnissen anzeigen2019-11-28T08:57:52ZGradl, TobiasSammlungsbilder mit Suchergebnissen anzeigenwenn keine Images bei einzelner Resource
*(from redmine: issue id 883, created on 2018-01-22tgradl, closed on 2018-01-24)*wenn keine Images bei einzelner Resource
*(from redmine: issue id 883, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/234Tagcloud is not filled2019-11-28T08:57:53ZGradl, TobiasTagcloud is not filledjust not enough data?
*(from redmine: issue id 884, created on 2018-01-22tgradl, closed on 2018-01-24)*just not enough data?
*(from redmine: issue id 884, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/235Use integrations -> gsmodel mapping for response items2019-11-28T08:57:54ZGradl, TobiasUse integrations -> gsmodel mapping for response items\_integrations
*(from redmine: issue id 885, created on 2018-01-22tgradl, closed on 2018-01-24)*\_integrations
*(from redmine: issue id 885, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/236Brand MWW2019-11-28T08:57:55ZGradl, TobiasBrand MWWcolors and shapes
*(from redmine: issue id 886, created on 2018-01-22tgradl, closed on 2018-01-24)*colors and shapes
*(from redmine: issue id 886, created on 2018-01-22tgradl, closed on 2018-01-24)*3.2Gradl, TobiasGradl, Tobiashttps://gitlab.rz.uni-bamberg.de/dariah/search/-/issues/239Trigger autocrawl on new collections2019-11-28T08:57:58ZGradl, TobiasTrigger autocrawl on new collections- immediately for rep
- at night for other collections
*(from redmine: issue id 895, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* duplicates #896- immediately for rep
- at night for other collections
*(from redmine: issue id 895, created on 2018-01-22tgradl, closed on 2018-01-24)*
* Relations:
* duplicates #8963.2Gradl, TobiasGradl, Tobias