Kotahi issueshttps://gitlab.coko.foundation/kotahi/kotahi/-/issues2021-04-27T09:37:00Zhttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/134DevOps - Prometheus & Grafana for Kotahi monitoring and alerting2021-04-27T09:37:00ZIlia EriomencoDevOps - Prometheus & Grafana for Kotahi monitoring and alertingEC2:
- [x] - create docker-compose for selfhosted stack "cAdvisor + NodeExporter + Prometheus + Grafana"
- [x] - deploy stack on eLife Kotahi prod server
- [x] - add Grafana dashboards for OS and Docker metrics
- [x] - add Grafana notifi...EC2:
- [x] - create docker-compose for selfhosted stack "cAdvisor + NodeExporter + Prometheus + Grafana"
- [x] - deploy stack on eLife Kotahi prod server
- [x] - add Grafana dashboards for OS and Docker metrics
- [x] - add Grafana notifications channel Mattermost
- [x] - send Grafana's admin credentials to Paul and user credential to Ryan,Ben and Vadim
EKS:
- [x] - install Prometheus in eks-elife-kotahi cluster and check targets
- [x] - install Grafana in eks-elife-kotahi cluster and check access
- [x] - import needed dashboards in Grafana (imported https://grafana.com/grafana/dashboards/10670)
- [x] - configure and check notification channel "mattermost-elife-kotahi-alerts"
- [x] - configure domain elife-grafana.kotahi.cloud and TLS certificate in ACM
- [x] - add Paul,Ryan,Ben,Vadim,Mihail accounts in Grafana and send credentials
- [x] - install Loki, Promtail in eks-elife-kotahi cluster for containers logs parsing
- [x] - create Loki datasource check Kotahi logs in Loki datasource over Grafana explore
- [x] - import and check "Loki Dashboard quick search" & "Logging Dashboard via Loki" in Grafana (https://grafana.com/grafana/dashboards/12019,https://grafana.com/grafana/dashboards/12611)
- [x] - note all installation/configuration steps in https://gitlab.coko.foundation/kotahi/endava-infra/-/blob/master/k8s/monitoring/README.md
- [x] - check connection Prometheus/Loki datasources from eLife k8s cluster to Grafana Cloud account
sources:
- https://liwei766.net/2020/10/06/set-up-loki-on-eks-to-get-full-visibility-into-your-cluster-logs-while-using-grafana/
- https://grafana.com/docs/loki/latest/installation/helm/
- https://grafana.com/docs/grafana/latest/datasources/loki/DevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/133DevOps - Kotahi monitoring and alerting (Cloudwatch and SES)2021-04-28T10:04:51ZIlia EriomencoDevOps - Kotahi monitoring and alerting (Cloudwatch and SES)- [ ] - develop and deploy Terraform Cloudwatch module
- [ ] - develop and deploy Terraform SES module
- [ ] - check Kotahi EKS cluster's Cloudwatch metrics and SES mail sending- [ ] - develop and deploy Terraform Cloudwatch module
- [ ] - develop and deploy Terraform SES module
- [ ] - check Kotahi EKS cluster's Cloudwatch metrics and SES mail sendingDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/132DevOps - create CI pipeline for eLife k8s2021-04-23T14:03:08ZIlia EriomencoDevOps - create CI pipeline for eLife k8s- [x] - define and add eLife k8s variables in CI secrets
- [x] - create .gitlab-ci.elife.k8s.yml and add to https://gitlab.coko.foundation/kotahi/kotahi/-/blob/main/.gitlab-ci.yml
- [x] - initial checking pipeline (get cluster-info)
- [...- [x] - define and add eLife k8s variables in CI secrets
- [x] - create .gitlab-ci.elife.k8s.yml and add to https://gitlab.coko.foundation/kotahi/kotahi/-/blob/main/.gitlab-ci.yml
- [x] - initial checking pipeline (get cluster-info)
- [x] - configure "elife-kotahi" Helm template
- [x] - test k8s deploy from Helm template
sources:
- https://github.com/sciety/sciety/blob/main/.github/workflows/ci.yml
- https://docs.gitlab.com/charts/
- https://aws.amazon.com/ru/blogs/containers/amazon-eks-cluster-automation-with-gitlab-ci-cd/DevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/131DevOps - develop gitlab-ci script and test new environment deploy2021-04-09T13:46:44ZIlia EriomencoDevOps - develop gitlab-ci script and test new environment deploy- [x] - create .gitlab-ci.yml
- [x] - prepare new NCRC k8s cluster in Terraform
- [x] - test manual NCRC k8s cluster resources on AWS
- [x] - test autodeploy NCRC k8s cluster resources on AWS via .gitlab-ci.yml- [x] - create .gitlab-ci.yml
- [x] - prepare new NCRC k8s cluster in Terraform
- [x] - test manual NCRC k8s cluster resources on AWS
- [x] - test autodeploy NCRC k8s cluster resources on AWS via .gitlab-ci.ymlDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/130DevOps - translate docker-compose to k8s manifests2021-04-07T08:15:46ZIlia EriomencoDevOps - translate docker-compose to k8s manifests- [x] - prepare docker-compose.yml from docker-compose.production.elife.yml in https://gitlab.coko.foundation/kotahi/endava-infra repo
- [x] - prepare .env for Helm template
- [x] - create "elife-kotahi" Helm template
- [x] - configure ...- [x] - prepare docker-compose.yml from docker-compose.production.elife.yml in https://gitlab.coko.foundation/kotahi/endava-infra repo
- [x] - prepare .env for Helm template
- [x] - create "elife-kotahi" Helm template
- [x] - configure putting from Secret/Configmap to environment variables in Deployments
- [x] - create Configmap for .env and link it in Deployments
- [x] - deploy Kotahi stack in EKS clusterDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/129DevOps - create environment manifest and unite all modules2021-04-06T14:16:30ZIlia EriomencoDevOps - create environment manifest and unite all modules- [x] - create terraform/tf/envs/kotahi/main.tf and add vpc,eks,dns,ingress,kotahi_prod_rds modules and kubernetes_secret.kotahi_prod_rds_postgres resource
- [x] - create terraform/tf/envs/kotahi/variables.tf
- [x] - create terraform/tf/...- [x] - create terraform/tf/envs/kotahi/main.tf and add vpc,eks,dns,ingress,kotahi_prod_rds modules and kubernetes_secret.kotahi_prod_rds_postgres resource
- [x] - create terraform/tf/envs/kotahi/variables.tf
- [x] - create terraform/tf/envs/kotahi/outputs.tf
- [x] - deploy all resources
- [x] - configure kubectl access to eks cluster and get cluster-infoDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/128DevOps - develop ses Terraform module2021-04-26T06:07:22ZIlia EriomencoDevOps - develop ses Terraform moduleDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/127DevOps - develop cloudwatch Terraform module2021-04-26T06:08:04ZIlia EriomencoDevOps - develop cloudwatch Terraform moduleDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/126DevOps - develop rds_db Terraform module2021-04-05T14:18:11ZIlia EriomencoDevOps - develop rds_db Terraform module- [x] - create module structure
- [x] - add in main.tf
- [x] - deploy resources on AWS
- [x] - check resources on AWS- [x] - create module structure
- [x] - add in main.tf
- [x] - deploy resources on AWS
- [x] - check resources on AWSDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/125DevOps - develop s3_attachment Terraform module2021-04-12T14:19:23ZIlia EriomencoDevOps - develop s3_attachment Terraform module- [x] - develop module code
- [x] - add in main.tf
- [x] - create variables.tf and output.tf
- [x] - check resource creation on AWS- [x] - develop module code
- [x] - add in main.tf
- [x] - create variables.tf and output.tf
- [x] - check resource creation on AWSDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/124DevOps - develop k8s_cluster Terraform module2021-03-23T09:16:37ZIlia EriomencoDevOps - develop k8s_cluster Terraform module- [x] - create module and variables
- [x] - add module in main
- [x] - check deploy- [x] - create module and variables
- [x] - add module in main
- [x] - check deployDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/123DevOps - cert-manager for k8s cluster2021-04-13T13:02:19ZIlia EriomencoDevOps - cert-manager for k8s clusterCert-manager
- [x] - deploy cert-manager in cluster
- [x] - create TLS issuer
Ingress
- [x] - create ingress rule 80 -> 3000 and check in on http://k8s.kotahi.cloud
- [x] - check ingress working without TLS
- [x] - add TLS issuer annot...Cert-manager
- [x] - deploy cert-manager in cluster
- [x] - create TLS issuer
Ingress
- [x] - create ingress rule 80 -> 3000 and check in on http://k8s.kotahi.cloud
- [x] - check ingress working without TLS
- [x] - add TLS issuer annotations in server-ingress
- [x] - check prod TLS certificate generation and handling on https://k8s.kotahi.cloud
- [x] - fix fake certificate generationDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/122DevOps - develop k8s_dns Terraform module2021-03-19T09:23:28ZIlia EriomencoDevOps - develop k8s_dns Terraform module- [x] - define cluster domain name (k8s.kotahi.cloud)
- [x] - configure resource "helm_release" "external_dns"
- [x] - add module "kubernetes_dns" in main.tf
- [x] - deploy DNS on AWS- [x] - define cluster domain name (k8s.kotahi.cloud)
- [x] - configure resource "helm_release" "external_dns"
- [x] - add module "kubernetes_dns" in main.tf
- [x] - deploy DNS on AWSDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/121DevOps - develop k8s_ingress Terraform module2021-03-25T07:58:37ZIlia EriomencoDevOps - develop k8s_ingress Terraform module- [x] - create module's code
- [x] - add in main.tf
- [x] - deploy resources on AWS
- [x] - test resources on AWS- [x] - create module's code
- [x] - add in main.tf
- [x] - deploy resources on AWS
- [x] - test resources on AWSDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/119DevOps - develop k8s_vpc Terraform module2021-03-17T15:04:27ZIlia EriomencoDevOps - develop k8s_vpc Terraform module- [x] - S3 bucket for tfstate
- [x] - create envs/kotahi with main.tf and define variables.tf
- [x] - check VPC deployment on AWS- [x] - S3 bucket for tfstate
- [x] - create envs/kotahi with main.tf and define variables.tf
- [x] - check VPC deployment on AWSDevOps - Permanent Prod Environment CreationIlia EriomencoIlia Eriomencohttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/608[NCRC] Manuscripts view dropdown labels not reflecting label name changes2021-08-30T13:08:12ZRyan Dix-Peek[NCRC] Manuscripts view dropdown labels not reflecting label name changes**Issue description;** Label name changes are reflected in the label within the context of the item in the list and form builder, but the dropdown menu does not reflect the label changes.
**Desired behaviour;**
- Dropdown menu should ...**Issue description;** Label name changes are reflected in the label within the context of the item in the list and form builder, but the dropdown menu does not reflect the label changes.
**Desired behaviour;**
- Dropdown menu should read; 'Ready to draft', 'Ready to review' and 'Ready to pubilsh'
- Label name changes should reflect in all instances where the label name occurs
![Screenshot_2021-08-20_at_08.09.30](/uploads/9cd3ce116cc5914e757fb46db50ab2b2/Screenshot_2021-08-20_at_08.09.30.png)NCRC MVPBen WhitmoreBen Whitmorehttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/589[NCRC] Topic and cross_post field mapping enhancement2021-08-06T12:51:00ZRyan Dix-Peek[NCRC] Topic and cross_post field mapping enhancement**Description;** Column D should capture the first topic as imported. If the user makes a change, Column P should capture ALL final topics being ticked and submitted (not just new ones).
References;
- [Spreadshseet endpoint](https://d...**Description;** Column D should capture the first topic as imported. If the user makes a change, Column P should capture ALL final topics being ticked and submitted (not just new ones).
References;
- [Spreadshseet endpoint](https://docs.google.com/spreadsheets/d/1sMU60q9qvMyvWEH352VvmxSRMZKklWAm_w78mpckzMQ/)
- ['NCRC_updated sheet](https://docs.google.com/spreadsheets/d/1CBkfIVhJA94aIAjlKkdZEBGEmUg0QqbdKQlE5Bt5MY4/)'NCRC MVPBen WhitmoreBen Whitmorehttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/586[NCRC] Field mapping and spreadsheet endpoint updates2021-07-30T11:34:29ZRyan Dix-Peek[NCRC] Field mapping and spreadsheet endpoint updates**Description;** further requests from the client after testing.
Please see updated fields in [NCRC_updated sheet, highlighted in yellow](https://docs.google.com/spreadsheets/d/1CBkfIVhJA94aIAjlKkdZEBGEmUg0QqbdKQlE5Bt5MY4/).
See test...**Description;** further requests from the client after testing.
Please see updated fields in [NCRC_updated sheet, highlighted in yellow](https://docs.google.com/spreadsheets/d/1CBkfIVhJA94aIAjlKkdZEBGEmUg0QqbdKQlE5Bt5MY4/).
See test feedback document; [Testing.xlsx](/uploads/68b61b3b130d1f6376b3ac99ce39360a/Testing.xlsx)
**Acceptance criteria (feedback verbatim from the client);**
- [x] Column N: Can we auto-populate this field with the letters “NA”.
- [x] Column T: I’m noticing that every row in Column T has a number in it. To be safe, can we auto-populate this field with the number “5” every time instead of a blank.
- [x] Study Design (Col V): In the Kotahi interface, we are missing two study designs:
Non-Randomized Trial (slug: non-randomized-trial)
Case Control (slug: case-control)
- [x] Subtopics (Column W): Correction...
vaccines_trial-protocols should be a singular protocol without an “s” (i.e., vaccines_trial-protocol)
- [x] Subtopics (Col W): Please add the following?
modeling_covid-19-mortality-morbidity-and-demand-for-healthcare-services
modeling_impact-on-healthcare-services-for-non-respiratory-diseases
modeling_masks
modeling_school-and-business-closures
modeling_stay-at-home-orders-and-mobility-restrictions
modeling_testing-and-contact-tracing
modeling_transmission-dynamics
modeling_symptom-screening
modeling_vaccines
non-pharmaceutical-interventions_stay-at-home-orders-and-mobility-restrictions
- [x] Column D: This field yields the original topic(s) that an object is marked when pulled in from the query. I’m not sure if the website automator uses this field, but for consistency, this field should select exactly ONE topic (if there are multiple) and push that slug to the endpoint. It can be literally any of the topics that the object comes in with.NCRC MVPBeatrice-Emanuela SuarasanBeatrice-Emanuela Suarasanhttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/572[NCRC] create PubMed query results csv2022-05-04T12:15:53ZRyan Dix-Peek[NCRC] create PubMed query results csv**Issue description;** client would like to review a comparative Pubmed query pull from both shiny app and kotahi. See PubMed query results attached.
Pull across 8 groups: 7734 pubmed papers; [kotahi_pull_compare.csv.gz](/uploads/5f8a2...**Issue description;** client would like to review a comparative Pubmed query pull from both shiny app and kotahi. See PubMed query results attached.
Pull across 8 groups: 7734 pubmed papers; [kotahi_pull_compare.csv.gz](/uploads/5f8a2679eb6323ce5867199c28a70596/kotahi_pull_compare.csv.gz)
**Acceptance criteria;**
- [ ] provide CSV of the Pubmed API query results pull for articles published between 6/6/2021 and 12/31/2021NCRC MVPhttps://gitlab.coko.foundation/kotahi/kotahi/-/issues/565[NCRC] Remove HTML from spreadsheet end point2021-07-26T12:14:08ZRyan Dix-Peek[NCRC] Remove HTML from spreadsheet end point**Description;** please remove HTML from AbstractEditor field content published to spreadsheet endpoint. The client has requested plain text only in order for content to publish successfully to Wordpress.**Description;** please remove HTML from AbstractEditor field content published to spreadsheet endpoint. The client has requested plain text only in order for content to publish successfully to Wordpress.NCRC MVPBeatrice-Emanuela SuarasanBeatrice-Emanuela Suarasan