Commit 9aa7749f authored by Jure's avatar Jure

Merge branch 'core_docs' into 'master'

docs: add more docs about PubSweet core parts

Closes #448

See merge request !576
parents d6b90baa 45207f7f
Pipeline #13053 passed with stages
in 13 minutes and 33 seconds
PubSweet comes with a built-in job-runner based on `pg-boss`: https://github.com/timgit/pg-boss.
The API exposed for jobs in `pubsweet-server` matches `pg-boss`'s API almost exactly. The only difference comes from starting and connection to a queue.
When the `pubsweet-server` starts, we will automatically call `startQueue()`, which calls `pg-boss`'s `start()`. This:
1. sets up the PostgreSQL database (e.g. adds a job table, migrates to latest `pg-boss` schema, etc.) to support job queues.
2. is a way for `pg-boss` to keep an eye on jobs, and manages storage (archival, deletion, etc.).
All other job queue users should use the exported `connectToJobQueue`:
```js static
const {
jobs: { connectToJobQueue },
} = require('pubsweet-server')
const jobQueue = await connectToJobQueue()
```
After this, the API is exactly the same as `pg-boss`'s: https://github.com/timgit/pg-boss
Check out the associated tests for examples of processing a job 'in-line' (https://gitlab.coko.foundation/pubsweet/pubsweet/blob/jobs/packages/server/test/jobs/jobs_test.js) or in a different process (https://gitlab.coko.foundation/pubsweet/pubsweet/blob/jobs/packages/server/test/jobs/jobs_different_process_test.js).
For completeness, one short, complete example (in a server component context) would be:
```js static
const {
jobs: { connectToJobQueue },
} = require('pubsweet-server')
const emailHandler = async job => {
return sendEmail(job.data)
}
const jobQueue = await connectToJobQueue()
const queueName = 'email-queue'
// Add job to the queue
await jobQueue.publish(queueName, {
subject: 'Hello!',
to: 'some@example.com',
content: 'Hi again!',
})
// Subscribe to the job queue with an async handler
await jobQueue.subscribe(queueName, emailHandler)
// Be notified on job completion with job result
jobQueue.onComplete(queueName, job => {
console.log('Email sent to: ', job.request.data.email)
console.log('Email job returned: ', job.response.data)
})
```
### Containers
While it is easy to create a docker container that processes jobs (equally easy as processing from a separate process, as shown in this test: https://gitlab.coko.foundation/pubsweet/pubsweet/blob/master/packages/server/test/jobs/jobs_different_process_test.js), it's not easy to sensibly manage these docker containers as a part of PubSweet's functionality. As such, I've decided to keep managing docker containers out of the scope of this MR:
1. it makes the MR quite complex and testing this the integration with Docker is not trivial
2. it's not clear this integration is needed, as using `docker-compose`, as we currently do, is quite powerful
3. this MR provides significant and much needed utility as it is, and should land as soon as possible
### Examples
An example of a long running process using this job queue is the XSweet conversion job. See https://gitlab.coko.foundation/pubsweet/pubsweet/tree/master/components/server/job-xsweet for details.
......@@ -36,8 +36,29 @@ module.exports = {
content: './content/technical_architecture.md',
},
{
name: 'Base model (data models)',
content: '../packages/base-model/README.md',
name: 'More PubSweet Core',
sections: [
{
name: 'Base model (data models)',
content: '../packages/base-model/README.md',
},
{
name: 'Database manager',
content: '../packages/db-manager/README.md',
},
{
name: 'Logger',
content: '../packages/logger/README.md',
},
{
name: 'Job runner',
content: './content/job_runner.md',
},
{
name: 'Command-line interface',
content: '../packages/cli/README.md',
},
],
},
{
name: 'Components',
......
......@@ -15,7 +15,7 @@ Migrations (folder `./migrations`) are automatically added if they exist.
If you use `@pubsweet/model-some-model` in your app (by specifying it as a component in the configuration), `typeDefs` and `resolvers` are gathered in server's `schema.js` to compose the app's entire GraphQL schema from three parts: 1. `pubsweet-server`, 2. app's components and 3. app's config.
# Support for extended data models (models based on another model)
### Support for extended data models (models based on another model)
Shown in (https://gitlab.coko.foundation/pubsweet/pubsweet/blob/master/packages/base-model/test/extended-data-model-component/src/index.js) is an extended data model (`extended-data-model-component`) for testing purposes. It exports the following things:
......@@ -31,7 +31,7 @@ module.exports = {
Things are exactly the same as in the non-extended data model, but there is one big exception, the `extending` property. This is a string, the name of the model that this extended data model extends. In this case `@pubsweet/model-extended-some-model` extends `@pubsweet/model-some-model` and what this means, in practice, is that `@pubsweet/model-some-model`'s GraphQL schema, resolvers and migration paths will be added to `@pubsweet/model-extended-some-model`'s. This happens recursively, so for example if you had a `@pubsweet/model-super-extended-some-model` that extended `@pubsweet/model-extended-some-model`, it would also include `@pubsweet/model-some-models`'s GraphQL schema, resolvers and migration paths. :curly_loop:
# Using standalone data models
### Using standalone data models
To use the above models, all you need to do is to add them to the `pubsweet.components` configuration, e.g.:
......@@ -43,17 +43,17 @@ To use the above models, all you need to do is to add them to the `pubsweet.comp
The data model's migrations will be added to the list of your app's migrations, and GraphQL queries and mutations will automatically be added to your API.
# API documentation
### API documentation
There are a few methods in the BaseModel that add utility features.
## save() - Saving a single model
#### save() - Saving a single model
```js static
const manuscript = await new Manuscript({ title: 'Test' }).save()
```
## saveGraph() - Saving a graph
#### saveGraph() - Saving a graph
```js static
const manuscript = await new Manuscript({
......@@ -62,7 +62,7 @@ const manuscript = await new Manuscript({
}).saveGraph()
```
## find() - Finding a single instance by id
#### find() - Finding a single instance by id
Passes parameters onwards to Objection's `findById` and supports its options:
......@@ -73,7 +73,7 @@ const manuscript = await Manuscript.find(
)
```
## findByField() - Finding instances by field value
#### findByField() - Finding instances by field value
Passes parameters to Objection's `where`.
......@@ -84,7 +84,7 @@ const manuscript = await Manuscript.findByField(
)
```
## findOneByField() - Find a single instance by field value
#### findOneByField() - Find a single instance by field value
Uses Objections's `where().limit(1)`.
......@@ -92,12 +92,12 @@ Uses Objections's `where().limit(1)`.
const manuscript = await Manuscript.findOneByField('content', 'Great success')
```
## all() - Returns all records
#### all() - Returns all records
```js static
const manuscript = await Manuscript.all()
```
# Examples in the wild
### Examples in the wild
Take a look at the [micropubs/wormbase](https://gitlab.coko.foundation/micropubs/wormbase/tree/master/server) application, for an example implementation of two models, `Manuscript` and `Review` using the `BaseModel` class, the supplied GraphQL connectors and Authsome.
# PubSweet CLI (command line interface)
PubSweet CLI (using `pubsweet` or `pubsweet help`) outputs:
# Contents
- [Introduction](#introduction)
- [PubSweet overview](#pubsweet-overview)
- [PubSweet modules](#pubsweet-modules)
- [Getting started with PubSweet CLI](#getting-started-with-pubsweet-cli)
- [Getting PubSweet CLI](#getting-pubsweet-cli)
- [Prerequisites](#prerequisites)
- [Installation](#installation)
- [Installation on Windows](#installation-on-windows)
- [Quickstart](#quickstart)
- [Using PubSweet CLI](#using-pubsweet-cli)
- [Displaying the commands (`pubsweet` or `pubsweet help`)](#displaying-the-commands-pubsweet-or-pubsweet-help)
- [Generating an app (`pubsweet new`)](#generating-an-app-pubsweet-new)
- [Running locally for development or evaluation](#running-locally-for-development-or-evaluation)
- [Running in production](#running-in-production)
- [Setting up the database (`pubsweet setupdb`)](#setting-up-the-database-pubsweet-setupdb)
- [Building your app (`pubsweet build`)](#building-your-app-pubsweet-build)
- [Running your app](#running-your-app)
- [Managing your app](#managing-your-app)
- [Adding and removing components](#adding-and-removing-components)
- [Adding a user to the database (`pubsweet adduser`)](#adding-a-user-to-the-database-pubsweet-adduser)
- [Contributing](#contributing)
- [Credits](#credits)
# Introduction
## PubSweet overview
**PubSweet** allows you to build state-of-the-art publishing platforms.
It's a modular and flexible framework consisting of a **server** and **client**
that work together, **components** that can modify or extend the functionality
of the server and/or client, and a **command-line tool** that helps manage
PubSweet apps.
### PubSweet modules
| repository | description |
| :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------------------------------------------------------------------- |
| [![pubsweet-server](https://img.shields.io/badge/PubSweet-server-51c1bc.svg?style=flat&colorA=84509d) pubsweet-server](https://gitlab.coko.foundation/pubsweet/pubsweet/packages/server) | an extensible server API |
| [![pubsweet-client](https://img.shields.io/badge/PubSweet-client-51c1bc.svg?style=flat&colorA=84509d) pubsweet-client](https://gitlab.coko.foundation/pubsweet/pubsweet/packages/client) | an extensible client app that runs in the browser |
| [![pubsweet-components](https://img.shields.io/badge/PubSweet-components-51c1bc.svg?style=flat&colorA=84509d) components](https://gitlab.coko.foundation/pubsweet/pubsweet/components) | pluggable extensions for server and/or client |
| [![pubsweet-cli](https://img.shields.io/badge/PubSweet-CLI-51c1bc.svg?style=flat&colorA=84509d) pubsweet-cli](https://gitlab.coko.foundation/pubsweet/pubsweet/packages/cli) | a suite of command-line tools for building and managing your platform |
# Getting started with PubSweet CLI
## Getting PubSweet CLI
### Prerequisites
- Node.js v10.0+
- Docker (recommended but not essential)
### Installation
The PubSweet command-line tools can be installed from npm or yarn:
```bash
npm install --global pubsweet
```
or
```bash
yarn global add pubsweet
```
### Installation on Windows
- Install Ubuntu on Windows:
[Windows Store](https://www.microsoft.com/en-gb/store/p/ubuntu/9nblggh4msv6#system-requirements)
- Launch the Bash on Ubuntu on Windows from the Start menu
- Install nvm:
[nvm install script](https://github.com/creationix/nvm#install-script)
- Restart your terminal/bash
- `nvm install 10`
- `npm install -g pubsweet`
### Quickstart
The sequence of commands for generating and running a sample app is as follows:
```bash
pubsweet new my-app-name
cd my-app-name
pubsweet start # (or yarn start)
```
## Using PubSweet CLI
### Displaying the commands (`pubsweet` or `pubsweet help`)
Outputs:
```
```static
Usage: pubsweet [options] [command]
......@@ -140,7 +45,7 @@ pubsweet new myappname
subdirectory with the name you supplied, and run `yarn install` to install the
app's dependencies.
## Running locally for development or evaluation
### Running locally for development or evaluation
```bash
cd myappname
......@@ -175,9 +80,9 @@ preferable to compile assets separately and maintain a startup script in your
app. A sample startup script is included in the root directory of the
`pubsweet-starter` repo (`app.js`).
## Running in production
### Running in production
### Setting up the database (`pubsweet setupdb`)
#### Setting up the database (`pubsweet setupdb`)
You need an instance of PostgreSQL 9.6+. Add your database connection settings
to the `pubsweet-server.db` key in the config in any format supported by
......@@ -191,7 +96,7 @@ pubsweet setupdb
If your database already has tables, `pubsweet setupdb` will not overwrite them
by default. You can force it to delete existing data using `--clobber`:
```
```static
$ pubsweet setupdb
error: Database tables already exist
error: If you want to overwrite the database, use --clobber
......@@ -215,29 +120,29 @@ pubsweet setupdb
--password correct-horse-battery-staple
```
### Building your app (`pubsweet build`)
#### Building your app (`pubsweet build`)
Use the `build` subcommand within your app directory to compile your app's
static assets with webpack. This command is useful for production scenarios
where assets need to be compiled in a separate step. For development, we
recommend using `pubsweet start`, which also takes care of building your assets.
### Running your app
#### Running your app
A sample startup script is included in the root directory of the
`pubsweet-starter` repo (`app.js`).
Ensure that the `NODE_ENV` environment variable is set to `production`.
## Managing your app
### Managing your app
### Adding and removing components
#### Adding and removing components
Components add models, UI, APIs, jobs, and much more to PubSweet. There are a number in core (https://gitlab.coko.foundation/pubsweet/pubsweet/tree/master/components), and you can create your own.
To add or remove a component, install it via `yarn add component-name` (or `yarn remove` to uninstall) and edit `config/components.json` in your app's directory accordingly.
### Adding a user to the database (`pubsweet adduser`)
#### Adding a user to the database (`pubsweet adduser`)
Run `adduser` within your app directory to add a user to an existing database:
......@@ -261,19 +166,3 @@ pubsweet adduser \
--email some@email.com \
--password correct-horse-battery-staple
```
# Contributing
Please read our
[CONTRIBUTING](https://gitlab.coko.foundation/pubsweet/pubsweet/blob/master/CONTRIBUTING)
guide.
# Credits
`pubsweet-cli` is part of the PubSweet ecosystem.
<a href="https://gitlab.coko.foundation/pubsweet/pubsweet"><img src="https://gitlab.coko.foundation/pubsweet/pubsweet/raw/master/assets/rgb-medium.jpg" width="300" /></a>
PubSweet is part of [Collaborative Knowledge Foundation](https://coko.foundation).
<a href="https://coko.foundation"><img src="https://gitlab.coko.foundation/pubsweet/pubsweet/raw/master/assets/COKO_logo.jpg" width="300" /></a>
# coko Theme
# Coko Theme
This theme is based on the values used by the coko
# Db-manager
Provides database management utilities to Pubsweet apps.
## Usage
### Usage
This package exports the following functions:
......@@ -11,7 +9,7 @@ This package exports the following functions:
- `dbExists`
- `migrate`
### `setupDb`
#### `setupDb`
This function
......@@ -21,7 +19,7 @@ This function
It can be called without arguments, in which case it will take its configuration from the following config keys:
```
```js static
{
'pubsweet-server': {
db: {
......@@ -41,23 +39,23 @@ Alternatively it can be passed the `dbManager` object as an argument, which will
The name of the database will be the full `dbPath`.
### `addUser`
#### `addUser`
This function adds a user to an already existing database. It requires a user object as argument:
```
```js static
addUser({
username: 'xxxxxx',
password: 'pppppp',
email: 'email@example.com',
admin: true // (optional)
admin: true, // (optional)
})
```
### `dbExists`
#### `dbExists`
This function checks if any database exists at the configured path. Returns `true` or `false`
### `migrate`
#### `migrate`
This function performs pending migrations.
# @pubsweet/logger
A module encapsulating standard logging features for pubsweet components and services.
## Usage
### Usage
Either:
```javascript
```js static
const myLogger = require('winston') // or something else
const logger = require('@pubsweet/logger')
logger.configure(myLogger)
......@@ -14,7 +12,7 @@ logger.configure(myLogger)
Or set the logger in config:
```javascript
```js static
const myLogger = require('winston') // or something else
{
......@@ -26,7 +24,7 @@ const myLogger = require('winston') // or something else
The configured logger can then be imported from anywhere:
```javascript
```js static
const logger = require('@pubsweet/logger')
logger.info('log')
```
......
To do list for UI components:
* Use `data-test-id` so that we're able to easily target components in tests (#416)
* Remove hardcoded references to react-feather. Icons should come with the theme (#411)
* Add proptypes to all components
* Add tests to all components
* Apply theme overrides wherever applicable
* Use validation color function from toolkit wherever applicable
* Update components so that they don't define their position in their parent components (eg. margin around them)
* Standardize how components are exported
* Add automated accessibility tests
* Define UI component contribution guide
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment