Compare commits

..

No commits in common. "fc90a89227f432f58e7dcc80311dbeb1151ff90f" and "7f58627864a713abca115057dd6540b46039fde2" have entirely different histories.

3 changed files with 43 additions and 59 deletions

View File

@ -57,7 +57,7 @@ CKAN_REDIS_URL=redis://redis:6379/1
TEST_CKAN_REDIS_URL=redis://redis:6379/1 TEST_CKAN_REDIS_URL=redis://redis:6379/1
# Datapusher # Datapusher
DATAPUSHER_VERSION=0.0.21 DATAPUSHER_VERSION=0.0.20
CKAN_DATAPUSHER_URL=http://datapusher:8800 CKAN_DATAPUSHER_URL=http://datapusher:8800
CKAN__DATAPUSHER__CALLBACK_URL_BASE=http://ckan:5000 CKAN__DATAPUSHER__CALLBACK_URL_BASE=http://ckan:5000

View File

@ -1,30 +1,27 @@
# Docker Compose setup for CKAN # Docker Compose setup for CKAN
- [Docker Compose setup for CKAN](#docker-compose-setup-for-ckan) * [1. Overview](#1-overview)
- [1. Overview](#1--overview) * [2. Installing Docker](#2-installing-docker)
- [2. Installing Docker](#2--installing-docker) * [3. docker compose vs docker-compose](#3-docker-compose-vs-docker-compose)
- [3. docker compose *vs* docker-compose](#3--docker-compose-vs-docker-compose) * [4. Install (build and run) CKAN plus dependencies](#4-install-build-and-run-ckan-plus-dependencies)
- [4. Install (build and run) CKAN plus dependencies](#4--install-build-and-run-ckan-plus-dependencies) * [Base mode](#base-mode)
- [Base mode](#base-mode) * [Development mode](#development-mode)
- [Development mode](#development-mode) * [Create an extension](#create-an-extension)
- [Create an extension](#create-an-extension) * [Running HTTPS on development mode](#running-https-on-development-mode)
- [Running HTTPS on development mode](#running-https-on-development-mode) * [Remote Debugging with VS Code](#remote-debugging-with-vs-code)
- [Remote Debugging with VS Code](#remote-debugging-with-vs-code) * [Updating the environment file for development mode](#updating-the-environment-file-for-development-mode)
- [Updating the environment file for development mode](#updating-the-environment-file-for-development-mode) * [5. CKAN images](#5-ckan-images)
- [5. CKAN images](#5-ckan-images) * [Extending the base images](#extending-the-base-images)
- [Extending the base images](#extending-the-base-images) * [Applying patches](#applying-patches)
- [Applying patches](#applying-patches) * [6. Debugging with pdb](#6-debugging-with-pdb)
- [_uWSGI_ command line arguments](#uwsgi-command-line-arguments) * [7. Datastore and Datapusher](#7-datastore-and-datapusher)
- [6. Debugging with pdb](#6-debugging-with-pdb) * [8. NGINX](#8-nginx)
- [7. Datastore and datapusher](#7-datastore-and-datapusher) * [9. ckanext-envvars](#9-ckanext-envvars)
- [8. NGINX](#8-nginx) * [10. CKAN_SITE_URL](#10-CKAN_SITE_URL)
- [9. ckanext-envvars](#9-ckanext-envvars) * [11. Manage new users](#11-manage-new-users)
- [10. CKAN\_SITE\_URL](#10-ckan_site_url) * [12. Changing the base image](#12-changing-the-base-image)
- [11. Manage new users](#11-manage-new-users) * [13. Replacing DataPusher with XLoader](#13-replacing-datapusher-with-xLoader)
- [12. Changing the base image](#12-changing-the-base-image)
- [13. Replacing DataPusher with XLoader](#13-replacing-datapusher-with-xloader)
- [Copying and License](#copying-and-license)
## 1. Overview ## 1. Overview
@ -47,12 +44,12 @@ The site is configured using environment variables that you can set in the `.env
Install Docker by following the following instructions: [Install Docker Engine on Ubuntu](https://docs.docker.com/engine/install/ubuntu/) Install Docker by following the following instructions: [Install Docker Engine on Ubuntu](https://docs.docker.com/engine/install/ubuntu/)
To verify a successful Docker installation, run `docker run hello-world` and `docker version`. These commands should output To verify a successful Docker installation, run `docker run hello-world` and `docker version`. These commands should output
versions for client and server. versions for client and server.
## 3. docker compose *vs* docker-compose ## 3. docker compose *vs* docker-compose
All Docker Compose commands in this README will use the V2 version of Compose ie: `docker compose`. The older version (V1) All Docker Compose commands in this README will use the V2 version of Compose ie: `docker compose`. The older version (V1)
used the `docker-compose` command. Please see [Docker Compose](https://docs.docker.com/compose/compose-v2/) for used the `docker-compose` command. Please see [Docker Compose](https://docs.docker.com/compose/compose-v2/) for
more information. more information.
@ -76,7 +73,7 @@ To start the containers:
docker compose up docker compose up
This will start up the containers in the current window. By default the containers will log direct to this window with each container This will start up the containers in the current window. By default the containers will log direct to this window with each container
using a different colour. You could also use the -d "detach mode" option ie: `docker compose up -d` if you wished to use the current using a different colour. You could also use the -d "detach mode" option ie: `docker compose up -d` if you wished to use the current
window for something else. window for something else.
At the end of the container start sequence there should be 6 containers running: At the end of the container start sequence there should be 6 containers running:
@ -108,7 +105,7 @@ dev script | description
`bin/generate_extension` | generate extension in `src` directory `bin/generate_extension` | generate extension in `src` directory
`bin/install_src` | install all extensions from `src` directory (ckan-dev does not need to be running) `bin/install_src` | install all extensions from `src` directory (ckan-dev does not need to be running)
`bin/reload` | reload ckan within the ckan-dev container without restarting `bin/reload` | reload ckan within the ckan-dev container without restarting
`bin/restart` | shut down and restart the whole ckan-dev container (use `bin/compose up -d` instead to reload new values from .env) `bin/restart` | shut down and restart the whole ckan-dev container (loads new values from .env)
`bin/shell` | exec bash prompt within the ckan-dev container `bin/shell` | exec bash prompt within the ckan-dev container
To build the images: To build the images:
@ -241,7 +238,7 @@ We want to install an extension like [ckanext-validation](https://github.com/fri
#!/bin/bash #!/bin/bash
# Create DB tables if not there # Create DB tables if not there
ckan -c /srv/app/ckan.ini validation init-db ckan -c /srv/app/ckan.ini validation init-db
``` ```
And then in our `Dockerfile.dev` file we install the extension and copy the initialization scripts: And then in our `Dockerfile.dev` file we install the extension and copy the initialization scripts:
@ -259,7 +256,7 @@ NB: There are a number of extension examples commented out in the Dockerfile.dev
### Applying patches ### Applying patches
When building your project specific CKAN images (the ones defined in the `ckan/` folder), you can apply patches When building your project specific CKAN images (the ones defined in the `ckan/` folder), you can apply patches
to CKAN core or any of the built extensions. To do so create a folder inside `ckan/patches` with the name of the to CKAN core or any of the built extensions. To do so create a folder inside `ckan/patches` with the name of the
package to patch (ie `ckan` or `ckanext-??`). Inside you can place patch files that will be applied when building package to patch (ie `ckan` or `ckanext-??`). Inside you can place patch files that will be applied when building
the images. The patches will be applied in alphabetical order, so you can prefix them sequentially if necessary. the images. The patches will be applied in alphabetical order, so you can prefix them sequentially if necessary.
@ -278,23 +275,9 @@ ckan
├── setup ├── setup
├── Dockerfile ├── Dockerfile
└── Dockerfile.dev └── Dockerfile.dev
``` ```
### _uWSGI_ command line arguments
The images use the application server [_uWSGI_](https://uwsgi-docs.readthedocs.io/en/latest/) to run _CKAN_. There are two environment variables, that allow to configure _uWSGI_ using [command line arguments](https://uwsgi-docs.readthedocs.io/en/latest/Configuration.html#command-line-arguments). The available options are described [here](https://uwsgi-docs.readthedocs.io/en/latest/Options.html).
For most use cases, the defaults specified in `ckan-X.XX/setup/start_ckan.sh` in `DEFAULT_UWSGI_OPTS` of the [ckan/ckan-docker-base](https://github.com/ckan/ckan-docker-base) image are fine. If required, you can either _overwrite_ the defaults or _append_ additional arguments.
| Variable | Description | Defaults |
|:--------------------|:------------------------------------------------------| :------------------|
| `UWSGI_OPTS` | If set, overwrites `DEFAULT_UWSGI_OPTS`. If not set, `UWSGI_OPTS` will bet set to `DEFAULT_UWSGI_OPTS`. | unset |
| `EXTRA_UWSGI_OPTS` | If set, appends its content to `UWSGI_OPTS`. | unset |
> [!IMPORTANT]
> These setting **do not** apply for the dev images.
## 6. Debugging with pdb ## 6. Debugging with pdb
Add these lines to the `ckan-dev` service in the docker-compose.dev.yml file Add these lines to the `ckan-dev` service in the docker-compose.dev.yml file
@ -310,7 +293,7 @@ command: `python -m pdb /usr/lib/ckan/venv/bin/ckan --config /srv/app/ckan.ini r
## 7. Datastore and datapusher ## 7. Datastore and datapusher
The Datastore database and user is created as part of the entrypoint scripts for the db container. There is also a Datapusher container The Datastore database and user is created as part of the entrypoint scripts for the db container. There is also a Datapusher container
running the latest version of Datapusher. running the latest version of Datapusher.
## 8. NGINX ## 8. NGINX
@ -328,9 +311,9 @@ This extension checks for environmental variables conforming to an expected form
For the extension to correctly identify which env var keys map to the format used for the config object, env var keys should be formatted in the following way: For the extension to correctly identify which env var keys map to the format used for the config object, env var keys should be formatted in the following way:
All uppercase All uppercase
Replace periods ('.') with two underscores ('__') Replace periods ('.') with two underscores ('__')
Keys must begin with 'CKAN' or 'CKANEXT', if they do not you can prepend them with '`CKAN___`' Keys must begin with 'CKAN' or 'CKANEXT', if they do not you can prepend them with '`CKAN___`'
For example: For example:
@ -338,7 +321,7 @@ For example:
* `CKAN__DATAPUSHER__CALLBACK_URL_BASE=http://ckan:5000` * `CKAN__DATAPUSHER__CALLBACK_URL_BASE=http://ckan:5000`
* `CKAN___BEAKER__SESSION__SECRET=CHANGE_ME` * `CKAN___BEAKER__SESSION__SECRET=CHANGE_ME`
These parameters can be added to the `.env` file These parameters can be added to the `.env` file
For more information please see [ckanext-envvars](https://github.com/okfn/ckanext-envvars) For more information please see [ckanext-envvars](https://github.com/okfn/ckanext-envvars)
@ -350,18 +333,19 @@ For convenience the CKAN_SITE_URL parameter should be set in the .env file. For
1. Create a new user from the Docker host, for example to create a new user called 'admin' 1. Create a new user from the Docker host, for example to create a new user called 'admin'
`docker compose exec ckan ckan user add admin email=admin@localhost` `docker exec -it <container-id> ckan -c ckan.ini user add admin email=admin@localhost`
To set this user as a sysadmin run
`docker compose exec ckan ckan sysadmin add admin`
To delete the 'admin' user To delete the 'admin' user
`docker compose exec ckan ckan user remove admin` `docker exec -it <container-id> ckan -c ckan.ini user remove admin`
In development mode use `bin/ckan` instead of `docker compose exec ckan ckan` for the above commands. 2. Create a new user from within the ckan container. You will need to get a session on the running container
`ckan -c ckan.ini user add admin email=admin@localhost`
To delete the 'admin' user
`ckan -c ckan.ini user remove admin`
## 12. Changing the base image ## 12. Changing the base image

View File

@ -1,4 +1,4 @@
FROM postgres:16-alpine FROM postgres:12-alpine
# Include extra setup scripts (eg datastore) # Include extra setup scripts (eg datastore)
COPY --chown=postgres:postgres docker-entrypoint-initdb.d /docker-entrypoint-initdb.d COPY --chown=postgres:postgres docker-entrypoint-initdb.d /docker-entrypoint-initdb.d