Ceph is a distributed open source storage solution that supports Object Storage, Block Storage and File Storage.

Other open source distributed storage systems are GlusterFS and HDFS.

In this guide, we describe how to setup a basic Ceph Cluster for Block Storage. We have 25 nodes on our setup. The masternode is a MASS Region and Rack controller. The rest of the nodes are Ubuntu 16.04 deployed through MAAS. The recommended filesystem for Ceph is XFS and this is what is used on the nodes.

This guide is based on the Quick Installation guide from the Ceph Documentation. This guide uses the ceph-deploy tool which is a relatively quick way to setup Ceph, especially for newbies. There is also the Manual Installation, deployment through Ansible and juju.

Prerequisites

Topology

  • 1 deploy node (masternode). MAAS region and rack controler is installed plus Ansible
  • 3 monitor nodes (node01,node11,node24). Ubuntu 16.04 on XFS deployed through MAAS
  • 20 OSD nodes (node02-10,12-23).

Create an Ubuntu user on masternode

It would be of convenience to create an ubuntu user on the masternode. with passwordless sudo access:

$ sudo useradd -m -s /bin/bash ubuntu

Run visudo and give passwordless sudo access to the ubuntu user:

ubuntu  ALL=NOPASSWD:ALL

Generate an SSH key pair for the ubuntu user:

$ ssh-keygen -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/ubuntu/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/ubuntu/.ssh/id_rsa.
Your public key has been saved in /home/ubuntu/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:t1zWURVk7j6wJPkA3VmbcHtAKh3EB0kyanORVbiiBkU ubuntu@masternode
The key's randomart image is:
+---[RSA 4096]----+
|       .E +**B=*=|
|        ..o==oOo+|
|       .+.o.o=.=.|
|      .. oo.o....|
|       .S..=oo.. |
|        oo += +  |
|       .  o  o o |
|                .|
|                 |
+----[SHA256]-----+

Deploy the /home/ubuntu/.ssh/id_rsa.pub pubkey on all the nodes (append in /home/ubuntu/.ssh/authorized_keys). You could add this pubkey on the MAAS user before deploying Ubuntu 16.04 on the nodes.

Set /etc/hosts

$ for ID in {01..24}; do echo "$(dig +short node${ID}.maas @127.0.0.1) node${ID}.maas node${ID}"; done > nodes.txt

It should look like this:

192.168.10.28 node01.maas node01
192.168.10.29 node02.maas node02
192.168.10.30 node03.maas node03
192.168.10.31 node04.maas node04
192.168.10.32 node05.maas node05
192.168.10.33 node06.maas node06
192.168.10.34 node07.maas node07
192.168.10.35 node08.maas node08
192.168.10.36 node09.maas node09
192.168.10.37 node10.maas node10
192.168.10.38 node11.maas node11
192.168.10.39 node12.maas node12
192.168.10.40 node13.maas node13
192.168.10.41 node14.maas node14
192.168.10.42 node16.maas node16
192.168.10.43 node17.maas node17
192.168.10.44 node18.maas node18
192.168.10.45 node19.maas node19
192.168.10.46 node20.maas node20
192.168.10.47 node21.maas node21
192.168.10.48 node22.maas node22
192.168.10.49 node23.maas node23
192.168.10.50 node24.maas node24

Now you can append the result in /etc/hosts:

$ cat nodes.txt | sudo tee -a /etc/hosts

Ansible setup

Use this setup in /etc/ansible/hosts on masternode:

[masternode]
masternode

[nodes]
node01
node02
node03
node04
node05
node06
node07
node08
node09
node10
node11
node12
node13
node14
node15
node16
node17
node18
node19
node20
node21
node22
node23
node24

[ceph-mon]
node01
node11
node24

[ceph-osd]
node02
node03
node04
node05
node06
node07
node08
node09
node10
node12
node13
node14
node15
node16
node17
node18
node19
node20
node21
node22
node23

Install python on all the nodes

$ for ID in {01..24}
> do
>  ssh node${ID} "sudo apt -y install python-minimal"
> done

Ensure time synchronization of the nodes

Install the theodotos/debian-ntp role from Ansible Galaxy:

$ sudo ansible-galaxy install theodotos.debian-ntp

Create a basic playbook ntp-init.yml:

---
- hosts: nodes
  remote_user: ubuntu
  become: yes
  roles:
     - { role: theodotos.debian-ntp, ntp.server: masternode }

Apply the playbook:

$ ansible-playbook ntp-init.yml

Verify that the monitor nodes are time synchronized:

$ ansible ceph-mon -a 'timedatectl'
node11 | SUCCESS | rc=0 >>
      Local time: Fri 2017-04-28 08:06:30 UTC
  Universal time: Fri 2017-04-28 08:06:30 UTC
        RTC time: Fri 2017-04-28 08:06:30
       Time zone: Etc/UTC (UTC, +0000)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

node24 | SUCCESS | rc=0 >>
      Local time: Fri 2017-04-28 08:06:30 UTC
  Universal time: Fri 2017-04-28 08:06:30 UTC
        RTC time: Fri 2017-04-28 08:06:30
       Time zone: Etc/UTC (UTC, +0000)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

node01 | SUCCESS | rc=0 >>
      Local time: Fri 2017-04-28 08:06:30 UTC
  Universal time: Fri 2017-04-28 08:06:30 UTC
        RTC time: Fri 2017-04-28 08:06:30
       Time zone: Etc/UTC (UTC, +0000)
 Network time on: yes
NTP synchronized: yes
 RTC in local TZ: no

Check also the OSD nodes:

$ ansible ceph-osd -a 'timedatectl'

Install Ceph

Install ceph-deploy

On masternode:

$ sudo apt install ceph-deploy

Create a new cluster and set the monitor nodes (must be odd numbered):

$ ceph-deploy new node01 node11 node24

Install ceph on master node and all other nodes:

$ ceph-deploy install masternode node{01..24}

Deploy the monitors and gather the keys:

$ ceph-deploy mon create-initial

Prepare the OSD nodes

Create the OSD directories

Create the OSD directories on the OSD nodes:

$ I=0;
$ for ID in {02..10} {12..14} {16..23}
> do 
>  ssh -l ubuntu node${ID} "sudo mkdir /var/local/osd${I}"
>  I=$((${I}+1))
> done;

Verify that the OSD directories are created:

$ ansible ceph-osd -a "ls /var/local" | cut -d\| -f1 | xargs -n2 | sort
node02 osd0
node03 osd1
node04 osd2
node05 osd3
node06 osd4
node07 osd5
node08 osd6
node09 osd7
node10 osd8
node12 osd9
node13 osd10
node14 osd11
node16 osd12
node17 osd13
node18 osd14
node19 osd15
node20 osd16
node21 osd17
node22 osd18
node23 osd19

Nodes 01, 11 and 24 are excluded because those are the monitor nodes.

Fix OSD permissions

Because of some bug we need to change the OSD directories owneship to ceph:ceph. Otherwise you will get this:

** ERROR: error creating empty object store in /var/local/osd0: (13) Permission denied

Change the ownership of the OSD directories on the OSD nodes:

$ I=0;
$ for ID in {02..10} {12..14} {16..23}
> do 
>   ssh -l ubuntu node${ID} "sudo chown ceph:ceph /var/local/osd${I}"
>   I=$((${I}+1))
> done;

Prepare the OSDs

$ I=0
$ for ID in {02..10} {12..14} {16..23}
> do
>   ceph-deploy --username ubuntu osd prepare node${ID}:/var/local/osd${I}
>   I=$((${I}+1))
> done

Activate the OSDs

For nodes 02 – 10:

$ I=0
> for ID in {02..10} {12..14} {16..23}
> do
>   ceph-deploy --username ubuntu osd activate node${ID}:/var/local/osd${I}
>   I=$((${I}+1))
> done

Deploy the configuration file and admin key

Now we need to deploy the configuration file and admin key to the admin node and our Ceph nodes. This will save us from having to specify the monitor address and keyring every time we execute a Ceph cli command.

$ ceph-deploy admin masternode node{01..24}

Set the keyring to be world readable:

$ sudo chmod +r /etc/ceph/ceph.client.admin.keyring

Test and verify

$ ceph health
HEALTH_WARN too few PGs per OSD (9 < min 30)
HEALTH_ERR clock skew detected on mon.node11, mon.node24; 64 pgs are stuck inactive for more than 300 seconds; 64 pgs stuck inactive; 64 pgs stuck unclean; Monitor clock skew detected 

Out newly build cluster is not healthy. We need to increase Placement Groups. The formula is the number_of_minimum_expected_PGs (30) times the number_of_OSDs (20) and rounded to the closest power of 2:

30x20=500 => pg_num=512

Increase PGs:

$ ceph osd pool set rbd pg_num 512

Now we run ceph health again:

$ ceph health
HEALTH_WARN pool rbd pg_num 512 > pgp_num 64

Still some tweaking needs to be done. We need to adjust pgp_num to 512:

$ ceph osd pool set rbd pgp_num 512

And we are there at last:

$ ceph health
HEALTH_OK

Create a Ceph Block Device device

Check the available storage:

$ ceph df
MapGLOBAL:
    SIZE       AVAIL      RAW USED     %RAW USED 
    11151G     10858G         293G          2.63 
POOLS:
    NAME     ID     USED     %USED     MAX AVAIL     OBJECTS 
    rbd      0       306         0         3619G           4 

Now we need to create a RADOS Block Device (RBD) to hold our data.

$ rbd create clusterdata --size 4T --image-feature layering

Check the new block device:

$ rbd ls -l
NAME         SIZE PARENT FMT PROT LOCK 
clusterdata 4096G          2

Map the block device:

$ sudo rbd map clusterdata --name client.admin
/dev/rbd0

Format the clusterdata device:

$ sudo mkfs -t ext4 /dev/rbd0

Mount the blobk device:

$ mkdir /srv/clusterdata
$ mount /dev/rbd0 /srv/clusterdata

Now we have a block device for data that is distributed among the 21 storage nodes.

Here’s is a summary of some useful monitoring and troubleshooting commands for ceph

$ ceph health
$ ceph health detail
$ ceph status (ceph -s)
$ ceph osd stat
$ ceph osd tree
$ ceph mon dump
$ ceph mon stat
$ ceph -w
$ ceph quorum_status --format json-pretty
$ ceph mon_status --format json-pretty
$ ceph df

If you run into trouble contact the awesome folks at the #ceph IRC channel, hosted on Open and Free Technology Community IRC network.

Start over

In case you messed up the procedure and you need to start over you can use the following commands:

$ ceph-deploy purge masternode node{01..24}
$ ceph-deploy purgedata masternode node{01..24}
$ ceph-deploy forgetkeys
$ for ID in {02..11} {11..23}; do ssh node${ID} "sudo rm -fr /var/local/osd*"; done
$ rm ceph.conf ceph-deploy-ceph.log .cephdeploy.conf

NOTE: this procedure will destroy your Ceph cluster along with all the data!

Conclusions

Using ceph-deploy maybe an easy way to get started with Ceph, but it does not provide much customization. For a more fine tuned setup you maybe better with the Manual Installation, even though there is a steeper learning curve.

References

  • http://docs.ceph.com/docs/master/start/
  • https://bugs.launchpad.net/ubuntu/+source/tzdata/+bug/1554806
  • http://docs.ceph.com/docs/jewel/rados/configuration/filesystem-recommendations/
  • http://www.virtualtothecore.com/en/adventures-ceph-storage-part-1-introduction/

Wiki.js is an elegant looking wiki based on Markdown. It supports LDAP and many more authentication mechanisms. In this guide we describe how to install Wiki.js on Ubuntu 16.04.

Prerequisites

  • An Ubuntu 16.04 instance.

Install curl, Node.js v8.x and build-essential:

# apt -y install curl
# curl -sL https://deb.nodesource.com/setup_8.x | bash -
# apt -y install nodejs build-essential

Install MongoDB v3.4

# apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 0C49F3730359A14518585931BC711F9BA15703C6
# echo "deb [ arch=amd64,arm64 ] http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.4 multiverse" | tee /etc/apt/sources.list.d/mongodb-org-3.4.list
# apt update
# apt -y install mongodb-org

Start MongoDB:

# systemctl start mongodb

Enable MongoDB at startup:

# systemctl enable mongodb

Install git

The version that comes with Ubuntu 16.04 fills the minimum requirements so there is no need to install it from upstream.

# apt -y install git

Install Wiki.js

# mkdir /srv/wiki.js
# cd /srv/wiki.js
# npm install wiki.js@latest

You will get this message:

> Browse to http://your-server:3000/ to configure your wiki! (Replaced your-server with the hostname or IP of your server!)
▐   ⠂    ▌ I'll wait until you're done ;)

Do as the message says. Let the wizard wait until we are done, and open another shell to work with.

Setup nginx

Install Nginx:

# apt -y install nginx

Create this VirtualHost configuration (/etc/nginx/sites-available/wiki.example.com.conf):

server {
    listen      [::]:80 ipv6only=off;
    server_name wiki.example.com;
    return      301 https://$server_name$request_uri;
}
server {
    listen 443 ssl http2;
    listen [::]:443 ssl http2;
    server_name  wiki.example.com;

    ssl_session_timeout 1d;
    ssl_session_cache shared:SSL:50m;
    ssl_session_tickets off;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS";
    ssl_prefer_server_ciphers on;

    ssl_certificate /etc/nginx/ssl/wiki.example.com.crt;
    ssl_certificate_key /etc/nginx/ssl/wiki.example.com.key;
    ssl_trusted_certificate /etc/nginx/ssl/CA.crt;

    location / {
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_pass http://127.0.0.1:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_next_upstream error timeout http_502 http_503 http_504;
    }
}

Enable the wiki.example.com VirtualHost:

# cd /etc/nginx/sites-enabled/
# ln -s ../sites-available/wiki.example.com.conf
# unlink default

Restart Nginx:

# systemctl restart nginx

Configure Wiki.js

After the installation you will be asked if you wish to run the configuration wizard. Select this and continue:

Yes, run configuration wizard on port 3000 (recommended)

Now browse to http://wiki.example.com/ and follow the installation wizard:

  • Welcome!: Start
  • System Check (if all good): Continue
  • General:
    • Site title: ExampleWiki
    • Host: https://wiki.example.com
    • Port: 3000
    • Site UI Language: English
    • Public Access: Not selected
    • Press: Continue
  • Important Considerations: Continue
  • Database: mongodb://localhost:27017/wiki
  • Database Check: Continue:
  • Paths:
    • Local Data Path: ./data
    • Local Repository Path: ./repo
  • Git Repository: Skip this step
  • Git Repository Check: Continue
  • Administrator Account
    • Administrator Email: admin@example.com
    • Password: MySecretCombination
    • ConfirmPassword: MySecretCombination
  • Finalizing: Start

Enable Wiki.js on startup

# npm install -g pm2
# pm2 startup
# pm2 save

Setup LDAP

This is an optional step for those wishing to integrate Wiki.js in their LDAP infrastructure.

Trust CUT IST ISSUING CA

Connect to the LDAP (AD) server and get all certificates:

openssl s_client -showcerts -connect dcs03ist00.lim.tepak.int:636 | tee ldap.log

Hit ‘Ctrl-C’ to end the command.

The certificate with the ID ‘1’ in ldap.log is the ISSUING CA certificate. Extract the CUT IST ISSUING CA certificate and save it in cut_issuing_ca.crt:

-----BEGIN CERTIFICATE-----
MIIF7DCCA9SgAwIBAgITcgAAAAakujIFDl5tvQAAAAAABjANBgkqhkiG9w0BAQsF
ADBNMQswCQYDVQQGEwJDWTEoMCYGA1UEChMfQ1lQUlVTIFVOSVZFUlNJVFkgT0Yg
VEVDSE5PTE9HWTEUMBIGA1UEAxMLQ1VUIFJPT1QgQ0EwHhcNMTUwMzA5MTAyMjAy
WhcNMjAwMzA5MTAzMjAyWjBeMRMwEQYKCZImiZPyLGQBGRYDaW50MRUwEwYKCZIm
iZPyLGQBGRYFdGVwYWsxEzARBgoJkiaJk/IsZAEZFgNsaW0xGzAZBgNVBAMTEkNV
VCBJU1QgSVNTVUlORyBDQTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEB
AMPHOqer+ovT+Y99lI4dSYa+XHAnCaup8vbLE9x4iKEjq9gfYq8Gs3Aujx6h32Y8
DLJcKHgxlzqwn6zzq2YSDziFPCka5bAZswaFqvD6fm22oujRmlUBDrW37OsP3nwJ
gT5GMUSzvE0ZvcdjotCm2iBDfGryJgU3PFgAvPXVyFq8bV1jTYBP8sqsWssIdOMg
doN8RLKj6gwJIwA1cvhvnuePU6a2HlHI314GaUqNtyX5PZ5VbUNIQXBt+McPNP0d
sg8yKzfBtl6V4U9xheKmdKxfIB0rN1L/uqzoewIrwUXab7l9kbSYoKqiPZkYnpEi
TQ8msXa/6TL0grQR085sugUCAwEAAaOCAbIwggGuMBAGCSsGAQQBgjcVAQQDAgEA
MB0GA1UdDgQWBBR0NThxWu0JH7JLfLf75h9dis6S6DCBnwYDVR0gBIGXMIGUMIGR
BgsrBgEEAYLiewEBATCBgTBOBggrBgEFBQcCAjBCHkAAQwBlAHIAdABpAGYAaQBj
AGEAdABpAG8AbgAgAFAAcgBhAGMAdABpAGMAZQAgAFMAdABhAHQAZQBtAGUAbgB0
MC8GCCsGAQUFBwIBFiNodHRwczovL3BraS5jdXQuYWMuY3kvcGtpL2Nwcy5odG1s
ADAZBgkrBgEEAYI3FAIEDB4KAFMAdQBiAEMAQTALBgNVHQ8EBAMCAYYwDwYDVR0T
AQH/BAUwAwEB/zAfBgNVHSMEGDAWgBSY/OKi6F/HKZRvR0574L5dNf4KwzA5BgNV
HR8EMjAwMC6gLKAqhihodHRwOi8vcGtpLmN1dC5hYy5jeS9jcmwvQ1VULVJPT1Qt
Q0EuY3JsMEQGCCsGAQUFBwEBBDgwNjA0BggrBgEFBQcwAoYoaHR0cDovL3BraS5j
dXQuYWMuY3kvcGtpL0NVVC1ST09ULUNBLmNydDANBgkqhkiG9w0BAQsFAAOCAgEA
IVJPKacKldP8WEFkCNGJq9PIFr5R0MuD6DUSifnnj94Z004ET29+C3sesUsWHMLd
p1LA+6pmF8JUAUg5Zzab2gV3vHVorYIPlPs2IorhTlnl5k9UCAjBuT3HV1kcCt2R
L601KYIYN3QIiVaA3k7xOPp+Y9nL/OlvjE3S6MVeC2WJYl1Ai+eZWQpL/SvAA8vH
NQzCqqNUnCT1zr9Uei71r0elgtoevsRmWRVWPmOl4Sft0amJ5fXDhYvN0KiktDTt
SeBy+YY48vzbhTRPFhUsj84ePyQcBqtLWQlyhmFs8b+gtSbmYWA1AnWgA+/Fd37Z
k75/x8qqnyZ4BFBKzEzukIKyapnRf8ARzL6oHo7XIXmkUb4cBzf5asNkqRRcUxrl
IEPGuGjr0+yo49f0uP3v+xE+Vyam27B3A4hhrgyqIe26/FxlbkyHT7BLhxCSu7kX
hymb9rZwGy8LR6JVCHhOllXm0eaKiRxFBYcgofekgUfjL5Coip6wAzabtrqE38zX
EkQ6sFi4hFM2ifVKA6lU1OUfeK1j5hEcmGzyCS+4aWXeY9/tTlDj28Z04cp34jla
Oc2Q4VBmO6yA1d6L22o9gTb4gvTcghHOJezWeo5PFsSD+GmJfq4JLvaHk9gsbb5D
099bze3eSDCISoR5ce6mdkD6pr5PfsVfZHhSktc4nk4=
-----END CERTIFICATE-----

Verify the certificate with:

openssl x509 -text -in cut_issuing_ca.crt

Add the CUT ISSUING CA in the trusted chain of the system:

cp cut_issuing_ca.crt /usr/local/share/ca-certificates/
update-ca-certificates

Configure LDAP for Wiki.js

Make these changes in /srv/wiki.js/config.yml:

  ldap:
    enabled: true
    url: 'ldap://ldap.example.com:389'
    bindDn: 'cn=wiki,ou=dsa,dc=example,dc=com'
    bindCredentials: 'MyLDAPCredentials'
    searchBase: 'ou=people,dc=example,dc=com'
    searchFilter: '(uid={{username}})'
    tlsEnabled: true
    tlsCertPath: '/etc/ssl/certs/ca-certificates.crt'

Give Access permissions to authenticated users

Visit the Admin URL:

https://wiki.example.com/admin

Click on ‘Users’. You will get a list of users. You can give ‘Read and Write’ access to them from the ‘Access Rights’ field and you can upgrade them to ‘Global Administrators’ from the ‘Role Override’ field.

NOTE: For LDAP the users need to login first before they are allowed to write.

Enjoy your newly created Wiki!

References

  • https://docs.requarks.io/wiki
  • https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions
  • https://docs.mongodb.com/manual/tutorial/install-mongodb-on-ubuntu/

LXD is a relatively new technology by Canonical. It is a container hypervisor that builds on top of LXC. It shares some similarities with Docker, but LXD focuses on full blown containerized systems, instead of containerized applications.

LXD/LXC creates lighter systems than VM hypervisors, and it is an easy way to experiment with different software without messing up with your system. You can also use it in production, to run Linux systems with much greater density than on VM hypervisors. The only drawback is that the container must be the same OS as the host (Linux in this case) and it depends on the host for kernel functionality (e.g. modules etc).

In this guide we will build an LXD container with two virtual NICs which are attached to the physical NIC of it’s host using bridged networking.

Prerequisites

  • An Ubuntu 16.04 host with two physical NICs

Prepare host networking

Install dependencies

# apt -y install bridge-utils

Setup networking on the host

There are two interfaces on the host enp3s0 and enp5s0. The first one, enp3s0 is the primary and connects to the main, DHCP enabled, internal network. The second one is connected to an isolated internal network where some critical services are running. There is no DHCP in the isolated network.

Prepare your /etc/network/interfaces configuration file like this:

# The loopback network interface 
auto lo
iface lo inet loopback

# The main Bridge
auto br0
iface br0 inet dhcp
    bridge-ifaces enp3s0
    bridge-ports enp3s0
    up ip link set enp3s0 up

# The primary network interface
iface enp3s0 inet manual

# The secondary Bridge
auto br1
iface br1 inet manual
        bridge-ifaces enp5s0
        bridge-ports enp5s0
        up ip link set enp5s0 up

# The secondary network interface
iface enp5s0 inet manual

Setup LXC/ LXD

Installation of LXD

# apt -y install lxd

Prepare a new profile

We need a profile with two NICs. First create a new profile:

# lxc profile copy default dualnic

Then edit the new profile with lxc profile edit dualnic:

config:
  user.network_mode: link-local
description: Dual nic profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: br0
    type: nic
  eth1:
    name: eth1
    nictype: bridged
    parent: br1
    type: nic
name: dualnic

The names of the virtual interfaces of the container will be eth0 for the primary and eth1 for the secondary.

Launch the container

Now we want to launch a Debian jessie container, which we will retrieve from the prepared images on the

# lxc launch images:debian/jessie mycontainer -p dualnic

List the container:

# lxc list
+-------------+---------+----------------------+------+------------+-----------+
|    NAME     |  STATE  |         IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-------------+---------+----------------------+------+------------+-----------+
| mycontainer | RUNNING | 192.168.0.122 (eth0) |      | PERSISTENT | 0         |
+-------------+---------+----------------------+------+------------+-----------+

The IP listed above was set by the DHCP server in the primary network.

Connect to the container:

# lxc exec mycontainer bash

Setup container’s networking

Edit /etc/network/interfaces inside our container:

auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

auto eth1
iface eth1 inet static
        address 172.16.16.100
        netmask 255.255.255.0
        network 172.16.16.0
        broadcast 172.16.16.255
        up ip route add 10.10.10.0/24  via 172.16.16.1 dev eth1
        up ip route add 10.10.20.0/24  via 172.16.16.1 dev eth1

The routes 10.10.10.0/24 and 10.10.20.0/24 are internal routes that need to be reached though the secondary network.

Take a clean snapshot

Before you start working on your container, it is a good idea to get a clean snapshot, so you can revert to it, in case you mess something up.

# lxc snapshot mycontainer

If you want to learn more about the and features I urge you to read the excellent introduction to LXD 2.0 by Stéphane Graber

References

  • https://linuxcontainers.org/
  • https://www.ubuntu.com/containers/lxd
  • https://insights.ubuntu.com/2016/03/14/the-lxd-2-0-story-prologue/
  • https://stgraber.org/2016/03/11/lxd-2-0-blog-post-series-012/
  • https://insights.ubuntu.com/2015/11/10/converting-eth0-to-br0-and-getting-all-your-lxc-or-lxd-onto-your-lan/

Less than a year ago I was attracted by the value for money offer of OnePlus 2 and became a proud owner. Being paranoid about Google tracking, I disabled most Google apps. Alas, some apps were impossible to disable. I assumed that this was caused by their dependencies with other core applications. Google play services was one them, but I never created an account with Google and never synced anything with them.

Time went by and it happened that data over mobile was accidentally enabled. And then I noticed from the mobile data traffic statistics, that Google Play services was creating traffic even without a Google account! Why does this happens its beyond my perception, but anyone is allowed to guess based on [recent history](https://en.wikipedia.org/wiki/PRISM_(surveillance_program)#/media/File:Prism_slide_5.jpg "PRISM Surveillance program"). There is an option to disable background traffic on the stock Android ROM (OxygenOS) but that works only when data over mobile is enabled. With Wi-Fi enabled, Google Play services keeps on sending suspicious traffic, back to mother Google.

So I decided to look for alternatives to replace OxygenOS with a Google-free Android ROM. These are the options I have investigated:

  • Replicant:
    This seems to be the more privacy respecting Android MOD. No support for OnePlus 2, alas!
  • AOKP:
    This is an interesting MOD with lots of features. There are recent nightly builds for OnePlus 2. I tried to install it my Phone but gave an error when I tried to sideload it.
  • CyanogenMOD:
    A popular Android MOD that was recently discontinued. A fork by the name LineageOS is taking over. No image downloads yet.<br />
    UPDATE (2-Jan-2016): There are some experimental LineageOS images here.
  • Paranoid Android:
    Another successful Android MOD with close ties to OnePlus. It boasts some interesting features. Officially supports all OnePlus models except the latest OnePlus 3T. I tried the OnePlus 2 image but the phone stack to the OnePlus logo boot screen.
  • Exodus:
    This is a minimalistic Android MOD. It is based on AOSP and is free from Google Apps. This is the only MOD I managed to get it working on my OnePlus 2.

So I decided to go with Exodus. This guide describes the procedure and preparations I followed in order to flash the latest Exodus nightly build for OnePlus 2.

DISCLAIMER: This guide could renter your device a luxurious brick! Backup everything before you continue. The author of this guide will not be liable for any damages you may cause on your device or any data lost. Proceed at your own risk!

Now we got that out of the way, let’s continue.

Prerequisites

  • A PC. In this guide we are using Debian/Ubuntu as the PC’s Operating System but this can work with other Operating Systems, with minor adaptations.
  • An OnePlus 2 smartphone. It could possibly be used for other devices too, but I have only tested this on OnePlus 2.
  • The Android Debug Bridge (ADB). This is part of the Android SDK. On Debian/Ubuntu you can install it with apt install android-tools-adb.
  • A Type-C USB cable to connect the phone to your PC.

Make your device detectable in Linux

There is some process we need to follow so that the OnePlus 2 device is detectable by ADB

Enable Usb Debugging

To Enable USB Debugging we need to first enable the Developer Options:

  • Go into Settings
  • Dive into the About phone option
  • Tap 7 times on the Build number option
  • Go back and tap on Developer Options
  • Enable USB debugging and OEM unlock

Enable detection of device under Linux

First ensure that the device is connected:

# lsusb
...
 Bus 003 Device 039: ID 2a70:f003  
...

The device ID for OnePlus 2 is 0x2a70. Set this in * ~/.android/adb_usb.ini*:

# echo "0x2a70" >> ~/.android/adb_usb.ini

Download the udev rules files (Thanks Nicolas Bernaerts):

# wget --header='Accept-Encoding:none' -O /etc/udev/rules.d/51-android.rules https://raw.githubusercontent.com/NicolasBernaerts/ubuntu-scripts/master/android/51-android.rules
# chmod a+r /etc/udev/rules.d/51-android.rules
# wget --header='Accept-Encoding:none' -O /etc/udev/rules.d/69-mtp.rules https://raw.githubusercontent.com/NicolasBernaerts/ubuntu-scripts/master/android/69-mtp.rules
# chmod a+r /etc/udev/rules.d/69-mtp.rules

Restart the udev and ADB services:

# systemctl restart udev
# adb kill-server
# adb start-server

NOTE: make sure USB Debugging is enabled otherwise you will still not be able to see the device!

Verify that the device is present:

# adb devices 
List of devices attached 
da0afea7        unauthorized

Seems OK.

Backup your data and stock image

Since we are not using Google’s services we will need to manually backup all important data such as:

  • Export contacts on a .vcf file.
  • Save Pictures and Videos from the phone on your PC
  • Backup every other Applacation data which is important to you.

You can also backup your entire system with ADB:

# adb backup -f oneplus2.bck -apk -shared -all

This will take some time depending on the size of your data.

You can later restore your system back to normal, using this command:

# adb restore oneplus2.bck

Prepare the recovery image

Download Prepare the First Aid kit

This is a bundle that includes all the files needed to bring life to a bricked OnePlus 2 device. It includes a recovery image which we will use to install Exodus (or any other image you wish) on the device.

# wget -O Firstaid_v_2_0.zip 'http://forum.xda-developers.com/devdb/project/dl/?id=16684&task=get'
# unzip Firstaid_v_2_0.zip

All the necessary files are in the newly created, Firstaid directory.

Download TWRP Custom recovery

The recovery image that comes with Firstaid is somewhat outdated. We will download a more recent version of it and replace it in the Firstaid directory. You can download the more recent TWRP image from here. Do not use wget or other command line tools, just a normal browser to download it.

NOTE: There are more recent versions than 3.0.2-0 like 3.0.2-1 and 3.0.2-2. You can try those if you feel lucky, but be warned! Those versions did not work on my phone. You can see all the past and resent images here.

Verify the downloaded image:

# wget -O twrp-public.asc https://dl.twrp.me/public.asc
# gpg --import twrp-public.asc
# wget https://dl.twrp.me/oneplus2/twrp-3.0.2-0-oneplus2.img.asc
# gpg --verify twrp-3.0.2-0-oneplus2.img.asc twrp-3.0.2-0-oneplus2.img

If you get Good signature from &quot;TeamWin &lt;admin @teamw.in>&quot; that means the image is correct. Now replace the TWRP image that comes with Firstaid with this one:

# cp twrp-3.0.2-0-oneplus2.img Firstaid/twrp.img

Flash the recovery image

Reboot to fastboot mode:

# adb reboot-bootloader

Alternatively you can poweroff the OnePlus 2 and press the Volume UP + Power buttons simultaneously.

Verify the presence of the device:

# fastboot devices
da0afea7               fastboot

Unlock the phone:

fastboot oem unlock

Flash Firstaid with the recent TWRP recovery image:

# cd Firstaid
# ./firstaid.sh

Now unplug the phone from the PC, poweroff and the press the Volume DOWN + Power buttons simultaneously. This will bring your phone into TWRP recovery.

Flashing the Exodus image

Download Exodus image and verify

Download the latest Exodus image for OnePlus 2 here:

# wget http://exodus-developers.net/exodus-6.0/oneplus2/exodus-6.0-20161111-NIGHTLY-oneplus2.zip
# wget http://exodus-developers.net/exodus-6.0/oneplus2/exodus-6.0-20161111-NIGHTLY-oneplus2.zip.md5 

Verify that the download is correct:

# cat exodus-6.0-20161111-NIGHTLY-oneplus2.zip.md5 ; md5sum exodus-6.0-20161111-NIGHTLY-oneplus2.zip
35dcf9ea73648682a36e673f5ed8f0eb  /android/exodus-6.0/out/target/product/oneplus2/exodus-6.0-20161111-NIGHTLY-oneplus2.zip
35dcf9ea73648682a36e673f5ed8f0eb  exodus-6.0-20161111-NIGHTLY-oneplus2.zip

Seems OK

Plug the phone back to the PC and ensure that it is detectable by ADB:

# adb devices 
List of devices attached 
da0afea7        device

Flash the Exodus image

Follow these steps:

  • Wipe old data: Select Wipe and then Swipe to Factory Reset. You may need to format the system if the previous system was encrypted.
  • Enter the Sideload mode: From the TWRP start menu select Advanced then ADB Sideload and then Swipe to Start Sideload
  • From the PC’s CLI run the following command:
# adb sideload exodus-6.0-20161111-NIGHTLY-oneplus2.zip

Wait and then reboot into the new Exodus system when done.

Optional steps

Install F-Droid:

By default Exodus has a minimal set of apps. Since we are not using Google Apps we will need to install F-Droid:

# wget https://f-droid.org/FDroid.apk
# adb install FDroid.apk

Lock your Phone

Setup a PIN or Pattern to lock the screen of your phone when not in use: Settings -> Security -> Screen Lock. Avoid fingerprint as it can be easily bypassed.

Encrypt phone

Encrypting your phone will protect your data if it gets stolen or confiscated. These are the steps Settings -> Security -> Encrypt phone. Your phone will need to be plugged to charger and charged to 80% or above, otherwise the system will refuse to start the encryption process.

Restoring the Stock OxygenOS System

If you decide that you don’t like Exodus or any other MOD that you have installed, you can restore your phone back to the stock OxygenOS. These instructions can also be used to restore your phone’s functionality after you have accidentally bricked it.

Download necessary images

Download the stock OxygenOS 3.0.2 image from here. Verify the download with:

# echo 46b1fde116275d83d05c2dd89422069f ; md5sum OnePlus2Oxygen_14_OTA_019_all_1606041303.zip
46b1fde116275d83d05c2dd89422069f
46b1fde116275d83d05c2dd89422069f  OnePlus2Oxygen_14_OTA_019_all_1606041303.zip

Download the upgrade patch for Oxygen 3.1.0 from here. Verify the download with:

# echo 9cfa9a2a4c7fada6f9db79ea660251d0 ; md5sum OnePlus2Oxygen_14_OTA_019-020_patch_1608262242.zip 
9cfa9a2a4c7fada6f9db79ea660251d0
9cfa9a2a4c7fada6f9db79ea660251d0  OnePlus2Oxygen_14_OTA_019-020_patch_1608262242.zip

Optionally you can download the SuperSU image if you would like your OxygenOS rooted:

# wget -O supersu.zip 'https://download.chainfire.eu/743/SuperSU/BETA-SuperSU-v2.52.zip?retrieve_file=1'

Flash Recover image

First we need to restore the recovery ROM from Firstaid:

# cd Firstaid
# ./firstaid.sh

Flash the Oxygen Images.

Before we install the OxygenOS image it is recommended to wipe the phone: From the TWRP menu select Wipe and then Swipe to factory reset. If the system was previously encrypted you will need to use TWRP to format it before the wipe.

Then we need to set the device into Sideload mode: Select Advanced -> ADB Sideload -> Swipe to Start Sideload.

First install the OxygenOS 3.0.2 image:

adb sideload OnePlus2Oxygen_14_OTA_019_all_1606041303.zip

Then re-enable ADB Sideload and load the Oxygen 3.2.0 patch:

adb sideload OnePlus2Oxygen_14_OTA_019-020_patch_1608262242.zip

Optionally you can also load the SuperSU application as well

adb sideload supersu.zip

Finally reboot into the new system and enjoy your freshly formatted phone.

References

  • http://bernaerts.dyndns.org/android/339-android-oneplustwo-oneplusx-enable-adb-mtp-detection-ubuntu-trusty
  • https://forum.xda-developers.com/devdb/project/dl/?id=16684
  • https://twrp.me/
  • https://twrp.me/devices/oneplustwo.html
  • https://github.com/TeamExodus/device_oneplus_oneplus2
  • http://exodus-developers.net/
  • https://github.com/TeamExodus/device_oneplus_oneplus2</admin>

Most registrars they provide a DNS service when you purchase a domain name. But having your own DNS servers means more control to you. And if you are an aspiring sysadmin, you will find it fun and educational. We will be using BIND 9 which is, by far the most popular DNS implementation.

Prerequisites

We will need two systems (VMs or containers) preferably on different geographic locations and different providers. One will be the master and the other system, the slave. This guide works for recent Debian or Ubuntu systems.

Some assumptions:

  • Master DNS:
    • Hostname: ns1.example.com
    • IP: 1.1.1.1
  • Slave DNS:
    • Hostname: ns2.example.com
    • IP: 2.2.2.2
  • Webserver:
    • Hostname: example.com (www.example.com is an alias)
    • IP: 3.3.3.3
  • Main mailserver:
    • Hostname: mx1.example.com ({mail,smtp,pop,imap,webmail}.example.com are aliases)
    • IP: 4.4.4.4
  • Backup mailserver:
    • Hostname: mx2.example.com
    • IP: 5.5.5.5

Preparations

Setup the /etc/hosts file

On master’s /etc/hosts:

1.1.1.1    ns1.example.com ns1

On slave’s /etc/hosts:

2.2.2.2    ns2.example.com ns2

Installing BIND

NOTE: all commands must be applied to both master and slave unless otherwise stated

apt-get -y install bind9 bind9utils

Allow port 53:

ufw allow 53

Configure the global options

Make the following changes in the /etc/bind/named.conf.options of both servers:

--- /etc/bind/named.conf.options        2016-12-12 14:44:57.163515708 -0500
+++ /etc/bind/named.conf.options.new    2016-12-12 14:52:29.749268250 -0500
@@ -1,5 +1,7 @@
 options {
        directory "/var/cache/bind";
+       recursion no;
+       allow-transfer { none; };

        // If there is a firewall between you and nameservers you want
        // to talk to, you may need to fix the firewall to allow multiple

Configure the local options

On master’s /etc/bind/named.conf.local:

--- /etc/bind/named.conf.local  2016-12-12 20:06:27.000000000 +0000
+++ /etc/bind/named.conf.local.new      2016-12-12 20:07:03.000000000 +0000
@@ -6,3 +6,8 @@
 // organization
 //include "/etc/bind/zones.rfc1918";

+zone "example.com" {
+    type master;
+    file "/etc/bind/db.example.com";
+    allow-transfer { 2.2.2.2; };
+};

On slave’s /etc/bind/named.conf.local:

--- /etc/bind/named.conf.local  2016-11-01 13:02:24.000000000 -0400
+++ /etc/bind/named.conf.local.new      2016-12-12 15:09:47.445235343 -0500
@@ -6,3 +6,8 @@
 // organization
 //include "/etc/bind/zones.rfc1918";

+zone "example.com" {
+    type slave;
+    file "db.example.com";
+    masters { 1.1.1.1; };
+};

Create the zone file

On master server create the zone file (/etc/bind/db.example.com):

$TTL 3600
@    IN    SOA    ns1.example.com.    root.example.com. (
                                            2016121201
                                            7200
                                            3600
                                            1209600
                                            180 )

; NS Records
@       300    IN      NS              ns1.example.com.
@       300    IN      NS              ns2.example.com.

; MX Records
@       300    IN      MX      10      mx1.example.com.
@       300    IN      MX      20      mx2.example.com.

; Address records
@       300    IN      A               3.3.3.3
ns1     300    IN      A               1.1.1.1
ns2     300    IN      A               2.2.2.2
mx1     300    IN      A               4.4.4.4
mx1     300    IN      A               5.5.5.5

; Alias (Canonical Name) records
www     300    IN      CNAME           example.com.
mail    300    IN      CNAME           mx1.example.com.
smtp    300    IN      CNAME           mx1.example.com.
imap    300    IN      CNAME           mx1.example.com.
pop     300    IN      CNAME           mx1.example.com.
webmail 300    IN      CNAME           mx1.example.com.

; TXT records
@       300    IN      TXT             "v=spf1 a mx ?all"

Verify the global configuration

Run the named-checkconf command on both servers. If you get any errors you should fix them before proceeding.

Verify the zone configuration

Run the following command on the master server:

named-checkzone theo-andreou.org /etc/bind/db.example.com 
zone example.com/IN: loaded serial 2016121201
OK

If you get ‘OK’ then your setup is correct, otherwise you should fix it before proceeding.

Restart bind and test your servers

systemctl restart bind9.service

Now test if your servers resolve your records.

The master:

dig smtp.example.com @1.1.1.1
...
;; ANSWER SECTION:
smtp.example.com.   300 IN  CNAME   mx1.example.com.
mx1.example.com.    300 IN  A   4.4.4.4
...

The slave:

dig smtp.example.com @2.2.2.2
...
;; ANSWER SECTION:
smtp.example.com.   300 IN  CNAME   mx1.example.com.
mx1.example.com.    300 IN  A   4.4.4.4
...

If you get any errors you can check your syslog file (/var/log/syslog).

Finalizing

After you have tested that everything is OK, it is time to tell your registrar about the new servers. This is called setting the glue records and you should consult the documentation of your registrar on how to do that. Ask support if you have any trouble.

References

  • https://www.digitalocean.com/community/tutorials/how-to-configure-bind-as-an-authoritative-only-dns-server-on-ubuntu-14-04

In this guide we will show how you can remotely decrypt a headless Debian or Ubuntu Linux system, that has been encrypted with LUKS.

Prerequisites

  • A LUKS encrypted Debian jessie or Ubuntu xenial system
  • Keyboard and monitor for the initial system setup
  • Allow SSH root access on the decrypted system using public key authentication
  • Use a different port for ssh (assuming port 4422) on the decrypted system

    NOTE: using a different port than the standard SSH port (22) serves a double purpose. For once, you will not received the scary WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! message every time you will try to remotely unlock the system and as an additional bonus you will get less SSH attacks on the active system.

Installing dropbear

Dropbear is a lightweight SSH server especially suitable for initial ramdisk (initrd) environments and other lightweight systems.

Install dropbear:

# apt -y install dropbear

Setup public key authentication for dropbear

Create the homedir for the root user and the SSH configuration directory:

# mkdir -p /etc/initramfs-tools/root/.ssh

Append your client SSH pubkey to authorized_keys:

# cat ~/.ssh/id_rsa.pub | ssh -p 4422 root@encrypted-system "cat >> /etc/initramfs-tools/root/.ssh/authorized_keys"

Setup a static IP for the unlock environment

This step is optional but highly recommended if you are setting up a static, permanent service. If you skip this step DHCP will kick in, provided you have a DHCP Server in your environment.

Run this command to update the /etc/initramfs-tools/initramfs.conf configuration file:

echo IP=10.0.0.67::10.0.0.1:255.255.255.0:encrypted-system:eth0:off >> /etc/initramfs-tools/initramfs.conf

Explanation of the different fields:
[host_ip]::[gateway_ip]:[netmask]:[hostname]:[device]:[autoconf]

NOTE: there are two successive colons (::) after the host_ip.

Setup the unlock script

Copy the following text in /etc/initramfs-tools/hooks/crypt_unlock.sh:

#!/bin/sh
#
# By Stinky Parkia
# https://stinkyparkia.wordpress.com/2014/10/14/remote-unlocking-luks-encrypted-lvm-using-dropbear-ssh-in-ubuntu-server-14-04-1-with-static-ipst/

PREREQ="dropbear"

prereqs() {
    echo "$PREREQ"
}

case "$1" in
    prereqs)
    prereqs
    exit 0
    ;;
esac

. "${CONFDIR}/initramfs.conf"
. /usr/share/initramfs-tools/hook-functions

if [ "${DROPBEAR}" != "n" ] && [ -r "/etc/crypttab" ] ; then
    cat > "${DESTDIR}/bin/unlock" < < EOF
#!/bin/sh
if PATH=/lib/unlock:/bin:/sbin /scripts/local-top/cryptroot; then
    kill \`ps | grep cryptroot | grep -v "grep" | awk '{print \$1}'\`
    # following line kill the remote shell right after the passphrase has
    # been entered.
    kill -9 \`ps | grep "\-sh" | grep -v "grep" | awk '{print \$1}'\`
    exit 0
fi
exit 1
EOF

    chmod 755 "${DESTDIR}/bin/unlock"

    mkdir -p "${DESTDIR}/lib/unlock"
    cat > "${DESTDIR}/lib/unlock/plymouth" < < EOF
#!/bin/sh
[ "\$1" == "--ping" ] && exit 1
/bin/plymouth "\$@"
EOF

    chmod 755 "${DESTDIR}/lib/unlock/plymouth"

    echo To unlock root-partition run "unlock" >> ${DESTDIR}/etc/motd
fi

Make the script executable:

# chmod +x /etc/initramfs-tools/hooks/crypt_unlock.sh

Apply the configuration

Apply the changes in the initial ramdisk:

# update-initramfs -u

Reboot the system:

# reboot

Remotely unlock the system

From your client, SSH into the initial ramdisk:

ssh root@encrypted-system 

If everything is correct you will be greeted by this MOTD:

To unlock root-partition run unlock


BusyBox v1.22.1 (Ubuntu 1:1.22.0-15ubuntu1) built-in shell (ash)
Enter 'help' for a list of built-in commands.

Unlock the system and boot into it:

# unlock
Please unlock disk sda3_crypt: 

You will get the following message and you will exit the remote shell if successful:

cryptsetup: sda3_crypt set up successfully
Connection to 10.0.0.67 closed.

You can now login to the active Linux system using the alternative port 4422:

ssh -p 4422 root@encrypted-system

If you can login successfully to your system you can remove the keyboard and monitor and hide your system somewhere where the Sun does not shine :).

Thanks to Stinky Parkia for the excellent guide and the brilliant unlock script.

References

  • https://stinkyparkia.wordpress.com/2014/10/14/remote-unlocking-luks-encrypted-lvm-using-dropbear-ssh-in-ubuntu-server-14-04-1-with-static-ipst/

LDAP/Fusiondirectory setup

In this guide we will be setting up LDAP (openldap) along with the FusionDirectory web management tool.

Preparations

  • Install Debian jessie (Ubuntu 16.04 should work too) on your server or VM
  • Setup the DNS records in your DNS servers:

    ldap.example.com.           3599    IN      A       1.1.1.1
    

    NOTE: replace 1.1.1.1 with you actual internal or external IP

  • Allow ssh and web services on firewall:

    apt -y install ufw
    ufw allow 22/tcp
    ufw allow 80/tcp
    ufw allow 443/tcp
    ufw enable
    

    NOTE: It may be a good idea to change the default ssh port from 22 to something less predictable

Setup LDAP

  • Setup ldap.example.com in /etc/hosts:
127.0.2.1 ldap.example.com ldap
  • Install OpenLDAP and FusionDirectory

    apt -y install slapd
    

    Enter and verify the administrator password for slapd.

  • Create An Internal Certificate Authority for Example LTD:

    • First install gnutls-bin:
      apt -y install gnutls-bin
      
    • Create the key for the internal CA for example.com:
      certtool --generate-privkey --outfile example.com-rootCA.key
      
    • Create a certificate for our internal CA:
certtool --generate-self-signed --load-privkey example.com-rootCA.key --outfile example.com-rootCA.crt
Generating a self signed certificate...
Please enter the details of the certificate's distinguished name. Just press enter to ignore a field.
Common name: Example LTD Internal ROOT CA
UID: 
Organizational unit name: IT
Organization name: Example LTD
Locality name: Limassol
State or province name: Limassol
Country name (2 chars): CY
Enter the subject's domain component (DC): 
This field should not be used in new certificates.
E-mail: admins@example.com
Enter the certificate's serial number in decimal (default: 6295758616856773074): 
The certificate will expire in (days): 7300

Activation/Expiration time.
The certificate will expire in (days): 7300

Extensions.
Does the certificate belong to an authority? (y/N): y
Path length constraint (decimal, -1 for no constraint): -1
Is this a TLS web client certificate? (y/N):
Will the certificate be used for IPsec IKE operations? (y/N): 
Is this a TLS web server certificate? (y/N): 
Enter a dnsName of the subject of the certificate: 
Enter a URI of the subject of the certificate: 
Enter the IP address of the subject of the certificate: 
Enter the e-mail of the subject of the certificate: 
Will the certificate be used to sign other certificates? (y/N): y
Will the certificate be used to sign CRLs? (y/N): 
Will the certificate be used to sign code? (y/N): 
Will the certificate be used to sign OCSP requests? (y/N): 
Will the certificate be used for time stamping? (y/N):
Enter the URI of the CRL distribution point: 
X.509 Certificate Information:
        Version: 3
        Serial Number (hex): 575f071b0d5a41d2
        Validity:
                Not Before: Mon Jun 13 19:19:27 UTC 2016
                Not After: Sun Jun 08 19:20:00 UTC 2036
        Subject: CN=Example LTD Internal ROOT CA,OU=IT,O=Example LTD,L=Limassol,ST=Limassol,C=CY,EMAIL=admins@example.com
        Subject Public Key Algorithm: RSA
        Algorithm Security Level: Medium (2048 bits)
                Modulus (bits 2048):
                        00:c0:75:c8:02:05:d0:0c:67:af:ac:0c:80:53:bf:cd
                        a3:80:76:cf:3e:14:19:99:5c:24:b4:fc:b0:42:8d:5a
                        03:5d:04:a5:85:c7:fe:e3:d4:30:6c:4c:26:90:76:c5
                        3e:a0:dc:a7:53:a7:eb:13:60:78:44:b3:0a:b2:77:0c
                        46:19:96:ea:d2:46:82:9c:11:2c:a5:e2:a1:57:38:f4
                        8e:4d:74:4f:f9:41:dd:11:f4:c2:f5:9f:b7:9a:93:7d
                        a7:f8:f3:dd:2e:08:6a:25:75:79:f3:63:e5:09:1f:bd
                        6a:38:45:85:f0:63:54:c0:08:68:41:15:66:a4:e3:84
                        49:7e:e5:c5:c7:6c:d3:c7:be:d5:5a:df:1a:1d:55:f8
                        35:73:bb:e3:ea:f7:66:af:d9:09:72:ca:17:5f:80:09
                        99:6a:49:e3:8b:f2:72:56:ac:f8:ba:60:49:d5:80:2a
                        07:e6:17:88:86:e4:3c:89:cd:af:2b:66:a1:af:53:f4
                        66:21:30:a3:22:af:a9:11:6e:98:e0:f7:6d:ef:8a:32
                        e9:0b:a4:82:7b:7b:db:2d:90:8e:bd:e4:54:04:a4:52
                        e8:cf:f6:2e:9b:97:46:ab:cb:38:06:23:33:db:42:0c
                        25:41:5a:d7:02:15:07:c6:e8:86:0b:a6:d7:7d:81:16
                        bd
                Exponent (bits 24):
                        01:00:01
        Extensions:
                Basic Constraints (critical):
                        Certificate Authority (CA): TRUE
                Key Purpose (not critical):
                        Time stamping.
                Key Usage (critical):
                        Certificate signing.
                Subject Key Identifier (not critical):
                        7a596f6dea4080e89c9e78a698d7126cd63dafa7
Other Information:
        Public Key ID:
                7a596f6dea4080e89c9e78a698d7126cd63dafa7
        Public key's random art:
                +--[ RSA 2048]----+
                |                 |
                |     . .         |
                |    . . .        |
                |   o .   .       |
                | . .+.  S o      |
                |  =o..o. + . .   |
                | o.o= .oo . o o  |
                | oo+.  .o  o o   |
                |o...  E+   .o    |
                +-----------------+

Is the above information ok? (y/N): y


Signing certificate...
  • Add the Example LTD Internal ROOT CA as trusted in ca-certificates:

    mkdir /usr/share/ca-certificates/extra
    cp example.com-rootCA.crt /usr/share/ca-certificates/extra
    dpkg-reconfigure ca-certificates
    

    Add the extra/example.com-rootCA.crt CA as a trusted CA.

    • Configure slapd:
    dpkg-reconfigure slapd
    
    • Omit OpenLDAP server configuration? No
    • DNS domain name: example.com
    • Organization name: Example LTD
    • Administrator password: ***************
    • Verify password: *************
    • Database backend to use: MDB
    • Do you want the database to be removed when slapd is purged? No
    • Move old database? Yes
    • Allow LDAPv2 protocol? No
  • Configure TLS on LDAP:

  • Create a key for ldap.example.com:

    certtool --generate-privkey --outfile ldap.example.com.key
    Generating a 2048 bit RSA private key...
    

  • Create a certificate for ldap.example.com:
certtool --generate-certificate --load-privkey ldap.example.com.key --outfile ldap.example.com.crt --load-ca-certificate example.com-rootCA.crt --load-ca-privkey example.com-rootCA.key
Generating a signed certificate...
Please enter the details of the certificate's distinguished name. Just press enter to ignore a field.
Common name: ldap.example.com
UID: 
Organizational unit name: IT
Organization name: Example LTD
Locality name: Limassol
State or province name: Limassol
Country name (2 chars): CY
Enter the subject's domain component (DC): 
This field should not be used in new certificates.
E-mail: admins@example.com
Enter the certificate's serial number in decimal (default: 6295762607454361711):

Activation/Expiration time.
The certificate will expire in (days): 3650

Extensions.
Does the certificate belong to an authority? (y/N): 
Is this a TLS web client certificate? (y/N): 
Will the certificate be used for IPsec IKE operations? (y/N): 
Is this a TLS web server certificate? (y/N): 
Enter a dnsName of the subject of the certificate: 
Enter a URI of the subject of the certificate: 
Enter the IP address of the subject of the certificate: 
Enter the e-mail of the subject of the certificate: 
Will the certificate be used for signing (required for TLS)? (Y/n): 
Will the certificate be used for encryption (not required for TLS)? (Y/n):
X.509 Certificate Information:
        Version: 3
        Serial Number (hex): 575f0abc2f81186f
        Validity:
                Not Before: Mon Jun 13 19:35:45 UTC 2016
                Not After: Thu Jun 11 19:36:29 UTC 2026
        Subject: CN=ldap.example.com,OU=IT,O=Example LTD,L=Limassol,ST=Limassol,C=CY,EMAIL=admins@example.com
        Subject Public Key Algorithm: RSA
        Algorithm Security Level: Medium (2048 bits)
                Modulus (bits 2048):
                        00:d0:15:8e:02:90:5f:4a:9f:90:ea:1e:35:e6:4b:eb
                        a9:8c:e5:bf:68:ec:83:0e:49:5b:d1:f0:08:4b:ac:b0
                        31:d2:e0:a7:eb:18:d3:ee:b8:38:b7:c4:0a:cc:97:cc
                        b6:ac:2d:29:c8:a8:c4:7c:cc:f1:36:5a:e9:6b:52:f5
                        1e:e5:4f:90:67:34:1f:8c:a8:17:72:ee:40:87:ba:ae
                        8b:f8:4f:f8:be:51:ee:ea:d5:e4:17:63:79:22:41:c0
                        19:43:33:55:bb:46:80:5c:b8:16:18:fa:fb:17:58:c2
                        ed:d2:14:10:3b:57:5d:de:7f:29:ab:66:c2:81:87:05
                        f7:b7:27:78:a9:c0:8e:4f:1c:3f:66:6f:dd:43:26:9f
                        84:59:fb:c7:21:3c:62:4f:8d:4a:25:ab:7e:f0:5f:7e
                        df:97:f7:79:f8:c7:2d:c8:5a:7a:de:ea:5b:c7:bd:e9
                        12:17:56:d3:47:ff:eb:fa:b5:6f:d9:56:8f:c7:e8:7a
                        46:92:75:cc:ff:de:0e:88:49:7d:d7:dd:6e:8d:3f:57
                        fa:0a:7a:3b:80:ec:0e:10:dd:70:d5:9a:8d:91:ce:72
                        44:06:21:d2:9d:e9:b8:91:13:68:4c:fc:e2:bb:4d:a8
                        97:ed:e9:a4:98:5d:e7:c0:ef:3e:9d:30:28:de:bd:10
                        01
                Exponent (bits 24):
                        01:00:01
        Extensions:
                Basic Constraints (critical):
                        Certificate Authority (CA): FALSE
                Key Usage (critical):
                        Digital signature.
                        Key encipherment.
                Subject Key Identifier (not critical):
                        6d8a173de01efa11a892dda76ccd7abc609a2707
                Authority Key Identifier (not critical):
                        7a596f6dea408aa89c9e78a698d7126cd63dafa7
Other Information:
        Public Key ID:
                6d8a173de01efa00a892dda67ccd7abc609a2707
        Public key's random art:
                +--[ RSA 2048]----+
                |                 |
                |                 |
                |        .        |
                |       . +       |
                |        A =      |
                |       +Y= .     |
                |      oo+kk+     |
                |      iii==o*    |
                |       .=B**o.   |
                +-----------------+


Is the above information ok? (y/N): y

Signing certificate...
  • Enable LDAPS in /etc/default/slapd:

    SLAPD_SERVICES="ldap:/// ldapi:/// ldaps:///"
    
    • Copy the ldap.example.com certficate and apply permissions
      mkdir /etc/ldap/ssl
      cp ldap.example.com.crt /etc/ldap/ssl
      cp ldap.example.com.key /etc/ldap/ssl
      chown -R openldap:openldap /etc/ldap/ssl    /
      
    • Create an olcSSL.ldif file:
mkdir /etc/ldap/custom_ldifs/
cd /etc/ldap/custom_ldifs/<br />
cat > olcSSL.ldif < < EOF
dn: cn=config
changetype: modify
add: olcTLSCACertificateFile
olcTLSCACertificateFile: /etc/ssl/certs/ca-certificates.crt
-
add: olcTLSCertificateKeyFile
olcTLSCertificateKeyFile: /etc/ldap/ssl/ldap.example.com.key
-
add: olcTLSCertificateFile
olcTLSCertificateFile: /etc/ldap/ssl/ldap.example.com.crt
EOF
  • Import the TLS configuration:
ldapmodify -Y EXTERNAL -H ldapi:/// -f olcSSL.ldif

Verify with:

slapcat -n0 | grep -i tls
olcTLSCACertificateFile: /etc/ssl/certs/ca-certificates.crt
olcTLSCertificateFile: /etc/ldap/ssl/ldap.example.com.crt
olcTLSCertificateKeyFile: /etc/ldap/ssl/ldap.example.com.key
  • Restart and verify slapd:
    systemctl restart slapd
    netstat -lnptu | grep slapd
    tcp        0      0 0.0.0.0:636             0.0.0.0:*               LISTEN      27665/slapd     
    tcp        0      0 0.0.0.0:389             0.0.0.0:*               LISTEN      27665/slapd     
    tcp6       0      0 :::636                  :::*                    LISTEN      27665/slapd     
    tcp6       0      0 :::389                  :::*                    LISTEN      27665/slapd 
    

Setup Apache

  • Install Apache with PHP:
    apt -y install apache2 libapache2-mod-php5 php5-ldap php5-mcrypt php5-intl php-pear
    

Setup Let’s Encrypt:

NOTE: this step is only necessary if you have a public facing ldap.example.com service

  • Setup Debian jessie backports:
cat >> /etc/apt/sources.list < < EOF
# jessie backports
deb http://ftp.debian.org/debian jessie-backports main
deb-src http://ftp.debian.org/debian jessie-backports main
EOF
  • Run apt update
  • Install Let’s Encrypt utility, certbot:
    apt -y install python-certbot-apache -t jessie-backports
    
  • Generate a certificate for all the domain ldap.example.com:
    certbot run -d ldap.example.com
    

    In the TUI add the admins@example.com email and agree to the ToS. Use 000-default.conf for now.

  • Download and trust all Let’s Encrypt Root and Intermediate CA certificates:

    wget https://letsencrypt.org/certs/isrgrootx1.pem
    for i in {1..4}; do wget https://letsencrypt.org/certs/lets-encrypt-x$i-cross-signed.pem; done
    cp *.pem /usr/share/ca-certificates/extra/
    for f in *.pem; do cp -- "$f" "/usr/share/ca-certificates/extra/${f%.pem}.crt"; done
    dpkg-reconfigure ca-certificates # Select all the newlly added CA certs
    

  • Setup FusionDirectory

    • Setup Fusiondirectory Repo:
      gpg --keyserver keys.gnupg.net --recv-key 62B4981F 
      gpg --export -a "Fusiondirectory Archive Manager <contact @fusiondirectory.org>" > FD-archive-key
      apt-key add FD-archive-key
      echo '# fusiondirectory repository' > /etc/apt/sources.list.d/fusiondirectory.list
      echo 'deb http://repos.fusiondirectory.org/debian-jessie jessie main' >> /etc/apt/sources.list.d/fusiondirectory.list
      echo 'deb-src http://repos.fusiondirectory.org/debian-jessie jessie main' >> /etc/apt/sources.list.d/fusiondirectory.list
      apt update
      
    • Install FusionDirectory:
      apt -y install fusiondirectory fusiondirectory-schema
      
    • Insert core FusionDirectory schemas into *slapd
      fusiondirectory-insert-schema
      

      Verify:

      fusiondirectory-insert-schema -l
      core
      cosine
      nis
      inetorgperson
      samba
      core-fd
      core-fd-conf
      ldapns
      recovery-fd
      
    • Create a Fusiondirectory Apache vhost (/etc/apache2/sites-available/ldap.example.com.conf):
      <virtualhost *:80>
          ServerName ldap.example.com
          Redirect "/" "https://ldap.example.com/"
          ErrorLog ${APACHE_LOG_DIR}/ldap-error.log
          CustomLog ${APACHE_LOG_DIR}/ldap-access.log combined
      </virtualhost>
      <ifmodule mod_ssl.c>
          <virtualhost *:443>
                  ServerName ldap.example.com
                  ServerAdmin webmaster@example.com
                  DocumentRoot /usr/share/fusiondirectory/html
                  ErrorLog ${APACHE_LOG_DIR}/ldap-error.log
                  CustomLog ${APACHE_LOG_DIR}/ldap-access.log combined
                  SSLEngine on
                  SSLCertificateFile      /etc/letsencrypt/live/ldap.example.com/cert.pem
                  SSLCertificateKeyFile   /etc/letsencrypt/live/ldap.example.com/privkey.pem
                  <filesmatch "\.(cgi|shtml|phtml|php)$">
                                  SSLOptions +StdEnvVars
                  </filesmatch>
                  <directory /usr/lib/cgi-bin>
                                  SSLOptions +StdEnvVars
                  </directory>
                  BrowserMatch "MSIE [2-6]" \
                                  nokeepalive ssl-unclean-shutdown \
                                  downgrade-1.0 force-response-1.0
                  BrowserMatch "MSIE [17-9]" ssl-unclean-shutdown
                  <ifmodule mod_php5.c>
                  <location></location>
                      php_admin_flag engine on
                      php_admin_flag register_globals off
                      php_admin_flag allow_call_time_pass_reference off
                      php_admin_flag expose_php off
                      php_admin_flag zend.ze1_compatibility_mode off
                      php_admin_flag register_long_arrays off
                      php_admin_value upload_tmp_dir /var/spool/fusiondirectory/
                      php_admin_value session.cookie_lifetime 0
                      #Include /etc/fusiondirectory/fusiondirectory.secrets
      
                  </ifmodule>
                  <directory /usr/share/fusiondirectory/html></directory>
                      Order Allow,Deny
                      # Insert your public IPs here
                      Allow from 2.2.2.2
                      Allow from 2.2.2.3
      
          </virtualhost>
      </ifmodule>
      
    • Enable mod_ssl, ldap.example.com and disable 000-default and default-ssl:
      a2enmod ssl
      a2ensite ldap.example.com
      a2dissite default-ssl
      a2dissite 000-default
      systemctl restart apache2
      
    • Setup FusionDirectory:
      • Install optional prerequisities:
        apt -y install php-mdb2
        
      • Point your Browser to:
        https://ldap.example.com/
        
      • Create a temporary token for the setup (taken from the first setup webpage):
        echo -n r9l1srnu0rvdeca4k826nq4e05 > /var/cache/fusiondirectory/fusiondirectory.auth 
        

        Click ‘Next’

      • In the ‘Language setup’ select ‘English’ and press ‘Next’.

      • If everything is ‘OK'(Green) on the ‘Installation check’ click ‘Next’

      • On the ‘LDAP connection setup’:

        • Location name: default
        • Connection URI: ldap://ldap.example.com:389
        • TLS connection: Yes
        • Admin DN: cn=admin(,dc=example,dc=com)
        • Admin password: *************************
        • LDAP schema check:
          • Enable schema validation when logging in: Yes
      • Keep defaults in ‘Look and feel’ except ‘Timezone’:
        • Timezone: America/Los_Angeles
      • Keep all the defaults in ‘Password settings’ except this:
        • Password minimum length: 8
      • In the ‘SSL’ field use these:
        • Key path: /etc/ldap/ssl/ldap.example.com.key
        • Certificate path: /etc/ldap/ssl/ldap.example.com.crt
        • CA certificate path: /etc/ssl/certs/ca-certificates.crt

        Click ‘Next’ when done.

      • In the ‘LDAP inspection’ page:

        • Press ‘Migrate’ (twice) in the ‘Inspecting object classes in root object’ option
      • Press ‘Create’ in the ‘Checking for super administrator’ option’, fill the fields, and ‘Apply’ when done:

        • User ID: ldapadmin
        • Password: ***************
        • Password (again): ***************
      • In the ‘Checking for default ACL roles and groups’ field, press ‘Migrate’
    • From the ‘Finish – write the configuration file’, download the configuration file on your PC

    • Copy the configuration file from your PC to ldap.example.com:

      scp fusiondirectory.conf root@ldap.example.com:/etc/fusiondirectory
      

    • Apply the correct permissions to the /etc/fusiondirectory/fusiondirectory.conf file:
    fusiondirectory-setup --check-config
    Checking FusionDirectory's config file
    /etc/fusiondirectory/fusiondirectory.conf exists…
    /etc/fusiondirectory/fusiondirectory.conf is not set properly, do you want to fix it ?:  [Yes/No]?
    y
    
    • Click ‘Next’ when done.
  • Use encrypted passwords in fusiondirectory.conf:

    • Enable the headers module in Apache:
      a2enmod headers
      systemctl restart apache2
      
    • Encrypt passwords in fusiondirectory.conf:
      fusiondirectory-setup --encrypt-passwords
      
    • Uncomment the following line in /etc/apache2/sites-available/ldap.example.com.conf:
      Include /etc/fusiondirectory/fusiondirectory.secrets
      
    • Restart Apache:
      systemctl restart apache2
      
    • If everything works as expected remove /etc/fusiondirectory/fusiondirectory.conf.orig
      rm /etc/fusiondirectory/fusiondirectory.conf.orig
      
  • Allow ‘.’ (dot) in usernames:
    • After everything is set, login as ldapadmin
    • Go to ‘Configuration’
    • Press the ‘Edit’ button at the bottom of the page
    • Un-tick the ‘Strict naming policy’ option and save.
  • Installing essential FusionDirectory plugins

    • Setup LDAP related plugins:
      apt -y install fusiondirectory-plugin-ldapdump fusiondirectory-plugin-ldapmanager
      
    • Setup the DSA plugin for managing service accounts:
      apt -y install fusiondirectory-plugin-dsa fusiondirectory-plugin-dsa-schema
      fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/dsa-fd-conf.schema
      
      • Allow the service accounts to read and write the password. First create the ACL definition:
    cat > /etc/ldap/custom_ldifs/service_accounts_acl.ldif < < EOF
    dn: olcDatabase={1}mdb,cn=config
    changetype: modify
    delete: olcAccess
    -
    add: olcAccess
    olcAccess: {0}to dn.subtree="dc=example,dc=com" attrs=userPassword
      by self write
      by dn.base="cn=admin,dc=example,dc=com" write
      by dn.children="ou=dsa,dc=example,dc=com" write
      by anonymous auth
      by * none
    -
    add: olcAccess
    olcAccess: {1}to dn.subtree="ou=people,dc=example,dc=com"
      by self read
      by dn.base="uid=test.user,ou=people,dc=example,dc=com" write
      by dn.base="cn=admin,dc=example,dc=com" write
      by dn.children="ou=dsa,dc=example,dc=com" read
      by anonymous auth
      by * none
    -
    add: olcAccess
    olcAccess: {2}to attrs=userPassword,shadowLastChange
      by self write
      by anonymous auth
      by dn="cn=admin,dc=example,dc=com" write
      by * none
    -
    add: olcAccess
    olcAccess: {3}to dn.subtree="dc=example,dc=com"
      by self read
      by dn.base="cn=admin,dc=example,dc=com" write
      by dn.children="ou=dsa,dc=example,dc=com" write
      by * none
    -
    add: olcAccess
    olcAccess: {4}to dn.base=""
      by * none
    EOF
    

    NOTE: Add two spaces before each ‘by‘ in the ldif above or you will spend endless hours in troubleshooting!

    • Apply the ACL
      ldapadd -c -Y EXTERNAL -H ldapi:/// -f /etc/ldap/custom_ldifs/service_accounts_acl.ldif
      
      • Setup the Systems plugin:
      apt -y install fusiondirectory-plugin-systems fusiondirectory-plugin-systems-schema
      fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/service-fd.schema
      fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/systems-fd-conf.schema
      fusiondirectory-insert-schema -i /etc/ldap/schema/fusiondirectory/systems-fd.schema
      
      • Setup a new server:
        • Name: ldap.example.com
        • Description: Communications Server
        • Location: My Datacenter
        • Base: /
        • IP-address: 10.134.31.94

        • MAC-address: 04:01:05:d6:63:02
      • From the ‘Services’ tab setup an LDAP service on the ldap.example.com server

    You are finished. You can now start connecting services to your LDAP setup like mail or asterisk.

    References

    • https://documentation.fusiondirectory.org/en/start</contact>

    These are the instructions for installing an ORCID1 authentication node. We will be using the simple-orcid-auth-node2 developed by the ORCID organization.

    Assumptions

    • An Ubuntu 16.04 server machine but works on 14.04x with some minor changes.
    • A FQDN, let’s say orcid.example.com.
    • Server IP is 10.2.2.2 in our case.
    • Create an orcid user: sudo useradd -r -m -d /var/www/html/orcid orcid.
    • For Ubuntu 14.04 it is better to use /var/www/orcid instead of /var/www/html/orcid.
    • Also use service servicename restart on 14.04.x instead of systemctl restart service.

    Installing simple-orcid-auth-node

    • Install necessary packages (as a privileged user):
      $ sudo apt -y install nginx nodejs npm
      

      NOTE: If you are using Ubuntu 14.04.x do not install the node package. This package is completely unrelated with nodejs.__3

    • Download and extract simple-orcid-auth-node (as the orcid user):

      sudo su - orcid
      wget https://github.com/rcpeters/simple-orcid-auth-node/archive/master.tar.gz
      tar xvzf master.tar.gz
      

    • Install the application:
      $ cd simple-orcid-auth-node-master/
      $ npm install
      
    • Test run the application (as the orcid user):
      $ nodejs client-app.js
      server started on 8000
      

      Looks OK. Now point your Hit CTRL^C and move on.

      NOTE: If you prefer using the legacy node client-app.js invocation, you need to install the nodejs-legacy package as well.

    Setting ORCID as an autostart service

    • Autostart using systemd4 (Ubuntu 16.04):

      • Create the /etc/systemd/system/orcid.service service definition (as the root user):
        $ cat > /etc/systemd/system/orcid.service < < EOF
        [Service]
        ExecStart=/usr/bin/nodejs /var/www/html/orcid/simple-orcid-auth-node-master/client-app.js
        WorkingDirectory=/var/www/html/orcid/simple-orcid-auth-node-master
        Restart=always
        StandardOutput=syslog
        StandardError=syslog
        SyslogIdentifier=orcid
        User=orcid
        Group=orcid
        Environment=NODE_ENV=production
        [Install]
        WantedBy=multi-user.target
        EOF
        
      • Reload systemd and start the service:
        $ sudo systemctl daemon-reload
        $ sudo systemctl start orcid.service
        
      • Verify that the service is started:
        $ sudo systemctl status orcid.service
        ● orcid.service
        Loaded: loaded (/etc/systemd/system/orcid.service; disabled; vendor preset: enabled)
        Active: active (running) since Wed 2016-04-27 09:00:16 UTC; 37s ago
        Main PID: 11141 (nodejs)
        Tasks: 5 (limit: 512)
        Memory: 24.1M
        CPU: 268ms
        CGroup: /system.slice/orcid.service
           └─11141 /usr/bin/nodejs /var/www/html/orcid/simple-orcid-auth-node-master/client-app.js
        Apr 27 09:00:16 orcid systemd[1]: Started orcid.service.
        Apr 27 09:00:16 orcid orcid[11141]: server started on 8000
        
    • Autostart using sysv-init (Ubuntu 14.04.x):
      • Prepare a sysv-init startup script or use mine for convinience:
        $ cd /etc/init.d
        $ wget https://raw.githubusercontent.com/theodotos/arena/master/orcid
        $ chmod +x orcid
        $ update-rc.d orcid enable
        $ update-rc.d orcid defaults
        

      Now orcid should be able to autostart after a reboot.

    Setting up nginx

    • Prepare this configuration:

      $ cat > /etc/nginx/sites-available/orcid < < EOF
      server {
          listen 80;
          listen [::]:80 ipv6only=on;
          server_name orcid.example.com;
          access_log  /var/log/nginx/orcid.access.log;
          error_log /var/log/nginx/orcid.error.log;
          location / {
              proxy_pass http://localhost:8000/;
              proxy_set_header Host \$host;
              proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
          }
      }
      EOF
      

    • Enable the orcid site:
      $ cd /etc/nginx/sites-enabled/
      $ sudo ln -s /etc/nginx/sites-available/orcid
      
    • Uncomment the following line in /etc/nginx/nginx.conf5:
          server_names_hash_bucket_size 64;
      
    • Restart nginx:
      $ sudo systemctl restart nginx.service
      
    • Verify nginx with sudo systemctl status nginx.service

    Now you can visit the http://orcid.example.com site and test your setup

    Going to production

    The default simple-orcid-auth-node is using the sandbox ORCID service which is ideal for testing. This is how the configuration file (helpers/config.js) looks like:

    module.exports = config = {
      // Config for OAuth2 
      CLIENT_ID: 'APP-O9TUKAPVLALU1SOJ',
      CLIENT_SECRET: '0eafb938-020e-45a6-a148-3c222171d9d8',
      AUTHORIZE_URI: 'https://sandbox.orcid.org/oauth/authorize',
      TOKEN_EXCHANGE_URI: 'https://api.sandbox.orcid.org/oauth/token',
      CODE_CALLBACK_URI: 'http://localhost:8000/authorization-code-callback',
      // General server config
      PORT: '8000',
      SERVER_IP: '127.0.0.1',
    }
    ...
    

    This setup will not work in production. You have to modify the CLIENT_ID and CLIENT_SECRET variables with your own credentials and change the AUTHORIZE_URI and TOKEN_EXCHANGE_URI to point to the production ORCID services:

    module.exports = config = {
      // Config for OAuth2 
      CLIENT_ID: 'APP-HSGSHJS335353GSGSG',
      CLIENT_SECRET: '56d4eb21-6622-8483-3422-f53f3fs53sfs35f',
      AUTHORIZE_URI: 'https://orcid.org/oauth/authorize',
      TOKEN_EXCHANGE_URI: 'https://api.orcid.org/oauth/token',
      CODE_CALLBACK_URI: 'http://localhost:8000/authorization-code-callback',
      // General server config
      PORT: '8000',
      SERVER_IP: '127.0.0.1',
    }
    ...
    

    Restart nginx and orcid when done:

    $ sudo systemctl restart nginx.service orcid.service
    

    References


    1. https://en.wikipedia.org/wiki/ORCID ↩︎
    2. https://github.com/ORCID/simple-orcid-auth-node ↩︎
    3. https://github.com/ORCID/simple-orcid-auth-node/issues/3 ↩︎
    4. https://www.digitalocean.com/community/tutorials/how-to-deploy-node-js-applications-using-systemd-and-nginx ↩︎
    5. http://charles.lescampeurs.org/2008/11/14/fix-nginx-increase-server_names_hash_bucket_size</service> ↩︎

    This article is about setting up you own ebook library with support for the OPDS catalog format . The COPS ebook catalog can read a Calibre library and expose it as a website. It is developed on PHP.

    Prerequisites

    • Prepare a Linux system. In this guide we are using Ubuntu 14.04.4.
    • Have a cops.example.com DNS A record pointing to the IP of your server.

    Install necessary packages

    $ sudo apt-get -y samba nginx php5-fpm php5-gd php5-sqlite php5-json php5-intl git
    

    Setup a SAMBA share

    Samba is needed for sharing the Calibre library directory with the library administrator. Alternatively you could use NFS or even OwnCloud.

    Note

    It is a security risk to expose Samba on a public server. Restrict it through firewall or use OwnCloud instead.

    • Tweak the Samba configuration file (/etc/samba/smb.conf):
      #  unix password sync = yes
         unix password sync = no
      
      #  pam password change = yes
         pam password change = no
      
    • Add a Samba share for calibre (/etc/samba/smb.conf):
      [calibre$]
      path = /srv/calibre
      valid users = calibre
      write list = calibre
      read only = no
      
    • Restart Samba services:
      $ sudo service smbd restart ; sudo service nmbd restart
      
    • Add the calibre user:
      $ useradd -m -d /srv/calibre -s /usr/sbin/nologin calibre
      $ smbpasswd -a calibre
      

    Setup Calibre

    • Now you should setup Calibre on your PC. On Debian/Ubuntu:
      $ sudo apt-get -y install calibre
      

      Windows users can download it from here:

      http://calibre-ebook.com/download_windows

    • Configure Calibre to use the calibre share as its catalog:

      • You can "mount" the calibre share on Linux or "map" the share on Windows.
      • Create an ebook Directory under the calibre share.
      • Create a new Calibre library inside the ebook directory. From the Calibre menu:
        • Callibre Library –> Switch/create library -> Create an empty Library at the new location and fill the path in the "New Location" text field.
      • If you get an error about "Corrupted database" just click ‘Yes" to rebuilt it.
      • Start adding books

    Setup COPS OPDS

    • Download the software:
      $ mkdir /var/www
      $ cd /var/www
      $ git clone https://github.com/seblucas/cops.git
      
    • Setup COPS:

      Under the /var/www/cops/ directory, copy the config_default.php.example to config_default.php:

      $ cp config_local.php.example config_local.php
      

      Make the following changes in config_default.php:

      < ?php
          if (!isset($config))
              $config = array();
      
          /*
           * The directory containing calibre's metadata.db file, with sub-directories
           * containing all the formats.
           * BEWARE : it has to end with a /
           */
          $config['calibre_directory'] = '/srv/calibre/ebooks/';
      
          /*
           * Catalog's title
           */
          $config['cops_title_default'] = 'My Ebook Portal';
      
          $config['calibre_internal_directory'] = '/ebooks/';
      
          $config['cops_full_url'] = 'cops.example.com';
      
          $config['cops_x_accel_redirect'] = 'X-Accel-Redirect';
      
          /*
           * use URL rewriting for downloading of ebook in HTML catalog
           * See README for more information
           *  1 : enable
           *  0 : disable
           */
          $config['cops_use_url_rewriting'] = '1';
      

    Setup nginx

    • Create an /etc/nginx/sites-available/cops.example.com
      server {
      
          listen 80 default_server;
          listen [::]:80 default_server ipv6only=on;
      
          server_name cops.example.com;
      
          access_log  /var/log/nginx/cops.access.log;
          error_log /var/log/nginx/cops.error.log;
          root   /var/www/cops;
          #index feed.php;
          index index.php;
      
          #Useful only for Kobo reader
          location /download/ {
                rewrite ^/download/(\d+)/(\d+)/.*\.(.*)$ /fetch.php?data=$1&db=$2&type=$3 last;
                rewrite ^/download/(\d+)/.*\.(.*)$ /fetch.php?data=$1&type=$2 last;
                break;
              }
      
              #Can break loading the images - if you don't see anything, comment
              location ~ ^/images.*\.(gif|png|ico|jpg)$ {
                      expires 31d;
              }
              #Can also break loading the images, comment if it happens
              location ~ .(js|css|eot|svg|woff|ttf)$ {
                      expires 31d;
              }
      
          #Not necessarily correct, it depends on distro.
          location ~ \.php$ {
             try_files $uri =404;
             include /etc/nginx/fastcgi_params;
             fastcgi_param   SCRIPT_FILENAME  $document_root$fastcgi_script_name;
             fastcgi_pass    unix:/run/php5-fpm.sock;
          }
      
          location /ebooks {
              root /srv/calibre;
              internal;
          }
      }
      

      Note:

      The feed.php setting was redirecting me to an XML site with this error: This XML file does not appear to have any style information associated with it. The document tree is shown below. So I am using index.php instead, as the index file.

    • Enable the cops.example.com site and disable the default:
      $ cd /etc/nginx/sites-enabled/
      $ sudo ln -s /etc/nginx/sites-available/cops.example.com
      $ sudo unlink default
      
    • Restart nginx and php5-fpm:
      $ sudo service php5-fpm restart ; sudo service nginx restart
      
    • Change permissions to let nginx write to the library:
      $ usermod -a -G calibre www-data
      $ chmod -R g+w /srv/calibre/ebooks/
      

    Now you can navigate to http://cops.example.com and enjoy your newly created ebook library!

    References

    • https://github.com/seblucas/cops
    • http://blog.slucas.fr/en/oss/calibre-opds-php-server
    • https://github.com/seblucas/cops/wiki/Full-example-with-Nginx

    This is a guide to setup a recent Ubuntu system on a Utilite ARM computer, to replace the official Ubuntu 12.04 OS.

    Prerequisites

    • You need to prepare a [kernel with cgroups](/?p=1073 "Building a new Linux kernel for your Utilite Computer") support.
    • An ARM bootable microSD or USB. You can use the official Ubuntu 12.04 [Linux Utilite Image](http://www.compulab.co.il/utilite-computer/wiki/index.php/Utilite_Linux_Image "Linux Utilite Image") on the removable device.
    • Backup any data you have on the original system!

    Preparations

    1. Boot from microSD or USB.
    2. Update the system and install debootstrap:

      $ sudo apt-get update
      $ sudo apt-get install debootstrap
      

    3. Mount the SATA root filesystem on /mnt:
      $ sudo mount /dev/sda2 /mnt
      
    4. Clear the filesystem:

      Note
      Make sure you save any data you need, before doing this!.

      $ sudo rm -fr /mnt/*
      

    Installation of an Ubuntu or Debian base system

    1. Use debootstrap to install the Ubuntu base system:
      $ sudo debootstrap --foreign --arch=armhf trusty /mnt http://ports.ubuntu.com/
      

      You can use vivid instead of trusty if you are feeling adventurous.

      For a Debian system use this command instead:

      $ sudo debootstrap --foreign --arch armhf jessie /mnt http://ftp.uk.debian.org/debian
      

      You can use stretch or sid instead of jessie if you are feeling lucky.

    2. Chroot into the new system:

      $ sudo mount -o bind /dev /mnt/dev
      $ sudo mount -o bind /dev/pts /mnt/dev/pts
      $ sudo mount -t sysfs /sys /mnt/sys
      $ sudo mount -t proc /proc /mnt/proc
      $ sudo cp /proc/mounts /mnt/etc/mtab
      $ sudo chroot /mnt
      

    3. Second stage debootstrap:

      After you enter the chroot jail, we need to complete the second stage of the base setup:

      # /debootstrap/debootstrap --second-stage
      

    Installing additional packages

    1. Setup the correct timezone:
      # dpkg-reconfigure tzdata
      
    2. Add repositories in /etc/apt/sources.list.
    • For Ubuntu:

      deb http://ports.ubuntu.com/ trusty main restricted universe multiverse
      deb http://ports.ubuntu.com/ trusty-security main restricted universe multiverse
      deb http://ports.ubuntu.com/ trusty-updates main restricted universe multiverse
      deb http://ports.ubuntu.com/ trusty-backports main restricted universe multiverse
      
      deb-src http://ports.ubuntu.com/ trusty main restricted universe multiverse
      deb-src http://ports.ubuntu.com/ trusty-security main restricted universe multiverse
      deb-src http://ports.ubuntu.com/ trusty-updates main restricted universe multiverse
      deb-src http://ports.ubuntu.com/ trusty-backports main restricted universe multiverse
      

    • For Debian:
      deb http://ftp.uk.debian.org/debian jessie main contrib non-free
      deb http://ftp.debian.org/debian/ jessie-updates main contrib non-free
      deb http://security.debian.org/ jessie/updates main contrib non-free
      
      deb-src http://ftp.uk.debian.org/debian jessie main contrib non-free
      deb-src http://ftp.debian.org/debian/ jessie-updates main contrib non-free
      deb-src http://security.debian.org/ jessie/updates main contrib non-free
      
    • Run update after adding the repositories:
      # apt-get update
      
    1. Install additional software.
    • We need to create a fake /sbin/initctl to prevent apt from breaking:
      # dpkg-divert --local --rename --add /sbin/initctl
      # ln -s /bin/true /sbin/initctl
      
    • Install the software you need:
      # apt-get -y install language-pack-en ssh isc-dhcp-client net-tools man lsof less
      
    • Remove the fake /sbin/initctl:
      # rm /sbin/initctl
      # dpkg-divert --local --rename --remove /sbin/initctl
      
    1. Setup users:
      # passwd root
      # useradd -m -s /bin/bash utilite
      # passwd utilite
      # usermod -a -G adm,cdrom,sudo,dip,plugdev utilite
      
    2. Setup a kernel:

      Now you need to deploy the [kernel we prepare earlier](/?p=1073#toc-deploy-the-tarball "Building a new Linux kernel for your Utilite Computer") and reboot.

    Choose a role for your system

    1. After you reboot you may find that networking is not working. If you are in a dhcp enabled network run dhclient to get an IP address:

      # dhclient
      

    2. Upgrade the system to the latest upgrades:
      # apt-get update
      # apt-get -y dist-upgrade
      
    3. Check the available roles
    • On Ubuntu you may want to install tasksel (already installed on Debian):

      # apt-get install tasksel

    • See the available roles on Ubuntu:

      # tasksel --list-tasks
      i server Basic Ubuntu server
      i openssh-server OpenSSH server
      u dns-server DNS server
      u lamp-server LAMP server
      u mail-server Mail server
      u postgresql-server PostgreSQL database
      u print-server Print server
      u samba-server Samba file server
      u tomcat-server Tomcat Java server
      u cloud-image Ubuntu Cloud Image (instance)
      u virt-host Virtual Machine host
      u ubuntustudio-graphics 2D/3D creation and editing suite
      u ubuntustudio-audio Audio recording and editing suite
      u edubuntu-desktop-gnome Edubuntu desktop
      u kubuntu-active Kubuntu Active
      u kubuntu-desktop Kubuntu desktop
      u kubuntu-full Kubuntu full
      u ubuntustudio-font-meta Large selection of font packages
      u lubuntu-desktop Lubuntu Desktop
      u lubuntu-core Lubuntu minimal installation
      u mythbuntu-frontend Mythbuntu frontend
      u mythbuntu-backend-master Mythbuntu master backend
      u mythbuntu-backend-slave Mythbuntu slave backend
      u ubuntustudio-photography Photograph touchup and editing suite
      u ubuntustudio-publishing Publishing applications
      u ubuntu-gnome-desktop Ubuntu GNOME desktop
      u ubuntu-desktop Ubuntu desktop
      u ubuntu-usb Ubuntu desktop USB
      u ubuntustudio-video Video creation and editing suite
      u xubuntu-desktop Xubuntu desktop
      u edubuntu-dvd-live Edubuntu live DVD
      u kubuntu-active-live Kubuntu Active Remix live CD
      u kubuntu-live Kubuntu live CD
      u kubuntu-dvd-live Kubuntu live DVD
      u lubuntu-live Lubuntu live CD
      u ubuntu-gnome-live Ubuntu GNOME live CD
      u ubuntustudio-dvd-live Ubuntu Studio live DVD
      u ubuntu-live Ubuntu live CD
      u ubuntu-usb-live Ubuntu live USB
      u xubuntu-live Xubuntu live CD
      u manual Manual package selection

    • See the available roles on debian:

      $ tasksel --list-tasks
      i desktop   Debian desktop environment
      u gnome-desktop GNOME
      u xfce-desktop  Xfce
      u kde-desktop   KDE
      u cinnamon-desktop  Cinnamon
      u mate-desktop  MATE
      i lxde-desktop  LXDE
      u web-server    web server
      u print-server  print server
      u ssh-server    SSH server
      u laptop    laptop
      

    1. Choose a role.

      If you would like a Desktop system, I suggest you choose something light like lubuntu-desktop or lxde-desktop.

    • Installing Lubuntu (Ubuntu):

      # tasksel --task-packages lubuntu-desktop | xargs apt-get -y install
      

      I prefer this method rather than the [TUI](https://en.wikipedia.org/wiki/Text-based_user_interface "Text-based User Interface") of tasksel, so as to have better overview and control of the process. Keep an eye on it every now and then, because it will ask you about keyboard-configuration and other questions. Select the default settings if you are unsure.

    • Installing LXDE Desktop (Debian):

      # apt-get install task-lxde-desktop network-manager
      

      For Desktop systems it may be a good idea to install network-manager as well.

    • Add the vivante GPU module on startup:

      # echo vivante >> /etc/modules
      

    After you finish you can reboot your system and start playing.

    References

    • https://github.com/umiddelb/armhf/wiki/Installing-Ubuntu-14.04-on-the-utilite-computer-from-scratch
    • https://wiki.debian.org/InstallingDebianOn/CompuLab/PC-Utilite/wheezy