Cloud OS images from Ubuntu, Fedora, Centos and most other Linux distributions have their password authentication disabled by default. So, every time when creating a VM instance on OpenStack with a Heat template, I have been using user_data directive to set the default password, without really understanding what’s going on inside.

vm_1:
  type: OS::Nova::Server
  properties:
    name: "my_vm"
    image: "bionic-server-cloudimg-amd64-v2"
    flavor: "my_flavor"
    networks:
      - port: { get_resource: oam_port }
    config_drive: true
    user_data_format: RAW
    user_data: |
      #cloud-config
      password: mypassword123
      chpasswd: { expire: False }
      ssh_pwauth: True

If you are like me, read on. You’d be surprised how much more you can achieve with this user_data.

So, what really is user_data?

The user_data property in OpenStack resource type OS::Nova::Server, accepts a multiline string.

In the example above I have included this string inline in the Heat template. But, you could place this content in a separate file and use the get_file directive to assign it to user_data.

user_data_format: RAW
user_data: {get_file: /home/ubuntu/my_user_data.txt}

This string of data assigned to user_data property is made available to the VM at instantiation time, via OpenStack config-drive.

The config-drive is a special disk drive that gets attached to the VM at boot. The VM can mount this disk, and read its data. You could implement your own mechanism to read data from the config-drive and take actions, but cloud-init provides an industry-standard approach for doing same.

Cloud-init

Cloud-init is a software package that is included in cloud OS images of all major linux distributions. When a VM is instantiated, cloud-init can read data from a data source such as config-drive, and apply configurations or execute tasks on the VM based on that data.

Config-drive is an OpenStack specific implementation, but you will find that other private and public cloud services also implement similar data sources.

What is cloud-config

Cloud-init accepts data in multiple formats. The most common and the simplest format is cloud-config, which is indicated by #cloud-config at the beginning of the user data string. Cloud-config has a rich syntax for executing many of the common tasks in setting up a new VM instance.

These tasks are implemented in a set of modules. Let’s explore what we can achieve with some of them.

Set passwords

As I mentined earlier, all the Linux cloud images have their password authentication of the default user disabled for security reasons.

We can use cloud-init Set Passwords module to enable password authentication.

#cloud-config
password: mypassword123
chpasswd: { expire: False }
ssh_pwauth: True

This cloud-config does three things.

password set the password of the default user. In Ubuntu this user is ubuntu, and in Centos the defaut user is centos.

By default this password is set to expired, so that the user has to reset the password at the first login. chpasswd { expire: False } disables this behavior, so password need not be reset.

ssh_pwauth: True enables password authentication for SSH login.

decoded

Configure key based authentication

If you need more security, you could set the authentication keys instead of enabling password authentication.

#cloud-config
ssh_authorized_keys:
  - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMXxvEtXqvqVJkYsJAhyppj5ZTStp0DEbeHWioziTlaZ0aTi0JHW9L3xDESyzPcmLxkGoGxXT6Tmx9AwUbhG4N1LrQw+8M96ZNj5e+repOUxR+ly+xl3UqxeVp46fllvSFnvXBwJCkyWR67Mbb/As9rVhld9SqzBmUriWtIhgJHv9MKqxT9Y4xxFyHGsg75WREtvzSA97QA9hHzfb86dn+t6Ir+hOvtVMq/V9og2CSbVqgPIElz0FxznpdhMPDGTRBZ/FL3XrSufQN+Ft9fqz78iskxv5AwmApmjT8mnoDBY+WDdnjJutkV/cwkBpF83nWdTEmXl2lDV5B1IBOTS1x ubuntu@my-server

ssh-authorized-keys accepts a list of public keys which get added to the authkeys file of the default user, so that we can use SSH key based authentication.

Creating Users

Cloud-init can create users and groups and assign users to groups.

#cloud-config
groups:
  - cloud
  - qubes
users:
  - default
  - name: cloudqubes
    shell: /bin/bash
    groups: cloud
    lock_passwd: false
    passwd: $6$rounds=4096$pd39WNugExZ2wI$LdHyr9jpullz7pmjl5WwEAULp5FK20BZLYg4N5YCUvX8QfS1depsNNh9RA7gNBIn0Vs5O4DzL4RM/jp5dMrwe1
  - name: indika
    shell: /bin/bash
    groups: [cloud, qubes]
    ssh_authorized_keys:
      - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEWi+eymLXDclcjc+ycQUm+mDmCw+yrE2D+EBKg+5Z2ugkMZ5ZrQHS4rWsEb9VorYcRJjJfC56u/gpooKRWzwqJtYjLMjxwHphc/+sCjwINMOuSOGLXksToRZPhSjTz5YjdajVdgacBjn/Pe5lU8HYfZxMeWx/thzpPUscJLiLXnlxKoQHm2urWyWbmvXxg4y1TYbeFUN8xH/2S9UFjm4QDAPNdsR3u7TYDJaGTTEoWz/l8n3qNnefZycWGnKihhobGOXsbNOf8Gb52J7kBmGfDgoRU8q9LMaiPPeXwTvdvhdLio4ce1tUb8CF3mFYwcszXiYPS+7SlhX0jzmP8DXZ ubuntu@palo-alto-jumpserver

groups key accepts a list of groups to be created.

users accepts an array of user objects.

The user is created with password authentication disabled by default, so you need set lock_passwd: false to enable password authentication.

The passwd within users key is a hash, and not the plain text password. This hash can be generated using mkpasswd utility which should be available in most Linux distributions.

$ mkpasswd --method=SHA-512 --rounds=4096 "mypassword"

Instead of password authentication, we can set the authentication keys for user with ssh_authorized_keys, which accepts a list of public keys for SSH authentication.

The groups key inside user object accepts a list of groups which the user belong to. These groups must either already exist in the system, or must be created with groups key in cloud-init.

The default directive under users instruct the system to leave the default user account intact. If omitted the system will make the first user under the users, the default user.

Package management

Different linux distributions use different package management software such as apt in Ubuntu, yum in RHEL/Centos, yast in Suste etc. Encapsulating these underlying variances, cloud-init implements a unified interface for package management, on all its supported Linux distributions.

Updating the package repository - the list of available packages and their versions - is a common requirement of a new VM. This is done via package_update directive in cloud-init.

#cloud-config
package_update: true

Cloud-init can upgrade installed packages as well. After upgrading some packages it may be required to reboot the system, which is handled by package_reboot_if_required directive. When set to true it reboots the system.

#cloud-config
package_upgrade: true
package_reboot_if_required: true

Package installation is handled by package directive, which accepts a list of packages. It is a good practice to update the repository before installing new packages

#cloud-config
package_update: true
packages:
  - squid
  - unrar

Additional support for APT

Sometimes, package manager needs additional configurations. Cloud-init has a module Apt Configure which supports user specific configurations for APT pacakge manager.

Using this module let’s configure proxy server for apt.

#cloud-config
package_update: true
apt:
  http_proxy: http://10.30.30.10:8080 
  https_proxy: https://10.30.30.10:8080

You need not be dissapointed for cloud-init not having such modules for other package managers, because you could implement a work-around with Write Files module.

Writing to files

The write_files directive can wirte to an array of arbitrary files.

#cloud-config
write_files:
  - content: |
      # My config file
      param1="xyz"
      parma2=50
    path: /etc/my-config
    permissions: '0644'
  - encoding: base64
    content: bXkgbXVsdGkgbGluZQpiYXNlIDY0IHN0cmluZw==
    path: /etc/my-base64-config
    permissions: '0755'
  - content: |
      10.199.254.115 ftp-server
    path: /etc/hosts
    append: true

The path specifies the destination of the file. The optional parameter encoding is set to text/plain by default, but can accepts a range of formats such as base64, gzip etc. In such cases content is decoded with the specified encoding format, prior to writing.

The content kay accepts a multiline string, which will be written to the file. Optionally, the permission of the file can be set, with permissions key.

There’s another optional key append which can be used to append the content to an existing file.

Run commands on first boot

Cloud config has modules to accomplish most common tasks reqiured for setting up a new instance. However, if you find any specific taks that cannot be achieved with one of those modules you can use runcmd module to run arbitrary commands at the first boot. runcmd accepts an array of strings as commands.

runcmd:
  - mkdir /disks
  - mkdir /disks/b

Cloud-init log files

When things go wrong, log files are your friend.

Config file /etc/cloud/cloud.cfg.d/05_logging.cfg specifies the logging configurations of cloud-init. It has two main log files.

The console output is logged at /var/log/cloud-init-output.log. This would be the first place to look, when troubleshooting.

Cloud-init debug log is written to /var/log/cloud-init.log. It can help you for more advanced troubleshooting.

Why use cloud-init?

Automation is a key aspect of cloud computing and NFV. Cloud-init provides a robust solution, for automating most of the deployment time configurations in VMs, also known as day-0 configurations.