Metadata service
When deploying Compute Instances, it's almost always necessary to perform additional configuration before you can host your website or run your workloads. This configuration might include creating a new user, adding an SSH key, or installing software. It could also include more complex tasks like configuring a web server or other software that runs on the instance. Performing these tasks manually can be tedious and is not ideal at larger scales. To automate this configuration, we offer two provisioning automation tools: Metadata (covered in this guide) and StackScripts.
Overview
The Metadata service provides a convenient method to automate software configuration when deploying a Compute Instance. It's an API that's accessible only from within a provisioned Compute Instance and provides relevant metadata to that instance. The Metadata service is designed to be consumed by cloud-init, an industry standard software that automates cloud instance initialization. This lets you use the same tool across multiple cloud providers, enabling a pathway for provisioning your systems as part of a multi-cloud infrastructure strategy.
The Metadata service provides both instance data and optional user data, both of which are explained below:
-
Instance data: The instance data includes information about the Compute Instance, including its label, plan size, region, host identifier, and more.
-
User data: User data is one of the most powerful features of the Metadata service and lets you define your desired system configuration, including creating users, installing software, configuring settings, and more. User data is supplied by the user when deploying, rebuilding, or cloning a Compute Instance. This user data can be written as a cloud-config file, or it can be any script that can be executed on the target distribution image, such as a bash script.
User data can be submitted directly in Cloud Manager, Linode CLI, or Linode API. It's also often programmatically provided through IaC (Infrastructure as Code) provisioning tools like Terraform.
When a Compute Instance first boots up, cloud-init runs locally on the system, accesses the metadata, and then configures your system using that metadata.
Comparison to Stackscripts
Similar to Metadata, StackScripts service can be used to automate system provisioning. However, the StackScripts service is Akamai-specific and does not interface with cloud-init. If you wish to keep your system provisioning tools cloud-agnostic and industry-standard, we recommend using the Metadata service.
Availability
Akamai's Metadata service is available in select data centers. Additionally, user data and cloud-init integration is currently only supported in a few distribution images. Supported data centers and distributions are listed below:
-
Data centers: Amsterdam (Netherlands), Chennai (India), Chicago (USA), Jakarta (Indonesia), London 2 (UK), Los Angeles (USA), Madrid (Spain), Melbourne (Australia), Miami (USA), Milan (Italy), Mumbai 2 (India), Osaka (Japan), Paris (France), São Paulo (Brazil), Seattle (USA), Singapore 2, Stockholm (Sweden), and Washington DC (USA)
-
Distributions: Ubuntu 22.04 LTS, Ubuntu 20.04 LTS, Debian 11
When selecting a distribution in Cloud Manager, the following icon designates distributions that fully support the Metadata service:
Compute Instances deployed in a supported region can always access the Metadata Service API to obtain instance data, regardless of the distribution. However, user data cannot be submitted for distributions that do not yet have cloud-init support.
Add user data when deploying a Compute Instance
The Metadata service is always active, so there's no need to enable it. User data can be provided to the Metadata service, which is then consumed by cloud-init when your Compute Instance boots up for the first time.
-
Cloud Manager:
-
Navigate to the Linodes page in Cloud Manager and click the Create Linode button. This opens the Create Linode form.
-
Fill out the form with your desired settings. Be sure to select one of the supported distribution images and data centers.
-
Expand the Add User Data section and enter your user data into the User Data field.
If you are unfamiliar with cloud-init, seeUsing cloud-config files to configure a server for help creating a cloud-config file.
-
Once you are ready, click the Create Linode button to deploy the instance.
-
-
Linode CLI:
linode-cli linodes create \ --label new-instance-with-metadata \ --region us-iad \ --type g6-standard-2 \ --image linode/ubuntu22.04 \ --root_pass [your-root-password] \ --metadata.user_data [your-user-data]
Replace [your-root-password] with a strong root password and [your-user-data] with the cloud-config data or script you wish to use. When using the CLI, user data must be a Base64-encoded string. Review the Base64 Encoded section below to generate the string.
-
Linode API:
Run the API curl request below, making sure to properly paste in or reference your API token.
curl -H "Content-Type: application/json" \ -H "Authorization: Bearer $TOKEN" \ -X POST -d '{ "label": "new-instance-with-metadata", "region": "us-iad", "type": "g6-standard-2", "image": "linode/ubuntu22.04", "root_pass": "[your-root-password]", "metadata": { "user_data": "[your-user-data]" } }' \ https://api.linode.com/v4/linode/instances
Replace [your-root-password] with a strong root password and [your-user-data] with the cloud-config data or script you wish to use. When using the API, user data must be a Base64-encoded string. Review the Base64 Encoded section below to generate the string.
When your Compute Instance boots up using a compatible distribution, cloud-init runs. If it detects that this is the first time running on this instance, it connects to the Metadata API and captures the instance data for that instance, including any user data that you added. It then uses that metadata to provision the software on the instance, including setting the hostname to the instance's label and executing the user data script.
User data can be added when creating a new Compute Instance, rebuilding an instance, cloning an instance, and restoring from a backup.
User data formats
User data can be provided in many different formats, with the most common being cloud-config.
-
Cloud-config script: cloud-config is the default syntax for cloud-init and can be used on any Linux distribution. It contains a list of directives formatted using YAML. Review the Using cloud-config files to configure a server guide for more details.
#cloud-config package_update: true package_upgrade: true packages: - nginx - mysql-server
-
Executable script: Cloud-init also accepts other scripts that can be executed by the target distribution. This includes bash and python. Since many commands (including those to create users and install packages) differ between distributions, providing these scripts may limit which distributions you can target.
#!/bin/bash apt-get update -y && apt-get upgrade -y apt-get install nginx mysql-server -y
-
Other formats: Review the User data formats guide within the official documentation to learn more about other types of formats supported by cloud-init.
Base64 encoded
When submitting user data through the Linode CLI or API, you first need to encode it into Base64 (without any line breaks/wraps). To do that, run the command below that corresponds with your local operating system. Replace [file] with the name (and path, if needed) of your cloud-config or script file.
-
macOS:
base64 --break=0 --input=[file]
-
Linux:
base64 --wrap=0 [file]
Modify cloud-init configuration and save a custom image
Our supported distribution images have cloud-init pre-installed and configured to interact with our Metadata service. Beyond submitting user data, you are not able to adjust cloud-init settings directly through Cloud Manager, Linode CLI, or Linode API. If you do wish to deploy Compute Instances using a modified cloud-init configuration, you can use our Images service.
-
Deploy a new Compute Instance using your preferred supported distribution image.
-
Log in to that instance using SSH or Lish and then modify the cloud-init configuration files (or add your own). These files are typically located in the
/etc/cloud/
folder./etc/cloud/cloud.cfg
: The distribution-provided configuration settings./etc/cloud/cloud.cfg.d/
: A directory containing other configuration files that are processed by cloud-init./etc/cloud/cloud.cfg.d/99-linode.cfg
: The Akamai-provided cloud-init configuration file. Since this is processed after the other configuration files, the settings here override any of the same settings that exist in those other configuration files.
-
Once you have configured cloud-init with your desired settings, create a custom image from that Compute Instance.
Now, when you wish to deploy a new Compute Instance, you can select your custom image. During the creation workflow, you can attach any desired user data for that particular instance. When cloud-init runs, your updated configuration settings will be used alongside any user data that you've added.
Access the Metadata Service API
In addition to being consumed by cloud-init, the Metadata service can also be accessed through an API. The API is available on industry standard link-local IP addresses (169.254.169.254
and fd00:a9fe:a9fe::1
) and returns only instance data and user data for that Compute Instance. For more details on using the API, see Use the Metadata Service API.
-
Log in to a Compute Instance that has been deployed in a supported data center using a supported distribution image.
-
Generate your API token by running the command below:
curl -X PUT -H "Metadata-Token-Expiry-Seconds: 3600" http://169.254.169.254/v1/token
Instead of receiving the token as an output string, you can save it directly to the
$TOKEN
environmental variable:export TOKEN=$(curl -X PUT -H "Metadata-Token-Expiry-Seconds: 3600" http://169.254.169.254/v1/token)
-
Query one of the following API endpoints to receive data from the API. If you did not save the API token to the
$TOKEN
variable, replace$TOKEN
in the commands below with your token.-
/v1/instance: Output information about your instance, including plan resources.
curl -H "Metadata-Token: $TOKEN" http://169.254.169.254/v1/instance
-
/v1/network: Output information about your instance's IP addresses.
curl -H "Metadata-Token: $TOKEN" http://169.254.169.254/v1/network
-
/v1/user-data: Output your user data.
curl -H "Metadata-Token: $TOKEN" http://169.254.169.254/v1/user-data | base64 --decode
-
Technical specifications
- The Metadata service is an API designed to be consumed by the cloud-init provisioning tool.
- All user data is encrypted and the Metadata service is only accessible from within the Compute Instance.
- Supports custom user data in the form of cloud-config scripts, shell scripts, and more.
- User data can be added when creating, rebuilding, or cloning a Compute Instance. User data can also be added when performing one of those functions using a custom image created from a compatible distribution image.
The Compute Instance must have a public network interface to access the Metadata service.
Troubleshoot metadata and cloud-init
View cloud-init logs
If you encounter issues with cloud-init or your user data, you can review the logs and output logs by running one of the commands below:
cat /var/log/cloud-init.log
cat /var/log/cloud-init-output.log
If you are not able to access your system through SSH, you can use Lish or boot your instance into rescue mode and mount your disks.
Run the cloud-init
command
cloud-init
commandThe cloud-init
command-line tool can be used to gather more information or perform certain actions. Here are some commands you can use to help troubleshoot cloud-init.
cloud-init status --long
: This provides information about the status of cloud-init. You will notice different output depending on if it is actively running or if has run in the past. See cloud-init status.cloud-init query v1
andcloud-init query userdata
: This outputs either the instance data or the user data provided by the Metadata service. Other metadata is also exposed by entering a different query key. See cloud-init query.
Updated 13 days ago